Category: Data Science

I made a simple AI system to create images

I created a simple AI system to create images of handwritten numbers. This is a relatively simple example but something I can build on when I have time to create more sophisticated images.

Here I have broken down the concept and hopefully given a simplified explanation. For those more technical, this explanation isn’t aimed at you.

To get us started, let’s admire these two images created by Bing Create using the prompts:

  • a sunny day with hills and a lake
  • a sunny day with hills and a lake and a killer robot

How does AI make images?

One of the most popular techniques for creating images using AI is something called Generative Adversarial Networks, more commonly referred to as GANs.

GANs are a type of artificial neural network.

Primarily, regular neural networks are trained using data where they are given some data and the outcome. For example, when I was going my PhD I would train them with medical data for a patient along with the outcome (did they die? did they come back into the hospital?). I would need to give it hundreds, sometimes thousands, of patient data. The neural network would try to learn the patterns between the medical data and the outcomes, therefore developing the ability to predict the outcome (death/readmission) based on new patient data. When we use both the input and output data we refer to this process as ‘supervised learning’.

GANs are different.

Firstly, they are not a single neural network, instead, they are 2 neural networks. Secondly, they are unsupervised – meaning they don’t know the outcome.

These two neural networks are called a Discriminator and a Generator.

So how do they work? (the non-technical explanation)

The Discriminator and the Generator are both trained at the very same time.

Using the handwritten numbers as an example…

We pass some examples of real handwritten images to the Discriminator model, which tries to learn the patterns.

At the same time, we feed in some random noise into the Generator which will try to turn that into a new image.

This new image is then passed through the Discriminator which essentially decides if the new image is what it would expect to see (from the patterns it had seen before). For example “does this new image look like a handwritten number?”.

It is not saying – “does this new image look like a number 8 or a number 9” – it is simply – “does this look like a number” or perhaps a better way of explaining it is “I’ve seen a lot of images, is this the same type of image I’m used to seeing?”

It’s a little like the hot and cold game. “You’re getting hot (closer)” or “you’re getting colder”.

The Generator uses this feedback to learn and try again, generating another image. Repeating the process over and over again.

Example

Using the one I built, below we can see how the AI system learned and improved over time.

Here it creates 32 handwritten numbers shown in a 8 x 4 plot.

In the first image (top left), each of the 32 images are just pure random noise. All of those images get shown to the Discriminator which has just started learning what real numbers look like and it says “those images you showed me look rubbish”. The AI system says they ‘look rubbish’ by giving them a bad score.

The Generator uses this bad score to update its internal workings and tries again (image 2).

Over time the system starts to generate images which look like real numbers. In this example, they look pretty good by the time we get to the 6th attempt.

The 7th image shows it got it wrong again, sometimes this happens. But given enough training time it gets back on track (see images 8 – 10).

After training for 10 cycles (called epochs in machine learning) we can see some numbers. As shown in the image below they are not perfect, but I can see a 1, 2, 5, 6, 7, maybe 8 and a 0.

Now my Generator is trained I can ask it to generate as many new images as I want.

So what does this have to do with art?

So there isn’t much difference between training a GAN to create handwritten numbers and training a GAN to create artwork.

However, these machine-learning approaches take a long time to train, so we often test the concept on smaller problems first. This handwritten example took 10 minutes to train on a 3060Ti graphics card. Training on a CPU would have taken significantly longer.

But using the same approach you can simply change the training dataset and create a whole bunch of new images.

Below is a very hi-res example from a GAN trained with the celebrity faces dataset [source]. Here it has created images of new faces.

And below is an example of a GAN trained with the work of Vincent van Gogh [source].

If you want to read a more technical understanding of GANs, check out this paper: Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (arxiv.org)

Summary

So, I created a GAN using Pytorch and trained it on handwritten number images. After 10 minutes of training, it could produce reasonable new handwritten numbers.

GANs have two neural networks, a Discriminator and a Generator. The Generator tries to make an image and the Discriminator decides if it was good or not.

GANs can be trained on any image data to create new images in the style of the images it was trained on. They are not copies but are instead close approximations

4 ways to gain experience (that work)

There’s this meme out there which goes something like – “entry-level position, applicants must have 10 years experience”.

It’s the chicken and egg scenario – you need a job to get experience, but you need the experience to get the job.

So how do you get that experience without having the job first?

Here are 4 ways I have done it in the past…

Find associated roles and work your way through them

Early in my career, I pivoted from engineering to IT. I primarily did this by completing a degree in Business and IT, but I knew I would still need some IT experience to help me stand out from the crowd.

Fortunately/Unfortunately – I had to work to fund my degree. One of the jobs I had was working in customer service in a call centre, and the product I was supporting was an ISP (Internet Service Provider). The role was simple, customers would call in mostly about account issues but we’d also handle basic first-line support issues too – “have you tried switching it off and on again”. It was a very basic role, but it was my first IT role.

This then enabled me to move on to my second IT role, Network Support Engineer at a company called NRM. At the time we supported one of the world’s largest gaming networks – doesn’t that sound amazing? Don’t get too excited, it was the Bingo gaming network – and yes, I would have to listen to a bingo game every night to make sure there weren’t any issues with the network.

By the time I finished my degree, I had experience in IT and it helped me stand out from the crowd, and get my first full-time IT position on a graduate management scheme (at the time called TCS).

Do work for FREE

Some people don’t like this at all, but it can be the quickest way to gain experience.

I did this to break into data science.

At this point I was studying for a full-time PhD, I was in my 30’s and for the last 5 years, I had been teaching at the local college. I knew I would be leaving uni in about 3 years’ time and that I would want to work in industry rather than academia. But I didn’t have any work experience on my resume in this field.

To fix this I signed to C4Di which is an incubator “that helps tech companies grow, and traditional businesses innovate”.

I would turn up one day per week, every week for around 2 years.

When things were quiet I’d just work on my PhD, other times I’d work on my own projects too. But every now and again I’d come across an opportunity to offer my services for free.

One rememberable example was when a startup asked how they could match candidates with jobs, for their startup idea. Now that was interesting, so I went away spent a week or so hacking some prototype together and gave them it. They were blown away and shocked that I had done this and had done it for FREE. I was happy for the experience, it became a talking point at future interviews but also that relationship became an important one with my next career move (a story for another time).

Another benefit from turning up and offering free services was that I got offered a bunch of opportunities to talk to people about their problems and ideas. One example is a company that wanted to use machine learning to solve a business problem. I listened, asked questions, came up with a technical solution and then told the guy – “I don’t think you need machine learning to solve that”. A few days later he offered me a job, which paid pretty good money (I didn’t accept the offer as I was still completing my PhD).

By the time I had finished my PhD, I had a bunch of experience in applying different data science and machine learning techniques to a bunch of different problems in a bunch of different domains.

If I had charged for my services? – then I would have made a little bit of money but had a lot fewer experiences.

Personal Projects

IDK if I’m just built differently to most people, but I always have some personal project going on.

During my 20’s I was obsessed with web development and was always playing with different web technologies. Building hacky websites in PHP, learning how to power them with MySQL.

During my 30’s I started to become very interested in game development. I even released a couple of mobile games, and built even more unfinished games just for the experience (I’ve even tried building Minecraft probably 5 or 6 times, in both Unity and Unreal Engine – which moderate success).

Now in my 40s, it looks like content creation is my newest hobby still sprinkled with a little bit of game development. But I’ve also built a robo-advisor, a news aggregator and recommender system, and a mobile app and played about with different cloud backends. And to be clear, that’s in the past 3 years.

But does this count as experience? In my opinion – of course, it does, especially if you don’t have anything else.

The goal here is to “have things to talk about during an interview”. 

Interview questions aren’t – “tell me a time when you sat at a desk in an office”, instead they are “tell me a time when you had to solve a complex problem”.

One example I did was conduct my own research in computer vision (during a quiet time when I was at C4Di). I did the full research – from collecting the data and experimenting to writing and publishing a paper.

Other machine learning research also looked at NLP (Natural Language Processing). I didn’t publish a paper on this but when I get asked the question “tell me about an NLP problem you worked on” – well I have a project to discuss.

Find experience in your current role

This is really dependent on your situation. If you’re flipping burgers at Mcdonald’s you might not have the opportunity to try building aeroplanes (2 random examples from my life).

In my current job, we’re very lucky. I’ve seen people go and work with other teams to try it out for a month or two. If you’re interested in a project then people will often welcome you in and often talk you’re head off because they’re so excited to share.

One example I have was gaining experience as a manager in big tech. I’ve managed before across my career, but managing in big tech is different. My old team was piloting an IC to a manager training programme for 3 to 6 months, and I had a very supportive manager who got me on that. In short, I did it, gained the experience, applied for a bunch of manager jobs and got one. All off the back of that experience.

An earlier example comes from that period in my 20s I discussed earlier. I was super interested in web technologies, the company I was working for had a problem tracking inventory – I wrote a tool/software system for that.

Summary

There is the chicken and egg scenario – where you need a job to gain experience, but you need the experience to get the job.

But there are some things you can do about it.

You can work your way through different roles, making incremental steps towards your goals. I did this in the earlier part of my career.

There are opportunities to gain experience by working for FREE. Payment doesn’t always have to be cash, you can get paid in experience. See it as an investment in the future.

Having personal projects has always paid off for me too. You’re not only learning, you are trying to solve problems while learning.

And finally, have a look around in your current role – you might be surprised to see opportunities to gain experience around you.