Editorials

Artist and engineer Ryan Murdock authored BigSleep, the foundational open-source notebook that paired BigGAN with OpenAI's CLIP. Less than a day after CLIP's release in January 2021, he posted a white clock that opened text-to-image generation to anyone with a browser. Murdock spoke with Peter Bauman (Monk Antony) about the lasting significance of one of the first widely available, open-source text-to-image techniques. They discuss the hacker ethos and story behind BigSleep's origins, Murdock’s own roots in Processing, glitch art and CPPN GANs, plus what the latent space reveals about ourselves.

Peter Bauman: I heard you say that you got into machine learning (ML) at 16. How did you hear about it and get started at that age?

Ryan Murdock: When I was 16 I was mostly doing procedural art. So it would have been Processing. I did a lot of visualizers back then with projectM.

A lot of what I was initially working on was old-school generative art, using stochastic input and RNG but with your hand-coded dynamics. You try to make something interesting and most of the time it will incorporate some randomness.

I was probably closer to 18 when I started doing ML generative art. In that transition, I remember watching a Coding Train tutorial by Daniel Shiffman doing a perceptron and thinking, “This is so neat.”

I was really interested in this idea that you could do learning with these relatively simple rules. They make it so the system eventually settles into this place where it has learned to do a task.

The first project that I did was adapting the perceptron and implementing backprop through a multilayer perceptron. I don't know if I ended up doing a convolutional net back then. I hooked up a webcam to the pretty small perceptron and visualized it.

Definitely in hindsight using Processing for backprop is a funny idea because it's kind of the wrong place to put it. I'm sure that TensorFlow and Keras and all of these other libraries already existed.

Peter Bauman: That whole MIT-Processing scene was also entrenched in ML with Casey Reas, Tom White and Golan Levin. So you got into ML at 18 and what year was that?

Ryan Murdock: The tutorial stuff with Shiffman was in 2018. I would have been about twenty at the point when I was doing CPPN (compositional pattern-producing networks) and GANs. So I'd been doing it for a little while at that point. I was really interested in glitch art for a very long time.

In some ways, generative and glitch art are adjacent to hacker culture, saying “We'll break things to try and make something pretty.” That definitely inspired the approach to working with CLIP.

The CPPNs were a predecessor to SIRENs. Basically, you give a point along a grid to the neural network and it produces a pixel color at that point. You're almost doing a shader where the computations are of neural networks. That was another one of the earlier things I worked on.

Maybe the first novel, or at least novel-ish, thing that I did was taking the CPPN and using it for a GAN. I think that people had been using CPPNs for a lot of interesting work, but I don't know if it had been as often applied to adversarial learning.

I did a lot of studies with that where I was training it on tree datasets and that sort of thing.