Jen Lowe and Patricio González Vivo on Democratizing Knowledge
Jen Lowe and Patricio González Vivo on Democratizing Knowledge
The co-authors of The Book of Shaders join Peter Bauman (Monk Antony) to discuss the book’s impetus and impact. They also explore how algorithms serve as a dialogue between artists across time.
Peter Bauman: I’d love to hear how you two first met.
Jen Lowe: We both ended up moving to New York in 2012, at different times, to be part of the creative coding world. I moved there for a data visualization job at Columbia and Patricio moved there to go to Parsons School of Design to do his MFA in Design Technology.
I was one of the cofounders of SFPC, which has gone through multiple iterations since then. Zach Lieberman was a cofounder and one of Patricio's teachers. Whenever I talked to Zach, he would just randomly bring up Patricio and say, “You should meet this person.” Then I saw Patricio's work come through; somebody retweeted it on my Twitter timeline. I was thinking, “Oh, my gosh, Zach keeps saying we should meet.” Then we met.
Patricio González Vivo: We met professionally because Zach said he was introducing me for a teaching position. We met for a professional coffee but then it became more of a dinner. We started talking and we ended up in a bookstore, talking about books. And we started dating after that.
Jen Lowe: We have a particular world in common. Zach was right. Whatever he was feeling, I don't know what it was at the time. I don't know that he remembers saying that.
Patricio González Vivo: We moved in together. And when I finished my studies at Parsons, I got a proper job as a graphics engineer.
Jen Lowe: I was very into Black Mountain College at that time, which was actually founded as a liberal arts school in the forest of North Carolina and ended up having a lot of Bauhaus folks moving over from Germany after World War II. Everybody just lived and worked together. I was inspired by that in SFPC—everyone making meals together and having an intensive life-study program with everybody spending time and learning together.
Peter Bauman: That's really interesting that Zach introduced you. That whole lineage between you two and Zach goes back even further to Golan Levin and John Maeda as well.
Patricio González Vivo: I feel part of that lineage. I met Golan through Zach, too. We also became very close friends with Theo Watson, who is another co-creator of openFrameworks, which I was really involved with. From the MIT Media Lab, there's this family tree of Scott Snibbe, Golan Levin, Ben Fry and Casey Reas, with Casey being a student of Ben and Zach being a student of Golan. I also had the privilege to work with Scott Snibbe, who is another former student of John’s. Also tied into that family tree is Daniel Shiffman, who invited Jen to teach a math class at ITP [the Interactive Telecommunications Program at NYU Tisch].
Peter Bauman: Can you also talk about how you got started collaborating on The Book of Shaders?
Patricio González Vivo: After finishing my MFA, I got my first job. At the same time, I was still teaching a class at Parsons.
Jen Lowe: He needed materials for teaching that class.
Then we thought, “If you're going to write the materials, why not make a big open-source book that's available to everyone?” Back then, there was not anything I’m aware of that made shaders accessible.
Patricio González Vivo: Shaders was this very niche knowledge. There were a lot of industry trade secrets between computer science, PhDs and very specific people working in the game industry. This knowledge was very gated. By then, I had been working with OpenGL and shaders for four years. Through the friction of having to learn everything and research how things were working, I decided that it would be awesome to democratize this knowledge. By then, Jen was pregnant with our first son so I had extra anxiety.
Jen Lowe: Patricio had a real compulsive need to write that. He put all of his new parenting anxiety into that project.
Patricio González Vivo: I came to Jen saying, I have this idea for a book of writing a gentle Shader tutorial in the spirit of The Inner Structure of the I Ching: The Book of Transformations, where each chapter was an evolution of the previous one. But I wanted to do it with an artistic twist through symbolic progression. There's something very algorithmic about the I-Ching because it's very binary. At the time, Jen was like, “I think you have three ideas all put together. Just make it easy for other people to learn shaders.” Jen brings clarity to the project.
Jen Lowe: Most of my professional background before I got into creative coding was in math education. I'd done a lot of teaching high school and college math and then I taught math for artists at ITP [NYU] and SFPC. I just have a real impulse towards making math accessible. I think it's fake that the way that it's taught makes it seem disinteresting.
I think a huge part of what I brought to it was saying, “Let's make this something that anyone who comes to it can understand.” That's always what I'm trying to bring.
You don't have to like it or want to do it but I don't want it to seem like math’s some special, magical thing that only some people can do.
Patricio González Vivo: She also helps me with it in my day job. I usually come with PhD papers with functions I don’t know and ask her to translate them for me.
Jen Lowe: I'm very into demystifying. There might be a lot of symbols but it's not actually hard.
Patricio González Vivo: There was another level of her needing to demystify myself because my natural way of talking is very Latin—this very Spanish tradition where you are very baroque, using floral language and metaphors all the time. And Jen really did...
Jen Lowe: I just brought some linearity. [They laugh.]
Peter Bauman: You mention the importance of demystifying complex concepts. For collectors who may not have a technical background but hear about them a lot, how would you explain shaders in an approachable way?
Patricio González Vivo: I think the choreography metaphor is perfect for that. If you are a single dancer, then you have a line of specific movements that you want to do—like a supermarket list. We start here and we go there. That is the sequential thinking that we're usually familiar with when we think about code, especially CPU-side code. Shaders introduce the idea of parallel programming. Now you are creating code that is going to be executed by several dancers in several performances.
You can think about it like an orchestra. It's not just important what one single instrument is doing. What they are doing collectively is important.
On the more technical side, we have computers that have one single brain that does this thing. But to do multiple tasks—like deciding the colors on your screen—having one person change all the colors is a lot of work to do. But if you have thousands of performers—with everyone choosing only one color—that is way easier and happens way faster. That's why GPU architecture has been so successful: because we are increasingly more visual in the use of our computers. Video and images are now ubiquitous. We expect to be able to do video and replay it on a device of any size. That's GPU. There is a relationship between this and AI, too, because it requires the same amount of processing power that needs to be done in parallel.
Peter Bauman: The choreography and orchestra metaphors are very helpful. What about for artists? I’ve heard some speak about shaders’ difficulty and a reluctance to explore them. What would you say to encourage artists to fight through that?
Patricio González Vivo: I can tell you what I love and what they have unlocked in my career. I'm a visual person, predominantly. Shaders is like a computational language designed for geometry, color and light—it's the language of light. Its architecture is very powerful because it's from the perspective of the most minimal components of an image—the vectors or the pixels. I love that it touches everything that is graphical and gives you full control over those details. GPU power has also been constantly increasing, and the parallel architecture of shaders is not only unlocking real-time graphics but also empowering AI and virtual reality. All the latest compelling and expressive technology out there is leveraging shaders or programs that run on the GPU. I think for everybody who is creative, it's all unlocked. It's an incredible tool to learn if you are creative.
Many people designing these languages with PhDs in computer science see it as a medium to construct but they don't necessarily have an artistic sense. They are not native visual thinkers. They come from this very mathematical verbal logic and are appropriating this technology and bridging it to creativity and visual thinking.
Artists can leverage this—what is natural to us—to contribute to exposing the assumed supremacy of language-based intelligence. There are so many people whose intelligence is visual first and not language first. It's a logic that, in tech, tends to suffocate the other logics. Now in AI, everything is prompt-based. That's because the people who designed and paid for these tools are very comfortable giving commands to other people—or a machine. It's a very verbal-mathematical logic. Creative thinkers—artists and designers—operate in different logics: collaboration, collage, visual composition, movement, sound and gesture.
Jen Lowe: For me, Dan Shiffman's book and the accessibility of Python ML tools made coding a lot more accessible. It went from something you learned in your computer science course to something you can play with yourself, make something cool and be experimental with. Part of what I think is really cool about The Book of Shaders is the fact that you put the code right in it. You have the opportunity to experiment. It's experimental learning.
I was a math major, but I took a computer science class in '96 and it was one of the worst experiences of my life. I never wanted to touch a computer again. It was horrible [laughs].
Then I found Dan Shiffman's book and it was like, “Oh, I can do this.” It wasn't because I couldn't do it; it was the culture that demotivated me. Dan’s book and videos made computing and learning to code more fun, accessible and experimental.
Peter Bauman: If somebody sees an art work and is interested in it for any reason, then they learn the work was made by shaders—what would you recommend they do next if they want to dig beyond the surface level and into the process of the work?
Patricio González Vivo: I recommend reading The Book of Shaders as a primer [laughs.] Then I recommend checking out the work of Inigo and what is out there in Shadertoy. There's a strong relationship between the shader world and the demoscene. A lot of old demosceners like Inigo Quilez, Mr. Doob and Jaume Sanchez (Clicktorelease). They all met through shaders. I have found them in my work career around places like Samsung, Apple, Adobe and Oculus. There are all these people who were in this petri dish of the demoscene. That bunch is very interesting because the mentality was very focused on visual and technical hacking in order to make cool, flashy demos.
I think for a collector, it will be interesting to see the difference between what is happening with people interested in challenging hardware and their own technical capabilities and what it means to go from there to something that feels expressive. I think that understanding that delta will be an easier thing, especially once you learn that
Dan Shiffman wrote about flocking, reaction diffusions, circle packing, vector flows and all these algorithms in his books. You see them over and over and rewritten again and again in the last twenty years. They have been reused and reused and reused and reused.
Jen Lowe: There's this particular creative coding pattern language with maybe ten different types of things that people tend to be remixing.
Patricio González Vivo: It’s important to understand the history of those algorithms, where they come from—all the way to the papers from the '90s to innovative or more technically focused implementations—and then how they were reimagined along the way. I'm thinking about vector fields and the work of Jeremy Rotsztain in the late 2000s. Then there's a relationship between that and Tyler Hobbs’s Fidenza. We see echoes of the same principles today and I think that’s interesting.
We are seeing the dialogue between artists—not just the output—in the same way that you see a work from Francis Bacon and you understand that he was in communication with somebody else.
For collectors and artists, the interesting part is how the artists are communicating with each other through their work. That will help you detect the “Hello World” version of certain techniques compared to something that feels mature.
And it's mature artistically because it's based on this lineage, on this conversation, which is exactly what I feel that Le Random is doing by interviewing and going back all the way to the beginning.
Peter Bauman: Thank you and I do appreciate your broader point of understanding a work through its genealogy. This goes back to what you said about the demoscene as well. You mentioned how parallel computing enables AI. What is the link between shaders and the development of AI and ML?
Jen Lowe: It all goes back to linear algebra. Linear algebra has built everything everybody's getting so excited about right now. The availability of large-scale computing plus linear algebra makes shaders, machine learning and AI possible.
Patricio González Vivo: In the Venn diagram, that is the core of everything. In terms of technology, it's funny because the concept of neural networks has been around for ages but has been in parallel with the development of GPUs. There are quite old theoretical papers that couldn’t be applied until GPU power caught up with the number of neurons—around 100 billion—in the human brain. We hadn’t seen them applied in the past because they required a wild amount of power.
Jen Lowe: The theory becomes applied based on the hardware. The theory is waiting for the hardware to catch up and then the application is possible.
Patricio González Vivo: I find it interesting because the original models are based on our brains yet our brains are basically arriving at the conclusion that parallel computation is the best. There is something lasting about the eternal love of humankind for ourselves and our attempts to replicate ourselves. This goes back to when we drew the first humans with stick figures or the Renaissance study of the human form and perception. We took God aside and we put humans in the center.
---
Jen Lowe is an artist, teacher, writer, researcher, and cofounder of the creative technology studio Maximal Expression.
Patricio González Vivo is a digital alchemist who creates artifacts of contemplation about time and space.
Peter Bauman (Monk Antony) is Le Random's Editor-in-Chief.