Exploring Shaders for the Web with Rik

Transcript:

Faith: Hi, Rik. Welcome.  Today we're going to do an interview with you on the shaders course, which is super exciting. Can you start by telling us what a shader is because some people might not know.

Rik: Yes. So a shader is a way to do things using the graphics card and basically a kind of like quick version of this is how to make almost interactive textures and colors and gradients  that you can change and really do new things with.

It's not a common thing on the web just yet.  It's a fairly cutting edge thing, but previously it was used for things like game development. So a lot of the same concepts of rendering textures and doing lots of weird things with colors is now being made available on the web.

So what we're doing is basically applying a lot of this game development and style of really getting into depth with how we make textures from scratch and going into that and applying it to a web.

Faith: Interesting. So, how did you go about creating the course? What did you want to show? I noticed 2D, 3D stuff. Tell us a bit about the projects. 

Rik: A lot of the shader stuff that you see on the web, it's very obviously cutting edge and a lot of it was done in the 3D realm. So using web GL using three dot JS, and what we kind of realized is that this was kind of leaking into 2D environments as well. So, you know, a quick example of that would be the Stripe homepage. The homepage has this kind of like texture, this gradient that moves and feels very subtle, but everyone who sees the Stripe homepage is like, wow, look at this, look at this cool little texture going on, how do they even do this? A lot of people are like, is this a video? Is this something else? What we saw is a lot of top websites now, and a lot of top agencies or top products, top product startups are starting to use this as well.

So we were like, how do we do this across lots of different types of style and how do we kind of apply things? It goes beyond just a normal Javascript-style website. How do we make it that cutting edge so that people can do lots of strange things or are they kind of almost quite artsy things with something that wasn't potentially possible a few years ago?

A quick example would be a slideshow. If someone was doing a quick slideshow, for instance, Usually what they would do is something like, click ‘next’ and then the image appears, which is fine. You know, having a new image of phases is good, but you know, if you kind of add that next level, like how do I do these transitions between them?

I might not just want to do a fade, I might want to do something like a pattern that waves in the next texture or do something completely different with that. It adds that nice extra element which goes beyond just being a slideshow. It feels a lot more interactive.

So that's something that we really focus on, taking the same things as what you do day to day on the web and thinking like, how would you add that next level to it? How would you go beyond the next step? You know, quick example would be, if we have one of the early projects, it could have just been a video background or it could have been just a static image.

Instead, we were kind of thinking, how do you make this interactive? How do we make something that looks like a texture interactive, you know, moving the mouse around, being reactive to things, move things or time, for instance, as well. Also running really, really quickly as well. So that's kind of where it came from: what do people do on the web right now? What would take it to that next level? 

Faith: Yeah, the projects are amazing. They look incredible. Which one did you have the most fun making, which are you particularly excited about? 

Rik: I mean, they’re kind of all my babies, so I like them all the same, but there's a few I really liked personally.

There's actually one or two that I was kind of like, I might keep this with my own little projects on the side! So, the third project called Alice McLeod is using this kind of like texture, which uses noise and randomness and a bit of grain in there as well, and it has this really dynamic background to it.

It's a simple landing page, you know, it's a contact page with links. Usually that'd be a fairly basic page, but if you add this moving texture, this randomness, if you refresh the page it has a different look and feel. It feels very, very interactive and it goes beyond being a very basic page.

And also it's a relatively quick thing to make as well. So I think that’s my favorite one, but also some of the later 3D ones as well, like we have a project called Jellyfish, which uses a lot of lighting, so we're kind of thinking, you know, where’s the light on this, how do we make it metallic? How do we make it have shadows on there as well? So, you know, that was something that we really wanted to show: how do we apply this game development-like lighting and metallic-ness to a website. That's really not done that often. This is where we wanted to push the boundaries a little bit more. 

Faith: Yeah, I love that one as well. That was put in later wasn’t it?

Rik: Yeah, there was a project that was too big. I wanted to really go in depth on one, one of these lighting projects to show the range of things that people could do with it, and also the control of it as well.That's something that I think this course really does well is like, it gives you very fine grain control over everything that you do. So if you want to make the lighting slightly different, it's your code that you're really interactive with and you can change it. It's your stuff. You want to change the lighting, it's your lighting. 

Faith: Cool, and there's 10 projects! That's a lot a for course. Do you need a lot of maths for this? It looks quite technical and places. Is it? 

Rik: So it is a technical course. There is a little bit of Javascript and HTML and CSS that goes with this just because it's interactive. From a shader point of view, it’s a bit of a weird concept, so it's not mathematical but you have to think about every single pixel individually. What I mean by that is when we usually do a web page, for instance, we kind of take it as a whole. So, you know, I have a web page and there's a button, if I click the button, something else happens. With shaders, it's per pixel, and it's thinking about what is happening at that time. And we basically write in a program for a pixel, which is kind of a weird concept because usually do from a webpage or a backend or something else.

This is kind of thinking like, how do you do this one thing? And it's not really connected to any of the pixels around it, which means that you're kind of limited in what you can do in terms of like, we don't know what the pixel is going to say, we don't know what the next frame or the last frame is going to be.

We have to think about things in a very logical way. So it's not really mathematical, it's more kind of thinking, okay, I have this thing in space and like a grid. What is this thing going to be at this time? What color is this pixel going to be at this time? So this is something that's a little bit different, it's a bit of a weird concept to get your head around at first. 

If you've seen this code for the first time, it looks quite strange to be honest, but we build it up piece by piece. So it started with this kind of like, how do we make these grading kind of effects? How do we go into noise and grain? How do we do these kinds of transitions as well? How do you have textures in there? So it’s not really mathmatical, but there's some technical parts to it too. 

Faith: Yeah, I saw a student said on Slact that they love the approach of this course because it does really feel like you're building and learning in chunks.

Rik: Yeah, we tried to make it almost like a story arc, like a TV series in a weird way, where you have these callbacks to previous projects and you can see things in different contexts, I suppose. We want to show people how they do it in their own projects, so you see the same kind of patterns appear again, and again, and one of the things that we've seen already is people using this in their own projects and they've done it really, really quickly. Some of the first people who've taken this course are building this stuff in real life projects so they understand how it works. They're doing it on their own. 

Faith: Is it mainly designers doing this, do you think? Or developers?

Rik: It feels a bit like digital art. And I'm kind of saying that at the moment of crypto art and NFTs. For me, it was kind of like, how do we make this stuff very fluid? So, you know, it's a mix between people who want to kind of take their designs to the next level. Some of this stuff you can't do with Figma. You can't really do this with other design programs. You have to get into the code and that should do it. That’s one of the fun things about it is, you know, you're getting your hands dirty and you're making lots of accidents along the way, so it’s very experimental. That's kind of how we approach it is like, how do we go about this thing from an art standpoint. Let's take one of the projects, like the green kind of background project that we have [Alice McCloud]. That was built up from lots of trial and error and lots of going in different variations and trying out different things. We kind of had a rough idea of what we wanted it to be in the end but it can go in different ways. That's one of the nice things about doing this stuff in the code. You do lots of accidents and you make something brand new and it's completely different. You could say it's better than what you actually had in your head. That's one of the fun things about this course, you change two or three things and it's something completely different. 

Faith: I can't wait to experiment! Did you come up with many kind of blockers? Did you break things? I guess some of those breaks and maybe a happy accident, but were there any kind of tricky bits that you came up against? 

Rik: I think the main thing for me is like what I said earlier, it's like getting your head around this idea of per pixel programs. For me, as someone who's worked in like web pages and backend code for a long time, I have this holistic view of everything where it's like, I know exactly what this whole thing is going to do, whereas this is very much like everything is running concurrently and it's a very strange concept. Getting my head around that kind of concept to begin with was like, okay, I don't have any concepts about what was next to me. That is quite strange for someone who's done JavaScript for a long time.

It was like, how do I make a texture using basically a million different, like mini programs all running at the same time? It really kind of messed with my head to begin with. Like why are we doing it this way?

I think one of the other things as well is the language that we use, GLSL, is very limited in scope on purpose because it runs so often, you know, it runs per pixel. So if you have a thousand by a thousand, that's a million times that it’s running and you have this limited palette, but using that kind of limited palette, you can actually make a lot of very, very creative things.

Faith: Amazing. And Web GPU, this is kind of said to be the future. How does that compare and, um, what's your view on those working side by side? Which is better?

Rik: It's very interesting because there's a lot of hard technology now, which is coming out from a web standpoint. Take something like Figma, that's all built on top of web technologies under the hood, but when you install it and use it, it doesn't look like that. It looks native, almost, so I think that's one of the things that all these kinds of new technologies give us, the ability to do things that we usually would be reserved for programs on a Mac or Windows that now we can do within the browser itself. It just does give us a power to actually do things that are more accessible from technology point of view. You're not dependent on Mac or Windows or iOS or whatever it is, you're now dependent on, can I access it in the browser? Can I go to a URL? That’s a lot more accessible So I think there's this movement in technology. Javascript for instance, is becoming that kind of native language of the web, but the web is now becoming more of a platform for a lot of tools. You know, the fact that Figma could do things very, very quickly is because it's a web technology. Compare it to something like Sketch, which is just Mac-only. If you have Windows, you can’t use Sketch, I think that's a big deal. Figma is very, very accessible because I can send you a link and you’re in that tool set very, very quickly.

So I think it's a great thing. The more that we have these kinds of inbuilt technologies in the web browser, the more accessible that this technology is for everyone. 

Faith: And it's KodeLife that you use for this?

Rik: So KodeLife is a program that we use at the moment to do some sketches, almost, so that we can really control what we can have as an input. KodeLife is basically an installment program by Hexler, a really great company. It’ss kind of like sandbox, I suppose, where you can play around and make your own shaders from scratch. They mainly do things for audio environments, so if you're a geek and there's lots of visual effects that are written in code, you can plug into the PA system. We're not really caring about music too much because we do things with the web, so we're using that as a basis and then transferring it into web technology.

Faith: Can you integrate music? 

Rik: Yeah, so it's a little bit more tricky just because audio on the web is not as common as just inputting things in the music player. You can do it pretty easily, it’s just, you get your audio tag, put it in your Javascript, read the things that you want to read, and then you do things with the shaders on top of that.

Faith: Maybe that could be a piece of content one day! So what are the limitations of this technology? Because obviously we've got all these shapes and effects we can make, it seems quite limitless at the moment, but there’s got to be some limitations…

Rik: I mean, it is a cutting edge thing, and I think if you're on an older browser, there might be some things that don't support certain technologies with that as well. I think at the moment, you know, we've looked into the amount of people that can actually use this technology, and I think it's 95% of the people who can actually see this stuff and feel like it runs really, really quickly as well. So I think that's something that we've been really focusing on.

It's like, how do we make sure this stuff runs really quickly in the browser? There's no point if it runs slowly, like, you can see something cool, but if it runs slower, then what's the point? Yeah, really making sure that it's accessible as well to as many people that want to see it. I think as we go forwards, and more and more of this technology gets added in, it just expands the toolset of what we can do. I'm very excited for the next few years to see what comes out. Okay. 

Faith: Is there a favorite shader that you've seen on the web? Something that you love (apart from the Stripe example!)

Rik: I was going to say the Stripe homepage! There's a few, I mean, there was one that I saw the other day where they used kind of like blur effects, so it had this kind of like 3D environment that as you move your mouse around the page, it almost looks like a water effect. It drags things around the page and moves images. There's a lot of things as well where you can scroll down a page and you get this kind of RGB split. I think the most powerful ones are the most subtle ones as well.

Like there's ones that you can get that really knock you over the head with it. And it's like, look at this really interactive site that has tons of like flashy things on there. I think it's really simple ones that do the best work, because you have those little sprinkles rather than all of the sugar on top. You're not trying to do too much with it, but you're just taking it above what it should be. 

Faith: I think you've said before that with shaders, you can achieve amazing things with quite little code.

Rik: Yeah, that's one of the nice things is because, like I was saying earlier, this stuff runs per pixel. The pixel doesn't really care about anything around it. So as humans who are kind of like looking at this whole canvas or output of what we actually have, you can get very, very interesting effects very quickly that, you know, a pixel, a pixel doesn't care about this stuff, it’s just running the thing that is told to run. You can make these kinds of really beautiful effects really, really quickly, and in under 20 lines of code. Some of the tutorial videos that we've got on our YouTube channel, you know, they're very, very short amounts of code. They're very interactive, very changeable, and they're very quick to make. That’s one of the powers of it, there's a lot of stuff you can do on top of this. In a weird way, it's almost like a styling language, almost like CSS, you have that power over what things look like.

That is a really kind of powerful concept because you do something really, really subtle and it can expand into lots of brand new things. There's so many variations, so many bits where you could branch off and do your own thing, it becomes very experimental.

Faith: Awesome, thank you so much. The shaders course is out now!