November 18, 2014
Developer Blogs: Editorial Planning and Distribution
Iris Shoor is the co-founder and VP of Product and Marketing at Takipi, a Heavybit member company that helps developers understand when and ...
About the Guests
Kyle Mathews is the founder of Gatsby.js. Kyle is dedicated to building Open Source web technologies and helping companies adopt them. Prior to Gatsby, he founded RelateRocket, and worked as Product Manager and Senior Frontend Developer at Pantheon Systems.
Brian Douglas: Welcome to another installment of JAMstack Radio. In the house, we've got Kyle Mathews.
Kyle Mathews: Hey, glad to be here.
Brian: Cool. And Kyle, you are the creator of Gatsby, the React static site generator, correct?
Kyle: This is correct.
Brian: Cool, awesome, which is why we have you here. So I guess we can just start, what is Gatsby?
Kyle: That's a great place to start. So,
Gatsby is a ReactJS website framework that lets you build entire websites using ReactJS.
Brian: Cool. So you say it's a React-based framework, not a static site generator. Is that correct?
Kyle: You can think of it as a static site generator as well. It's kind of an open question on the branding so far. Yeah, we like to call it like a framework because it does much more than kind of a traditional static site generator, and that kind of can confuse people.
Brian: But how would it compare to something like a Jekyll?
Kyle: So, you think of like Gatsby as Jekyll++.
I think this is really powerful because you can combine kind of the app and the website paradigms into one thing, which I think is really important. In this day and age, the internet's bigger and like, more competitive, and if you're building a website, it's really nice to be able to take advantage of everything that kind of modern app tools give you.
Brian: So at Netlify we personal use Hugo for our marketing site. I do have a Gatsby project for internal stuff that we haven't shipped yet, but it's also internal at the moment. But Hugo, what we do is we inject, well, my coworker that built the marketing site just injected a bunch of jQuery in there.
Kyle: Exactly, yeah.
Brian: With Gatsby, the way I understand it is that it's actually React first. That's what's doing all the rendering for you. Is that correct?
Kyle: Yeah, it's React only.
Brian: React-only, yeah.
Kyle: Yeah, and that's what so great. I mean,
Brian: Yeah, which is great, not having to do any context-switching. So if your actual app is already a React app and you're doing all the normal React stuff in there, but then you have to throw together something else, actually even zooming out, you could actually do your entire app in Gatsby. Is that correct?
Kyle: Yeah. Yeah, there are actually people who are building kind of marketing sites combined with their normal application. Like, there's several SaaS companies that like, everything is Gatsby.
Brian: Interesting. Yeah, yeah, because yeah, we have some issues having separate repos, having a Hugo context and then having a React context. I think we've solved a lot of those, but I think if everything is Gatsby, I think like the whole login thing, like, you can log into the marketing site, and then it's going to switch you over to the app site, which would be pretty awesome.
Kyle: Yeah. And you can like share components then across everything.
Brian: Yeah, your style library actually would be shareable as well.
Kyle: Yeah. because one of the companies I worked at after a year or so there the marketing side did this big rebrand, and then all of the sudden, the app looked completely different than the marketing website. And that sort of stuff was really common. It's hard to keep everything in sync.
But if you have kind of like a component library and a design system that can work both in a marketing context and in kind of like the app context, then it's really trivial to kind of keep everything looking very similar and like, feeling very cohesive.
Because marketing is always desperate for engineering resources, but engineers don't want to like, jump into whatever the marketing people are using, because typically, it's some CMS thing that's like ugly and repulsive.
Brian: Or a jQuery mess.
Kyle: A jQuery mess. Yeah, or been worked on like a dozen different people in a dozen different contexts, and so forth. And it's just not fun. But if the whole company can use like, this common core set of React components, then it's pretty straightforward to keep moving forward quickly.
So, can we zoom back out a little bit more about Gatsby? I want to find out, like, so where did it come from? Why did you decide to build a static site generator?
Kyle: So, great question. So, a bit about my history. I actually started in programming in college back in the 2000s doing Drupal. That was kind of like my first exposure to the programming. So I did that really intensively for a few years, actually. I went to a bunch of Drupal cons and did a bunch of consulting in college and a bit after around Drupal and so forth. So, I was pretty heavily steeped in kind of the CMS website world.
And then after college, a few years after college, I worked at Pantheon, which is the company here in San Francisco that does Drupal. They did just Drupal then, and now WordPress as well. They build like really nice developer and hosting tools.
And so anyway, so I just fell completely in love with that whole paradigm of having these really, decoupled atomic components that you could just mix and match together and have really brilliant reuse patterns.
And I was like, "this is amazing. I'm so productive. I can like, ship things without bugs that are exactly what I want." I could think of something and build it, like super quickly, and I just never had. It just felt like a dramatic improvement over what I was using before, which was, Drupal in the past, and then, Backbone and jQuery. Anyway,
I wanted to like, ship HTML, but build it with React components. So I was like, okay, how do I kind of like, blend these two? Like basically, how do I like, take React and webpack and use that to build a static site generator?
And anyways, and that's kind of where Gatsby came from. And it was basically just like, it started to just like, oh, it'd be cool if this existed sort of thing, can I build something like that, because if it existed, that'd be great. I would use it. And that was about mostly what I was thinking. But then I open-sourced it and it kind of just evolved from there.
Brian: Yeah, and it basically took over your life?
Kyle: Basically, yeah.
Brian: So, did you have plans on making this, when you open-sourced it and said the "here's Gatsby thing", did you have plans on like, "oh, this going to be the thing I want to do and build projects in this form," or were you just like...
Kyle: No, I mean, I'd been working on the startup then for about almost a year, so we were pretty set on making that a success. It was just, React was early and there were a lot of people just experimenting. Like, "hey, there's this brand-new thing. What can it be?"
And that was, in a way, I could contribute to kind of the developing React story. It's like, "hey, here's this way of taking React and using it in kind of a novel, interesting, potentially valuable way." And I mean, early in any kind of experiment I had an inkling that it could be something pretty significant, but that wasn't the goal.
Brian: Yeah, so I remember, I used Gatsby for some small little side projects and just like, checked my couple boxes somewhere. The reason why you built it was like, see if this works, and so it seemed much like, oh, cool, there's a lot of things that work together. This is awesome.
And so you recently released 1.0, so that was like a long time coming, and there's a lot of new things in that that are really cool that are built into 1.0. One being like the plugin system, the other thing being a lot of the PWA, like progressive webapp enhancements, so like the offline thing. But like, keeping that, that idea first. Do you want to talk about some of those things you added in 1.0?
Kyle: So, kind of like carrying on from the story, so when I open-sourced it, it was," hey, this would be awesome, but like most hacks, you're focused on getting to like the first stable point you can find, not necessarily like the global maximum type thing. So anyways, v0 of Gatsby had a lot of limitations.
One of the big ones was it wasn't like a progressive webapp in the sense. One of the big things about progressive webapps is that it kind of like progressively boots up. It kind of pulls in code and content and code and data as necessary to make the app work. But
So that was one of the big limitations that I wanted to address, is that I wanted to have built-in code and data splitting for Gatsby. So no matter how big your site gets it'll still boot up really quickly. It'll still load really quickly and then only kind of gradually pull in additional code and data as needed as you kind of like navigate around the site.
And so the idea of Gatsby plugins is that it abstracts that low-level configuration behind a plugin system. So, you can just say, "I want SaaS set up correctly to work with Gatsby and webpack," and you just install it, and then that's it. And then people who then care about and are knowledgeable about how SaaS should work the best way, then they kind of maintain the actual plugin code.
So, the Gatsby plugin system is really powerful. It can do anything to webpack and set up Babel plugins and a bunch of other stuff. So it's really kind of fun to see that people who are really new to web development, they can just add plugins, and things just work. And they can get going without a lot of effort.
Yeah, so then kind of the last big thing was a new data sourcing transformation system. So, one of the other big limitations for static site generators is that they only work with files on disk. Which is great if you're a programmer hacker dude and you're like, I want to build everything with Git and Markdown and files on disk, and I just want to like, pull up my text editor and code away. But the vast majority of websites don't work that way.
You actually have other people who don't sit there all day working on it. And so there's like, of course, a whole world of tools for, CMS's, generally, content management systems, of all flavors, for letting people edit, create and edit content.
Gatsby's data layer lets you integrate Gatsby with basically any data source. So, if you have an API, you can write a Gatsby source plugin which goes to that API and pulls down data and lets you then pull it into pages on your site.
So, there's already source plugins for like WordPress or Drupal, for Firebase. Somebody built one for Lever and Workable, which are two jobs kind of management tracking tools, so that they can for their website, they can like pull down the current list of open jobs.
Brian: Was there was a recent Contentful plugin?
Kyle: Yeah. So Contentful has a really nice plugin as well. So they're pretty easy to write, and they're really powerful. They kind of like recreate all the data from this remote system right in your Gatsby environment.
Brian: And it is exposed in some sort of like, so in Hugo they call them taxonomies or some sort of variable. If you use the plugin, it's exposed in like a Contentful or like a Gatsby data?
Kyle: One kind of interesting constraint/advantage of Gatsby is, because any code you write for Gatsby runs on both, during the build process, but then also runs in the climate. Which means that, so, like Hugo and Jekyll and other places, in all your templates, you have access to all your data. Basically, they just collect all your data and like, stick it into the template and like, "here's everything. Do whatever you want."
And with Gatsby, that doesn't work, because word data you put to a React component also has to be shipped into the client to be available there, because it's like running the exact same thing, both at build time and also when it gets loaded into the browser or the smart phone or whatever, wherever the site's running. And so because of that, we have kind of anyways, a different way of pulling data into components. And we use something called GraphQL.
And GraphQL is a kind of, it's a query language sort of similar to like, SQL. So Gatsby uses GraphQL to query whatever data your source plugins bring in. So, if you have a Contentful source plugin, then you can query the different content types that you're pulling in from Contentful. So you can have a post type and a product type and upcoming events type, and then those will be automatically turned into a GraphQL schema, and then you can just write, say you have a page with side bar with like a list of upcoming events.
You can have a React component, which, in that React component has a query for a list, like the next, the next five upcoming events, sorted by their date, and then you can query some other fields, like the location and this, that, and the other thing. And then really easily turn that then into a component, which you can then drop anywhere on the site.
Brian: I haven't messed around with any of the GraphQL stuff yet for Gatsby. I've been intrigued enough to eventually get there. I just have to get the framework up. I'm trying to build like, so, we have a marketing team at Netlify, and we have a lot of notifications that need to be maintained. Not maintained. People just need to be updated on like, what is the status of this, what's the status of that.
So as I build an internal tool, and I wanted to use Gatsby because why not and to be able to provide like a list or a stream of data to show off and stuff like that. And so, the data will be sourced from multiple places, GitHub and probably Slack, I think. I'm not sure. I haven't thought through it. Trello is another one that we also used.
Kyle: We need source plugins for all those.
Kyle: So, each plugin can take options. So, typically, if you're connecting to Remote API, you need some sort of access key, access token. So the plugin, it just like takes that from the options that the user passes in, and then it just does an API call. Sometimes APIs will have like, client libraries, which will kind of smooth over the process, or other times you'll just get an HTTP library for Node.js and issue a call by pulling stuff. And then it's just kind of up to the source plugin to Gatsby-ize the data.
How Gatsby thinks about data is in the terms of nodes or individual chunks of data. So if you use WordPress and say "post 10 pages", the WordPress source plugin takes each post and turns that into a Gatsby node.
You have a few required feels like ID, and then like a type, and then you just then add all the rest of the data that's like applicable to that field. And then anyways, and that's basically all you do. You just say, there's a Gatsby function called createNode. And you pass in the data, and you just create all your nodes using that, and then Gatsby then kind of stores all that data and then uses that to A, infer the schema.
Anyways, it's storing the data. It creates a schema, and then you can run queries against that schema which then pull out data from the store. So, from an implementation, like a source plugin implementation standpoint, it's really straightforward. because all you have to do is say, "hey Gatsby, here's data," and then Gatsby handles the rest of it.
Brian: Okay, cool. Yeah, it sounds like a fun weekend project. You're still using webpack as part of your build system as well?
Brian: Okay, cool. And that's not abstracted away. Like, if I wanted to add things to my webpack, do I go there, or do I reach for a Gatsby plugin?
Kyle: Gatsby plugin is advised, because typically, they've had a lot of attention to make sure that they're correct and so forth, and also it's nice because then you're abstracted away from webpack version changes. So we're getting fairly close to upgrading to webpack 3. That'll be the next major release of Gatsby. And if you've directly reached in and manipulated the webpack configuration, then it's up to you to update that for webpack 3, which could involve some serious pain.
Brian: Yeah, maybe.
Kyle: But if you're just using a plugin, then it'll be upgraded for you, and you don't have to worry about it.
Brian: Okay, very cool. Yeah, I'll keep that in mind. I know I got that call out when we were at React Rally. I built a template for Gatsby, and we used style components, but we didn't use the actual Gatsby plugin. So, noted.
Kyle: Yeah. And that's the cool thing about plugins, there's all sorts of potential optimizations that can be done for different things. And so, the plugins are great, because they kind of hide that complexity from you.
Brian: Some of the maintainers, they're doing some refreshers on their React docs, and it seems like they're using Gatsby for it.
Kyle: Yeah, yeah.
Brian: So, have you been involved in that conversation or those efforts?
Kyle: So Brian Vaughn on the React core team at Facebook, he's been leading up that effort. And anyways, we've been friends for a while, so I've been helping out with that. Gatsby v1 is still new enough that any significant project is probably going to run into bugs, slash, missing pieces. And anyways, so I've been helping smooth over that and make sure thats smooth process.
Brian: Yeah, we wrote some code because they're hosting it on Netlify at the moment, so we wrote some code to help support them. So we've been well aware of that happening, but it's nice to see that they're getting closer and closer to making that switch.
Kyle: Yeah, it's pretty close. I think the idea is to launch that along with React 16.
Kyle: Of course, we'll all know while you're listening to this, since I guess React 16 will be out by the time this is out.
Brian: Yeah, it'll be out, and you'll probably be looking at a Gatsby page in the React docs, so it's all going to be fun.
Kyle: Yeah, you'll be like, "this is so amazing. It's so fast."
Brian: Awesome. So what's next for Gatsby? Now you have 1.0. I know you mentioned webpack 3 is going to be, so, 2.0 is going to be webpack 3? Are you going to make that as a breaking change?
Kyle: Yeah. So that's the main thing. We'll also upgrade to React 16 and Babel 7 as part of that. That's probably most of what we'll try to tackle in that upgrade, and the community members have actually done a lot of the work on the webpack 3 upgrade.
So I just need to carve out some time soon to focus on that. And so that's Gatsby 2, and then Gatsby 3 is starting to, starting to think more seriously about that. Gatsby 3, the plan is to focus on scalability. So Gatsby v1 was a big bump to our scalability. Like, v0, if you got past a few hundred pages, you're in trouble, and like v1, you can comfortably get to 1,000, 2,000.
It's getting increasingly painful and somewhere around 5,000, 6,000, maybe just kind of starts to crap out. But Gatsby's always been designed that it could scale to essentially any number of pages. Anyways, I thought a ton about it just when I was working on v1. And anyway, so
I want to start tackling that problem as well, just like figure out how to unleash Gatsby on the really big sites of the world. because it'd be awesome to have sites of 100,000 pages, a million pages running Gatsby happily.
Brian: Yeah, that'd be a really interesting case study, slash, I'd love to watch the release notes as they get closer to being done.
Kyle: Yeah, so yeah. So, looking for kind of basically partners, I guess, large websites that want to jump into kind of the React Gatsby world.
Brian: For perspective, do how many pages the Facebook React docs are?
Kyle: I would guess a few hundred.
Brian: A few hundred? Okay.
Brian: Awesome, cool.
Brian: Yeah, I think we just about covered, unless there's anything else you wanted to add in there. I know you guys are open-source, so, you probably really appreciate PRs, issues etc.
Kyle: Oh, totally, yeah. It's been really phenomenal to see just the growth of community since v1 was launched. I just actually just published a new blog post earlier about kind of the growth of community since v1 launched. And anyways, I was just counting some numbers. So there's been 100, 110 new contributors in like, the last 77 days, so a bit over two months. And there's been, like, 400 MergePRs in that time as well.
Brian: Wow. And, are you the one reviewing them all, or do you have anybody else helping you out as like some sort of a help maintainer?
Kyle: There's a few other people, but I think I'm doing most of it right now.
Kyle: But, I mean, it takes a while to groom good maintainers, if you will. I mean, they have to have enough experience with different parts of the code base. Plus they have to like, clearing. They have to have enough time, I guess, to jump to it and so forth. So, so far, it's been me, mostly me. But there's definitely some community members who are starting to contribute more and more, who are like maintainer-type stuff. And people are like, contributing in deeper and deeper ways, which is really great to see.
Brian: Yeah, which is good, and it's going to set you up nicely to get that 1,000-page website.
People, when choosing tools, they really love general tools. They just want this is my Swiss Army knife, and it can do anything that I need it to. It's like always this comfortable, familiar tool that can perform well in lots of situations. And so, for Gatsby to get to that point, it just has to have gazillions of contributors that have exercised it in every possible situation.
so that there's not like this blessed path that, if you stay in there, it's comfortable and works well and it's not bugs, but like you step off it, and then, well, hey, why did you step off of it? We only care about this sort of thing. So I want Gatsby to be kind of that general-purpose tool.
Brian: Yeah, yeah, cool. Well, hopefully, I'll get it inside Netlify and I'll start being able to one.
Kyle: Netlify your soul.
Brian: Yeah, Netlify my soul. I'll be able to hopefully get a plugin or two out of the way.
Kyle: Yeah, that'd be awesome.
Brian: You'll see me around.
Kyle: Yeah, looking forward to your PRs.
Brian: Cool, and yeah, so, that's the conversation. And I kind of want to transition us to JAMpicks, so if you're familiar. Anything you're jamming on, things that keep you going.
Kyle: Yeah, so, I thought I'd pick some albums that I've been, music albums I've been really enjoying lately. I listen to a lot of music. So I switched to Spotify a few years ago, and whenever you make kind of a major music platform change, you just lose music for a while. But it's kind of fun, though, because eventually, you run across them again, and you're like, "oh, yeah, I used to listen to that in college when I like, had CDs and stuff, or whatever."
Brian: What are CDs?
Kyle: I can't really remember. They're like circular and they smell funny if you put them in the microwave. That's the main things I remember now. But anyways, CDs, CDs, yeah. So anyways, so two that I've recently rediscovered are RJD2, Deadringer. RJD2 is the band. Anyways, Deadringer's the album.
And it's kind of like early, early-ish electronic music. It was pretty innovative. And anyways, it still holds up super well 10-15 years later. And then another one that I ran across again, this one's actually really old, but I discovered it like 10 years ago. But it's from 60s, 70s. Anyways, Crosby, Stills, and Nash.
Brian: I've heard the name of it. I can't name a song.
Kyle: Yeah. Another one I've really been enjoying recently, Lorde's new album Melodrama.
Brian: Yeah, it's pretty good.
Kyle: Yeah. I really liked her first album, but this one feels like a big step forward, just kind of the maturity of her songwriting. She's a very, a very adept songwriter. Those are my JAMpicks.
Brian: Cool. Awesome. I've got some stuff to listen to. So my picks actually, yours sound like polar opposites. So, one thing I love, I love this like, new, I don't even know if it's new. It's actually, it's country music, but it's like country music as if it was like 40 years ago. I think they call it alt-country.
So, Sturgill Simpson is a guy I've been listening to a lot, and I can't pick an actual song. I use Spotify as well, so I'll just like put in, follow that artist and just play them while I'm on the train. So he's just a guy I love listening to. I think I like country. But I don't like, name-brand country. I like this weird underground indie country thing that's going on right now.
Kyle: Yeah, that sounds, or, I don't know, my little sister likes country. Whenever I'm listening to it, I'm not that interested.
Brian: Yeah, I come from the south, but I'm not a big fan of that kind of, like, proper, Florida-Georgia line bro country, whatever. I'm not into that. But Sturgill Simpson. So, check that out. And then there's this other artist who, I'm not even sure if she's an artist. Well, she's an artist.
She has a song out. Her name's Cardi B, so it's like Bacardi, but Cardi B. And she's an interesting person. She's more Instagram famous than she is singer famous, and so she used to be a stripper, and then she became an Instagram model, which, whatever that is. It's just people who take pictures of themselves, put them up there, so they get a lot of following.
Kyle: Yeah, I missed the boat on Instagram.
Brian: Yeah, and so then, I guess she was at a photo shoot and someone was listening to her yell at somebody on the phone, and they figured out that the cadence that she was yelling in was actually really good for rap. So now she's a rapper. So she's like, very much in like the Nicki Minaj vein, but she's got this song which is Bodak Yellow. And that's like her hit.
So I think her story is more interesting than the actual song itself, but the song's actually pretty good if you're into that sort of style. So, check out Cardi B. I think she's going to be like the next, like, Kim Kardashian, or I don't know. She might have a, she might be in the reality space. I'm not even sure. She just came out of nowhere. I just kept hearing her name, and I finally listened to the song, and I was like, "oh, that's her."
Kyle: Yeah, I find it really interesting how celebrities these days are so, they like, so easily morph from category to category now. There's like, no boundaries. They're just like, I'm famous, and you can kind of jump anywhere.
Brian: Yeah, so the philosopher Snoop Dogg, his saying is, "get in while you fit in." So, that's pretty much where it is, and I think he's also been a really good testament to that. On top of that, I do want to mention GraphQL as one of my picks. If you haven't tried it, I think you should really check it out. As I mentioned before, I will be talking at GraphQL Summit on some of the stuff I've been doing at Netlify for GraphQL.
And, so it's going to be a short lightning talk. But it's going to be a lot of fun. It'll probably be out by the time this comes out, so just check it out, Google it, Google my name. And Kyle, thanks for coming into the Heavybit studio and talking about Gatsby with us.
Kyle: Yeah, my pleasure.
Brian: Awesome. So, that's it, and keep spreading the jam.