June 21, 2018
Ep. #18, Collaborative Security with HackerOne’s Marten Mickos
In episode 18 of The Secure Developer, Guy meets with Marten Mickos, CEO of HackerOne, a platform for vulnerability coordination and a bug b...
In episode 37 of JAMstack Radio, Brian is joined by Ben Schwarz, founder of Calibre, to discuss the shortcomings and frustrations of the modern web experience for customers, and the tooling available to improve it.
About the Guests
Ben Schwarz is the founder of Calibre, a company offering powerful web analytics aimed to improve customer experience. He is also a technical author and speaker whose work has been featured on CSS Tricks and in Smashing Magazine.
Brian Douglas: Welcome to another installment of JAMstack Radio. On the line we've got Ben Schwarz coming in from Melbourne Australia. Hey, Ben.
Ben Schwarz: Hey, how are you?
Brian: Awesome. Ben, do you want to tell the audience what you're doing right now?
Ben: Absolutely. I'm the founder and CEO of a company called
Calibre. We're building tools to help companies
make the web slightly faster than it is today, and we're doing
that by providing monitoring that tries to get
into some real world user situations, where
we test from all over the globe from 14 different locations
around the world under different network conditions.
We can make it seem like the user, it's all simulated user stuff. We can say "This is on a slow phone on a 3G connection in Mumbai, India." Or, you can pop right over to Singapore, or back over to California if you want to.
Brian: With Calibre, you say that you could provide emulation for all these instances?
Ben: That's right. The idea is that you can check to see what conditions your customers are in, you can have a lab where you effectively monitor those user situations, and you get a good understanding of if you are improving things for them or maybe if performance is degrading.
Brian: OK. I assume with monitoring you get logs, and if there is some instances that need improvement, is this all part of the whole Calibre experience?
Ben: Yes. Part of the experience is that you get screenshots of everything. Of the load progress of your site, you get a little video, if you want to, of any particular test. As well as a whole stack of metrics, and also recommendations that are coming from some other tooling as well. Like Google's Lighthouse project is built into Calibre for example, and we have those recommendations of "If you were to do these two or three things your performance would improve from there."
Brian: Awesome. The question I'm wondering is why Calibre, why performance? Do you have some anecdote or experience that you had prior to Calibre that pushed you towards building a company around web monitoring?
Ben: I would say that
I'm similar to a lot of other product
builders in that I care about user experience, and
I want things to be fast and fluid and feel good for somebody using
it. But that interest has been around for a long time.
Brian: That's pretty early on if you came around the time of Jekyll. I imagine this is still actively used and maintained? Bonsai, it is?
Ben: It's not maintained at all, and I don't think the last
commit was definitely in the last few years.
But there was this small community of people who were using it, for sure.
They were using it for every site that they built.
I definitely used it for it for a bunch of stuff that I worked on back in the day.
I was working as more of a consultant for all sorts of companies all over the country and all over the world. I used it for an office furniture company one time. That was cool because it had hundreds of pages, and I had to do a whole bunch of optimization on the tool itself to be able to generate those hundreds of pages within a reasonable amount of time.
Brian: That's pretty cool. I won't condone using it, but I'll definitely check it out on GitHub and try to figure out what I missed eight years ago. Because I don't think I remember-- I don't think I was programming eight years ago, I didn't start until five years ago. I would have missed it entirely. I've--super fast ramp up here in San Francisco.
You go to a boot camp and then you're senior engineer is how it goes.I'm kidding, it doesn't go that way. There's a lot more hard work that goes into that.
Brian: You're learning from doing Bonsai, basically building web apps up until now? That's how you got into the whole performance and monitoring piece of it?
Ben: I worked as a consultant for
about 10 years, I would say, and I was a Ruby
developer since way back.
I'd always been that person who worked in between the
designers and the developers, the smart people and the
people who were good at designing and building interfaces.
Of course I was interested in both sides, and my skills
bled in both directions and through
my consulting work what I found was that
I'd go into a team and I would
meet all these people that I thought were great and fun to be around.
Six months later I would leave because that's how consulting
works. I still see some of those
people years later, and that's awesome, but
I've got so many people that I worked with. But I hated that I didn't
own anything and I didn't feel like--
Even you would make a recommendation and you would say,
"You would probably want to do this.
I'm not even going to be here to be responsible for it later.
Maybe that's what you want to do."
I hated that, it felt like a cop out. It felt like I wasn't doing enough. With Calibre I didn't set out to start it, it started as a hobby project something that I was working on, and I was doing a bunch of open source work at the time.
I had a rule with myself where I wouldn't work on something if I didn't think I was going to release it.It was a rule, it was like "Don't waste your time on stuff that you're never going to release. Only do stuff that you release." With Calibre, I was so interested in it and I didn't think it was going to go anywhere, but I forced myself to do it. Then I launched it on my birthday, which was a fun thing to do, "Here's this thing I've been working on for six months in the quiet," launched it on my birthday and people started signing up. People that I didn't know started signing up, I added a credit card form eventually. And the start of 2017, I stepped back to say "OK, there is a viable business here." I went full time and here we are almost two years later.
Brian: There's a popular correlation between, in
general, the JAMstack in general, as well as
consulting. I had Jeff Escalante here talk about
Roots, and also he came through consulting too
as well, over there in New York and Philadelphia. He has a static site
generator. Then Carl Matthews who's now leading a
company around Gatsby, also came through consulting.
There's a lot of people out there who are building sites quickly and really fast, and I'm the opposite of you where I never ship stuff. I work on stuff just for the sake of working on stuff, then never ship. Mainly because the main product that I work on, it solves a lot of my needs. I'll carve out some time to plan a hobby project but never ship it, so I appreciate you being persistent in actually getting to the end of the release process. Because you released something.
Ben: Look, those rules are good to live by if it's a
project that you're going to invest a lot of time into. But
also the other side of that is I have projects that I've worked with,
friends of mine, I have one called GIFCity.
GIFCity was a thing that we built in the
six years ago, so maybe that wasn't as common then.
But all that this thing did was hit Tumblr, and you could tell it
the Tumblrs that you liked, and it would do a
full screen GIF.
It would just look for GIFs on Tumblr.
We added some keyboard controls, and we were playing this at parties and
projecting it on buildings across from the rooftop
while somebody was playing music or whatever, and we had
buttons to change the GIF or to never show
Or, we also did this mirroring thing where they were mirror-reversed
out on the side, so you could get a widescreen
One of the other guys that I worked on it with did a web version where he had pixels that were part of the gifs that were being thrown at the screen, and those projects are the most fun. The things where you get to pull a few passions together and build something that's ridiculous, but fun.
Brian: I agree.
My daytime job is developer advocate, so I tend to
attach myself to a project and build a lot of examples.
Most of my examples are a little zany and off the wall.
I have an example which was going to be my pick, which we
are-- At GitHub where you have GitHub actions, I'm going to build an action to
look at people's PRs and if it's over
15 files then it'd be like--respond
automatically when you open up the PR to say, 'What are you doing?'
I don't know what I'm going to call it, but that was my idea to do a one-off. It's too much. Another couple of examples that I have, I've mentioned this quite a few times on the podcast, but I'll mention it one more time. Which is my Hustlin' app, which comes to mind with your GIFCity, where I built an API to tell me when there's a baseball game to avoid the BART and avoid the city at all costs whenever there's baseball, which will be coming out in a couple months because spring training is coming up again.
I definitely agree with scratching an itch, but also having fun with it. It's easy to approach but also it helps keep you on the horse to continue to push and ship it.
Ben: For sure.
Brian: Considering the JAMstack, what would you need to focus on with JAMstack or static site generator sites, as far as performance goes? What are some sort of low hanging fruit you can solve?
Ben: That's the coolest thing about the definition
of JAMstack in the first place.
It's centered around the idea that it should be
performant, in the way that you're building a static site.
Sure, you're going to use an API and that could slow it
down, but you should statically render
it on a server side so that there's no API wait time.
That's excellent, but that's baked in because if they didn't
even say "You shouldn't-- Static site, you
could pre-render it if you wanted to."
But we'd be losing a lot of performance there.
It's so cool that that's a foundational thing of
JAMstack for sure.
But all the same rules still apply.
There are few areas where the web is slow,
and I can talk about them if you want to.
Brian: I'm all ears. I do the Lighthouse thing for most of my sites. I have a site that I shipped yesterday for a hackathon in Nigeria. So top of mind is "Let's make sure the thing is as fast as possible, and let's not break Android phones over in Nigeria, w hile people are trying to look at this thing." Lighthouse is my go to as far as checking the boxes, but I'm curious to what corners of the internet that need to be improved.
Ben: For sure. At Calibre, we
have hundreds of customers who are testing thousands and thousands of pages,
and they're doing it a lot. We see
a lot of common issues,
but they do fall into a few small
categories. One, of the most
people are confused by it, a lot of people have been building it for years but maybe
not using the most modern techniques that are around as
well. There is every level of the spectrum, but there's a
few important things to remember.
wire you've got to remember that it might be compressed, it might
What happens after that? It
gets down to the device, the device has to decompress it
firstly, then it has to analyze it.
But remember-- Maybe your average size of
a script on the web today is about 600K.
The average site has about 600K of script.
Once you've decompressed that, that can be somewhere in the region of 2-2.5, maybe even 3 megs. Depends on the file, right? Suddenly you've got this wad of script, and that needs to be downloaded-- After it's downloaded it needs to be interpreted, turned into an ASCII and then it needs to be executed.
The thing with browsers is they're doing that in a single threaded environment.
So if your page is-- You're trying to paint some stuff so the
user can actually see something, and
in, then there's third party scripts that are also being parsed and
downloading at the same time.
What you're doing is affecting the CPU performance of the page and the TL;DR of that is that for a user, you could be sitting on a couch scrolling along your feed or reading an article for a recipe you're about to cook, something like that. We've seen it, right? It's the thing where the page stops scrolling and your finger keeps moving, then it catches up and you scroll to the bottom of the page and you've clicked on an ad, and you've opened another browser window. It's the most frustrating, annoying thing.
But it's most notable on a mobile
rather than a desktop, because they have slower
CPUs. Your $3,000 MacBook, which is
super fast, if you've got a $2,000 iPhone maybe
it's the same experience.
Those things are fast as heck.
What happens when you use a 3 year old
phone, or a phone that you bought at the post office because
you can't afford something else?
Or, whatever your conditions are.
Brian: Yeah. Let's solve this problem.
Ben: We can solve this, it's not--
Ben: I actually had that as a-- I gave a talk
over in Denver last year, and I had a
table where--the idea was that when you're
talking about doing performance work, maybe write down
the thing and how much performance you think is in that thing.
There's five seconds in this, and then have an estimate,
then have what you achieved from the work that you did.
But the comment under, at the bottom one, at the
There's 15 seconds there, and the problem is that you
lose all your revenue and it's not a
realistic thing to do.
So, you can't do that.
What can we do?
We can reduce the amount of scripts.
If you've got third parties, maybe you can try and get rid
of some of them.
If that's maybe a realistic-- Or, maybe you can delay the
time at which they get loaded.
If you're trying to load everything with all of your
scripts at the same time, that's putting a lot of pressure
on the device.
If you can delay some of those things to happen, some of them can
happen asynchronously, so that when the device is able to process it, it
will and you can delay some of them until later.
That's where the web should be. We shouldn't be trying to load everything, the whole website all at once. We should be trying to give people just what they need.
Brian: You'd mentioned that Calibre, it ties in with the Lighthouse open source project from Google, and you mentioned code splitting. My experience of code splitting is that it works on brand new projects that are like, "I just threw this up together and I was watching a tutorial," or whatever, "I was hand fed what to do to make code splitting work." But I found in realistic projects that I've already had, I've been working on, and I've discovered code splitting after the fact. It's hard to approach at that point. Does Calibre provide any education or handholding? Not for code splitting, but for some of these other things? Service workers, etc. to push you towards making a better experience on the web?
Ben: For sure.
In terms of on the Calibre side, we try to make
recommendations and some of those recommendations come from Lighthouse as well.
If we're doing those sorts of things, but we also write a fair bit of
content. We spend a hell of a lot of
time looking into browser tooling.
I spend a lot of time reading Chrome source code to figure out bugs
and that sort of stuff.
We tend to find about, I'd say, somewhere between two and
three Chrome bugs per month. They
can be subtle small things, or they can be maybe slightly bigger.
We work quite closely with the Lighthouse and Chrome [devrel]
team, so we get an understanding of these issues and we
understand how the browser works, and we try to demystify some of that through
One of the posts that comes to mind that I did
last year, or maybe even the year before, could be
where we upgraded to React 16 which
was brand new at the time.
We went into Chrome dev tools and we were like,
"OK. Let's try and figure out why this
thing is slow.
Let's step through and audit together, piece by
piece, and instead of going
'The website is slow, reduce the amount of scripts you
have,'let's see what that script is doing."
This article, which is on CalibreApps.com/blog, is all about how to use the tools to understand what is happening and what you can do next. It doesn't matter how big your code base is, you can still do that. You can still trace, you just have to know how. That's the most--
Brian: That's the biggest challenge. There's a plethora of content and documentation that you can copy and paste from the Chrome dev blog, but a lot of it's like, you have a perfect project and you have a perfect situation to copy and paste this code and put it in there, and whenever I'm outside of that perfect situation the world's lacking that content. So I'd be excited to thumb through some of that stuff and maybe have some of the listeners as well, jump through that and see if they can apply it to their projects today.
Ben: That's what we're trying to do at Calibre at the moment. We think our tool is great, and we know that it needs to get better, and we know that other tools need to get better too. In the meantime we need to help educate people and try to have this understanding of what is important, because it isn't easy to know what to do next. We're trying to help people make that leap where maybe they don't know something, but they know how to learn. That's one of the most important things that we can do.
Brian: Excellent. Have you heard of the conference PerfMatters?
Ben: I have. It's coming up, isn't it?
Brian: Yeah. It's coming up here in the spring down in Redwood City, I believe. I would definitely want to mention that in case the listeners are interested in finding content as well as last year's conference, there's a lot of good little tidbits you can find from that, for sure.
Ben: I know Estelle, who runs it quite well, and she's incredible. It's the second year of the conference and the lineup this year looks fantastic. There are people who are doing amazing work in the performance world and that's definitely-- If you're in the area you should go. I would be coming, but it's my birthday on the day of the conference.
Ben: Yeah. I was talking to Estelle about it, I couldn't make it last year either, but--
Brian: Would that make a Calibre 1 year old then, at this point?
Ben: No, not Calibre's birthday. Calibre's going on two years now.
Brian: Two years? OK. Because you had mentioned you had shipped on your birthday.
Ben: On a birthday, yeah. I've been full time for two years, I was working on it a little bit before that, but I don't count that.
Brian: Cool. I wanted to ask you a bit about Australia too, since you're already there, and also considering you're in the future. What's the dev scene like in Australia? Because I know, I'm here in the Bay Area, most people migrate or find themselves here at least once or twice a year, or at least move here for work or whatnot. What's the dev scene in Australia between Melbourne and Sydney?
Ben: Huge. Huge and active, I would say.
We've got a bunch of newer companies, like Culture Amp
and Canva, and they're large.
I saw Culture Amp bought a company from
yesterday, so that's pretty cool, and I know the founders there.
It's super active, and we have some of the most
talented people in the world for sure.
Thinking about the people who I'm lucky enough to have
around me, we've got people who started the
starred components project for CSS or the
people who created the CSS modules, or the people who
are writing and speaking about working with
Figma or Sketch and building a design system and that sort of stuff.
These people are at the forefront of where we are as builders for the web, no question. I'm quite proud to be an Australian. I know we're not, in terms of some of our things that we're doing environmentally and to people are not the best things, but in terms of the dev world we're talented and we should be proud of that.
Brian: Cool. Awesome. If I ever make it out to Australia, I'll hit you up. I'll figure out where Melbourne is first, and then I'll hit you up.
Ben: Do it for sure, because it's one of the most beautiful countries in the world for sure, but you've got the added advantage. I know you are in SF, so it's not cold or anything ever there. Melbourne is quite famous for its weather, it's similar to San Francisco but I know it's freezing over in New York right now. But today it's a completely perfect blue sky, it's summer, it's beautiful.
Brian: Excellent. Anything else you want to add about Calibre or web performance that maybe listeners can check out?
Ben: Firstly, I want to encourage you even if you don't use Calibre, I want you to start to think about monitoring or performance testing in general.
Because the thing is you can't be everywhere and you can't be in Frankfurt on a 3G connection, you need somebody to be able to validate or something to be able to validate what your customers are seeing and what they're feeling.
The other thing is that you need metrics that tell
you something about experience, and Calibre and
other tools have metrics that try to
describe what a user sees and feels, and in a
couple of cases-- I'll give you a couple examples.
Firstly, we have paint metrics that describe when the
first thing happens.
When a user gets drawn into your experience and sees that
something is happening, we have a metric to tell you about that point in
time. We also have a metric to tell you when a page stopped
rendering, when it visually stopped changing.
That's the point when a user would be like,
"OK. I'm going to interact with this thing now."
Brian: Excellent. Before we transition to picks, I want to make
a quick mention.
I messed up on Jeff Escalante.
I'd mentioned he was on here and his project was
Roots, the one we talked about was actually Spike. I
discovered that while we're talking, but I just wanted to insert that. So,
listeners, episode 20.
Check out Jeff's episode up on Spike.
Then let's go ahead and transition to picks.
These are Jam Picks, things that we're jamming on. This could be food related, tech related, unrelated as well. We've had quite a few of those on the episodes. So feel free to pick something unrelated, but if you don't mind I do want to mention my Pick. Which first, is JamPicks.netlify.com, someone tweeted me and was like "Is there a list of all the Picks that we've done on the podcast?" I said no.
So I have now created JamPicks.netlify.com. It's a Gatsby site and it has every single episode, it's super non-CSS. It has no design at all, but it will show you every episode including this one and the Picks that we pick. It's a little meta, but definitely check out that. I also alluded to, in the conversation with GitHub Actions, Ben have you seen GitHub Actions yet? Or have you heard of these yet?
Ben: Yes I have. We're actually building some tools for that right now.
Brian: Cool. Send them my way if you want some exposure or if you need some engineering help or partner help, or anything like that. I can hook you up. I know somebody.
Brian: But for the listeners, if you haven't seen GitHub Actions, these are
scripts that you can run in GitHub so you can literally run
code in GitHub on GitHub.
Under the hood it's Docker containers, so you have 53 minutes of
runtime, very similar service functions or Netlify
functions. You could ship code without needing to host it
But it's a little more than that, so any event
that you have on GitHub, let's say every time you push you want something to happen.
I know a lot of other services within the JAMstack, they automatically deploy for you,
but imagine every time you push or every time you merge
the master, like you want to cut a release, you can set that up through GitHub A
ctions and then some as well.
We've seen a lot of people running Ruby code directly
As soon as something happens it's run a script of Ruby.
Definitely check it out, as I mentioned I'm running this trivial
action that if there's more than 15 files it
comes up with a nice little funny quip and
says "Maybe consider breaking up these files
into a smaller PR."
I'm still working on whether it's 15 or whatever, 15 is arbitrary at
this point, but I'm mainly trying to have fun with the projects I work
on and have a little banter in there.
I had one last pick.
Quite a few episodes ago we had [inaudible] on and
inside that episode the picks I chose was corn
tortillas and how to cook them on your stove.
I've recently gotten to doing a lot of baking.
2019 is my year of making food.
I've made flour tortillas, and it's surprisingly
easy to make.
It's literally three cups of flour and two tablespoons of baking
soda, and then two tablespoons of lard and
You put that together, throw it into a cast iron skillet and you have a flour
tortilla. It's beautiful.
I've only been eating my own tortillas since the beginning this year and it's been great. You can freeze the dough too as well. I just throw them in the freezer. Little balls of dough, then thaw them and make a little tortilla with some eggs or some cheese, or-- I kind of make them Chalupa-like. I made Chalupas too as well, which if you guys aren't familiar just checkout Taco Bell. They've got Chalupas there. Other than that, those are my Picks. Ben, do you have any Picks?
Ben: I do. I was just updating my Picks as you were speaking, because you made me care about things so much. Firstly, GitHub Actions is cool and I had it as one of my Picks as well. I saw a good one, performance related. There was a GitHub Action that would run in your pull requests, and it would go through the images in your pull request and automatically compress them for you.
Ben: It's incredible, and it posts a diff of the byte improvement for the image, and there's no visible differences or artifacts for those images. I think that's where we should be automating things.
Brian: That's amazing.
If we want to build good products, let's make the hard stuff that's annoying. Let's just make that easy to do.
Brian: Talk about annoying. I was working on a blog with other people on my team and one of the biggest annoyances was images, because we have three gigs of images for every single blog post we wrote for like three years, and it became problematic. Having something automate the process of this swap out this image with something better, or-- Was this on S3? Were they using S3 to do that?
Ben: I don't know what they were doing.
Brian: OK. I vaguely remember somebody using S3 to do something with an action, but I'm not sure we're thinking of the same one.
Brian: That sounds like an amazing use of automating stuff. What was your next Pick?
Ben: Absolutely. It's summer in Australia, so cycling at the
moment is the best for me.
I've got a couple of friends who just bought new bikes and we're
heading out every day if we can pretty much, after
morning calls or whatever, it's what I'm going to do after this call.
Cycling in the sun and going to the river
and watching animals, and that sort of stuff.
There's few things that are better in the world.
Apart from that, in food I've been into bagels recently. I'm using a recipe that is called "soft chewy bagels from scratch," it's on ChefSteps.com, and I don't know. I've been making delicious bagels every few days here at home, and I love a bagel for breakfast.
Brian: That sounds intimidating. Tortillas is the most creative I've been as far as my cooking so far, and I looked at the recipe for croissants and I was completely intimidated not to do it because the amount of work it goes into. But I'm going to check out your recipe and I'm going to make a bagel.
Ben: I've done a bit of stuff with doughs recently, like I've been making flatbreads or bagels. I've tried some basic stuff. I always found dough difficult to work with, but this recipe, maybe you can link it up in the show notes or something. I'm getting good results with that, and I'm starting to understand how the whole thing works. Pizza base too, which you can make pizza at home.
Brian: Yeah. When you can have pizza on a bagel, you can have pizza anytime.
Ben: I'm into that.
Then my final thing is I'm really into
building Calibre, the product at the moment.
We just had somebody start with us full time and
they're kicking ass, and I'm so enjoying what I'm
Just building a product and trying to help make the web faster than it is today is what gets me up every day, and I'm super passionate about that.
Brian: Awesome. Congratulations on the new hire.
Ben: Thank you.
Brian: Thank you for coming on and chatting about performance. It was insightful and I look forward to tinkering with your project, as well as reading some of your content as well. Ben, enjoy your cycling ride. Listeners, keep spreading the jam.