1. Library
  2. Podcasts
  3. To Be Continuous
  4. Ep. #10, Programming Languages
light mode
about the episode

In this episode of To Be Continuous, Paul and Edith talk about programming languages and why they continue to evolve.

For more on Continuous Delivery you can follow To Be Continuous on Twitter at @continuouscast.

A Bet Against Erlang
The Wrong Language for the Job
Little Languages
When a Language Goes Extinct
Open Source and Pay-to-Play Games
LinkedIn and the Co-Metric
Angry Screeds and Authenticity
Raising the Abstraction

transcript

Paul Biggar: It's official now. Continuous delivery has a downside.

Edith Harbaugh: I think we're in agreement as we always are.

Paul: If you learn Erlang, you're probably super into making reliable distributed systems. C++ is exactly the sort of thing that you don't want to be writing in distributed systems in. When you switch from an unsuitable language to a suitable language, you often end up with amazing advances in productivity and in reliability.

Edith: The techniques that worked in the past are used up. You need to find new techniques. And I think that's the same with languages.

Paul: Hi, I'm Paul Biggar, founder of CircleCI.

Edith: I'm Edith Harbaugh, CEO and co-founder at LaunchDarkly.

Paul: And you're listening to To Be Continuous, a podcast about continuous delivery and software development.

Edith: You can get in touch with us anytime at our Twitter handle, at continuouscast.

Paul: The show is brought to you by Heavybit. To learn more, visit Heavybit.com. And while you're there, check out their library, home to great educational talks from other developer company founders and industry leaders.

Edith: So today was our episode 10, and Paul and I took a pretty sweeping retrospective at where languages come from and why they continue to evolve.

A Bet Against Erlang

So Paul, believe it or not, I have a bet about a computer language.

Paul: Okay, that sounds interesting.

Edith: So I was catching up with a friend who started a company, and he's a CEO, and he told me that the company had decided to have their main application written in Erlang. And I know the various popularity of all languages because I look at this myself when I see what things we're gonna build an SDK for. Erlang has very low adoption. I'm not really making a judgment on whether it's a good or bad language. I'm just saying when we looked at what we were going to support, Erlang was not at the top of the list.

Paul: I think you've got something which is factually correct there.

Edith: So my bet with my friend who's a CEO is that he will regret having picked Erlang.

Paul: Oh, interesting. Okay.

Edith: Not, as I said, any judgment about whether or not it's a good language, but that it will be increasingly difficult to hire people who know Erlang.

Paul: Okay, well, this is something I definitely have an opinion on. Circle, as you know, is built in Clojure. Clojure is a niche language at best. It was growing rapidly, but it's probably kind of flattening out at the moment.

But niche languages are amazing in many different ways, especially functional ones. And there's so many reasons why they're amazing. But the first, and I imagine hiring is something that your friend was worried about or something like that.

Edith: Oh, that's what I worried about. I mean, we chose a niche language, we're in Go. But, you know, my co-founder John's like, it's a niche language now, but it is only picking up.

Paul: Right. So that one is growing. But niche languages are actually fucking amazing for hiring, because when every startup in the world is written in Rails and you're trying to hire a Rails developer, you're competing against every single company that's out there and you've no differentiators.

Edith: Well, it's hilarious, because I remember in San Francisco in two thousand seven I went to a party. And they were talking about how Rails was their differentiator.

So, like, I literally remember this.

Paul: And it would have been in 2007.

Edith: Yeah, it was like, how do you manage to hire engineers? They're like, oh, we're Rails. Everybody wants to work in Rails. And then I remember, like, eight years before that, in nineteen eighty nine, Java. Java was the hottest.

Paul: '89?

Edith: Oh, I'm not that old, Paul. Come on.

Paul: I thought you said '89, '99. Yes, exactly. So actually Paul Graham had an essay about this. I don't remember the name of the essay, but he talked about how at some point Java had been, you know, an exciting differentiator. And then you could find the great hackers by the fact that they knew Java.

Edith: Yeah, I mean, I remember going to JavaOne when it was the hotness. And now it's like the notness.

Paul: Now it's old and corporate, right. There were cycles, you know, Python used to be a thing that you only knew if you went and learnt it yourself.

You couldn't learn about it in college. There weren't these boot camps. No one was doing it except because they were super into learning the latest things, and there's this correlation of great hackers being super into great things, which you may or may not, or new things. You may or may not agree with that, but you know, it's an opinion that's out there.

So if you're into a niche language, like the people who learn Erlang are the people who, well, okay, there's all the telecoms people who learned Erlang, and I don't know if that's gonna be helpful or not. But if you learn Erlang, you're probably super into making reliable distributed systems.

Edith: Yeah, but don't languages kind of start to peter out? I mean, if they don't start to catch fire, if they're not getting new blood, I mean, somebody who's really into the cool new thing like Erlang, haven't they already moved on to Scala?

Paul: If they're only into it as the cool new thing, then yes. But if Erlang, you know, appealed to their certain sense of who they are or a certain sense of how they think, certainly Clojure for me was the certainly the language that I hate the least out of all languages I've tried, with Python a distant second. And certainly, like, in the future when I'm, well, I probably won't be looking for jobs, but theoretically, if I was looking for jobs, you know, Clojure would be a major differentiator for me.

Edith: Because you've tried other languages and this is the one you like.

Paul: I've tried so many other languages. So, Clojure's amazing. And I haven't tried Erlang, to be fair.

But my friend used to work at this video game startup. In fact, I used to work at this video game startup. It was called Demonware.

Edith: Did you forget you'd work there, or just like...

Paul: I wasn't there for very long. But so it's now Activision Europe or Activision Ireland or something along those lines.

But it used to be called Demonware, and they made this matchmaking software. And so matchmaking is you're playing online gaming and you're trying to make a match. There's all these bunch of people. And they'd written the software initially in C++.

Edith: And I hear you're pretty familiar with that kind of software.

Paul: Very familiar with C++.

Edith: And more with the matching.

Paul: C++ is exactly the sort of thing that you don't want to be writing in distributed systems in. And it was riddled with bugs, it was awful. And then this guy Vlad, and I don't know how apocryphal this story is, but this is how it was told to me, and I know Vlad and he's very smart. Vlad came along and he rewrote their matchmaking software in about nine days, I think, in Erlang, and it solved all of the problems that they had ever had in C++.

Because Erlang is a system that's designed for building these things. And when you switch from an unsuitable language to a suitable language, or from a lower level language to a higher level language, or whatever it is, you get closer to the domain that you're trying to solve, you often end up with amazing advances in productivity and in reliability versus doing it the old shitty way.

Edith: I think you said something very interesting, which is basically that languages are evolving for custom purposes. 'Cause what you said about Erlang and how it's really perfect for this high scalability use case. I mean, that might not be true at all if you're looking for, like, say, a mobile app.

Paul: Absolutely. So Erlang is something which is designed for telecoms. And telecoms is something where fundamentally, you know, packets are coming in and you're trying to do some transformation on them and get them out to another place.

Edith: Trying to decimize them.

Paul: Something along those lines. And the whole purpose of it is be super reliable. Don't, you know, this is telephone switches. You know, don't let some sort of systematic failure take down the entire system.

And, you know, when you compare that to something like Rails, Rails is something which is built for making websites super quickly. And if you're gonna make a website super quickly, like, you know, start now, finish before lunch, Rails is your friend and Erlang is not gonna be your friend. And if you're looking to make telephone switches, Erlang is your friend and Rails is not going to be.

Edith: Yeah, I mean, this comes back to something we've talked about before, just the trade-off between, like, time, quality, and features.

Paul: Right, right. I mean, it's all tied to it, and those languages have certain properties that make it easier to build certain features in certain niches.

The Wrong Language for the Job

Edith: Have you ever seen a case where somebody just had the wrong language for the wrong use case?

Paul: Oh, all the time. It happens absolutely all the time. People rarely pick languages for the domain that they're writing it in. People typically pick languages because it's the language that they know or it's the language that they can hire in.

So you have all these big enterprises who are writing everything in Java. Because that's what you do. Like, there's an unwritten rule. In fact, it might even be written somewhere that if you're an enterprise you have to write things in Java, or C# if you're a Microsoft shop.

Edith: Yeah, this is hilarious to me because I just remember in two thousand I was working for a company and Java was very counterculture back then.

Paul: Right. And if you look at the companies that were built around ninety nine that were counterculture, like Google, for example, has tons of software written in Java, right? And they were like one of the biggest counterculture companies there is. And then they had this foray into Python which never really went anywhere, and now they're kinda super into Go. But they were, you know, building in this counterculture language that eventually became the enterprise language.

Edith: Yeah, it's funny because I was at Vignette, which was built on Tcl. Which is actually very good for...

Paul: Holy shit.

Edith: And then they tried to move to Java because Java was the hotness. And it turned out that Java was just absolutely just not the right thing.

Paul: Interesting. Were they doing, like, a lot of string processing?

Edith: Well, they were content management and they were deploying between different systems. It was when the Java standard was really early.

Paul: I see. Okay.

Edith: So it was basically a mass revolt around all of our customers who'd built V6, was our Tcl version, and then we tried to shove this V7, which was Java, with no migration path, and we're like...

Paul: Oh, God. Well, I mean, there's the question of the big rewrite. The big rewrite never works. Spolsky has this big article about the Netscape rewrite that was supposedly a disaster.

Edith: Yeah. To tie this back into continuous delivery, that's one argument for continuous delivery, is there's never a big rewrite. There's just many...

Paul: Many small rewrites. You take one system, you move it off into another service, you write it in its own specific language or technology. And then you do the migration, and then you delete all that code after the migration is done.

Edith: Yeah, it's actually how, so my company LaunchDarkly is picking up customers.

Paul: Oh, really, you have a company called LaunchDarkly?

Edith: Yeah.

Paul: Well, what does it do?

Edith: I'm really happy you wore the t-shirt today, by the way.

Paul: I have several at home now.

Edith: You could get a hoodie too if you want.

So my company LaunchDarkly is feature flags as a service. So basically the ability to roll features out to your own users, to permission them for different users, and to experiment with them.

And what we're seeing with some of our own customers is they built this custom for themselves. But they just built it deep into their app. And now when they're trying to migrate to a new language, they're still like, wait a second. We're gonna have to rewrite not only our app, but our framework too, and we don't really want to rewrite our feature flag framework.

Paul: So it's funny. We see this exact problem. So we at CircleCI just switched to using LaunchDarkly. And before that we had, like...

Edith: Thank you for wearing the t-shirt.

Paul: Thank you for the t-shirt. So we had, I think, maybe five different feature flags implementations in our code base. In different points. One was for users, one was for organizations, one was for different machines, one was for a different version of software.

There was tons and tons of feature flags. You could opt into experimental stuff. We could opt you in. They were all over the place.

And, I mean, we also see this when we're selling CircleCI, that we go into an organization and they've got a bunch of different CI systems. One team is on, you know, BuildBot from 2003 or something like that. Someone's using Jenkins, a bunch of teams have moved to Circle, and they want to get everyone else over to Circle.

You see the same, I mean, adoption cycle throughout the enterprises. You have some teams that are leaders in, and that are explicitly asked to be the bleeding edge, like the labs teams who are using GitHub and who are using Heroku and who are using CircleCI and that sort of thing. And then there's the rest of the teams who are using ClearCase and C# and that sort of thing, and trying to learn from the labs people to be able to ship faster.

Edith: And that's interesting because that's very meta, because one of the things that LaunchDarkly enables is labs features.

Paul: Right. Where did we start?

Edith: Erlang.

Paul: Erlang, okay. So there's kind of this idea in talking about Erlang about the shifting role of languages in the space, right? So you were talking about people using Tcl, people are using Python, people were using Java and Rails and Go and Clojure and all these things.

So what we see is almost like a continuous delivery in the programming language space.

Edith: Yeah. Well, and then does it reach its limit where you have something that is too custom built? Because I have a couple examples.

Paul: Oh, go for it.

Edith: Well, because you said a language for every use case. But I think if you get too niche, you get to the point where you simply can't hire people or nobody's interested in your language.

Paul: I agree and I disagree, but give me your examples.

Edith: So I'm not gonna name names, but companies that have no customers but have spent two to three years trying to come up with their own language, because it's just then a barrier to adoption. I mean, 'cause it's like not only do you have to have this product that nobody wants, if you want to use it, you have to learn a language.

Paul: Right, right. Now you could say that, but isn't that how 37signals built Rails? I mean, maybe they almost certainly didn't spend two years not shipping a product, but they built Rails for their own thing, and now it became this massive thing and they benefit from that.

Edith: Well, I think there's a difference between building it because you need it to build your own product versus building it for other people to know how to use your product.

Paul: To know how to use, so hang on. This company, like their customers had to understand something about that language in order to be able to use it.

Edith: Yep.

Paul: Well, that was clearly an error.

Edith: Yeah, so not only did you have to buy their software, you had to, like, train somebody on a new language to use the software.

Paul: I see, I see. Okay. Well, I mean, user adoption is a major problem for startups. So if you add new obstacles, a new programming language, a new language of any kind, in front of users before they can use your software, I think that's gonna be a problem.

Edith: Yeah, it's one of those things, it's like never get involved in a land war. It's like never create friction, like having to learn a new language.

Little Languages

Paul: So I've seen lots of cases where domain-specific languages have been very effective. So domain-specific language is a little language for a little domain. I mean, I've created a bunch of little languages myself.

Edith: Oh, let me know.

Paul: Well, so when I was doing my PhD we had to do code generation. And so I was generating code, and basically when I started by generating code, it was in the there were strings in the files and they were being concatenated and there were variables in it and all that sort of thing. And it's just like it was very, very unreadable.

I was trying to copy, like, large paragraphs of code into a place and I couldn't tell, you know, the different layers of escaping and all that sort of thing that you get when you're doing string processing in various languages. And what I was trying to generate, I had a C++ compiler, or a compiler that was written in C++, which was compiling PHP into C.

Edith: Oh.

Paul: And so there was all these different...

Edith: Yeah, I was like, but why?

Paul: Because this was where we were doing it. And so every statement in PHP ended up being like six lines of C depending on certain various things. And so what I wrote was a little language where I could write that paragraph and basically do pattern matching on the basis of properties of PHP that may or may not have been present in the...

Edith: Is this like a compiler of compilers?

Paul: It was, yes. I mean, that little language had a parser. And it had, like, sort of a code generator and a sort of an interpreter sort of thing.

Edith: This is very...

Paul: Very meta.

Edith: Yeah, we throw around that word and I'm like, wow.

Paul: Yeah. But it was really useful because it raised the level of abstraction for which I was writing software. It got rid of a major source of bugs, and it reduced my cycle time and the time to deliver new features on the basis of this incredibly...

Edith: Well, what was the adoption?

Paul: I mean, it was only for me. So it wasn't being built for other people.

Edith: So you have a truly micro-language?

Paul: Yes, yes. I mean, essentially I wrote a library which abstracted a certain function, and the abstraction was good, and therefore I gained productivity.

Edith: Does that qualify as a language, though?

Paul: I mean, yes. If you write a parser for it, it's a language.

Edith: That's like Biggar's Law.

Paul: Yeah.

Edith: If you write a parser for it, it's a language.

Paul: Well, so if you look at the design patterns book, the Gang of Four Java design pattern thing, one of the design patterns was an interpreter. And, I mean, this is essentially that, right? A little language, a domain-specific language, an interpreter, very similar kind of pattern. I certainly wasn't the first to come up with this.

Edith: That's funny. I think it was Jeff Atwood, though it could be somebody else, who said anything that can be written in JavaScript eventually will be.

Paul: Right, right. I think that's a different sort of law. I think he's talking about things like Emscripten, where you can compile things down to JavaScript, and I mean, basically, you know, everything will eventually be on the web.

Edith: Yeah. I was in London, I was visiting different people who are using feature flags. And one of the companies I visited was Import.io. And their entire business is they take web pages and they strip out it to make it back into structured data. All web pages start...

Paul: As structured data.

Edith: It was like basically they're recompiling everything back down into a database. Which was pretty funny. It was like everything just reduces.

Paul: So we were talking about migration patterns, or, you know, people doing migrations. And I think this is, that's almost part of the migration thing. You know, you're waiting for someone to get an API and they don't have an API yet, so you use something like Import.io or, you know, scraping of some kind. And then eventually they get an API and then you need to migrate over to it. But you could never have built your business waiting for that API because you needed to get to market faster or something along those lines.

Edith: Yeah, I mean, you could say basically all TripIt was that, that there is no universal reservation API.

Paul: Of course not, no. So you use the emails.

Edith: Yeah.

Paul: Yeah, genius. It had a parser. It was a language.

Edith: It was regex.

Paul: Sure. But, I mean, you could argue that there is a subset of English which is emails which have flight confirmations in them, for which you wrote a parser.

Edith: Well, not just flights, hotels, cars. And then, and I guess it is true, because then sometimes people would want to know how to structure an email to send it to us so that it would get parsed.

Paul: Oh, wow. Okay.

Edith: Well, 'cause like if you're an admin or something, or like if you're trying to add a...

Paul: You just want to forward it.

Edith: 'Cause, like, even I did this, if you wanted just to add an activity, you know, it was a hassle to go to the website and, like, hand key everything.

Paul: So you wanna just forward to the, yeah.

Edith: Yeah. So it was, I guess, a language.

When a Language Goes Extinct

So Paul, you talked about how Erlang was actually good for hiring because it attracted niche people, but there's also the point where a company will find that their language has started to go extinct. Like, I'd put in this category Fortran. Like the Voyager is now struggling because the space mission was programmed in Fortran. Now people don't really know Fortran so much.

Paul: Right, right. I think everyone saw this in the two thousands with the dot com crisis and the COBOL. No one knew COBOL anymore, so the greybeards were brought out of retirement for massive salaries and that sort of thing to you know, add two bytes to something for your two thousand crisis, that's what I was talking about.

Edith: Why do you think Y2K never really materialized?

Paul: Because it was handled.

Edith: So nobody remembers the crisis that was handled.

Paul: I mean, if you went through it or if you were just alive at the time, you remember everyone talking about how these two extra zeros are gonna make all the fucking difference. But anyone who had a problem with it just, you know, handled it. And also everyone shut everything off at midnight.

Edith: Yeah, I went camping.

Paul: Were you sure when you were coming back that there was gonna be a civilization to come back to?

Edith: Well, I went to Yosemite. I was like, I like it there. I was like, I can hang out here for a while.

Paul: It's good, because when the marauding hordes of post-apocalyptic zombies were destroying San Francisco, you'd be out in Yosemite already.

Edith: Yeah, you know, it's a fun place to hang out.

Paul: It was well thought out.

Edith: Yeah, you know, I got a lot of food.

Paul: So languages very obviously die there. There's a life cycle of languages. And being in San Francisco, I think we see the very start of the life cycle especially, the shininess, the early adoption, and even, you know, kind of the early mainstream languages, if we're gonna take a Crossing the Chasm sort of terminology to it.

And then we get bored of the late mainstream, and I've actually never looked at the latter part of that graph, so I don't know what the bottom of the rightmost thing on the Crossing the Chasm graph is, when everything's in decline. I don't even know what you call that.

Edith: I think it's called retirement.

Paul: Retirement. There we go. So there's all these languages which are in retirement. And there's all these languages which are in the late majority stage, which I think is the unexciting phase if you're here. And those are obviously the, you know, the Java and C# and C++ and that sort of thing, which are relegated either to career programmer places where you never need to change your technology stack, or legacy technology which is still relevant, and I think Chrome is maybe a good example of something which is written in C++ and is still very, very relevant.

Edith: Yeah, it's interesting. I think you're very accurate to describe it as that cycle. It's exactly right. And I think some languages just never cross the chasm. Why do you think some languages just never make it across?

Paul: It's an adoption thing. There's a very interesting talk by Simon Peyton Jones. He's the sorta head of Haskell, and he talked about how almost all academic languages die, and somehow Haskell managed to survive.

And I think he doesn't quite know how. And I think that that's the case with all kind of adoption. You don't really know why this particular one. You can have theories in retrospect, but there's no real good reason why one survives and something which is roughly equivalent doesn't survive.

Open Source and Pay-to-Play Games

Edith: Yeah, so there's a big heated topic now about the backlash around open source. Which is basically, a lot of developers are tired of not getting paid

Paul: To work on open source.

Edith: To work on open source. And it's the same sort of cycle. Like, I hung out with my really good friend on Friday.

He's super excited because he's maintaining this framework, and like people are pinging him with questions and he feels like he's getting a lot out of it. On the other hand, like, he's at his job and his job is paying him for his time. And then three years later I've seen people on the other side of that who are like, I'm not getting paid for this. It took over my weekend to answer this person.

Paul: Well, then stop answering the person. I can't say I'm... Without talking about open source, because I think that there's a contentious topic there, let's talk about the games industry. There's lots of people in the games industry who are worked, like, on death marches, terrible working conditions, shitty salary, and they get burnt out.

And then they complain about the situation. And when they eventually leave the industry, there's thousands of not yet jaded people waiting to take their place.

Edith: Yeah, it's like Hollywood.

Paul: Right. Exactly like Hollywood. You know, if you want to be an actor, you have to, you know, work the shitty jobs and get your name out there and pass your resume around, and once you make it, great.

And if you don't make it, or you make it into like a sort of a... This isn't a very good analogy for open source. I feel it's getting further and further from it. But I liked your story, so I was just going with it.

So I have an open source project that I'm de facto maintainer of, because I couldn't find anyone else to take over. This is my PHP compiler. Every now and then I get an email or a GitHub issue or something about it. And basically, project's dead. I'm not doing anything.

And I went through a very, I mean, it wasn't particularly stressful, but, you know, there's a period where lots of people were asking for it, and I had to decide, was it alive or dead? You know, I don't have time to help people with it. I tried to answer emails, tried to get other people to contribute. And it's like, at the end of it, I let it die.

It was like, if your open source thing is good enough, someone else will take it. Someone else will fork it, whatever. You hand over the reins or just accept every pull request that comes. And if you're tired of it, be tired of it.

Just don't work on it anymore. And I think that this breaks down when you're talking about larger projects, when you're talking about Linux or GCC or something like that, where most of the contributors are paid. Or you're talking about something like Rails or Ember or something like that, where, you know, there's a lot of kind of early adoption and maybe not a lot of money in it.

Well, programming languages, I think, are good. Like NPM and Nodejitsu went through a big sort of like, oh, we can't afford to keep this infrastructure that everyone is relying on. What are we gonna do?

Edith: I think open source is worthy of its own episodes, so we'll just put a semicolon there and then say I think Zynga was really successful in gaming because they were the first gaming company to really adopt continuous delivery.

Paul: Oh, interesting.

Edith: Like, 'cause the old console games were like, you put all your effort, when you talked about the death march, it was because you were putting so much effort in because you had to get it right.

Paul: Yeah, that's really interesting. So you think that the entire creation of casual gaming and social gaming and that sort of thing was really predicated on the fact that they were able to release things quickly?

Edith: Yeah, I mean, it used to be, so I'm leaving aside the social aspect, 'cause you can have a social aspect in a console game, like as you just talked about. Yeah, absolutely. You can have matching, like there are good games, like Xbox with social.

Paul: You know, I'm not sure you can, because I think a major part of casual games was that they were able to iterate really quickly on the game mechanics.

Edith: Yeah, and that's what I mean. I think we're in agreement, as we always are. But the reason why they were so successful is they could iterate.

Paul: Right. So this is interesting. I think this is the first place where I've ever seen me be unhappy about continuous delivery, because it created, you know, casual games and the pay-to-play and all the bullshit which makes gaming awful these days.

Edith: Wait, you're unhappy or you're happy?

Paul: Unhappy.

Edith: Unhappy.

Paul: Unhappy. I mean, the casual games are like, I sat down to play some casual games, someone told me it was awesome, and I downloaded onto my phone and I played it, and then I couldn't get anywhere because I wasn't willing to spend money into it.

Edith: Oh my god. The horror.

Paul: Well, I'd have happily paid twenty dollars, but I hate being asked for ten cents here and two dollars there, and, like, that model of gaming, like, fundamentally offends me as a consumer. And I understand it's what has to be done, but it's what has to be done because of Zynga, and because it exists, it's a sort of a race to the bottom.

Edith: Yeah, well, they've iterated their way into this, that this is the best way to extract value, is not to ask for somebody twenty bucks up front, but just instead to get them sucked in and be like, quarter, quarter, quarter.

Paul: Well, it's official now. Continuous delivery has a downside.

Edith: Well, it's the parking meter versus the parking garage metaphor. Like something that you keep feeding quarters into versus you just go and park overnight.

Paul: I see it.

Edith: Wait, so it's official, continuous delivery makes Paul...

Paul: Makes Paul unhappy for the first time.

Edith: Unhappy for the first time. Oh, Paul, so continuous delivery makes you unhappy.

Paul: Well, only in this situation so far. I don't think I've seen another situation.

LinkedIn and the Co-Metric

Edith: Well, this is our tenth episode, so I have another situation for you.

Paul: Excellent.

Edith: So LinkedIn. So LinkedIn is getting a fair amount of flack right now because they're very aggressive about invites and re-invites. And so, you know, there's this meme floating around, like

Paul: Oh, the polar bear sticking his head into the bus. Do you want to connect on LinkedIn?

Edith: Yeah, or like, you know, Back to the Future Part three, you know, the Western Union rolls up and says Marty McFly, do you want to connect? And you know the reason why they're doing all this stuff is because it works.

Paul: Does it work? There was a fascinating talk by Keith Rabois, or I don't know how you say his name. But it's the unintuitive way of pronouncing it. Who is, he's at Khosla, but I think he's, is he COO of Square, or he was the COO of Square?

I think he's at Khosla now. He was the COO of Square.

Edith: And he was at PayPal before.

Paul: He was at PayPal, right. And so he gave this talk in the Y Combinator Startup School talks.

Edith: Did you see it in person?

Paul: No, no, it was the one where you had to be part of the class, like the Stanford class. And he gave a talk, and so he talked about every, like, KPI or every metric can't have a co-metric. And that was, like, one of his big three lessons. Other lessons, like, have a dashboard.

But the idea was that every metric can be juiced.

Edith: Of course.

Paul: And the way to avoid juicing leading to problems is to have a co-metric.

And it feels a little bit like there's a co-metric for LinkedIn's engagement thing that they're not, or they've got the wrong co-metric, or no co-metric. And it's like people are getting very unhappy. People who don't see a lot of value from LinkedIn are getting very little value from LinkedIn. They're still kind of on LinkedIn.

Like, I mean, I consider myself to be one of these people. Like, I'm on LinkedIn. I never really use it. You know, they've never really found a product market fit with me apart from having a profile on it. But...

Edith: Oh, I love LinkedIn.

Paul: What do you do on LinkedIn?

Edith: Every time a new person signs up, I go and look them up on LinkedIn so I can know their role and what they do. It's like we had Sam the other week from Rapportive.

Paul: Right, right. And so that's value for you, but not so much value for them. So LinkedIn offers a lot of value to the people who are using the sales packages, the recruiting packages.

Edith: Or just anybody who wants to know more about their customers.

Paul: Know more about their customers, their colleagues, the people they want to date, all this sort of thing. But for the people themselves, there's not necessarily a massive value. And especially, like, if you're in a high demand area, you're gonna get tons of recruitment spam.

I get tons of spam for people who want me to use their software or who want to sell me their software, and especially who want to sell me consulting services. So the signal to noise ratio is complete shit. And at the same time, you know, I keep getting invites from people I don't know, or maybe I do know them. It's like, did I meet them at a conference?

I don't really know. Like, I just don't get a great lot of value out of LinkedIn. And so it's trying to juice up its metrics and trying to make me, you know, better LinkedIn person or whatever that is. Without, I feel, making LinkedIn a product that I actually want to use or that I want to use more.

And it's funny because, like, we talked about Rapportive. Rapportive is like the greatest product I've ever used. Or one of the greatest products I've ever used.

And it's owned by LinkedIn, and it's amazing. But LinkedIn the product is not a great product. I'm happy it's in the world, but I don't see myself wanting to engage with it more in the way that I do with Facebook or Twitter or Instagram or whatever.

Edith: Well, I mean, I disagree. I use LinkedIn daily. I mean, I find it immensely...

Paul: You're in a sales role.

Edith: Yeah.

Paul: And you're in a recruiting role. But if you're not in one of these roles, I don't think you're gonna find it useful even slightly.

And what's that old adage about, if you're not paying, you're the product? It's literally half of, not even half, but 98% of LinkedIn is the product to the two percent who are selling or recruiting.

Edith: Oh yeah, I mean, it was funny because a guy tried to ask me out over LinkedIn. I never met him and I was just like, dude, what are you doing? This is LinkedIn, not, like... Not Match.

Paul: Maybe LinkedIn should add a dating package.

Edith: Yeah, I don't know. I was just really puzzled. I was just like, what are you doing? I really like the idea of the anti-metric. What are some examples that he gave?

Paul: I don't remember. It was a good talk, though, I'd recommend looking at it. But I think the, like, if you're sending out email invites or something like that, you know, probably your co-metric is, you know, the dissatisfaction of some sort. So number of people who unsubscribe is probably a good co-metric for number of people who re-engage.

Edith: Yeah, I think there's a fine balance, though, that, you know, there's always gonna be somebody who's unhappy. Like, so an example I heard was, like, so Gmail, whenever they do a canary release, i.e. they're pushing out new code to some people. One of the things they monitor is the prevalence of "Gmail sucks" on Twitter.

Paul: Right, right. Exactly.

Edith: And they always accept...

Paul: Beautiful co-metric.

Edith: But they always accept that there will be some level of it.

Paul: Right, right. And I think it's fine for there to be a level of it. Like, you're trying to hit certain things, you're gonna break a certain number of eggs. And there's always lots of people who are, you know, angry about change, and angry about breakage.

Angry Screeds and Authenticity

Edith: I read this great blog post by this guy. You know, he talked about there's always people who don't want progress. There's always somebody who would say the Sistine Chapel is good enough without that painting. I think you wrote that.

Paul: I think I read that. It was good.

Edith: I enjoyed it. But what was your, was there a specific inspiration to write that post, or was it...?

Paul: I mean, that was the "It Really Is the Future" post, talking about why everyone was hating on Docker. I think it's sort of tangential to the current thing. But there's a little bit of a correlation there. Like, people don't like change. People like things to be the way that it is. And you see people talking who send out angry screeds about, you know, LinkedIn should be doing X, or better example, Facebook, Twitter,

Edith: Twitter.

Paul: Exactly. Twitter should be doing whatever. Twitter shouldn't have launched Moments, they should have launched Moments, they shouldn't have hearts, they should have hearts, you know, that sort of thing.

Edith: It's like, they should be able to close.

Paul: Fundamentally, you know, there's someone at the other end who appreciates your screed and reads it and, you know, combines it with the five hundred other, 10,000 other elements of feedback that they've gotten, and combines it with the metrics that they have and the co-metrics that they have to decide, you know, what they actually want to do.

Edith: Yeah. I mean, that's the job of a product manager.

Paul: Right, exactly. Exactly.

Edith: You can't make everybody happy all the time.

Paul: And so the people who are complaining about LinkedIn, I think fundamentally they're right, in the sense that every complaint is right. Every complaint is accurate.

Edith: Yeah, I mean, the thing that I didn't like, so the thing that I didn't like is they recently switched their messaging from something more like an email so now, like, this horrible, like, IM type style, which when I'm doing a business communication, I don't like doing.

Paul: Oh, interesting.

Edith: Like, when I write somebody in a business, I, like, start with, like, dear or hi.

Paul: Well, there's a question, should you be doing that? I'm not sure you should. One of the kind of fundamental parts of doing modern business is authenticity.

And if you're going to be talking to your customers, if it's a very formal interaction versus a very sort of informal, where we're friends, you're a part of the community, you're a part of the product. I'm very much a latter, a form of the... a fan of the latter.

Edith: You're a former fan of the latter?

Paul: I'm a fan of talking to your customers with authenticity, which very often means dropping the "Hey John" at the top and the "Best" at the bottom and that sort of thing. And more IM-like and less.

Edith: Yeah, it's funny because actually for our biggest enterprise customers, we give them a Slack or HipChat channel.

Paul: Right, exactly.

Edith: Yeah. Where, like, they're on, and they ask us for stuff and we write them back. It's cool and fun and liberating.

Paul: I had an interesting conversation with one of our very good customers. You're familiar with Rainforest QA?

Edith: Oh yeah, love them.

Paul: They're our customers, and I know them socially and that sort of thing. And we sent out some email, I don't remember exactly what the email was, and Fred, who's the CEO, yeah, I'm telling the story. It was sent out over Intercom, so sent out to all of our customers. It goes back to, like, a, you know, Intercom inbox where it's monitored by product managers or support people.

Edith: We use Intercom too, by the way.

Paul: Awesome. And so Fred replies, and I think it might have been an apology for something, or maybe there's some downtime or something. And his reply was, no, like, "Dear Paul" or, you know, "Best" at the end of it. In fact, he didn't basically say anything. He said the exact phrase, "fuck you."

Edith: That's a very Anglo-Saxon phrase.

Paul: It is a very... Fred is English.

Edith: I think that's one of the oldest Anglo-Saxon...

Paul: Right, right. And so I responded to Fred. So first my support people said, oh, I think we've got an irate customer here. Paul, maybe you should talk to him. And it was routed to me.

And I replied to Fred and I said, how about you go fuck yourself?

Edith: So, first of all, do all irate customers get routed to you, or just Fred's?

Paul: I think he was routed to me because I think you could tell on the software that I knew him or that I talked to him before or something like that.

But the point that I was making is not that I hurl insults at customers, but that the closer you are to friends with your customers, the longer they're gonna be your customers.

Edith: Yeah, I don't have any that I would respond like that to.

Paul: Right. I have about one that I would respond that way to. But if I was to say, you know, "Dear Fred, I'm sorry that you feel this way." Perhaps we could have, I don't know, whatever. Perhaps we could arrange a call to discuss your, well, you know, whatever. Like, it's an inauthentic conversation.

Edith: I think Fred is a special use case that, like...

Paul: It's an extreme example of making sure that you have authentic conversations with all of your customers.

Edith: I think it is true to be authentic. Like, I remember, so I was working at an Internet of Things company, and this is before there was Intercom, so literally every time somebody registered their device I would just handwrite them an email. 'Cause I was so excited and I just wanted to know more. And then I would get emails back like, are you a bot? And I'm like, nope.

Paul: Yeah. No, I did exactly the same thing before. The first fifty or hundred customers, I wrote them an email. I mean, the first twenty, I wrote them an email before they became customers, but I said, like, you know, why are you using, where did you find us? You know, tell me what you think.

And then when I eventually switched that to Intercom, when I was spending, like, twenty percent of my time writing those messages and we had enough customers to be that... And then, you know, still try to make it as authentic as possible. Customers are developers, so I put in the bottom, you know, while this was automatically sent, you know, I'll personally read and reply to all replies.

Edith: Yeah. It's funny because I think everybody's kind of on this game now of personalization. I actually got a personalized pitch on LinkedIn. From somebody who said they listened to the podcast.

And, like, they didn't just say they listened to the podcast, like, they quoted from a couple episodes. And then they tried to pitch me.

Paul: Yeah, I mean, that's the standard thing, right? You write a paragraph that shows your connection to the human being. Hi, how are you? I see you just had a child.

That's awesome. What lovely pictures you had. And then you go to the pitch.

Edith: And hopefully they've actually had a child.

Paul: Right, right. I don't think you should apply that to everyone.

Edith: Yeah, no, I mean, the worst is when you get something like hello brackets first name. And the inside it says first name, not your first name.

Paul: Right. So it's interesting, you see, have you read the book Predictable Revenue?

Edith: Yeah.

Paul: So everyone's read Predictable Revenue now, right? It's the modern sales techniques. And so you start to see the emails that were in Predictable Revenue taken almost verbatim. And I get those emails all the time.

Edith: It's funny, so I was talking to my friend, she's awesome. She's done sales at Yammer, Responsys, Mixpanel. And there's just this constant evolution of sales pitches too. Like, one of the most liberating things the HubSpot VP of marketing said is that what he said was not content marketing is God, which was what I expected him to say.

The techniques that worked in the past are used up. You need to find new techniques. And I think that's the same with languages.

You can mourn the past and say, oh my gosh, why doesn't, why is... You know, I remember when SEM was really good and now everybody does it. I remember when content marketing was really good, and there's still niches. And same with languages. I mean, where do you think languages are going?

Raising the Abstraction

Paul: Yeah, that's quite the spin around from product in sales things. Where are languages going? I mean, I think languages are fundamentally about raising the abstraction.

Edith: Yep, we agreed.

Paul: Yeah. There's a question of what is the next level of abstraction.

Edith: Actually, though, do you know Jesse Robbins? He had an interesting interview in ReadWrite Web about saying that he had to basically custom build everything for his new company. Because he couldn't really use anything that was out on the market.

Paul: Okay. Like, I mean, it's a hardware device, is that what you mean?

Edith: Yeah, like, so he had to build his own data center, all this stuff. So it's kind of the opposite of... He said it was the opposite of abstraction.

Paul: Oh, interesting. There wasn't enough latency, or, yeah, he had specific constraints around latency or something along those lines.

You often get this sort of thing where the fundamental differentiator of your company is that you're able to take some process and do that process much, much better, whereas everyone else is using commodity components.

And in fact, you know, Google, that, you know, commodity components was their thing, whereas everyone else was using specialized components. And Google was able to succeed because they were, or one of the reasons, many, many reasons that Google succeeded is that they use commodity components and everyone else is using

So they discovered a new way of creating data centers, or, you know, whatever it is that you're doing, achieving business, being full stack, being less full stack, you know, abstracting the right things, de-abstracting the right things is a major differentiator in most businesses. They found one process that's better or different or gives them more success.

Edith: No, Google's kind of legendary for building everything in-house.

Paul: Right, right. When Google started, all their servers, like, used, everyone else was buying you know, Oracle on IBM, expensive multimillion dollar things, and Google was putting, like, you know, the Dell that you were using in your homeroom, and they just figured out how to make fault tolerant software. And they were ahead of the world because that's what everyone is doing now.

Fred Stevens-Smith: Hi, this is Fred from Rainforest QA, ex-customer of Paul and CircleCI, and still offended human. You've been listening to the at best mediocre podcast To Be Continuous.

Edith: To learn more about Heavybit, visit Heavybit.com. While you're there, check out their library, home to great educational talks from other developer company founders and industry leaders.

Subscribe to Heavybit Updates

You don’t have to build on your own. We help you stay ahead with the hottest resources, latest product updates, and top job opportunities from the community. Don’t miss out—subscribe now.