October 13, 2020
Ep. #27, Ready For Observability with Corey Quinn of The Duckbill Group
In episode 27 of o11ycast, Liz and Charity are joined by Corey Quinn of The Duckbill Group. They zero in on the point at which a system need...
In episode 52 of To Be Continuous, Paul and Edith discuss the changing landscape of recruiting, focusing specifically on remote vs. local hiring, and how a company’s personnel structure changes as it grows.
Paul Biggar: So, Edie. LaunchDarkly is all local?
Edith Harbaugh: We're transitioning right now.
Edith: How about Dark?
Paul: Dark is all local, but we are-- Are we eight people, or nine people?
Edith: Interesting. So you've done--? Every time I ask you, it's a different answer. You've done five startups now?
Paul: It's four, four startups. I am not counting a couple of things, but four that incorporated, let's say.
Edith: All right. What's been your evolution of local versus remote?
Paul: So I'm not going to count any of the early ones, Circle was the first one that really counted, and that was not so much remote as distributed.
So we didn't have another office, we had the Circle office, and frequently the Circle office was-- I think it's been about half the company has been at headquarters and half the company has been remote.
Now the company is 230 people or something like that, it's huge, and the engineering team in particular is extremely distributed.
Edith: Tell me about why you did that with Circle, and why you're not doing that with Dark?
Paul: With Circle we knew what we were building. In 2012 it was very clear that there was a CI-shaped hole in the market and we knew what to put into that hole.
Edith: Like, a circular--?
Paul: Yeah. It was a circular hole.
Edith: There was a peg.
Edith: There was a Circle-shaped peg that you could insert.
Paul: Right, and many people were trying to fill it with other shaped pegs, but Circle was the one that fit the market the best.
And with Dark, we're building a thing that we don't really understand, so there's an exploration.
There is figuring out-- Very high bandwidth communication is required for a lot of things, and we're not planning on staying like that forever.
In fact, I imagine that when we raise our next round that we will probably shift partially because it's so hard to hire.
Edith: Yeah, I completely agree with you. Not about Dark and Circle because I don't know them so well, but about our own company, about LaunchDarkly.
At the beginning there is a lot of discovery and I think it really helped us that we were all at Heavybit.
Paul: That you were all in the same room, or specifically at Heavybit?
Edith: Both. I remember John, my co-founder and I, the CTO, would go out to some--
I don't even want to call them "Customers"because they weren't paying us, but we would go out to some meeting and we'd hear people's problems.
We would come back and, this is the Heavybit part, we would all get lunch with our engineers.
And when I say "All get lunch," I mean we had two engineers and the four of us would go get lunch.
It wasn't super fancy, and we would talk about how the meeting had gone.
I had come from a world before big enterprise companies where you would write these elaborate PRDs, product requirements.
I don't know if you've ever been in a big company where--
Paul: No. That sounds like hell.
Edith: Yeah. When I was a product manager at a big enterprise company you'd write a 40 page spec dock that nobody would ever read, and then you'd have to explain "This is why I care about this." And the engineers would--
Paul: But the spec is what you have to do. Like, I feel that the value of that meeting is "Here are the learnings."
Edith: Yeah. But you would try to synthesize a month of meetings into one spec, versus in the early days we would just go to a meeting and we come back like "Here's what people are thinking about."
Paul: At Circle when we would have those meetings we would frequently--
But I think we probably only did this in about half the cases, but we would write up what happened and just email it to the company.
It's like, "We met this person. They had these interesting insights, X, Y and Z." It wouldn't have any necessarily any answers, just good questions to be thinking about.
Edith: That's certainly what we do now that we're bigger, but I think a huge strength of us looking back is that we could do it in real time.
John and I would be heatedly discussing like, "What do they mean by this? What do they mean by this?"
Our engineers are like, "What do they mean by this?" And there wasn't an intermediate step of "Let me wait to have 40 minutes to put together a grammatical paper."
Paul: We are co-located, but we also have like a little more process than that, which is something different than we did a Circle where we were like "We are extremely low process."
And so we have sessions where we discuss customer feedback and we write down customer feedback and have a place that we sit down and, "How is this going to turn into product requirements? Should we even do this thing, like where does it go on the road map?"
Edith: We of course did that when we got bigger, it's just at the very early stage when we're under 10, we didn't really do that.
I'm finding a lot of value from doing it even at the small size. As someone who is extremely against process, now that there is good process-- This is what you get from having a PM as a co-founder.
So, I'm sure John is pretty blessed with the same thing.
You end up with process that actually works, and so the whole anti-process thing that was going around in the 2012-13-14's.
I think it's terrible, but I hadn't personally seen a better thing to do and now that I have, it's wonderful.
Edith: Paul, I have been continuously biting my tongue for last three minutes to not say "I told you so."
Paul: Yeah, no. I'm sure you did, probably multiple times. I don't listen to old episodes, but presumably many an episode had been like "Paul, did you ever think about having a process?"
Edith: No. What I said in the past is "You always have a process. It might just not be a good process, or one that you acknowledge."
Edith: If your process is you don't write anything down and the engineers build whatever they feel like building, that's still a process.
Paul: Yes. That was our explicit process for a time at Circle, and I think actually that's one that doesn't play that well with a distributed team.
Edith: It's awful. Because you have this huge time lag, you have perhaps a culture barrier where somebody doesn't want to question.
Paul: There was a lot of expectation that you would figure out what was valuable to work on, and in a distributed team that's terrible.
In a co-located team, it probably would have been much better, but still not good.
Edith: Yeah. I think that process of "We didn't write much down," We did write stuff down, worked till about eight and then we had a transition to more process.
Paul: We write specs for everything now, and they're not like particularly long specs. They're often just a couple of paragraphs being like, this is--
Edith: Yeah, we would write a paragraph but we wouldn't write volumes.
To go back for a second, I don't want to say that we didn't have specs, they were just-- I think the thing we did not have to do was defend our rationales behind them.
Paul: You mean as founders, or as early employees? It was the same for everyone, or was that just a founder thing?
Edith: I'm not going to name a name, but there's a company you and I both know very well that's based in Ireland. They still haven't built SSO.
Paul: A company that we both know that is based in Ireland, I can't imagine what that is.
Edith: No, me neither. Because they're like, "Why would we build this? Nobody needs it."
And anybody who's spent a lot of time out in the field is like "Every big company needs SSO."
So the thing I mean about high bandwidth is when it came to "Why does LaunchDarkly need to be able to SSO? Why is security important? Why are we building this audit log feature, which might sound extremely boring?"
Our engineers were right there with us because they're like, "Yeah. We've heard you come back from this meeting and you talked about why the customer wants this, we get it."
Paul: Right. Fundamentally, you want to get everyone at your company on the same page.
There are different challenges at different sizes, I would say it's not absolute. Like, "The absolute one is better," or--
But in particular, what I think that you get from a distributed team is that you are forced to write things down, and there's a lot of benefits from that. The people who find out about it aren't just the people who sit nearest you or are in the right meetings, everyone in the company gets aligned on the same page.
There's a lot of forcing functions around things like accessibility, things that you have to do if you're distributed because of so many people who aren't in the company, whereas typically you would get someone who is a certain second class citizen if they're the only remote person at the company.
Edith: Paul, you're just saying everything I would say and I'm just--
Paul: Another podcast where we completely agree with each other? I am shocked.
Edith: No, four years ago you would've said "Everybody should just get it."
Paul: You think I would've said that?
Edith: I could replay that episode for you.
Paul: I would rather not. I'm strongly trying to forget all of that.
Edith: No, but I totally agree.
What I'm trying to say is that at every stage you need a slightly different thing, like up until we were 8 people we had a daily stand up with the entire company.
That was extremely valuable, and I still remember our last stand when we were about 15 people and we were like, "OK. This is the end of daily stand ups with everybody."
Now we have to like start doing different group meetings and we'll write down our group statuses.
Paul: Yeah, we're still doing the daily stand up and it's going to get to the size where--
One thing we do actually that's really good is we have this contact sharing meeting on Thursdays, and that's basically if you're newer than someone else, which means everyone, you can ask for a particular document to be discussed from the past.
"Why did we do this this way?" It could be like part of the code base, it could be product spec, it could be just like "I don't understand this concept."
And then we talk about the concept, and what we're especially finding now that we have marketing and other folks that they gain a huge amount from this and much more than the engineers, who gain some.
Edith: I think it's also really valuable, and this is what I would try to push our team towards, is don't venerate the past.
Does that make sense? Like, "We might have done it this way two years ago, but two years is a long time."
Paul: Yeah. That's actually one of the great things about the context sharing, because we can share "Back then we thought this and that was wrong. It was a terrible idea and we regret it entirely."
Edith: I try to avoid judgment, like I try to say "This was something we tried two years ago. Maybe this was a good idea two years ago, but did it work?"
Like, I'll give an example. Our first meet ups, we got very low attendance. They didn't work at all.
When we started to get a bigger name and a bigger brand our meet ups are great now.
Paul: Interesting. It's just the size.
Edith: Yeah, there's just stuff that comes with size that works.
Paul: But you probably got something out of the small ones, like some evangelists, some people who like really like to--?
Edith: We got a lot of pizza that nobody ate.
Paul: All right.
Edith: The worst was we accidentally scheduled it during Salesforce Dreamforce and Warriors season final, and we got two people who showed up.
Paul: It'd be better that no one showed up.
Paul: If no one shows up you can go home.
Edith: We have one person who I knew through our accelerator who's basically come to every one of our meet ups.
Paul: Aw, that's cute.
Edith: Yeah, but if a lesson from that is "Never do a meet up--"
Paul: Right, it's not the lesson at all.
Edith: The lesson is like, "That didn't work. Let's put this on pause for a little bit and then try it again."
Paul: So, you moved to Oakland at some point?
Edith: Yes. You came to our moving in party.
Paul: I remember, but the people listening to the podcast did not come to your moving in party, so I'm setting up context for the discussion that ensues.
Edith: Thanks Paul, I thought you were trying to cover up the--
Paul: My terrible memory?
Edith: But you did remember to wear your LaunchDarkly shirt.
Paul: Of course, I always remember that. So you were in Heavybit and then you moved to Oakland?
Paul: At what size?
Edith: We were four people when we moved in, because I remember we literally could pick up everything, pack it in a hatchback of our engineer's car and drive it the three blocks.
Paul: This was when you moved into Heavybit?
Edith: Yeah. Because we were up at market three blocks away, and so we put all our monitors in.
Paul: Where was that, runway?
Edith: We had gotten coworking space at the Yammer office, and I remember because I was like, "This is the easiest move we're ever going to do."
Paul: So you were here till I seem to recall like 18-16-20, somewhere around there?
Edith: After we closed our A we moved to Oakland, and that was spring of 2016.
Paul: Was that for office space, or for hiring, or for standard of living? What was the purpose of moving to Oakland?
Edith: We'd always wanted to be in Oakland.
We'd actually started the company in Oakland, so John and I-- John my co-founder had started at a coworking space in Oakland, and then we got into an accelerator based out of Yammer.
Then we got in at Heavybit, but whenever we hired somebody new we always said, "We're going to move to Oakland."
Paul: You told them from the start?
Paul: Gotcha. So there was no pushback when you finally decided to move?
Edith: The only pushback was people kept wanting us to move, and I said, "Until we get our A we don't we don't have the money to move."
Paul: So people wanted you to move far sooner?
Edith: They'd already gotten apartments over in Oakland.
Paul: Right, OK.
Edith: They were ready, and the day the A closed they were like, "Great."
Paul: "It's time to go." Have you found it easier to hire in Oakland?
Edith: I think so. I don't know if there's another variable, which is that we're a more successful company.
When you're trying to hire when you're 8 people and just getting off the ground, versus we have 800 customers worldwide. We've raised $77 million dollars.
Paul: How many customers?
Edith: 800 worldwide.
Paul: 800 customers?
Edith: $77 million dollars in venture funding. That's very different than when we were sitting here with $2 million in venture funding and scraping to get a customer.
Paul: We should probably pause and recognize that those are fucking crazy numbers.
Edith: Which one?
Paul: Both of them, actually. $2 million dollars is just so much money, and then $77 million is just an absolute flabbergasting amount of money.
Edith: That literally happened, I was on a sales call this morning which is a funny side story--
But I was selling to this CTO and I'm going through our brag slides, and he's like "$77 million?"
And then you hear the voice on the phone, like "Yeah. Things are different now."
Paul: What did that mean?
Edith: I think he was just flabbergasted that we had raised $77 million for feature management.
Paul: Did you explain to them that you had 800 customers worldwide and that they paid you a phenomenal amount of money for the thing?
Edith: They pay us an intrinsic value based, much like Circle, who is a--
Paul: Would you describe its intrinsic value as a value minus or cost plus?
Edith: I'm lost in your logic now.
Paul: One of the reasons that everyone moves to Oakland and other places is that they can't afford to hire in San Francisco or Palo Alto or wherever it is that they are anymore.
Edith: Let me loop back to the sales meeting for a second.
It was really funny because it was a sales meeting in San Francisco, and we made a big push to show up in person because it was a really important customer that we were trying to--
Paul: Did you still live in San Francisco?
Edith: I used to live in San Francisco, my co-founder lives in Oakland. It's a customer we have some usage at and we're trying to get more.
So it was a meeting with their CTO, and I was like, "OK. You show up in person if it's the CTO."
So it was myself, my co-founder John, our salesperson, and we get there bright and early and we're sitting in the conference room and everybody's remote.
Edith: Everyone else on the meeting.
Paul: Wow. I've done this, I've gotten to a meeting at Google and it's like, "Why did I come here? We could have-- I came in person. What was the point of this?"
Edith: Yeah. We'd all shown up, we'd put on our nice clothes, we're like "We're going to show the customer the report," and then we walk in--
Paul: But you could've been sitting at home in your pajama bottoms, you put on a shirt and makeup and the rest doesn't matter.
Edith: You put on your makeup?
Paul: I put on my makeup before a call. I mean, it depends on the lighting of course.
Depends if I have time to set up the lighting before the call or not.
Edith: Yeah, but it really reminded me-- Have you seen , there's this movie from the 80s with Val Kilmer.
Paul: Is it Top Secret?
Edith: It's the one where he's a student at a rip off of Caltech.
Paul: That's not the one.
Edith: Everybody wants to sleep in, so they just go and they set up-- This is from the 80s.
They set up a tape player to record the lecture, so there's this wall of tape recorders in the lecture hall, and then the end of it is that the professor also sets up his.
So the finish of the story is I literally thought we'd rush all the way to support customers thing to present in person.
Paul: That's a real power move, not showing up and it being remote. Was it like someone showed you to a conference room?
Edith: Somebody showed us to a conference room--
Paul: Got you some water--
Edith: Got us some water--
Paul: Maybe a snack.
Edith: We set up and it's a customer we're trying to sell to, and then there's people on the phone and we're like, "OK. There's somebody--"
They were legitimately remote, there was somebody in Ohio, there was somebody in India.
Paul: But the CTO was also remote?
Edith: No, he's in San Francisco.
Paul: Was he there?
Paul: He didn't show?
Edith: It turned out he'd gone to the wrong building.
Paul: So he dialed in?
Edith: No, he showed up about six minutes late being like, "I'm so sorry.".
Paul: That could be much worse.
Edith: Like, "I'm so sorry. There's multiple buildings."
But it was hilarious because I was just literally thinking, "Why didn't we dial in?"
Anyway, that's part of the power of local versus remote, is just to show that--
Paul: You can show up.
Edith: And that you care. You get a lot from body language.
Paul: I think one of the things about Circle being in San Francisco was that we were the CI company that was in San Francisco, and there was a CI company in Amsterdam and in Berlin and there was one in Australia and there's a bunch of them everywhere.
We're the ones that people have personally met because we're at all the meet ups and we know the people and we meet random people, and so on.
You'd be at a party and you'd just be talking to someone, and they're like, "You do CI?" Whatever.
Edith: You sound like real fun at parties, Paul.
Paul: I definitely go to parties and talk about work all the time.
Edith: Like, "Let me tell you about CI." It's like, "No--"
Paul: I got uninvited from all the other parties. I think being in San Francisco or at least like San Francisco adjacent, Oakland, for dev tools is this very important thing.
Edith: Yeah, I didn't realize it and then we sold to Circle because I started to know you through Heavybit, and that didn't count for anything.
Then you made us go talk to all your engineers.
Paul: I think by the time we were doing it, I just wasn't doing anything at Circle.
Edith: But we could go into the office and talk to Jim, and the CTO. I was like, "Why do we have to go do that? They could have looked at the website."
It's like, "You have to show up."
Paul: Yeah. You make the connection and then you
managed to convince them, because a lot of the time in terms of feature
flags we could build that ourselves.
We could use the open source thing, or we could try it out and maybe get to it eventually. It's somewhere on the road map.
But when you show up in person, someone is like, "No. Actually that sounds like it could solve our problems," and you get to sell.
Edith: So for Dark, when do you think you're going to start to go remote?
Paul: We have transitioned a little bit from the original company, which was like "Figure out what we're building."
We have figured out what we're building, and we're now in the path of going to launch and going to search for product market fit.
Edith: Strange Loop?
Paul: We are having a launch party at Strange Loop. You should come.
Edith: What's the date?
Paul: September 13th, on the evening of the conference.
Edith: Must be pretty exciting.
Paul: Yeah, and you can sign up at DarkLang.com/launch.
Edith: What stage do you think you'll start to hire distributed remote?
Paul: After the next round.
Already it's extremely challenging to hire in San Francisco, everyone that we talk to is talking to like 15 other companies. There's a lot of competition here, there's a lot of companies here.
It's not that isn't true in other places, but there's a lot more people here that you can hire and a lot more people who have the expertise that you're looking for.
Edith: I think Oakland is an advantage for us. Our managers come out of Atlassian, so my co-founders connection, and they say it is easier to hire at LaunchDarkly in Oakland than Atlassian in SF.
Paul: That makes more sense.
Edith: Why do you think so?
Paul: SF is saturated. If you're in a high demand place, then a higher supply isn't all that useful because the demand is so high.
But people want to have low commutes, because commutes affect people's happiness and how much they enjoy working at a company.
If you have an hour long commute for a shit day versus a 20 minute commute for a shit day, that's going to be materially different.
If you have a wonderful day and then you have an hour long commute to home, you're still going to be like "Fuck.".
Edith: Oh, my gosh. That just adds up.
Paul: Yeah, and some people don't mind the commute.
Some people are built in a different way that that makes them OK with the commute, but even in that case people with long commutes would rather "Maybe I can work from home a couple of days a week."
And when you're working from home, there's a material difference between "It was OK for me to occasionally work, but I had to miss the meeting."
And "I have an experience that is like 98% equivalent."
Edith: I do think it's possible to live too close to work.
Paul: No. How close would be too close?
Edith: Like, for example, if you had a house office.
Paul: Office houses are great.
Paul: I would say the ideal startup experience is having your office in your living room.
Edith: What do you like about it? The hypothetical "You" of course.
Paul: The hypothetical me, yes. This was definitely not our situation until recently.
If you're in the very early stages and you're in the stages where you work a lot, then you have the exact same experience no matter whether it's midnight or during the day. You have all your stuff, you have the same environment, same whiteboards.
Working from an office house is fucking ideal. Sadly, we had to give up our office hours because we hired too many people and then we went to Heavybit, and then we just got our own office near your house actually.
Edith: Yeah, I've heard about it.
Paul: You haven't seen it yet?. You should you come by and see our office house.
Edith: Was there an office warming?
Paul: There should have been an office warming, but we did not have an office warming.
Edith: It must be very cold.
Paul: Actually we have air conditioning and San Francisco has been insane recently, so it's been freezing.
Edith: You said something interesting before about the transition between everything being spoken versus written down.
I think we're at the same phase, because we crossed 100 employees.
It's definitely a phase where I'm like, "OK. Don't assume that people know this is the way it works."
Paul: I think that's the corollary to writing it down, is that you have to organize it.
One of the things I curse at the moment is Google Docs, because your entry point to your documents is like this directory structure.
Edith: It's awful.
Paul: It's awful.
Edith: The search for Google is awful.
Paul: Yeah, and with other things, I don't know which ones because we use Google Docs unfortunately-- You can organize a hierarchy.
You start in this page and this page tells you all the places to go, and you can customize it, and you can have the other things still sitting there but not really being relevant or whatever.
That ability to organize people's experience around how they consume the documents that matter to your company is huge, especially for new people who are coming in.
I have a theory that your culture and knowledge is as good as the last person you hired.
Paul: Not sure if I agree with your theory, but carry on.
Edith: That they don't know anything about your culture, they don't know anything about your knowledge, so they're at this point the strongest and weakest link.
Paul: Why are they the strongest?
Edith: Because they are going to convey stuff to the next person you hire.
Paul: OK, so you're saying that if you don't manage to have a good context setting for the current for the newest hire, this is why people have extensive boot camps?
Paul: Do you have a boot camp for new hires?
Edith: We don't call it "Boot camp," we definitely have onboarding. I think "Boot camp" has a lot of overhead as a word.
Paul: Do you have a planned experience with some sort of coach?
Edith: We have 30-60-90 plans, so this is a holdover from stuff I liked when I was a new hire.
Paul: Yeah. We do 30-60-90 as well.
Edith: Back when I was looking for jobs, I would ask my new manager for 30-60-90 before I accepted the job. They would be surprised sometimes.
Paul: Right. Because people usually don't like to be managed.
Edith: I wanted my objectives. I didn't think of it as being managed, I was like "What am I going to do? Cause I don't want to give up my current good job and go there unless I know what I'm getting into."
If you don't know what I'm going to do in the first 90 days, I don't want to show up and flail around.
Paul: Is that still each person is having an individual learning experience within their team, or is there a place that all the new hires go to acclimatize?
Edith: We have some stuff that's across all functions, and we've definitely started to specialize based on different roles. Like a 30-60-90 for a developer.
Paul: I don't mean a 30-- I'm trying to get back to the boot camp. Do you have something that is like roughly a boot camp-like program for new hires?
Edith: Yeah. But beyond that what we expect you to do is individual by function. How about you?
When you're still at this stage where you are hiring one person a quarter, do you have--? How do you add on board?
Paul: Yes. I think we are hiring a little bit faster than one person a quarter, but we're roughly in that area and it's very much a custom plans per person.
The context sharing thing really helps, it was originally started as an engineering thing and we've realized that it has as much if not possibly more value for non engineers.
Other than that, we do 30-60-90 and we set up one-on-ones.
There's a bunch of things we do to onboard people, but I would say that we definitely don't have any program or that sort of thing.
With engineers we do two weeks of pairing at the start, probably spend five days paired with two people.
We might be maybe tweaking that, it might be like four days with three people or something like that.
Get you acclimatized to the code base, we use an unusual language, so we use Caml for both the front end and the back end.
That's a little bit weird for people, so we try to get people through projects where they get to experience a large part of the code base and get to understand how it all works.
Edith: Paul, this all warms my heart.
Paul: That we don't just give people a desk and be like, "Here's the thing. Go figure out how to build it, and then build it?"
Edith: Or worse yet, the famous "You give somebody parts of a desk and say--"
That was what Amazon was famous for, was making everybody assemble their desk.
Paul: Right, God. So we actually have a really good technical onboarding experience, like you get your laptop and installing the Dark software takes about an hour.
It's an automated script and it just runs.
Edith: Yeah. It's a huge thing for us. We want to make people feel productive and happy as soon as possible.
Paul: At Circle we always tried to have people commit code on their first day.
I think that can be a lot to ask, just simply sometimes there isn't a task available that is going to be committable in five minutes of coding. But I find that ideal if possible.
Edith: Yeah, I think there was an anti-pattern about 20 years ago of-- Here is the exact anti-pattern, "Let's hire a lot of really smart people and have them figure it out."
Paul: Oh, my God.
Edith: No, that was like a real management.
Paul: It was, yeah.
Edith: It was it was like, "We'll hire these brave grads out of wherever and they'll figure out what the business is doing, what our core values are and what customers want."
And it's like, "No. You hire a bunch of really bright people and it's Lord of the Flies."
Paul: Right. We had this at Mozilla, it was very much a sink or swim culture.
And I'm like, "Fucking great, a meeting. We can understand what everyone is working on. We can get context."
But the thing is that most people in that team had been there four or five years, they knew exactly what everyone was working on, they had so much context over the code base and they reviewed everyone's code and so on.
So they had one meeting and everyone's like, "We don't want this meeting." And I'm like, "I'm completely fucked."
Edith: As the new hire?
Paul: I wasn't even the new hire at that point, but I hadn't I hadn't figured out large parts of things, and Mozilla was very much sink or swim.
The problem is that then, the problem for me was that I picked way too many projects and then they were all over the place and then they didn't--
But with little guidance and my manager had 35 direct reports.
Paul: That is the number that I said.
Edith: How do you even physically do that?
Paul: He also had some IC work.
Edith: 35 direct reports? That's like a goldfish.
Edith: You don't even know where they all-- They're like in this other thing getting eaten by a shark.
Paul: I think I've said this before on this podcast, I feel that we all got fucked by open source.
That there was this idea that you just show up with a pull request and that's how the software is designed and built.
And I think a lot of open source things have internalized that thing of just "Anyone shows up and if you're employed there, you're just showing up 40 hours a week with things that you figured out to do yourself."
But that was 10 years ago, I'm sure they've figured some of that stuff by now. Fingers crossed.
Edith: That just sounds awful. Not all open source, but just the idea that everybody can figure out everything without any context.
Paul: Right. If you look at open source projects, the
vast majority of them provide no context to
their potential contributors and people come up with
these drive by PRs that are absolutely not useful.
It's very few really well-run open source projects that do well at providing that context.
The knock against a/b testing used to be that it meant throwing spaghetti at the wall, but this sounds worse. This sounds like throwing random food stuffs in a wind tunnel.
Paul: At least when you're when you're throwing spaghetti against the wall there is a well-defined success criteria for your a/b tests.
It's like "It makes us more money, or less, or conversions or whatever."
The problem with this thing, with the "Sink or swim" or however you think of it, everyone figuring out their own their own success criteria is that everyone's on a different page.
Edith: They're not on the same page, they're all in different universes. Like, "Am I improving performance, am I improving usability? Am I improving the API? What am I even doing?"
Paul: I released an open source project recently, and one of the things that I really wanted to do before shipping it was giving a rough roadmap for how I wanted--
What are the goals of this?
What is success criteria? What is not done?
And, how can you contribute to this in a way that is useful and meaningful?
As a result, every contribution I made was meaningful and helpful, so every contribution that anyone made was actually aligned with these objectives because we told them how.
Edith: Because otherwise every check-in, it's like you could be doing this really neat thing in the UI that brings down performance by 10 %--.
Paul: Yeah, but it's really neat.
Edith: It could be that performance is so great that it's OK.
Or it could be like, "Performance is the paramount thing."
It could be like you're doing something really cool that increases concurrency, but destroys the UI.
You're just sending people out to just-- It's like the random walk of all random walks. That sounds really frustrating to come to work every day to.
Paul: We've been talking about open source, and obviously open source is this great distributed project.
It's like having a distributed company, and it should be clear that--
If you have a distributed company that has the same processes as a bad open source project, you're going to have the outcome of a bad open source project.
Edith: What do you think are the same outcomes?
Paul: The negative outcomes are just it's gone in a bunch of different directions, no one's work gets merged, people kill work at the last minute if it doesn't gel with the vision, or worse, merge it when it doesn't gel with the vision. Both of those are horrible outcomes.
Edith: "Vision" or "Context," and it's like, "Who's vision or who's context?"
Paul: Right, exactly. So as a distributed company, and I really learned this the hard way because we didn't do any of this in early Circle and they do this much better in late Circle after I left. I'm seeing it from the outside.
If you're going to have a distributed team you need to be super clear on context setting, on goal setting, on directions.
You need everyone in your company to be able to figure out, or company project or whatever, to figure out "Are we going in the right direction? Is there even a right direction? What does it mean to be the right direction? What are success criteria, or what criteria that you will know that it is the right direction?" Etc.
Edith: Yeah, I completely agree. I just feel sad about all those people who showed up at work every day and worked hard, and then it's like "My code is not what anybody wanted."
Paul: Yeah, imagine you spend like weeks or months on a thing and you're given no feedback along the way, so you finally show up with the finished thing and people are like "That doesn't match what we're trying to do."
Edith: Your manager has 34 other reports. You're like, "Who is even the arbitrator on this?"
Paul: I think with Dark, the reason that we specifically didn't go distributed at the start was that we needed this extremely high bandwidth communication, and there's some stuff in distributed systems that isn't well solved, like whiteboards.
Edith: Yeah. I've seen hacks for it.
Paul: But that was one that we used a lot, there's a lot of "I'm just going to draw this on the whiteboard and try to convey to you exactly what's happening here."
Edith: So, part 1, why do you think a whiteboard is good?
Paul: It's so cheap, they're everywhere, and there's a marker in front of you, and you can just draw the thing that you mean and there isn't that in any distributed thing. There are occasionally like--
Edith: Is it for swim diagrams, or what are you--? Or is it for code snippets?
Paul: What is a "Swim diagram?"
Edith: For flow between different parts of a system.
Paul: It could be for anything. It could be like you're drawing the architecture of your of your system, or it could be just like "I'm drawing a quick UI."
I found that when I'm talking to someone at my desk, I just pull out a piece paper and do it on a whiteboard just as if I could.
But it's much harder to do that over Slack.
Edith: Yeah, it's impossible to do on Slack.
Paul: Especially if you've got an open plan office and you're trying to have a phone call with someone in another place, or trying to have a video call with someone in another place.
You have to go find a quiet spot to have that, and all the conference rooms are taken because you have an open office, there's a lot of problems that are caused by the overlap of two cultures, the distributed and the co-located.
Edith: I just got back from a big customer trip to Australia and Europe, and amongst the funniest things was sometimes they'd have these elaborate video conference setups and they would have a big sign like "This is not a whiteboard."
Because apparently somebody had tried to write on it.
Paul: Every sign has a story.
Edith: Yeah. "This is not a whiteboard. Do not use white--" Like, it's this very special thing for video conferencing and you have to use a different pen.
Paul: Right. Does it work?
Edith: I was terrified. I'm not going to touch that thing. I'm like, "I'm a vendor, you don't cause damage to a $5,000 piece of A/V equipment."
Paul: So, do you think you're going to go distributed?
Edith: We already are. Not distributed, we still have our headquarters in Oakland but we're hiring more salespeople in the East Coast to get more customers there.
We're going to put more people in different regions because we have customers there. I still think there's a lot of value in having a nexus.
Paul: If you're going for a distributed team, then a nexus works against that.
Edith: I don't think we're going for a distributed team, we're just trying to go for a "We have people in different parts of the country."
Paul: That's a distributed team, and a distributed team needs all the accommodations of a distributed team.
When it's easy for someone in the office to do a thing that it's not easy for someone remote to do, that starts to breed problems.
Edith: It would drive me nuts, I was working at a San Francisco company that got bought by a company in another region and we would do calls and we couldn't understand them.
Paul: Were they foreign?
Edith: No, they were down in Texas or Seattle and the equipment in our office was not that good, and the equipment in their office was not that good.
We were like, "We can't hear you. We literally can't hear you." And they would say, "We don't care."
Paul: I think I found a bigger problem.
Edith: Yeah, and they're like "You're the satellite office. That's not important for you to hear."
And we're like, "OK. I guess you're just paying us all to sit here and not understand what you want us to do."
We weren't trying to be cheeky, we were just like "We literally cannot understand what is happening on this call."
Paul, any final thoughts?
Paul: I think the major thing is a distributed team has to be designed for being distributed, and making the transition from one to the other.
It needs a well-planned and properly executed plan for making distributed people part of your team.