June 14, 2016
Beta Testing for Early Developer Teams
UX designer and former Treasure Data product manager Luca Candela discusses defining your feature tests, cohorts and research experiments. H...
The title of this talk is based on a book I read about parenting. It's a really great book that covers how to talk to other human beings very effectively. It kind of shaped the way I look at what I'm doing.
So if you know who I am, I'm just going to cover a little bit of history. I have this blog, Coding Horror, which I started in 2004, which is 11 years ago. And a lot of people who know of me know of me through my blog.
My blog became this weird ball of energy where a lot of people were looking at it, like 20, 40, 50,000 people a day. To the point that my day job, which was at Vertigo Software in Point Richmond, started to seem kind of quaint. I could go into work, I could have a little bit of effect on the world, or I could write a blog entry and have this big, disproportionate effect on the world.
I was trying to figure out what I wanted to do with this ball of energy, when you have all these people paying attention to you. That led to Stack Overflow. I contacted a bunch of people that I respected online, one of them was Joel Spolsky. Joel said, "Hey, I got this great idea. There's this site, Experts Exchange. I like what it does, but it does it in the worst possible way, and I think we can do better."
I said, "That's a brilliant idea, and it's exactly the kind of idea I was looking for that I could point my community towards and that I could work on." For example, the name Stack Overflow comes from a survey that I put on my blog, "What should we name this thing?" Stack Overflow was by far the most popular choice.
In the process of developing Stack Overflow, that was about 2008, and, it's funny because also in 2008, we went to CUSEC which is a Canadian developer student conference at a college in Montreal. First of all, it was really cold. We went to Montreal in February, which I think was very cruel to do to me.
This was one of the slides I presented at that talk, and one of the things I said was, "Look, when you're young and in your 20s, you need to take these big risks because this is the only point in your life when it really makes sense to take those risks, because once this happens to you, you're not going to be taking so many risks. You're just not going to do it. Maybe you will, but there's a time to experiment in your life."
Paul Graham was always talking about this, and I think Paul Graham was really right. "You're young, take crazy risks. Do crazy, sometimes irrational things and see what happens." So I thought that was great advice. And then as the network progressed, in 2008 we launched Stack Overflow.
I started to think of it as, because I'm a geek, this is like the Star Wars trilogy where we have Stack Overflow, but we also want to launch a companion site for sys admins, which was Server Fault. And then a site for just computer enthusiasts that we called Super User.
So we were exercising the engine and saying, "Hey, this engine works so well for Q&A, for software developers, we think it's going to work for other topics. I'm not exactly sure what topics there are, but there's some very low-hanging fruit in the geek community around system administration, which isn't technically programming. It's really close right? Hey, I love computers, and I'm not a programmer, but I can write some scripts." So those are the three sites that we launched, and we call that the Stack Overflow trilogy. We also have our own Christmas special, which is meta.
I had a weird phobia when we started Stack Overflow. I didn't really want to talk about Stack Overflow on Stack Overflow. I had a very "Fight Club"-like attitude towards this. "Look, come to Stack Overflow, and let's talk about programming and how we solve each other's programming problems and how we learn from each other as programmers. But don't talk to me about 'why does the voting system work this way? How can we do X or Y?' or about how the community works.
I realized in retrospect this was a huge mistake to the point that the community said, "Hey, we need this so bad, we're going to launch a phpBB forum about being on Stack Overflow." At that point I was like, "Okay, that's like the 'nuclear' option. You have forced my hand. I'm not going to be responsible for more phpBB being in the world, that's just irresponsible. So we launched Meta Stack Overflow.
We internalized this so much that every site we launched after that had a meta, which was the place where governance forms, where people who want to talk about the community can go, because this is your leadership. These are some of the most important people on your site.
This is really a key finding that drives a lot of things that I do downstream.
So after that, in July 2010, Joel had sort of experimented with letting people pay to launch Stack Exchange sites. And this basically didn't work at all, which didn't surprise me but surprised Joel. We retooled it to more of a community tool. We said, "Hey you, the community, tell us what sites you want on our engine. You vote for them, and you sort of put your reputation weight behind those sites and commit to those sites."
Say, if there was a site about flying, you know, being a pilot, I would go there as a Server Fault user, I would go there as a Stack Overflow user, and then we know the site's going to work, because they know the engine, they understand our very strict rule set and they were willing to come to the site. So that worked pretty well.
There's like, I want to say 150 Stack Exchange sites? But there aren't thousands. Joel was always like, "There could be 10,000 Stack Exchanges." And I was like, "No, I don't think there will be." I was thinking more like 10. So the answer was really closer to what I thought it would be. It's not 10,000, it's not 100,000, it's more like 150, which is still a very very good number. And those sites are very good at what they do.
Now, the idea is you're playing this "six degrees of Kevin Bacon" game. The people who are programmers are also pilots. The people who are pilots are also cooks. So you're sort of walking that net of relationships of people who are using your Q&A engine and like it and see that it works really really well for reasons that I'll get into, and want it on other subjects.
The other thing we spend a lot of time doing at Stack is explaining why things are working the way they're working. It's a great engine, but it's a very strict engine. It's very strict in what it'll accept. It's very strict in what it'll ask you to do. And the community over time trends towards strictness because of the reputation system. It's kind of a side effect of reputation systems.
People start to get pretty strict about who has stuff because they want the value of their stuff to increase, and in this case, the stuff is reputation. And that's actually okay, it's fairly safe, it's manageable. It's better than the alternative, which is, post whatever you want and see what happens. We know where that road goes, and it's not to a good place.
So after I left Stack Exchange they built this really beautiful tutorial that explains, "Hey, this is how it works. Here's what we do. If this is your first time visiting, check it out." It's like a little tour. It's interactive. It animates. It's really cool. The key bit is that "This is not a place we come to discuss stuff, okay? This is a place we come to find answers to questions, okay?"
There's a difference between these two things, and this is what we ended up fighting and teaching, well, fighting is a strong word, but teaching people about a lot. And this quote is massively overused. This is the guy who gave us iTunes, first of all, so if you're saying no to everything, you should have said no to iTunes 20 years ago.
But he's right, it's really really hard.
Saying no is the most important job that you have.
It's not a fun job to tell people, "Hey, that fun thing you want to do? You need to stop that, like, now, or else I'm going to kick you out." That's nobody's idea of a good time, unless you're pretty antisocial.
I think that was really a key element in what we were doing. Can you say no to your users in a nice, educational way? This is why we do it the way we do it, because it produces these amazing result sets. That's why people want to come to these pages, that is why you're even here, "you" being the person on the site.
So the first thing you do is come to our site and tell us, "Hey, I would like to screw all this up." That doesn't go over well for obvious reasons. They have to understand why, but they don't know why. So you have to educate them.
In 2012 we're pretty deep into Stack Overflow, we're pretty deep into getting everything working, we have all the network sites up, the engine is really coming along. We're on version 3.0 or whatever, and then this happens to me.
You need to be very careful when this happens to you. This is not an easy thing to deal with. One baby is a lot, for the record. Two babies at the same time, if you know how parallel programming works and how much more dangerous it is, this is that, okay? They're doing great. That's June and Maisie. They're super fun.
We knew at Stack Exchange that we were launching a lot of sites. The community was telling us, "Hey, we want this site. We want this site." And if enough people signed up and said, "Hey I would support that," we would launch the site.
There would be a beta period that was a very controlled. It was almost like a process of introducing a bill on Capitol Hill. It's very bureaucratic, but intentionally so, because we, at the end of this, want to arrive at something that people are actually going to be able to use and go to and work.
The odds of success are starting to be high. But even with all this vetting, we had sites launch, and you can go to Stackexchange.com, go to the site directory and scroll to the bottom. And that is where you will find the sites that are not working. They're nice sites, there's nothing wrong with them, but they're not really working. And you might say, "Well geez, Jeff, why aren't those sites working?"
The answer is because what Stack Exchange does is essentially data, fact, and science-based.
Not all communities are amenable to an approach where everything you talk about is about the data, the facts, and the science.
In fact, a lot of the things they want don't even make sense or would be shut down immediately on Stack Overflow. I mean, if you posted, "Hey look at this cool code I wrote, isn't it awesome?" Gone instantly, right? Because it's not a question.
Maybe you just want to share a story. There was a very popular question on Stack Overflow called, "What's the funniest comment you've seen in source code?" It was very very funny. But it wasn't really something you could learn from. It was just, "Hey, cool story bro," you know? And that's okay, but that's just not what the site is about. So the strictness is what made it work, and the strictness also evolved over time, because we didn't know.
When we launched Stack Overflow, we didn't know what was working, what wasn't working. But the community showed us. I was just sitting in there with my ear to the wall going, "What is the community doing?" Observing them, watching them. Seeing what's working, seeing what's not working, and letting them tell me how to build the site. These are all the things that no longer fit on the Stack Exchange engine, and never really did.
This led to another problem. I left Stack Overflow and Stack Exchange in 2012 because I had the twins and other life events, and it was a much bigger company at that point. Plus we had really solved our fundamental problem.
Does anyone get Experts Exchange results in search anymore?That's victory, right? You build a better path and people forget the other path even existed. That's the way it's supposed to work, so that's great.
I'm not at Stack Overflow, I'm not at Stack Exchange, but you know my company was successful, so people come to me for advice. I'm not a good person to ask for advice, for the record, which has complications on the parenting side.
The first thing I would say to them is, "Well, you know, I probably don't use your site. Is it something I naturally am logged into? Why does my advice matter so much? Why am I an expert? What makes me an expert? What makes me the person that's going to tell you the right thing to do here? I'm just a voice like many other voices."
I would turn around and say to them, "Look, you need to be asking your community this stuff. You shouldn't be talking to me. You should be talking to your users, your customers, your fans, your audience. Whatever you call it, you need to talk to them and see what they think of what you're doing. Because they're the people that use it every day. I don't use your stuff every day. Your investors probably don't use your stuff every day. But your users really do, and your customers."
If you don't have a community around your product, or not one that you can point to or touch or see or feel or get to in any way, you need to go fix that. The first thing you might find is that, "Well, geez, nobody actually cares about our product." That's your problem! You need to fix that first.
Before you start asking anyone for advice, figure out why nobody cares enough to have a community around your product.
The smarter companies I would talk to would say, "Okay, that's good advice, Jeff. You're right, we should talk to our community. How do we build a community? How do we put in tools so we can talk to our community?" So I started looking at all the tools that were available for talking to your community, traditional forums like Lithium, phpBB, Vanilla to some extent, Get Satisfaction or customer support tools.
That's not really what I'm talking about, I mean it is a little bit. You know, issue trackers, bug trackers. None of this really fit. And I was thinking, "Well, this doesn't work. There's nothing here I would install if I had a company and I wanted to do this." I was receiving the same advice that I was giving this person. I would be embarrassed to install this software for my users. It would not be something I would want to use.
So I started to think, "Maybe I'll change sides in this war on discussion." Maybe I need to be on the other side of that war and see what it's like. What we need is a tool that lets us reach all these key people that we think want or need our product.
You know, the audience, the fans, the customers, the users, the friends, the creators, whoever it is. We need to be reaching these people and talking to them, having them talk to each other, even more importantly, right? Not just talk to us, but talk to each other. And this is what led to the creation of Discourse.
Discourse was me saying, "Look, I built this highly specialized engine, Stack Overflow and Stack Exchange. It works on a very narrow set of topics and works in very specific conditions. But I want something that works everywhere." I was thinking, "Okay, going with my geek trilogy analogy, rather than the X-Wing, which is a superiority fighter, this is a TIE Fighter. It's essentially disposable. There's thousands of these things, right? Some of them will succeed, some of them will fail. It's just the nature of the beast. And they're also cheap and easy to deploy." None of that was true of Stack. It look a lot of discipline to get a Stack up, and it wasn't an open-source tool chain. So this kind of changed all that.
We launched a Discourse for Imgur recently. I have a relationship with Imgur because we were one of the first customers, enterprise customers, for Imgur with Stack. And I love the company. I love what they're doing, I view them as a next-generation Reddit. Basically, it's like what Reddit eventually will be, is Imgur.
When I was talking to them about like, "Hey Jeff, what should we do?" "I'm just one guy giving people advice, right?" I said the same thing I always say, "Well, you have an amazing community! You have an incredible community at Imgur, right? You just need to unleash them."
And we did, we set this up and they went crazy for this thing. It clearly met some really unmet need in the community of "Wow, now we can talk to the developers about the product. Now we can talk to each other about Imgur, how is it working, what are the problems we're seeing." All this governmental stuff that I wanted them to do about organization within the community they started doing.
You can go to Imgur right now and look at the categories and the product feedback categories are incredible. They're filled with amazing product feedback. Way better than I could provide. Plus, if I'm going to provide product feedback to Imgur, I want to ask the community, because I'm a very infrequent user of Imgur, so I would just post on the site and say, "Hey look, I think the upload dialogue is really weird in the way it works." It still actually is on Imgur and I'm proposing a restructuring of it.
I want to propose it to the community first and see what they think about it. And then I would take it to Allen at Imgur, and see what he thinks. This is how I think these things should work because everything I've done in my career up to this point for the last 11 years has been me listening to what the community is saying.
Now, you don't do everything the community says, because that would be insane. But filtering out what they're saying into like, "Wow, 10% of these ideas are amazing and very, very good ideas," is hugely powerful. And that's the basis of Discourse, that's what I hope people get out of Discourse.
Just today I was browsing as I was building the presentation. This is just another topic that somebody started. It has an animated GIF actually of what he's talking about within the product. And this stuff to me is gold.
You know 90% of everything is crap, that's true of everything in life, including feedback.
But your most avid users are here. They know if you're watching, and they know if you're participating. It's a very, very strong signal.
Now, the difficulty here, and we've actually had discussions with Imgur about this specifically, is the community was a little too popular in the sense that they used it as a social tool to just engage and hang out, which isn't wrong, but it's not on the core Imgur product. That was kind of unintentional. The way this is supposed to work is communities like this are supposed to be a little bit fun for people to go to.
It's not all business like Stack Overflow. That's why Stack Overflow isn't as fun as it could be, because it's all about learning. It's not about having a great time, it's about coming in and sitting down in the classroom and listening to the teacher a little bit, then responding in some reasonable answer order. This is more like a dinner party where people just come and they start talking, and some of the talk will be very useful. But a smaller percentage.
The way I like to think about this in my head is like search results. Not SEO or any of that stuff, but when you do searches, what sites do you end up on that helped you or were useful or solved your problem, whatever it was that you're having, the thing that you're looking for?
Chat logs are absolutely the lowest of the list. It's really rare. It happens maybe once every several months that I end up on a chat log as a search result. So that tells you something about chatting. I love chat as an entity. We use it at Discourse, obviously. But in terms of generating artifacts that are useful to people, chat is not good.
Now, forums are still very common hits for me, that's one of the reasons I took on the Discourse project. I felt like, "There's a lot of these things out here, and they help me a lot." In fact, one time I was headed to an investors meeting. I don't have that many investors meetings. I had to get gas in my car. It's a car I don't use very much, we have two cars. It's a VW Sportwagen, and I couldn't get the gas cap open. I was like, "How dumb am I? I can't operate a motor vehicle?"
So I'm stuck. I'm like a little bit late to this meeting, and you don't want to be late to VC meetings. It's really bad, for the record. And so I was panicking. I was like, "What do I do? I know this should open. I've got to be doing something really, just obviously wrong here."
I searched Google and I got a result that was a forum that said, "Hey, the gas cap tends to stick on this model, and you just kind of have to pull it." I was afraid it was going to damage the vehicle, so I actually got a result on my way to my investors talking about the problem that I was trying to solve.
This is why I'm doing what I'm doing. This stuff actually works, right? Of course blogs are great for finding answers, that's why I started my blog. Seeing people writing a bunch of stuff that was useful to me, I was like, "Wow, this isn't just my diary. This is useful information that's being put out into the world" moving up to Facebook, Twitter, and then of course Stack Overflow being sort of at the top of this list we optimize for efficiency.
We will tell you to go away if your problem only helps you. We're not interested in helping you.
That sounds cruel, but you gotta think about the goal. The goal is to help as many programmers as possible with the problem. So you need to convince us that your problem matters. You need to talk us into this problem being an interesting problem that would help not just you, but other programmers that would come behind you. None of that really matters for discussion. That's just me breaking it down.
Stack is kind of like a scalpel. It's a really great tool, but it's kind of dangerous. When people asked me for it, I was like, "Eh, I don't want to give you the scalpel, because you're going to hurt yourself, and you're going to hurt your users with this tool because they're going to be angry that you shut down their questions. But that's how it's supposed to work, right?
If you look at Yahoo Answers, if you look at the other historically embarrassing entries in this category, and sometimes even Quora is like this now, they don't have enough rule sets.
They're not strict enough in what they will accept. And this dilutes the power of what they're trying to do.
Stack is a scalpel. It's not open source, it's very narrow and does a very specific thing very, very well. But it's dangerous.
Discourse is a table knife. You sit down to dinner, you have a table knife, there's probably several restaurants on the street. Whereas Stack is a school. How many schools are on the street? There's a big disparity in what we're trying to do. It's like I want to write something that's going to reach as many people as possible because the tool is a general-purpose tool that works for everybody, and it's not going to hurt you.
If you deploy Discourse and it doesn't work, nobody's going to be angry with you. It just won't work. Whereas if you deploy Stack and it doesn't work, you're going to have angry people sort of beating down your doors about why their stuff isn't good enough to be on your site.
So where we said at Stack, you know, is this useful? I know it's a totally different set of criteria. Unless it's illegal it's basically allowed, right? That's the set of guidelines plus whatever guidelines you add on top of it. Stack is a system of "no." It's me telling you, or the system telling you, "No, this isn't good enough. Come back, we have standards here."
Just like a community college might have a standard, like the University of Arizona might have standards. Like your local high school might have standards. Because, again, the analogy is school. We have standards about behavior here because we're trying to be a school. We're not a restaurant where, as long as you're not killing anyone, you're allowed to be here. Our standards are higher.
But Discourse is nice for me because it's a system of "yes." I can say, "Yes, take it, it's open source. It doesn't cost you anything. Go install it tomorrow and have a great time. And your users are probably going to enjoy it because it enables them to have nice interactions with each other that are not gated by anything other than what they want to do.
Now this is a little bit like turning to the "dark side," where I spent all this time working for this strict system and now let it all hang out. So it's kind of liberating on that level.
We call Discourse a system of rainbows because it's trying to be all things to all people, and it's also like a generic building block of community.
When you think of any type of group online and them trying to organize and talk to each other, this unit of work of the paragraph, not necessarily chat, but the unit of work of the paragraph, is a really strong one. It's usually captured in email, which is a silo, which is bad, or mailing lists. And to some extent Google Groups and things like that.
We're trying to say, "Look, no, we want to be sort of like the Wordpress of forums where it's a great ecosystem, it's a community ecosystem, and it sets your community free. It sets your data free. It doesn't cost you anything. We don't take anything from you. It all belongs to you. And there is some subtlety here, because these systems of opinion are very different from the systems of data, fact, and science that we saw on Stack.
So if someone said, "Who's your favorite X-Man?" There's no right or wrong answer. There's no reason to downvote opinions, because they're fundamentally opinions. They can't really be wrong, you can just disagree with them.
So you'll see on Discourse we don't have the system of voting things up and down. We immediately removed that because we can't have that on systems of opinion. But there is a trust system. So even though we don't have a visible reputation score like on Stack, where everywhere you go there's a reputation number next to someone's name, we did away with that because we didn't want people to judge the discussion based on the perceived reputation of the participants.
We want it to be about the content. And that's true of Stack as well, but the reputation is supposed to drive certain really strict behaviors. And we didn't want strictness, we want a looseness here. We don't want people arguing about what's allowed, what's not allowed. We want them to just have a good time and have a good experience on the site.
Now, this concept of immune system I think is very important. One of the things we built at Stack was a system that was sort of self-regulating in the governmental sense. In terms of they actually elect moderators there, which is super rare on the Internet. I actually tried to do prior art on that, and couldn't find anything. Nobody else was actually electing moderators like we do systematically at Stack. And the idea that you need a discussion site to protect itself.
You might say, "Well, from what? What are we protecting ourselves from?" First is cruelty. The people that are mean to other people. That has to be pushed out. You have to have tools to get rid of the meanest people in your community, or reform them, whichever it takes, right? If you can't push out the meanness, if you can't push out the spammers, there's a lot of places overrun with spam and people you can't trust, then you don't have a community. It's not going to survive.
One of the interesting things we found on Discourse that I didn't appreciate early on was that how much you're willing to read is a huge indicator of how engaged you are in that community. Not necessarily how much you talk, but how much you listen.
If you know conversational theory, or if you have conversations with human beings, you know this is true, you have to listen at least as much as you talk, right? Ideally a lot more listening than talking. So we had to build systems that let us incentivize listening. Just coming every day, reading, not necessarily posting a lot.
A lot of the traditional forums got this so wrong that next to the name they would say, "This is your post count, this is how many times you've posted." So that's immediately the thing that's incentivized in all these systems, and it's the wrong thing. You do not want to incentivize talking. You want to incentivize listening. That's way more important.
For example, "trust level 1" on Discourse. If you ever go to Discourse, we put you in the sandbox when we start, because we're like, "Whoa, whoa, whoa, somebody just showed up on our front porch. We don't know this person. You can post, but there's a lot of limits around that sandbox. If you want to get out of the sandbox, these are technically the rules.
You'll see that the rules are really about, "Hey, did you actually even look around here? I mean, just a little? I mean, five topics, 50 posts, 10 minutes?" This is not a big ask. First of all, this gets out 99.999 repeating percent of spammers, gone based on this rule.
But you'd be surprised how many people show up at your community and it's like, "I'm just going to post this and I'm not even going to make any attempt to read what it is you're actually doing here." That always surprises me, right? Your best users are like, "Hey, let me look around here and see what's going on."
It's like being a tourist. You look around, you see how they're doing things there, and then you participate in kind. There's no way you can do that by showing up on day zero. minute zero, and start a post. You have no sense of what that community is, what they do, and what they're about. So it's a really really powerful reason to incentivize this stuff.
So the other thing, and again this is the iceberg, the first time you post we actually pop up and try to educate you about this concept of not just civility, that's a baseline right, be civil. But actually to be kind is a level on top of that, right?
Civility is because you believe that vaccines are dangerous, I'll be civil to you, but I'm not going to be nice to you. Because I think that's actually a really dangerous belief. But I will be civil.
Now, kindness takes that a step further, and says, "Hey, I'm going to empathize with this person, whatever problem they're having and the reason they would believe that. I'm going to try to walk a mile in their footsteps."
This is a tricky thing to encourage. So one thing we do, there's a whole host of systems in Discourse around this, is when a person shows up for the first time and starts typing, I am a big believer in what I call "just in time" behavior reinforcement. Because at that point I figure for some dudes, somebody's wrong on the Internet, right?
The reason I'm typing is because somebody's wrong. I mean, I've done this. I know everybody has done this. I want it to come up at the time, it's like, "Hey, I know this dude is totally wrong and you're going to show him what's what. But while you're doing that, remember that this is actually a human being. Whatever crazy belief they may have or whatever thing they're wrong about, there's still a person on the other end of that ethernet cable somewhere. And you should really consider that when you're typing."
You just forget. People forget that they're talking to other human beings.
They think that they're talking to some chat bot or something that doesn't even really exist. That one level of indirection. There's all this research around this, if you put one level of indirection around money, people will spend more.
We know this from billions of mobile games. You're not spending money, you're spending "floos." So it's like wow, I'm just spending floos, no big deal! This is a really powerful concept, that one layer of indirection. Either for good or for bad. So we want to bring it back in and say, "Look, this is a person. Bring the level of indirection back and appreciate that this is a human being."
There's some other subtle things that we're trying to fight at Discourse, one is being off-topic. We try to provide tools so people can branch conversations. It's still a tricky thing to do, we're still working on it. But one thing I didn't like about existing discussion systems is we'd get into one discussion, and then all the sudden, it's like a totally different discussion.
You're like, "I don't care about this other discussion. I didn't sign up for this. The title was X, but I got Y. The title was 'funniest dog pictures', and now you're talking about cats. Maybe I don't like cats.
Or what I call "Hacker News syndrome," where you go into Hacker News like "Wow, interesting article," and the first entire conversation thread is just like, "I don't care!" You're on some crazy tangent, and there's hundreds of people on this crazy tangent, and even worse, they don't let you collapse it. Which Reddit does. They really really need it, because I'm like, "Man, I'm just like scroll, scroll, scroll, it's all these things I don't care about. It's like, shut up already! Give me a different tangent, this is a bad tangent.
So that's the purpose of that system in Discourse, is to sort of let people branch conversations. And they also bi-directionally link to each other. It's like these things reference these things. That's something we learned from Stack later, was how important these bi-directional links were. Kind of obvious in retrospect.
Of course spam, easy tools. What I call putting a trash can on every street corner. There's a couple different manifestations of this to basically make it easy for people to do the right thing if they see bad activity, if they see a spammer, if they see something that's really just sort of over the line. It's one or two clicks to do it. And it's a really easy, transparent system.
It also does what I refer to as the "just in time training" thing of, "Oh, you clicked flag. Now let me teach you about flagging, because you indicate an interest in flagging. I know you're not going to read the FAQ, nobody does this.
You're going to go in and start playing the game. And while you play the game, we're going to teach you about the game at the time that we think you're ready for it.
So the information is in front of you.
"Thanks for helping to keep our community civil!" That's a suggestion from the community. I had something else like "Why are you flagging this?" and people said, "Whoa, whoa, whoa! "You can't say 'why are you flagging this?'"
I never thought of that! I never thought of how intimidating that would sound, right? "Why are you flagging this?" It's like, no, no, no, thanks for helping us to keep this community civil. I was like, that's awesome. And that came directly from the community. That wasn't me, that was me listening to what the community was telling me to build.
So you see how it sort of teaches you what to do, and also it indirectly teaches you these are the problems that you should be looking out for. If it's not on this list, you can probably ignore it. But those are the big ones, right? Now the other thing that's a little bit subtle about discussion is that it actually is supposed to be fun. You actively cultivate fun.
This is why companies that come to me, "I want a customer support forum!" I'm like, "I don't think you do, and I'll show you why. This is why you don't want a customer support forum." Look at my mole. Right, is that a fun party? No!
I don't think anyone would say that's a really fun party for a doctor. Doctors have doctor-y things they want to talk about, they want to talk about things that aren't doctor-y, they want to talk about, I mean, who knows. They might get into some medical stuff, but it has to be natural and organic, and it has to be fun for the doctors.
It can't be a bunch of people coming in and sort of begging for support, and you know, the blind-leading-the-blind people who know just enough to be dangerous telling people who don't know enough to be dangerous the wrong thing. So this is a tricky thing to do.
You really do want to optimize for fun, and you need your employees and your company to be in there alongside them, having fun in the space, and talking about the product.
If you go to Meta Discourse, I've internalized this so much, because it's what I do. I went to Meta Stack Overflow every day for four years. And I was there alongside the users, talking about the product, talking about how it works, engaging them. And that's what I do in Discourse. So my view is very, very biased on this.
It's like if you have a company, if you have an organization, you have an audience, you should have a tool where you talk to your audience. And you should be in there, or representatives, official representatives, talking to the people that use your product on pretty much a daily basis.
So that's sort of my pitch. And I'm a little bit of a one-note person on that because I think it's the only way to build a company. Or maybe it's the only type of companies that I like to build. It's like the community powers what we're doing. And one thing I like to say is that WordPress is our spirit animal.
When I installed WordPress for the first time, this was, gosh, 2007-2008 maybe? I thought it was terrible. I was like, WordPress is not good. It's taken over my CPU, it's kind of confusing. I just didn't like it, and I wrote a whole blog entry about it. Matt Mullenweg emailed me, and he was like, "Whoa, whoa, whoa, it's actually pretty good." And he sort of walked me through it.
But one thing I noticed after that, and I wasn't necessarily convinced by the way, just because Matt Mullenweg emailed me, I thought it was nice. What did convince me though was every year WordPress got so much better because there was such a vibrant community around the product. And the key insight I had then, it's the same thing you have with Firefox and stuff, the community makes the product.
The community is the reason you want these products.
For the same reason that when I go to Amazon, the first thing I do is look at the customer reviews of the product. That to me is the most important thing on that page. Maybe specs, maybe how much it costs, but then, like, is this terrible? You know, I don't know! I need other people to tell me if this is actually going to be a good product or not.
Sometimes they're reliable, sometimes they're not, that's the game we play with this stuff. But these are the most important elements on the page to me. And the community around Amazon is what makes it so powerful just like WordPress. So that's what we're trying to do with Discourse, is have this really powerful community ecosystem with a tool that lets you build something like that for yourselves if you want it.
My job has been to sometimes teach companies about the value of this. "Why would I want to talk to my community?" Which to me is like asking, "Why do I want to be a human being?" I don't even understand the question. But it can be hard. Sometimes communities can be difficult to engage. They can be challenging communities. Maybe they're hostile communities, I get that.
So as we've been working through this, all these companies provide us feedback to Discourse, and it's open source. Everybody benefits from all the changes we're making as we build up our customer portfolio, as we engage the community. It becomes a better and better tool and meeting the needs of more and more companies that we're working with. So I love this.
I love what I call "going on safari" with my users. That's basically what I do every day. I play interactive, multi-player games where you type paragraphs to each other, occasionally a picture.
These are the games that I play. So I'll join gaming sessions on each one of these sites, and say, "Hey, how do I feel about this community? Am I happy or sad that this page is on the Internet?" To me, that's the way that I look at whether we're succeeding or not at Discourse. Are we making the Internet better or worse? And, that's all I have.
Well, it's sort of like the way you get to know people in life, like you basically start to recognize them. Like we're big on avatars. We actually have a little job, one of the just in time reminders is, if you post enough and you don't have an avatar, we'll say, "Hey, you've posted a lot, that's awesome! I want to get to know you. Can you please add an avatar, and we'll link directly to the profile page.
Right, that makes sense, because why would you even bother with an avatar? Say you drop out on meta Discourse, you're like, "Hey Jeff, your product sucks." And then you don't need an avatar at that point, because you're minimally engaged. But once you're engaged more, I want to get to know you.
And when I see avatars on Discourse, this doesn't necessarily scale obviously, I'm like hey, awesome, I'm excited to click on that topic because I know the person who posted that is really good in terms of their feedback that they give me is very very good and very actionable. And sometimes not so much. It's like okay, that wacky person. It does happen.
So I think you get to know them the natural way, by interacting with them, you know? I mean that's sort of the goal. Now, granted that doesn't always scale, if you had thousands of people sending you feedback, it becomes more challenging to funnel that all in. And you'd need like a team of people, or some other way of sort of triaging feedback. But getting to know your customers I think is really powerful.
Well I think a lot of human interaction is basically story telling, right? And stories have a sequence. They start off at the beginning, stuff happens, some exciting thing happens, and then there's sort of termination, right, there's an ending to the story.
So I think storytelling is really what we're optimizing for. And that is a really linear process. I mean, you could flip it upside-down, I'm not super, necessarily opposed to that. The threading view I am a little opposed to, because Discourse is sort of what I call a hybrid-threaded system, meaning, it's kind of like Twitter in that if you drill into Twitter you can sort of see the context of the little conversation that went on.
But it falls apart pretty fast, right? And that's kind of intentional. I mean if you're having long conversations on Twitter, I can tell you pretty much unequivocally you're doing it wrong. Like you shouldn't do that. It's just impossible to follow for everybody in the world.
But I do want people to think when they write something, to provide enough context that if I looked at your post, I could understand what was going on, because you included enough context, whether quoting or just explaining the situation, so that I don't have to read 20 things upstream to understand what you're trying to get at.
So I'm also trying to teach people about writing, basically. It's like, carry the context along, just the right amount of context, And that's kind of a burden that's supposed to be on the writer.
We have this attitude of write once, read many.
And it's true! You know, the writer has a responsibility to the reader, to write something that's understandable. And that will have some lifespan beyond, I wrote this, you know? Whereas if you're in chat, it's a totally different experience. Chat, you know, I'm typing disconnected sentences. But here you should be typing paragraphs.
So we want to kind of encourage some of that stuff. So those decisions are kind of intentional. I think there are ways to build hybrid-threaded systems that are not as bad, there's just trade-offs on both sides. I think if you look at classic forum software, which is where we come from, we were trying to avoid building Google Wave for the record. Because then you look at it and you're like, "I don't even understand what this is."
If you look at a lot of traditional forums, they're very flat. So it's part of our DNA to say, look, we're taking this flat model, we're adding a little bit of threading hybridization, you can click and see what this was in response to, you can see the downstream replies, you can sort of expand it like an accordion, right? But beyond that it was kind of a conscious decision. I don't think we'll ever add a true threading system.
So the question was about the difficulty of necessarily finding things in chat systems. If you do a Google search, you're almost never going to end up on a chat transcript. That's just the way it is. And you can see why, even though Slack is very good, we use Slack, I love Slack, it's a great great product, but I have the same searching challenges that you do.
What I've found works for me on Slack specifically is it's all about discipline on the entry side, and chat is all about no discipline. So these are fundamentally incompatible concepts. I'm going to type some words that came into my brain. Right like, in any random order. Not even going to fix the misspellings. This is not going to end well for searching later.
So what I do is I create a file snippet sometimes. Files I think are good artifacts, at least you can search for the file. But you're right, it's like you want the file, and you want the context that went around the file, and Slack is still pretty bad at this. Despite all the funding, I'm sure they'll get there. But part of it is built into the system, you're right.
You know because there's no obligation, you can just type three words, I could type 10 messages that are just single words, and it's like how would you even search for that? They're 10 disconnected words, it's not a sentence, right? Whereas what I think is a huge power of forums, the reason they've been around so long and work so well is we want to talk to each other and tell stories and to do that you have to, the paragraph is the fundamental unit of work. And that's a disparity in those systems.
When a sentence or words are the fundamental unit of work, you're just not going to get as good of results as when the paragraph, or the sentence at least, is the unit of work. So I think they'll get better but I don't think they can ever cross that chasm.
So the question was how do you, if you have a large user base, how do you get them involved in building a community, and the short answer is, this is like government, man.
You give them power. You give them the right to vote. You give them the right to speak. You give them the right to make laws.
Then they're part of the government, it's a democracy to some extent, right? And Discourse is built around this premise of yes, we come with a lot of out-of-box defaults that help you get it right. But the idea is, there's a category called meta. Out of the box in every Discourse instance, it tries to explain, here, this is meta, talk about the community.
And you know, in advance of the change you would start beating the drums like, "There will be change, we're changing to this new system." because change is super hard. I want to be clear, if you have a community and they're using an existing tool, it's what I call Atwood's Second Law.
It's like the difficulty of change is like the square of the community size, times the number of years they've been using that product. And this is regardless of how good or bad the tool may be. It could be the worst tool in the world, there would be so much friction.
So you need to involve them early on and say, "Hey there might be a change, "what do you think of this change?" Here's where we're going." And then, say you switch to Discourse, it doesn't even matter, if you switch to some other software, it's about giving them the power to set the rules, determine what the rules are, determine what they like, what they don't like, and build out the community alongside you.
And that works for us at the product level, obviously. But just in terms of like, scoping the community, like this idea that they're doing this with you. It's not like, "Here, my subjects, "you shall have what I bequeath upon you." It's more like, "Let's do this, it's for you." This is a house we're building for you guys, "where do you want the stairs?" "Where do you want the door?" And some of that could be just the color of the walls, could bike shedding stuff, but that's fine, right?
Choosing the color of the walls in your house is an important decision. So I think that's the way to think about it, and that's the way to structure it.
I think all the traditional methods, you know if you have a blog, blog it. I mean we saw this with Imgur, we see this with a number of our partners, if you have a newsletter mention it in your newsletter, link to it from your site.
Another thing I think is very strong is having essentially pseudo, mini-celebrities on your site.
Think about it, to your community, you guys and gals are celebrities.
You're the people who created the product that they're interacting with. That matters, when I talk to people and say, "Hey, I'm the founder of Discourse, and I'm going to talk to you about the product." They're like, "Holy crap, that's great!" So I think that one-to-one interaction with people. It's like you're talking to the people that built the thing. We're making time to talk to you because you matter, because we want our product to matter, and without customers it doesn't matter.
So I think there's ways to build trust around you participating and being an example of what you would want to be there. Think about what the ideal user would do when they came to your discussion site, your ideal customer in general actually, and be that, do that. Show examples of that, create a path for them to follow.
You think about it in terms of where your house is. If you're in San Francisco hopefully you own a house, or you have a place to live. But how much time do you actually spend in the place that you live, right? You spend a lot of time traveling, going to work, doing stuff.
So your home, it's your home, it's on your domain. It doesn't belong to Zuckerberg, it doesn't belong to Ev Williams, it doesn't belong to Stack Overflow, even though that's creative commons, that technically belongs to everyone. So you want to cultivate your garden the most.
But you still want to be doing stuff on Stack Overflow, stuff on Twitter, stuff on Facebook, this is just how stuff gets done in the world. These are just, you want to be where everyone else is, to some extent. But I think you ultimately want to tend to your garden, and one thing I like about Discourse is we ostensibly sell it, but it's open source, it belongs to the community, it belongs to you guys that are running it, and to the people that show up there every day.
I think that's a powerful message to have, this is our thing, you know?
And although you have the other things, you have Stack Overflow, you have Twitter, you have Facebook, this is our place, man. And this is where we fly this flag. And this is how it's going to be, because we set the rules here.
I think you want to think about it that way, and sort of sell it that way. They're all valuable, and I would still put stuff on Stack Overflow for quick and dirty, I have a problem, I type something in Google, press enter, Stack overflow's still going to be better for that.
We're evolving towards some Q&A-like features, but we will never be Stack Overflow because that's not the point of what we do. But the idea that you have, "Oh this is the correct answer." We will definitely be adding that. But we're never going to do sorting by votes, or anything like that, right?
I want to address that specifically, because you said, is there such a thing as essentially being too open in your conversations with your customers, and sure there's stuff that's hidden. I'm not going to talk about the salaries of people that work at Discourse. I mean there's stuff I won't talk about on our Discourse instance.
I try to keep everything public as much as I can, it's part of my schtick. But you're right, there are limits. I do want to point out, there's this other company, I'm not even going to mention who it is, but they essentially forked Vanilla, which is an old PHP forum, it's still pretty good, one of the better sort of variants. They forked Vanilla, and then what they do is they go on Twitter and anybody who mentions one of their competitor's product, like they go and talk to them.
All this does, is make them look like dicks, right? So like, I don't think you need to worry. All you have to worry about is doing the right thing. Like when you talk to your customers, do other people come into your site, see, wow, they had a good interaction with this customer.
They had a problem, they're like, "Wow we take your problem seriously, "we're going to go and fix it, in fact we're going to fix it so nobody else has this problem again, I'm really sorry this happened." And now the product is better for everybody, and they're like, "Wow, cool. The founders of the company were here, they had a problem, and they fixed it. And it was a really nice interaction."
That is what they're going to see. These people that are theoretically sniping, are really only hurting themselves. People see through that really really quickly, in terms of like good actors versus bad actors.
So the whole reason of putting these public artifacts out there, is to say, "Look, we are good actors. We will engage with you, sometimes we don't get it right. But we fix it, and here's the evidence that I can point to of that."
Like for example, we're working with a customer now that wants Discourse to run in a subfolder. Normally we run in a subdomain which is way easier for hosting. But they're insistent that they want to be in a subfolder.
So I went on Fastly's community, Fastly is the customer. And I said, "You know we don't really know if we're doing this right." So I went on Fastly's site, and I didn't name the customer by name but I said, "Look, I want everybody to understand how to do this, I think this is kind of a weird problem, where you have to route all subfolder traffic to another server, it kind of really complicates hosting."
I wanted advice and I wanted feedback. And even though it makes Discourse look like "Oh, Discourse doesn't know what they're doing", it's like well, this is how I figure stuff out. I put it out there and other people chime in, and hey, now we all learned something.
I think that's what you want too, your customers will see that too. Now, not all customer bases may be interested in talking to you, but that's something you want to figure out. I don't, it's hard to say the nature of your audience. But even if you guys use it internally, like let's just discuss our product, forget anyone else is here, we're going to go in here and discuss direction of our product, I think that's really powerful.
The people who are interested in that would say, "Wow, I can go in here and have an influence about what these guys and gals are doing to their product." Again, very powerful to me. And I think the smarter customers pick up on that.
So the question is about moderation, like if stuff disappears from Stack Overflow, it's community driven, sort of opaque in terms of what's happening. The general built-in philosophy of Discourse is, when you post something unless it's wrong in a very clear way that no one would disagree with, it pretty much can stay.
We do have a clause, if you look at the guidelines, it says, "Moderators may remove any content or any user for any reason at any time." That's just kind of a hedge, because we don't know what your moderation policies are going to be actually. But on most sites that we have, they're very very liberal moderation policies.
Unless there's active harm, and this is the kind of thing you would discuss. It's like, "Hey I want to remove this topic, because I think it's old and people searching find it, and it's just confusing, and we can do better, right?" So you would start a meta topic, discuss it, and then just delete the topic.
It's in your house, it's on your domain, on software that you control as much as we control, because it's open source. So there's no editorial oversight built into Discourse, except for spam and stuff and community flagging.
I should mention community flagging. So one of the immune system things that happens is if three users, not new users but established users, flag the same thing, it is hidden. But this is a process where, when the post is hidden, a private message goes out to the person who wrote that post and says, "Hey, the community had an issue with your post. And if you edit it, it'll come back immediately."
All it takes is a little bit of commitment to have that interaction where the person is listening, and says, "Oh I messed up." And I've actually seen this work, which is amazing, when you flag something, see the person get the PM, and go in and fix their post, is a really really good feeling.
When it's just something's obviously, they just went too far, and they know it. But all this is visible from the outside. It's not super common. We don't find that people in Discourse like to flag all the time. We have to actually encourage people to flag, we had the opposite problem actually.
I don't think you'll find the community deciding they want to kick people off the site, that hasn't happened to us so far. And it's built into Discourse, unless your post is patently offensive, it should generally stay.
The question was about how you go from being a small company, where you have good engagement with the community because you're on the discussion site frequently, you get larger, you get busy. How do you continue to have intensive engagement with your community on the discussion side.
What we see a lot of companies do is, essentially you anoint someone as like community liaison. And that person's job is to be present, be everywhere. If you look at Imgur, Allen's sister, Sarah, who's super nice, essentially has that role.
There are multiple people they're hiring, because they have all this funding. But that's okay, the community knows Sarah and they love her. They have a special name for her, everybody would say "Sarah please", so her name is Sarah Please, Sarah Please. So it's cute, right?
You get this interaction of having someone that's sort of anointed as the public figure. Basically Matt Cutts at Google who's a great guy, has this public visibility, he's like, "I'm the face of this part of Google."
Obviously there's hundreds, maybe thousands of people behind Matt doing all this work, and nobody thinks it's just Matt doing all this work. Nobody has that idea, but you have a public face. And I do think it's good to anoint somebody into that role.
The other thing to think about is, two other things I'll add to that. You do have to still make it your house. Even if it's just a room in your house, like you have a bigger house now, right? You got a lot of rooms in your house.
Make sure you visit there occasionally. Don't let the room go unused.
Have a policy of just check in, get a feeling of how things are going, statistically sample, go into some conversations, reply to some things, just have it naturally be a part of your house. Don't get so isolated that it's like, "Oh it's that building way down the street that we never go to anymore." Don't do that. That's unhealthy.
The other thing is to realize that this is really about the community learning to talk to each other as much as to you. The long-term established communities, you could walk away and they'd be like, "We don't care, this is our place. This is our house, we don't need you to be here. We'd like it if you were here, but we're here to talk to Joe and Sally, cause we like Joe and Sally, and we just come here for Joe and Sally."
That's completely valid, and that's the kind of clubhouse you're trying to build up for people, to eventually become self-sustaining. It's like, "Wow, these people are cool and interesting. This is a great party, I want to come here at least once a week."
Think about ways you can sustain that. And that's more of a long-term goal. But long-term survivability is definitely a goal of Discourse. It's like how can we teach people to do this.
Thank you, thanks for coming.