February 27, 2015
Heavybit’s Wheelhouse Presents: Michael Coté on Continuous Delivery
Pivotal Labs' Michael Coté (formerly 451 Group) offers his "State of the Industry" presentation to Heavybit's Wheelhouse attendees includi...
2019 was an amazing year for the open source community. More and more companies are not only relying on the work of the open source community, but are contributing back both in terms of financial donations as well as engineering time investment. Even Microsoft, which has notoriously been focused on proprietary development for ages, has finally started embracing the paradigm shift, and it looks like the trend is not going to slow down in 2020. That being said, with success and growth come many challenges. Most open source projects rely heavily on volunteers and their time and focus is a rare commodity. Wrangling an army of unorganized developers, many of whom are only in it for self-serving goals, can be a frustrating endeavor which results in burnout and fatigue. As projects get bigger and more popular — the problem gets exponentially harder.
In this article, we’re going to explore some practical solutions you can employ to help keep your open source projects relevant and growing while taking the burden off the core contributors.
Before we dive into some strategies for deploying a successful open source project, you might be wondering if going down this path is right for you. If you are a founder or an early employee of a start-up, you might be curious how open source development fits into your revenue strategy and you’re probably (rightly) skeptical about the time sink that it may create for your engineering team.
Embracing open source is great for several areas of your business:
If, however, you’re hoping to set up an open source product for commercial gain, I would caution you. The prospect of having other people contribute to building your product may sound great, but in reality this is not how open source projects typically succeed. It’s unlikely that your team will move faster as a result of open sourcing a piece of your code — in fact, it will likely move a lot slower as a portion of resources will now go into project management, documentation, reviews and building an ecosystem. Some open source projects have been able to realize great ROI, but it’s just like starting any other business venture — there is a lot of luck involved, and the payoff may not be financial; it may be purely educational.
With that in mind, if you decide to go down this path, let’s take a look at some tips and tricks to make your open source project as effective as possible.
Let’s say you’re a lead engineer at a large tech company and your team is building a new feature for your product. Your design team wants to try some new innovations or push some boundaries of existing technical limitations. You look around the room and unfortunately nobody has built anything quite like that before. You roll up your sleeves and come up with a proof-of-concept that just might do the trick, but there’s a lot of work to do before your tool is ready for production. At this point, you might turn to the open source community — you publish the code on GitHub, you give a few talks at your local city meetups, and it happens to catch the right eyeballs. Several people are intrigued by your ideas and want to contribute some of their time to help you polish the prototype into something that will be reusable. The project takes off!
As an engineer, you now find yourself in a strange position. You’re no longer writing much code. You’re doing a lot of management-y type stuff: communicating with different volunteers, arguing over technical approaches in pull requests, debating technical decisions in issues, writing documentation, creating issue tickets that describe your goals, researching tools you can use for sharing a roadmap, etc… In addition to this, your contributors likely aren’t working on exactly the same product feature that you are, so your project is naturally going to start to get pulled into different directions.
This is where a project management solution can help get you organized and on track. Tools like Trello and Asana are great for that — as is ZenHub, as it allows you to stay within the GitHub ecosystem (where all your volunteers are doing their work). You want to pick something low friction for both your team members as well as consumers of the tool (who are going to be filing bug reports, feature requests and questions).
No matter the tool you choose, let’s explore some simple project management tricks we can use to help you keep your project moving in the right direction.
Your project’s README file should clearly describe what problem you’re trying to solve, and how you’re going to approach it. Kent C. Dodds does a great job of this in all his projects. He usually puts it right at the top of the file. This simple clarifying statement is going to help prevent people from trying to push your project into a direction that is solving a problem you don’t have. In addition to this, providing a list of problems that you project deliberately ignores or will not attempt to solve is also helpful. You want to be wary of project bloat. The more things your tool is going to do — the harder it will be others to adopt it (and it will likely start to suffer from performance issues as well).
Next, include a list of other projects that you found that do something similar but weren’t quite right for your needs. Do the research and scope out your project’s competitive landscape. The amount of stuff you can find on GitHub these days is immense and someone likely already tried to solve something very similar to what you’re doing. Proving this information upfront will help answer many questions that your users are naturally already wondering about as they look to compare your tool against another one they’ve come across. As your project gains traction, the volume of incoming questions will consume you. An initial investment in providing answers to common questions will help reduce that, and keep you focused on what really matters — building something great.
Finally, have a think about the longer-term vision and document this in a roadmap. Finally, have a think about the longer-term vision and document this in a roadmap, either with a dedicated roadmapping feature or a simple document. This will give your audience a clear sense of your direction and it will also help you internally prioritize the work that’s coming in. Before accepting any pull request or code change, should always be asking yourself if that change contributes to something on your roadmap. If it doesn’t, it might not be the right time to bring it in. This will give your audience a clear sense of your direction and it will also help you internally prioritize the work that’s coming in. Before accepting any pull request or code change, should always be asking yourself if that change contributes to something on your roadmap. If it doesn’t, it might not be the right time to bring it in.
Writing code is hard stuff. It’s especially hard when you have to contribute to someone else’s code. Different people also have different skill sets and are at different levels in their engineering proficiency. For this reason, it’s important to put some effort into making your open source project as approachable as possible. You should try to empower your contributors and make it clear how people can contribute, and where you specifically need the help. There are a few ways to do this.
A simple way to get started with this is to label issues with a “Good First Issue” tag which lets new volunteers easily find tasks that are simple and require little upfront knowledge. These can be small, simple bug fixes, adding missing tests, or even small trivial updates to your documentation files. Your project should try to have a handful of these items tagged and visible for new contributors to get familiar with the process of making contributions. For bonus points, you can set up a dedicated pipeline in your task management system to highlight them. In fact, you can set up pipelines for a variety of categories: tasks that have been approved and are ready for work, tasks that are controversial and require further discussion, feature that were considered but were rejected, etc… You can then also sort the issues in the pipeline to specify priority, making it clear that items on the top should be picked up first.
Many open source projects will be extra vigilant about deleting stale issues, whether they are inactive or simple items that you don’t believe are worth exploring. This helps keep things clear and removes noise. Although it’s fine to mark them as such, I would strongly recommend against deleting them from the system entirely and keeping them visible to your users through search. This will prevent duplicate requests from being filed in the future — reducing the amount of triage you need to do.
Architecture and technical decisions can also result in polarizing discussions. Software engineers are often highly opinionated and love to get into online feuds. Don’t stress about this too much and don’t try to squelch the conversation. As the maintainer of the project, it is ultimately your decision to listen to the feedback or not, but the conversation is important because it (again) serves as documentation. Mark these issues as “Needs More Discussion” to highlight the fact that you’ve reviewed this, considered it, but not intending to put anything into action just yet.
A successful open source project is more than just good code. It requires building a community, driving a vision, and staying razor-focused on your goals. Growth is not necessarily what is going to lead you to success if the project ends up being pulled into several directions, accumulates bloat and becomes stale or inefficient. The most successful projects have to be able to stay lean and committed to tackling their niche market.
The tips above help ensure that contributors of your project know what to work on and understand the guidelines within which they should make changes. A success project has a good blend of individual freedom and initiative with organization and management. Contributors should have the power to decide how features are implemented, while leaving project owners with the responsibility to define what features are implemented. Unfortunately this means that the owners need to be extremely transparent, communicative and clear about the direction. I hope some of the techniques and strategies described in this article can help you begin a framework for providing this clarity.