November 29, 2018
Deployment Best Practices
Join HashiCorp CEO Mitchell Hashimoto, Heptio Co-founder Joe Beda and LaunchDarkly CEO Edith Harbaugh as they discuss the impact of architec...
Every startup team has limited resources. You have questions to answer and hypotheses to test before committing engineering resources to build the scalable version. Practicing lean launches helps your team use those resources effectively. Ghostbuilding is a great way to check the market acceptance for your products before actually building.
One of the key principles of lean launches is that you want to optimize for cycles of learning. If you can get product feedback before you invest engineering resources, you’ll be able to learn as much as possible up front, so you can ensure that more of your engineering resources are focused on building a market-accepted product.
In this article, I’ll explain what Ghostbuilding is and how you can use it to build products more efficiently.
Ghostbuilding is the act of creating just enough illusion of the product to test your hypotheses. This approach helps you gain higher confidence on market-acceptance of your product by the time you build your MVP.
A Ghostbuild is a light-weight product that tests your most important hypotheses. The customer sees the product working and may think that the functionality is automated by technology. But, like Wizard of Oz’s man behind the curtain, humans are powering the product behind the scenes. The product maturity progression is pictured below. You can see where Ghostbuilding fits in the products maturity lifecycle:
A Ghostbuild sits somewhere between a prototype and a proof-of-concept. A prototype is useful for a company who is going through early stage fundraising. A prototype is also useful for selling a complex idea to an enterprise customer. The prototype has a very limited functional scope. It may need to be used only under supervision.
Conversely, the proof-of-concept is a more expensive build-out that can be used “in-the-wild” by the customer who commissioned the POC. Enterprises may pay $25k to $100k for a proof of concept. The notion does not exist for B2C companies, who typically fund their own way to their MVP.
Ghostbuilding requires many of the same software and tooling you’d need to design and launch a product. WordPress websites are useful for approximating a user interface. Other software can be used to emulate the product’s functionality, like chat integrations (Intercom or Slack), spreadsheets with macros or Google Sheets with Google Apps Script, MailChimp and IFTTT.
In addition to tooling, there is one more requirement for Ghostbuilding: a team of humans with great communication skills. The Ghostbuilding phase is full of valuable user data and feedback. Making sure that your team is collecting and processing all that valuable feedback is essential.
So what does a Ghostbuild really look like? Here are examples of different ways of Ghostbuilding:
Reporting dashboards is a common feature across many different SaaS products. Different users often need different insights. The challenge here is learning what information is most useful for which user persona. A project like this doesn’t necessarily need any engineering work at all — the simplest way to test this is to simply send human-created PDF reports to customers. Your customers will give you feedback on what else they’d like to see on the PDF. By the time you’re ready to build out the dashboard, you’ll be very close to knowing what the customer actually needs.
This is a relatively simple example of Ghostbuilding. But what if you need to test something more complex?
If I want to test the market demand for this, I don’t need to actually build the AI engine. I would build a WordPress website and integrate Intercom. Then people can message in their requests for restaurant recommendations. I would hire a few humans, probably customer service agents, give them a playbook on how to answer questions, and let them go at it. The playbook information will be based on some research that I do ahead of time on the scope of questions an AI engine could answer, and extrapolate slightly from there. Because I don’t intend to use real humans in my eventual product, this approach lets me collect data on a feasible version of the product.
After a week of testing this Ghostbuild, I would analyze the data. Sample metrics for this project might be:
This will all help guide me in deciding whether to build this product, and if I do, what I should optimize for before I invest in any deep engineering to build out this AI engine.
Where Ghostbuilding will have the most impact will vary depending on if you are selling to consumer or enterprise. In enterprise, you have the option to acquire a POC contract. Basically, you can have one customer pay your way to an MVP. In that case, you need to build light, but may choose not to Ghostbuild because you have a committed user and you need more robust automation. But Ghostbuilding is still possible, especially if your product needs a bridge path to a real product, like AI. Or, if your customer is rolling out a limited pilot, you can offer a lower cost proof of concept to them by Ghostbuilding.
For a consumer audience, you likely have to pay your own way to an MVP. Because customer acquisition cost is lower than in enterprise, it really makes sense to Ghostbuild.
Keep in mind that when you Ghostbuild, you might have to discontinue the service temporarily while you transition to the real product, so you don’t want to get to the point where customers are relying on your Ghostbuild for too long.
While Ghostbuilding is a way to conserve engineering resources, you will need a person with product sense and user research experience to lead the effort. In more complex projects, you may also need some engineering to Ghostbuild.
Typically, you’d spend 1- 3 months Ghostbuilding. Depending on your Ghostbuild path, You may need a software configuration expert (IT), which may be in the range of $5-10K per month. Or you can do this part yourself by googling how to configure certain software. In the reporting PDF sample above, you might do this entirely without engineering. For a more complex project, you might need one front end engineer and half a backend engineer’s time, or one full stack engineer for $10K-30k/mo, to help you hook things up. Properly scoping the project before you begin will help you estimate how much to budget for a Ghostbuild.
In summary, if you’ve decided to Ghostbuild, consider the overall cost of the project, including:
Ghostbuilding shares many of the same pitfalls of classic product building. For instance, you might try to build too large of a scope. To avoid this, start with determining the questions that you need answered before you start building. Make a list of the variations you’re considering, and look for ways to quickly Ghostbuild.
Another common pitfall is creating too magical of an experience with Ghostbuilding that cannot be recreated in the real version. Remember, you are using humans to underpin some of the functionality that you will automate later, and some of these functionalities might not translate that well into automation. If you can, consult the technology architect halfway through your Ghostbuilding phase. They should be able to give you a better sense of how the real version will be built later on, and whether it’s feasible to support the features you want.
Keep in mind that the point of the Ghostbuilding is to validate hypotheses, not to serve customers in a sustainable way. It’s easy for teams to forget that they’re Ghostbuilding and try to use that code to support users at scale. As soon as you have your questions 80% answered, then it’s time to start actually building. When hypotheses are proofed and the spec is ready, hire a real engineering team with an architect and start over.
To learn more about best practices for early-stage product strategy, check out other pieces from Connie Kwan in the Heavybit Library, and learn more about her upcoming talk on Early-Stage Product Storytelling.