September 19, 2018
Ep. #48, Resolving Accidental Complexity with Dark
In episode 48 of To Be Continuous, Edith asks Paul what it took to found his fourth startup, Dark.
Hard to believe it’s almost time to close the books on 2018. As we reflect here at Heavybit on everything that happened this year in the developer tools ecosystem, I identified a set of emergent trends that I expect to increasingly dominate the discussion in 2019. In no particular order:
It’s been just over four years since AWS forever changed the course of IT infrastructure with the release of Lambda and serverless has been rocketing up the hype cycle ever since. While the long-term prognosis is still strong, I believe that 2019 is when all the heady platitudes start getting tempered with a dose of reality. This is ultimately a good thing, it signifies more and more organizations attempting to solve real-world challenges with a serverless approach. Expect to hear about more organizations pulling back from pure serverless and adopting a serverless first approach where serverless technologies are preferred but teams readily fall back to more traditional container or even server-based approaches when performance or other design constraints dictate it. For a deeper dive on some of the fundamental issues with a pure serverless approach check out this excellent paper by Hellerstein et al.
One could argue that 2018 was already the “Year of Kubernetes” but I think that clear convergence on service mesh as the dominant architecture pattern for managing microservices and Envoy as the canonical data plane implementation has primed it to accelerate even more in 2019. Expect continued innovation in the control plane around security, configuration, and more. Check this out for more on data plane vs. control plane. Istio has a lot of early mindshare as a control plane but big questions remain around whether it joins Kubernetes and Envoy on the Mount Rushmore of CNCF or if control planes become a more niche/specialized piece of the puzzle with many alternate implementations. You’ll also be hearing a lot more about GitOps as the preferred mechanism for managing change boundaries in Kubernetes clusters even when it comes to 3rd party software.
While they’ve been bubbling beneath the surface for several years now the tensions between venture-backed companies building OSS infrastructure software (e.g. databases, messaging queues, caches, etc) and large public cloud providers boiled over in a big way at the end of 2018. Both RedisLabs and Confluent recently changed the licensing of significant (albeit non-core) parts of their offering with the singular intention of preventing the public cloud providers from competing with their hosted offerings. Bryan Cantrill wrote two thoughtful blog posts on why it might not be so simple as that and Adam Jacob just launched the Sustainable Free and Open Source Communities (SFOSC) project to try and provide a more formal framework to reason around both sides of the issue. One thing is certain, the insatiable appetite of the major cloud providers to monetize traditionally licensed OSS has permanently altered how I think about funding the development and maintenance of said projects. I believe the end result of that impact will be much clearer by year’s end.
2018 was the year that GDPR’s impact became more than theoretical, Node developers were repeatedly owned by their byzantine dependencies, and public data breaches reached new heights. Not to imply that I’m necessarily critical of any of these actors/ecosystems (I love you Node developers!), I recognize that this is the new normal. It’s critical that engineering organizations identify the right set of tradeoffs for their specific threat models and formalize their secure software development lifecycles. Imperatively this must be done without giving back all the gains in agility we’ve made in the past decades. Tools like portfolio company Snyk that plug seamlessly into existing workflows while dramatically reducing the time to detect/remediate vulnerabilities will become must-haves.
And that’s a wrap! Happy holidays to everyone and I can’t wait to see what happens in 2019!