Remember the PaaS days?
Back 5 to 6 years ago, when PaaS was something up and coming many (including me) were thinking ops is going to die. Who would want to mess with servers and infrastructure when you can just outsource the whole thing and get on with what you're good at?
Now, 6 years on and many PaaS failed stories later, I know I was wrong. PaaS is not the future as it doesn't take away the pain of infrastructure, it only postpones it.
With a product that requires involvement of the vendor at the very beginning of the project lifecycle, PaaS vendors spend money and resources nurturing customers from hundreds of dollars to thousands of dollars per month, just to see them leave and build their own servers on a cloud provider.
The truth is public PaaS has become the land of tinkerers running lean startups in the earliest days of their lives, with the winners of the game leaving PaaS to run their workload on their own servers on a wide range of IaaS vendors.
I've written before as to why I think something as innovative and popular as public PaaS has not taken over the world, so I'm not going to repeat those arguments. But there is one point I want to dig a bit deeper into as I think we're about to make the same mistake again, and that is around the assumption of the demise of ops.
Believing what we see and read from public PaaS to developers doing everything that's needed to run the ops (can I call that DevOps?) it's not far fetched to think ops, as a discipline is always at the cusp of extinction. I think this thinking mostly comes from those of us who have been lucky enough not to work in a larger and older organization, or look at this from a technology standpoint rather than an organizational one.
Any organization will have ops work. If they don't, they are either renaming it as something else (like Devs wearing ops hats and doing ops things as part of their duties) or temporarily delegating it to someone else like a public PaaS provider, for the time being.
This is not entirely a function of technology, but about business structure. Businesses have different life-cycles for innovation (dev) and keeping the lights on (ops), and that means devs will never take over everything, for many good reasons. Any technology developed by developers that wants to take off in the enterprise in a big way, has to acknowledge that.
This, in my opinion, is the biggest reason public PaaS never took off: as businesses grow, they need to regulate and streamline their operations, and this generates a discipline of ops in the organization, which will take over from the external PaaS provider (or in better words, puts developers in a different cycle of business).
Conversely, this is why private PaaS like Cloudfoundry is doing much better than their public cousins: they are tools for ops, catering for devs. They are successful because they acknowledge that ops exist by becoming a resource management utility for ops, and not a tool to take ops out of the equation.
Let's not forget Ops again
I however feel we are about to make the same mistake we made over public PaaS in the container world. Most of the solutions for running and managing containers in production are too developer-centric. They don't have a clear separation of where application specifications stop, and where infrastructure requirements begin.
Tools like Docker Swarm are examples of the first steps towards splitting these tools between ops and devs at a conceptual level, but there is still a lot to do when it comes to application specification: what are the services?, how do they interact with each other?, what are the contracts they should adhere to when talking to each other?, etc... These are application specification questions.
The next set of questions are ops questions: how to bring resources into our infrastructure?, how to build the inventory of what we have?, how to put constraints on where each service is going? and how they find each other that reflects the infrastructure setup?, and so on.
Things are encouraging when I look at tools like Swarm and Compose in the sense that they are separate tools, and have clearly differentiated functions. However they have not made a great distinction between devs and ops in their API. For example, Compose (still work-in-progress) suggests supporting all of Docker CLI options and many (if not all) of Docker's CLI options are purely dev-centric.
One of the best things about Docker is the clear boundary it draws between devs and ops: anything inside the container is devs, and anything outside is ops. The current API of Compose and to some extent Swarm, is a step back in this clear distinction.
We are trying hard to make this distinction clear, to reflect the reality of what's going on in the enterprise. The split between manifest.yml and services.yml, or the way our UI is built, are some of the starting points for how we're trying to push this vision through. I'd love to hear your thoughts and feedback on that.