With the death of public PaaS, infrastructure and development teams are looking for ways to replicate the ease of deployment offered by public PaaS. Along the way, teams are finding that the complexity and cost of a roll-your-own DIY PaaS is more costly than originally estimated. This article examines the layers required to support a private PaaS, the key decision points for choosing a private PaaS solution, and the options available for DIY PaaS.
The layers of private PaaS
While Platform-as-a-Service offerings, such as Heroku, appear to be simple and easily replicated with some simple scripts, they offer a considerable amount of functionality. The full PaaS stack of functionality is divided into seven layers:
Layer 1 : Physical Infrastructure - the actual hardware used for deployment, which may be owned or leased via the cloud
Layer 2 : Virtual Infrastructure - the virtualization of hardware for sharing, provided by a virtualization vendor or cloud vendor
Layer 3 : Operating System - the choice of one or more operating systems that will be used to deploy containers, including Ubuntu, RedHat, CoreOS, or Amazon Linux (EC2 only)
Layer 4 : Container Engine - the engine that manages running containers on the operating system, which may include container image management or just raw container services
Layer 5 : Container Scheduling - manages the lifecycle of containers, including starting, stopping, and destroying them as required
Layer 6 : Container Orchestration - enforces the defined requirements for a set of containers (e.g. min/max of each container type), performs appropriate scheduling to enforce those requirements, and offers simple services for logging and statistics
Layer 7 : Application Workflow - supports the creation of container images from source code, identifies and configures service dependencies, routing, and service provisioning using the Container Orchestration layer
Each layer builds upon the last, enabling a complete application workflow from code commit to deploy.
The difficulty with public PaaS, as pointed out previously, is that the configuration, routing, and service provisioning is limited to what they support within the 7th layer of their PaaS. You cannot deviate from their built-in assumptions and restrictions of how their platform works.
The growing need for private PaaS
With the desire to deploy applications easily and with the flexibility not currently available with public PaaS, many teams are looking for an alternative solution. Enter container-based deployments.
For those with existing data centers or that simply want flexibility in consuming one or more cloud infrastructure providers, they are looking toward container-based deployments, combined with commercial or open source alternatives. However, not all of these solutions handle a seven-layer PaaS out of the box. This results in companies moving toward a DIY PaaS model.
The emergence of DIY PaaS
As a result, many groups are trying to build out their own internal PaaS, often termed Do-It-Yourself PaaS, or DIY PaaS for short. Teams combine a variety of products, open source libraries, and scripts to glue together their own PaaS that supports automated deployment of their applications via containers using tools such as Docker.
Container-based deployments allow development teams to build, test, package, and deploy their software easily. Development, test, and production environments all run with the same container images, with only configuration changes required to support their deployment needs. At least, that is the assumption up front. The reality happens to be much different.
How complexity grows in a DIY PaaS
As teams begin exploring the construction of their own, container-based PaaS, they often begin with a single server. As their installation and needs grow, more nodes are required to support the application. Multiplied by the number of projects that need to use the infrastructure, the result can grow quickly.
Nano - Deployment and orchestration of containers on a single node, often using Docker Compose; limited or no scale-out available; no need for self-healing as the single node is quite manageable
Micro - Deployment and orchestration of containers across 10s of nodes; logic to handle scaling of deployments across nodes is required; manual support possible given the lower number of nodes
Medium to Large - Deployment and orchestration of containers across 100s (medium) or 1000s (large) of nodes; node count is higher, so manual intervention isn't possible, requires built-in monitoring and healing; robust scheduling required to fully utilize server resources
At some point, teams start to realize that building out your own PaaS produces exactly the wrong result. Instead of making it easier for you to deploy your applications, it converts you into the wrong kind of software company - one that is focused on the business becoming a PaaS, rather than a software company.
Can't we just automate our way to private PaaS with scripts?
Scripting languages like Chef and Puppet have their roots in pre-cloud days and are great tools to describe our desired infrastructure. But as we buy more of our infrastructure as services, scripting our desires is going to mean less and less and therefore the old tools that were suitable for the days of full stack control are not going to cut it anymore.
Additionally, as the number of external dependencies in the new datacenter grows the probability of your scripts breaking grows with it. As we lose full control over the entire stack, we are going to risk being more exposed to unwanted and unannounced changes we are not prepared for.
Because of these shifts since the release of these scripting language tools, we take on a significant business risk by trying to build our private PaaS on a foundation of quicksand.
Counting the cost of private PaaS
When planning a private PaaS solution, there are three options available. Let's examine each one to find the one that best fits your needs by looking at the benefits and costs of each one.
Option 1: In-House Built DIY container-based PaaS
The first option is to build a complete DIY container-based PaaS internally. While this provides the most flexibility regarding tool selection and workflow customization, it is also the most costly. Teams will be required to select solutions, install, and integrate a variety of tools and libraries across all but the first two layers of the PaaS:
Est cost: The average devops engineer averages $105k USD x 2-4 engineers = $210-420k, not including licensing and support costs of selected tools and continuing maintenance costs.
Pros:
- Fully customized to your company's needs
- Can utilize existing hardware infrastructure or cloud provider relationships
Cons:
- Large up-front cost
- Longer time-to-completion
- Likely distraction from core business/product strategy
Option 2: Private container-based PaaS
The second option is to seek out a private PaaS solution that offers most of the foundational layers but still require some customization to add application workflow support:
Tools such as Google's Kubernetes, Pivotal's CloudFoundry, and RedHat's OpenShift offer this style of solution. The amount of customization required to meet the needs of your team will vary across these products. However, they generally offer the most flexibility while reducing the amount of work required by building an in-house DIY PaaS.
It is important to note that for small dev teams, private PaaS solves some of the scaling problems by mostly forcing good practices onto the development team. However, it also shifts the maintenance time spent by the team from the app to the private PaaS to customize its behavior.
Est cost: A devops engineer averages $105k USD x 1-2 engineers = $105-210k, not including licensing and support costs of selected tools and continuing maintenance costs.
Pros:
- Greater flexibility than a public PaaS
- Less expensive than an in-house DIY PaaS
Cons:
- Still requires dedicated staff to install, customize, and maintain the application workflow and support the private PaaS solution
- Work may exceed the ROI threshold for many organizations that only need to deploy and scale a handful of applications
- Some solutions only support a single cloud infrastructure provider, requiring extra work to support cloud failover
Option 3: Container Management as a Service
So, what are teams supposed to do if public PaaS is dead and a DIY PaaS is too expensive? Our recommendation is to utilize a third option: container management as a service.
A container management service, such as Cloud 66, allow developers to focus on building the code, defining their infrastructure, and deploying new versions - without scripting or cloud provider lock-in. Everything from packaging of code into a container image, to private container image hosting, deployment, and service discovery are handled by the service. For those who want to automate the manual setup steps, an API is also available to interact with the Cloud66 service.
Est cost: No devops engineers required for infrastructure scripting. With the modern, scalable cloud native application requiring at between 10 and 20 servers across a variety of roles (app, db, etc), the overall cost would be between $100 ($19 first server + 9 * $9) and $190 ($19 first server + 19 * $9) per month - assuming a monthly management service cost of $9-19/server and excluding monthly cloud resource fees.
Pros:
- Least expensive
- Quickest time-to-completion
- Solution grows with the number of applications deployed, with little or no change
- Many solutions support multiple cloud vendors for failover
- Faster setup time than many code deploy solutions offered by cloud infrastructure vendors
Cons:
- Works with public cloud only, preventing use of existing hardware infrastructure
- May require more initial setup work than Heroku
- Paid tiers only, no free tier available for early stage applications like those found with most public PaaS solutions such as Heroku