Kubernetes (K8S) Deployment Options

6 min readJan 7, 2021

Doing It Yourself v/s Using Managed Platforms

Containers Everywhere

With microservices pattern being adopted increasingly for greenfield applications as well as transforming the legacy applications, containerised deployment of these applications has become the norm due to obvious benefits, mainly abstraction of application and its dependencies. While there are applications with single or a limited set of containers (not needing complex container management), majority of enterprise scenario container deployment tends to have multitudes of containers as well as more being added for newer applications all the time, leading to a complex estate.

Managing Containers

As containerised deployments continue to grow, the need for container management and orchestration platform has come to preoccupy the minds of application developers and IT operations engineers. While there are multiple solutions available, Kubernetes (K8S) has become the de facto standard for container management and orchestration. K8S is an open-source system for automating deployment, scaling, and management of containerised applications. K8S groups containers that make up an application into logical units for easy management and discovery.

Options for Kubernetes Deployment

Do It Yourself — Use the K8S project to build your own Kubernetes implementation
Managed Kubernetes Services — Use the vendor provided managed Kubernetes implementations (GKE, AKS, EKS etc.)

1.1 Kubernetes — Do It Yourself

While one can go for DIY implementation, it needs to be done keeping the following in mind:

Deployment Time

Creating your own K8S implementation and setting up a production-ready highly available cluster can take a long time (5–6 weeks). It can get quite complex soon if the infrastructure base is diverse with multiple geographic locations involved. To make the matters worse, your team may need to bring together multiple components, services, plug-ins and add-ons. Most importantly, it will require experts who are scarce, otherwise you are looking at a steep learning curve and even longer deployment time.

Monitoring

It is imperative to set up the cluster with continuous monitoring to allow it to be healthy and stable. This will require various aspects to be built and configured with your K8S deployment e.g. log collection, workload monitoring, certificate management, network isolation, SSL termination.

Management and Troubleshooting

As with any other system, K8S cluster itself and/or the workloads running on these clusters may develop issues which need to be taken care of. Finding out the issues, researching the solution and fixing them can be a daunting task and requires significant effort from expert engineers. On top of that, K8S is an open source solution that keeps evolving and hence the deployment needs to be upgraded and patched continuously requiring a dedicated expert team of K8S expert engineers.

DIY Setup Cost

Given the complexity of DIY implementation and managing of a K8S cluster, you need to have an idea of the key activities involved in developing and managing your own K8S cluster, some of which are mentioned below.

Cluster provisioning and configuration with usage of tools like Puppet or Chef. This needs to be done for each environment and every cluster.
It is important to build in processes and tools for monitoring the K8S infrastructure and hook that with your organisation’s ITSM processes and tools.
K8S undergoes frequent upgrade cycle to deal with vulnerabilities as well as bringing new features and keeping the K8S implementation updated will require custom scripts.
K8S cluster need to use resources available from multiple infrastructure (bare metal, public cloud etc.) and this will require to be built in.
Additionally, features like load balancing capabilities, high availability, auto healing, security, integrating with IAM etc. will need one-time work to be done to make the K8S clusters production ready.

As per one estimate, this will require significant one-time effort (~350 person days == $350K) and an ongoing effort to maintain the K8S platform (~50 person days per month == $600K per year). This cost only includes the manpower cost to develop and maintain K8S clusters. There is going to be separate cost for infrastructure and IT administration associated with that, which has been fully discussed in section 2 while comparing the DIY K8S cost with managed K8S platforms.

1.2 Kubernetes — Managed Platforms

Public Cloud vendors saw an opportunity in the market to alleviate a lot of the administrative issues associated with Kubernetes by managing the control plane and Master nodes. This means that administrators no longer need to worry about installing Kubernetes or upgrading the cluster themselves. All major public cloud vendors have come up with their own managed K8S platforms and in this section, we are going to compare 3 major platforms — GCP managed Google Kubernetes Engine (GKE), Azure managed Azure Kubernetes Service (AKS) and AWS managed Elastic Kubernetes Service (EKS).

Feature Comparison

Based on the information above, there is not much to differentiate among these leading services, and all are packed with relevant features.

Managed Platform Cost

Managed Kubernetes Platforms generally incur cost for running control plane [ and not the VMs etc.] and then the cost for worker nodes. A benchmark cost comparison (per year) has been given below with the assumed load of 100 core/400 GB k8S cluster when deployed to different managed platforms (see references for details).

2. Cost Comparison — Managed Platforms v/s DIY

So far, we have looked at the considerations for implementing DIY K8S platform, effort and cost involved in doing that setup. We have also looked at the managed K8S platforms available from leading public cloud vendors in terms of their features and cost of running a specific size of K8S clusters on them.

Now, let us look at how the managed platforms compare with DIY implementations in terms of running a specific size K8S cluster. For sake of completeness, we’ll consider infrastructure cost in totality including servers ($14375/server for 5 years lifespan) and ancillary cost including network ($500/year), power($1080/year) and rack space($8400/year). We’ll also add IT Administration cost to manage the K8S cluster ($8160/year for cloud and $32640/year for on-premise). [These figures have been taken from a study mentioned in references below].

As we can see, managed K8S are always cheaper (only slightly) as compared to DIY deployed on same cloud vendor. A DIY K8S set-up on-premise is the costliest option and best avoided as it has got complexity, capability and ongoing maintenance considerations. However, DIY deployed on cloud infrastructure is competitively priced.

3. Kubernetes Implementation Options — Bringing it All Together

For any specific Kubernetes deployment, teams must choose from all available options considering feature availability, flexibility in K8s setup, timeline, cost considerations, skills availability, and vendor platform portability.

To summarise, a general guidance for choosing a between DIY and Managed K8S platform should be based on considerations most important to your team and the organisation and decision can be made as presented below.

Finally, you must make your choice based on considerations most important to you, your team and your organisation.

References:

Kubernetes (K8S) Deployment Options

Doing It Yourself v/s Using Managed Platforms

Written by Satyajeet Singh

No responses yet