Safer Kubernetes Deployments with Spinnaker: With Integrated Solutions for Problems Like Config Management and Service Discovery

Kubernetes has fundamentally changed how we view applications that run in the cloud. With integrated solutions for problems like Config Management and Service Discovery, Kubernetes provides a relatively holistic approach to running cloud-native applications. One common misconception about Kubernetes is that it’s a deployment tool, which is it not.

Kubernetes is a portable container “cloud” that can live anywhere and creates a standard interface for infrastructure, similar to how AWS creates a standard interface for their data centers via tools like EC2.

One of the most significant problems affecting the community today is deployment. While getting applications running on Kubernetes is somewhat simple, managing the promotion, update and lifecycle of these applications in a safe and repeatable way is a problem that the Kubernetes community is just starting to discover. Safety and reliability are two critical aspects of Continuous Delivery and Spinnaker is precisely positioned to solve this problem for Kubernetes. Kubernetes does an excellent job at maintaining the availability of your application, but it isn’t so great at ensuring that your applications are functioning as they should and leave implementing those requirements as an exercise of the user. In this post, I’ll explain how the new Kubernetes provider for Spinnaker seeks to solve this problem and aims to make your Kubernetes deployments even safer.

Deploying to Kubernetes

Most applications deployed on Kubernetes tend to follow a similar pattern. First, the running application is represented as a Deployment, defining the containers, their environment variables and the compute resources they require. This Deployment references a Secret or ConfigMap as a way of sourcing environment variables of other environment’s specific configuration. Each time a new Pod is started, the current state of these resources mount to the container, and their keys inject where necessary. This Pod function is really powerful because Kubernetes exposes resources that help us build applications that follow the principles of 12 Factor Apps. One of the limiting factors, however, is that these resources aren’t versioned alongside the Deployments that use them. Practically, this means that configuration in your ConfigMap and Secret have a high likelihood to be incompatible with previous versions in the event of a rollback.

To help illustrate the point, let’s take a look at an example:

apiVersion: v1
kind: ConfigMap
metadata:
name: app-config
data:
FOO: bar
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-app
spec:
template:
spec:
containers:
- name: app
image: my-application:1.0.0
ports:
- containerPort: 80
envFrom:
- configMapRef:
name: app-config

In this example, our Deployment (my-app) is referencing a ConfigMap (app-config) which contains a list of environment variables that our application depends on, specifically the FOO environment variable. Imagine a scenario where our application no longer depended on the FOO environment variable but began depending on a BAR environment variable. To roll this change out using vanilla Kubernetes, you’d need to update the contents of the ConfigMap and then apply the Deployment manifest. If a critical bug surfaces in this deployment, the best way to resolve this is to rollback to the previous version. You could do this by using kubectl rollout undo but that only reverts the Deployment. The changes made to the ConfigMap persist which breaks our application since it’s missing previously required FOO environment variable.

One of the primary principles of Spinnaker is Immutable Infrastructure. The tools that Spinnaker exposes for your deployments are all pre-disposed to favor this style of application deployment. By following this principle, you can be sure that an application functions just as it did before in the event of a rollback.

While the Deployment follows the principle of Immutable Infrastructure, the rest of the Kubernetes resources do not, and that is one of the problems that Spinnaker is aiming to solve with the Kubernetes V2 provider.

Manifest Versioning

To solve this problem, Spinnaker inspects every Kubernetes manifest that it deploys. If it detects a non-versioned resource is deploying alongside a versioned resource, it updates the name of that resource to include a version. For example, in our previous scenario, we updated the contents of our ConfigMap, but the name remained the same (app-config). Now, when we change our ConfigMap, Spinnaker creates a new ConfigMap to represent that change. So, if we start out with app-config-v00, the next change creates app-config-v001. This change ensures that changes to this resource do not affect previous versions of the application if they were to appear again.

In this way, Spinnaker improves the experience of deploying to Kubernetes by making it safer. To achieve the Immutable principle of Spinnaker, Spinnaker makes sure that each piece of a deployment is versioned such that the whole, each piece of the puzzle, is a single unit that works together. These units, comprised of Deployments and ConfigMaps for example, work together to model your application running on Kubernetes.