Spinnaker tricks: Restarting a deployment automatically

What problem are we trying to solve?

Spinnaker has always understood the need to do “Orchestration” type tasks. These are tasks that you can do via the UI to do things like rolling restart a deployment, scaling it up or down manually or similar. These types of operations are actually stages that are executed similarly to pipelines, but many such operations are not visible when modifying a pipeline.

A common case I’ve encountered recently is the need to restart a deployment to pick up a change. Typically this is a DNS change when my app does NOT support DNS updates (often a situation in highly secure environments), or a password change from a config that’s not re-read automatically. Normally, you’d have to login to a system, get credentials, and call a kubectl rollout restart -n namespace deployment my-webapp. This can be time-consuming and a pain. There are automations you can do (e.g. a Runjob internally, or some other workload automation). BUT let’s use the power of Spinnaker to solve this since it natively supports some options to make these kinds of tasks MUCH easier.

Today’s tip is how to restart a deployment via a simple trigger (webhook in this case) by manually creating a pipeline for one of these operations – the rollout restart stage.

Let’s get started…

First, create your pipeline. Note you’ll need to adjust settings to match your environment. The full pipeline is available here:

https://github.com/jasonmcintosh/spinnaker-work/blob/main/pipelines/restart-pods/pipeline.json

Let’s look at one piece of the config that I’ve shortened for this blog:

{
      "account": "k8s-namespace-prod",
      "application": "demo",
      "cloudProvider": "kubernetes",
      "location": "prod",
      "manifestName": "deployment demo-web-app",
      "name": "Restart pods",
      "type": "rollingRestartManifest"
    }

The configuration above that is key is location, type, account, and manifestName. This is how Spinnaker identifies the type of operation, the location (aka namespace) and the resource. Note that the “manifestName” in this case is a combination of the kind “deployment” and the resource name to restart. You can now via a very simple curl trigger an automated restart of a deployment (DO note, it’s recommended to use validation to verify the contents of a webhook!):

curl -X POST -v -H "Content-Type: application/json" http://spinnaker.example.com/api/v1/webhooks/webhook/restartSomeDeploymentService -d '{"restartCreds":"isSomeSecuredEntry"}'

You can see the results of a restart below:

You can see the results of several rollouts here:

The pipeline in this example was only used to restart a single manifest and do nothing else but you could easily add this as a stage in any pipeline if you wanted for some reason to trigger a restart.

Final tidbits

This was done via a very simple webhook, but you could use any of the other spinnaker “triggers” – e.g. a pubsub message, a git commit, cron, or something else entirely. There are other operations than just rollout restarts you can use with this method. For example – this method can be used to scale up a deployment! And a reminder – watch the permissions and validation of your triggers! This is essentially exposing an automated operation against your infrastructure. It’s powerful, but with great power comes great responsibility!