Measuring Continuous Deployment (CD) with Spinnaker and Splunk

It is now commonly understood that data-driven decisions yield better results. We can ingest and listen to signals from our users to determine our next paths and products. Services can be baselined and improved upon, which across the board improves how a business functions. Businesses can become more tactical and gain competitive advantages over others in their market. Security practitioners can hunt and identify threats by detecting anomalous behavior or deviations from the normal. DevOps and SRE teams can derisk software deployments and user impact by automating the data produced while pushing new code from Spinnaker.

Spinnaker, the leading tool for Continuous Delivery (CD), has an extremely rich dataset that allows detailed information about software deployments across all clouds. Today, Armory.io and Splunk announced a technical partnership to make this data actionable for DevOps teams. This will allow all companies to identify bottlenecks in their Software Development Lifecycle (SDLC) and leverage Spinnaker to automate those most time-consuming stages. It also improves the software deployments with allowing for real-time data analysis and self-healing rollback capabilities.        

If you’re new to the DevOps space, where should you start in measuring DevOps? Some of the guidelines have been already established for us to point us in the right direction.

DORA Metrics: DevOps Research and Assessments 

• Deployment Frequency (DF)

How often does your organization deploy new code? Public cloud allowed developers to deploy new code without waiting on an Ops team to provision dependencies and capacity. Microservice-based applications increase frequency dramatically by decoupling the application components from each other. Each is on its own release frequency, which does not impact other teams.

• Mean Lead Time for Change (MLT)

A higher level measurement that is a sum of all the stages in Value Stream Mapping. Essentially what is the SDLC measurement from code commit to production deployment? Spinnaker specifically addresses this by automating a lot of processes into a stateful delivery pipeline.

• Mean Time to Recovery (MTTR)

I laugh a little at this one because it’s always been presented to me in a dollar amount. “Our outage last night cost us $10,000 a minute!” The true measurement is the time to get back to the last known good state where user experience is great and revenue (usually) is growing.

• Change Failure Rate (CFR)

This is a metric that must be tracked and should always be improved with Spinnaker. Shifting left and providing the right features and guardrails for developers should improve the hygiene of not only your software, but also how you continuously deliver code. What we see at Armory is as our customers’ journey with Spinnaker progresses, their CFR goes down.

Value Stream Mapping

Breaking down the SDLC and reviewing to establish “current-state” and determining a “future-state” that removes time-consuming bottlenecks without compromising safety. Armory actually does a great half-day exercise with our top customers which gets everyone in a room who knows anything about how code gets out to production and facilitates a building of the “current-state.” We then break down that chain into bottleneck identification, approval mapping, and script deficiencies. Spinnaker has been proven as a platform to automate these and get customers to the best possible “future-state.”

Continuous Verification

Injecting the best practices for monitoring a service into the entire SDLC, especially production. Again this is something that Splunkers have been doing for over 15 years. Today we use the term “observability,” which is a combination of logs, metrics (KPIs), and tracing. We want to leverage all the time series and machine learning processing we can to ensure our current state in production meets our service level objectives (SLOs). Armory Splunk App for Spinnaker allows us to marry the Splunk data engine, with Spinnaker’s data driven Continuous Delivery, Canary Verification, and Automated Rollback.

To learn more about installing the Armory Splunk App for Spinnaker and sending Continuous Delivery data to Splunk, please view our documentation. Also, come learn more at the Splunk DevOps Open Source Insight and Innovation event on July 22nd.