Demystifying Deployment Frequency | iftrue
The Meaning of ‘Success’ in Deployment Frequency
The term “success” in the context of Deployment Frequency often refers to the completion of the deployment process, regardless of potential software issues. The aim of the metric is to push teams to have reliable, repeatable, fast deployment processes. When the deployment completed with success, we count it as deployed and don’t check if it has bugs or issues on running code. Change failure rate will measure the bugs/failures on the running code, so if the deployment is a success than it is counted as deployed.
Considering Issues and Regressions
As we mentioned above, the post-deployment effects should be measured on another important metric called “Change failure rate”. It is difficult to measure that properly and has many debates on how to measure it. Our aim with the deployment frequency is to help teams to deploy every day; however, the change failure rate can be in a range between 0-15% of the changes made.
Distinguishing Deployment Frequency from Code Change Issues
We saw some teams tried to measure only deployments that have zero issues or failures after deployment. However, this will make deployments happen less. Deployment and changing code or writing code are two different aspects of shipping code to production. The benefit of measuring those two is to help teams to focus on separate processes individually. Instead of merging two different processes, “deployment” and “code change failures”, we measure them separately and focus each of them individually.
The Natural Throttle: Quality Issues
Quality issues can serve as a natural throttle to cycle time and deployment frequency. Dora Metric system is by nature have correlation between 4 keys. All metrics, “deployment frequency”, “change failure rate”, “mean time to recover”, “lead time for changes”, have correlation between each other and even successfully measuring one of them helps teams to predict the others. This is the beauty of the Dora metric system.
The Role of Deployment Failures
As we discussed above, deployment failure can affect all 4 metrics on the Dora metric system. The failure of the deployment will be reflected to a higher change failure rate; if the deployment failure causes an outage on the system, then that will increase the time to restore service metric. Lead time for changes will also increase with the failure on deployment for a very similar reason as the time to restore a service since the lead time is calculated by measuring the time between pull request creation to deployment to production.
This is again the beauty of Dora metrics; one metric can have a significant impact on others and can tell more than numbers. Teams should monitor their CI/CD pipeline carefully. The main goal should be having less changes per deployment with higher deployment rates so that problems arise and are solved granularly.
The Big Picture
You might have calculated the Dora metrics or not, but the Dora metric system can identify multiple problems and help teams to have a better developer experience. Each metric has impacts on the others and opens new ways to deploy better.
EA
Erdinc Akkaya
Founder
Building iftrue. Former engineering leader.