Today’s demanding business environment requires that mainframe houses adopt the same DevOps practices used for distributed platforms. Measuring the key performance indicators (KPIs) of the mainframe helps ensure that the platform is active and reliable at all times, which is vital to the organizations that rely on it to process millions of transactions a day. This has also created a need for KPIs in DevOps—regardless of the platform—grouped into three categories:
- Velocity: understanding how fast it takes to go from development to production and to each stage along the way.
- Quality: ensuring consistently good quality down the pipeline.
- Efficiency: comparing the effort expended against the work being done.
To measure these KPIs, the DevOps Research and Assessment (DORA) team came up with four key DevOps metrics, now called the “DORA metrics.”
What are DORA metrics and why are they so important?
These four metrics are essential indicators of progress and improvement in:
- Lead time for changes: how long it takes for your pipeline to deliver code once a developer has checked it back in. Not tracking this delays getting innovations to market.
- Deployment frequency: how often you deploy to production, delivering the innovation the business demands. The goal is to deliver small batches of work that improve quality and put innovation more quickly into the hands of your customers.
- Change failure rate: assesses the percentage of code changes pushed back into production that are failing and/or must be rolled back. This metric can be used in conjunction with the prior metrics to understand where the sweet spot is for delivering code. Delivering code too fast or in large batches may cause this metric to rise.
- Mean time to recover (MTTR) service: tracks how long it takes to fix problems that occur in production. This is tied to the previous metric. A higher change failure rate may cause this metric to rise, as well.
As a company improves, it can deploy more frequently while keeping the change failure rate the same (see Fast, Good, or Cheap—Get All Three with Continuous Improvement by Mark Schettenhelm).
Maturity and elite status
Alongside deployment frequency metrics, organizations are also rated at low, medium, high, and elite levels of maturity.
- Low deploys between once a month and once every six months, typical in mainframe.
- Medium deploys between once a week and once a month.
- High deploys between once a day and once a week.
- Elite organizations like Google, Facebook, and Netflix deploy multiple times a day. They expect their teams to push code into production on day one, and in terms of MTTR, can fix a problem in less than an hour.
DORA metrics for the mainframe
Mainframe shops must stay up to date and using DORA metrics is an efficient and standardized way to measure that modernization and get to that next level of maturity. If DORA metrics are not used, it becomes difficult to know whether you are as efficient as you can be, and to benchmark against industry peers.
The four metrics are interconnected like cylinders in an engine. Organizations must remember that despite the attention given to cloud and distributed technology, the mainframe is probably the most important server they have.
How do you capture DORA metrics for the mainframe?
Mainframe can capture DORA metrics the same way a distributed environment can: through DevOps tooling and/or system logs, which are then parsed into the four measurements. This should start as early as possible. We’ve made this easier for teams by offering BMC AMI Adviser dashboards, which eliminate the need to plot and consolidate data and provide immediate perspective on your industry positioning.
How BMC AMI DevX tools blend with DORA metrics
In addition to zAdviser, additional BMC AMI DevX tools can help provide a complete picture of your metrics.
BMC AMI DevX Code Pipeline lays the groundwork, capturing the original source code management (SCM) function, which facilitates editing, testing, and preparing to deploy by gathering metrics all along the way. ISPW is at the core of developer activities, allowing people to check code out, edit it, and check it back in. ISPW then compiles the code when it is ready. If, during testing, an error is found that forces a re-edit and another compile, it passes through ISPW until it is promoted and approved for deployment. During this time, metrics are applied from check out to deployment to production.
zAdviser is a software-as-a-service (SaaS) solution that simplifies the process of gathering metrics by making data easier to review—which also helps bring organizations closer to elite level. zAdviser captures data around how people are using BMC AMI DevX mainframe tools and applications such as ISPW and BMC AMI DevX Abend-AID. This parallels the practice common to the distributed world, largely through the use of Git. Customers thinking of migrating to Git and outside of ISPW should be aware that ISPW allows a company to have some teams using Git with ISPW while others continue to use just ISPW.
Abend-AID automatically detects and diagnoses problems across multiple environments, addressing issues the first time they occur. In the context of DORA Metrics, Abend-AID provides supporting data and helps seek out the root cause. It is possible to obtain DORA Metrics without Abend-AID input, but you can’t improve them without it.
BMC AMI DevX is the only solution that can take data in from SCMs and allow customers to see how they are developing their mainframe applications today, which tools they are using, the pieces of code their developers are working on, and how that code is going through the lifecycle.
DORA metrics and the four KPIs help management measure and understand the performance of their time to delivery and their development teams while BMC AMI Adviser KPI Dashboard for DORA Metrics allows them to leverage that data to continuously improve their DevOps efforts.
To learn more about tracking DORA metrics on the mainframe, download our eBook, DORA Metrics for Mainframe DevOps.
