Customer satisfaction is a top concern for all levels of an organization, from the service desk to developers, and from management to the executive suite. An organization must provide services that not only fulfill its customer needs, but that are also reliable and as free from bugs as possible. Even the smallest bug can create dissatisfaction and negatively impact a brand’s reputation.
Development teams and managers can gain insight into mainframe software quality with BMC AMI zAdviser. Offered free to customers with current maintenance, this software-as-a- service (SaaS) solution captures data from BMC AMI DevX products and uses machine learning (ML) to develop key performance indicators (KPIs), giving users advanced metrics to help improve their organization’s mainframe DevOps processes and outcomes. These metrics can then be compared to benchmarks determined by blending anonymous data from all zAdviser customers to show how development teams are performing relative to the BMC AMI DevX ecosystem as a whole. In short, zAdviser gives development managers a view of what “good” looks like.
Two new KPIs have been introduced as part of BMC’s October release. Part of zAdviser’s Quality Dashboard, these KPIs help determine where and when abends are occurring, giving valuable insight into development quality and aiding shift-left testing.
Escaped Abend Ratio
Abends that escape to production could be customer-facing, creating dissatisfaction and damaging an organization’s reputation. The Escaped Abend Ratio KPI can be thought of as an escaped bug ratio. It shows the percentage of unique abends occurring in production compared to abends in all of an organization’s logical partitions (LPARs).
Ideally, all abends would be caught in the development testing environment, but that is never the case. This metric enables developers and management to see how many abends escaped into production and then adjust test cases accordingly to catch them in the future. The goal is to see the percentage of abends in production decrease. The same or higher number of abends in development indicates that testing is catching what it should, leading to a higher quality of product in production.
Median Time to Detect Since Compile
When a program abends, zAdviser collects a variety of data, including the amount of time since the program’s last compile. The Median Time to Detect Since Compile calculates the median (50th percentile) of this time for all abends. In a development environment, a low number is favorable—as the old adage goes, “You want to fail fast.” For production environments, the opposite is desirable; longer timeframes between compiles and abends might indicate that the program abended due to a unique confluence of events, or a unique use case that wasn’t taken into consideration during testing.
Early Warning
With these new KPIs, the Quality Dashboard becomes a sort of early warning system for developers and development management. Teams can gauge the quality of code as it is developed and deployed rather than waiting for a complete degradation of quality or a complaint from a customer. When development teams see abends increasing in production, they can refer to zAdviser’s BMC AMI DevX Total Test dashboard to see what tests were executed, when, and how they failed, then adjust test cases to make sure bugs are caught before they leak into production.
In addition to using the Quality Dashboard of zAdviser to see what “good” looks like, managers can see what “quality” looks like, as well, with new KPIs that help organizations improve the quality and coverage of shift-left automated testing and the overall quality of software. Quality products, after all, lead to happy customers.
