The DevOps journey started with the idea of removing barriers between Development and Operations. Today DevOps has evolved into a set of practices and tools that are automating all aspects of application development as well as how the applications operate in production. Now companies are implementing advanced approaches using artificial intelligence (AI) techniques such as machine learning (ML) to maximize value. In this blog, I’ll share some new research from Enterprise Management Associates (EMA) and perspectives about the future of workload automation in DevOps. We’ll look at how enterprises are benefiting from combining AI/ML, DevOps and automation to reach new levels of productivity, quality, and reliability.
In its infancy, the Dev part of DevOps got most of the focus, with major investments made in continuous integration/continuous development (CI/CD) and infrastructure automation. In the last five years, the Ops side of the house has received its due focus, and the emergence of Workload Automation as part of DevOps practices is particularly noteworthy. This practice involves automating and orchestrating complex and interdependent application workflows and data pipelines to deliver critical business services in production. In fact, new research from Enterprise Management Associates (EMA) reveals 80 percent of enterprises have integrated the automation of application workflows into their DevOps processes instead of dealing with it in an ad-hoc style at the end of a release cycle.
If you are among the 80 percent of enterprises that have already integrated application and data workload automation into your DevOps initiatives, you don’t need me to explain the benefits. The results are clear. Building workload automation into applications, data pipelines, and business services when they are being created already has helped many organizations reduce their development cycles and time to production, increase reliability and performance, and free up time that can be spent creating new digital business capabilities instead of maintaining existing assets. Now we’re learning – from experience and new research – that those benefits can be enhanced through next-generation DevOps automation that infuses AI/ML into the process.
Let’s start by looking at the current state of workload automation in DevOps, which is itself pretty exciting. Today, 73 percent of companies are automating workflows while applications are being developed, according to the recent EMA study. This is often referred to as Jobs-as-Code. The automation of application workflows and data pipelines by treating jobs as code has become a mainstream practice in just a few years. This is a testament to the proven value it provides, and how easy it is to adapt. Jobs-as-Code is practice that was pioneered by Control-M from BMC. Many of its early adopters such as Carfax were instrumental in proving its value to the market.
“DevOps is part of our culture and has been a very long time now. We’re a very time-to-market company, you really can’t afford to wait. It used to take weeks or months to get a process through development and into production. As we’ve moved more to the Jobs-as-Code, DevOps, open philosophy, we’ve really reduced that timeframe. We can actually go down to minutes.”
Robert Stinnett, Carfax
How will DevOps automation continue to evolve? It will become smarter, with help from AI/ML. While building application and data-pipeline automation into the DevOps process may have been a big leap for many organizations, introducing AI/ML should be more like a series of incremental steps. Leading workload automation products today provide policy-based automation, where automation engineers can set rules based on analytics from the workload automation tool’s own metadata.
For example, a policy for times of peak processing where compute utilization exceeds a threshold would have the workload automation product interface with an infrastructure automation solution to provision additional compute and process capacity there, and then terminate the provisioned resources when processing is complete. The natural evolution of this will be for workload automation solutions to have embedded AI/ML algorithms that can self-learn such patterns and then recommend remedial actions, which will take the system closer to a self-managed concept.
Sound too futuristic? Not to us, or to EMA. Analyst Steve Hendrick wrote: “ML will drive improvements in quality, which will drive wide-ranging benefits on application performance, reliability, OpEx efficiencies, cost savings, and customer satisfaction. AI will follow as a second wave of innovation built on ML that will enable expansive automation. This automation can mean improved speed in delivering new applications and updating applications, and improved employee productivity—all of which reduce SDLC cycle time and leave more time for addressing higher-order development work.”
The majority of IT operations professionals EMA surveyed believe introducing AI/ML to DevOps will improve application performance, application innovation, and design, and overall business revenue growth. Many developers agree, and the majority believe AI/ML will speed new application delivery. Both groups cited many other expected benefits.
The market isn’t quite there yet though. While many enterprises have placed high priority on using artificial intelligence and machine learning, those that are exploring their use for workload automation in DevOps are encountering some obstacles. Of course, there are the obvious suspects that typically tend to limit innovation – difficulties in setting strategy and finding skilled staff. But EMA’s research also found that, according to developers and IT operations staff, a specific challenge to further automating DevOps is finding tools with AI/ML capability, and integrating them into the toolchain and processes. Look for future versions of workload automation solutions to have more built-in cognitive technology features to address these needs.
Bottom line, the application and data pipeline workload automation market is continually evolving to meet enterprise needs, and Control-M has been at the forefront. The current desire for artificial intelligence and machine learning provides a clue to where the market may go next. Stay tuned, and until then, visit the Control-M website to browse other analyst research reports, learn about the Jobs-as-Code approach, and see testimonials from customers that have brought workload automation to DevOps.
Want to learn more?
- Read EMA’s complete report – AI & ML: Driving the Next Generation in DevOps and Workload Automation
