In this article, we’re giving you an overview of the most recent DevOps reports from Puppet, BMC, the DevOps Institute, and Google Cloud. Then, read on for the steps successful organizations can take in adopting DevOps. Jump to what interests you:
- Puppet’s State of DevOps
- BMC & Puppet discuss the State of DevOps
- The DevOps Institute
- Google’s Accelerate Report
- Steps to starting DevOps the right way
(Learn all about DevOps in our multi-part DevOps Guide.)
Puppet’s 2021 State of DevOps
Every year, Puppet reports on the State of DevOps, and it recently released its 10th edition, sponsored by BMC and other industry leaders. Over the last decade, the reports have collectively polled more than 35,000 technical pros, and its annual results hold a lot of sway.
(Source)
For the 2021 report, Puppet received 2,657 responses from around the world, with the majority—32 percent—falling into the 1,000 to 10,000 employee category and 40 percent in the tech sector.
The report, issued less than a year since the last one, looks at the historical evolution of DevOps and highlights the importance of automation, cloud, and people and culture to its success. Puppet notes that DevOps has evolved to the point that:
“many of the teams that are ‘doing DevOps’ well don’t even talk about DevOps anymore—it’s simply how they work.”
And yet, for the organizations still stuck halfway through their journey, it comes down to getting a better handle on technology and people in order to keep moving forward.
Puppet finds that across that middle group, the blockers and challenges change from a mix of technical, culture, and organizational at the low end to almost entirely cultural (organizational buy-in, risk aversion, imperfect feedback loops, sub-optimal team definitions, etc.) at the high end.
The report also level sets by examining the history of DevOps since it began issuing its reports, and looks at the big picture of what DevOps is and isn’t.
- DevOps is not just automation. Highly evolved firms are far more likely to have implemented extensive and pervasive automation, but being good at automation does not make you good at DevOps. Sixty-two percent of organizations surveyed say they’re stuck in mid-evolution on their DevOps journey despite high levels of automation.
- Almost everyone is using the cloud, but most people are using it poorly. Organizations should not expect to become highly evolved just because they use cloud and automation. While 65 percent of mid-evolution organizations report using public cloud, only 20 percent of them are using cloud to its full potential.
- Team identities and clear interaction paradigms matter. Enterprises are held back from evolving to the highest levels by organizational structure and dynamics. Organizations that are “good at DevOps” have teams with strong, well-understood identities, clear responsibilities with a high degree of autonomy, and well-defined communication channels with other teams.
Other key findings include:
- While cultural blockers such as organizational resistance to change, legacy architecture, skills shortages, limited-to-no automation, and unclear goals or objectives still exist for mid-evolution firms, they are most acute for low-evolution organizations.
- The presence of “DevOps teams” is confusing for the industry and many organizations, and in most cases doesn’t help organizations evolve. Puppet finds that organizations with less ambiguous team names and more clearly defined responsibilities are far more likely to have a higher performing IT function.
- Good security practices and better security outcomes are enabled by DevOps practices. As DevOps practices improve, DevSecOps naturally follows.
- Platform teams can accelerate DevOps transformations by leveraging existing automation, and the most highly evolved organizations in Puppet’s DevOps models are adopting a platform model that enables self-service for developers and curates the developer experience.
- Modernizing older (legacy) IT infrastructure and applications requires analysis, sorting into easily understood categories, and setting explicit goals and action plans.
DevOps webinar with Puppet & BMC
While “many of the teams that are ‘doing DevOps’ well don’t even talk about DevOps anymore—it’s simply how they work,” Puppet found that other organizations are mired at the halfway point.
In a follow-on webinar The State of DevOps: Stuck in the Middle, BMC Product Manager Jon Stevens-Hall spoke with Puppet CTO Nigel Kersten about the report and how organizations need to get a better handle on technology and people in order to keep moving forward. Key takeaways from their conversation include:
- How size affects DevOps success
- Defining legacy for your organization
- Getting deployment frequency right
- Management’s key role
Size matters in DevOps
First, they talked about how size matters when it comes to DevOps.
“DevOps in large enterprises is very different from the DevOps in smaller companies and startups, and that seems to be a big theme of the state of DevOps report,” says Stevens-Hall.
“The people I was meeting [at conferences] back in 2015 were smallish companies and the biggest and most prominent company…was Spotify…who…have been a very big pioneer in terms of team organization and things. Just four years later…it was banks, it was telcos. They felt they’d done agile development, but the ops side hadn’t really emerged as agile.”
“The most interesting thing for me was that [they said] legacy isn’t going away…overnight. [The bank said] a lot of their transactions [might] involve the newest technology, but they also involve a lot of legacy products that can’t, or won’t be changed for years…they need new work practices, new technologies to [meet] that challenge of allowing DevOps to thrive amongst all this governance and complexity.”
Legacy is a variable term
Kersten adds that “legacy” is also a variable term. “When customers start to talk about the legacy environment, it’s not all the same. It is not actually the oldest stuff they have. Often there’s these old mainframe systems, which are really well run by small teams that operate in a very, very DevOps mindset just without sharing those practices across the rest of the organization,” he says.
“Legacy often tends to refer to those half-baked applications built in the nineties and two thousands [by people] who are all self-taught [so] there wasn’t a lot of discipline. Everyone was working this out as they went along. I think we have this whole generation of applications that are much harder to monetize than many of the things on the mainframes…And I think there’s many categories of what kinds of problems people have within legacy.”
(Learn how to succeed at application & software modernization.)
Getting deploying frequency right
In addressing the report’s categorization methodology, Kersten explained, “We introduced this idea of low, medium, and high evolutionary groups…deployment frequency at the low level [occurs] monthly [to every] three months, six months to get a deployment from done to actually shipping value to an end user, medium is between daily and weekly, and a high group is on demand.”
“Deployment frequency [can] be a vanity metric if people focus on it too much [but] there is something transformative for the business moving from monthly to every couple of days, it’s a really big jump.
“[When] we look at the data from the survey itself…there’s relatively low-hanging fruit in most organizations to move from low to medium. And then there’s a relatively concrete set of tasks to move from medium to high. The low group is getting smaller. The high group is getting larger, but there is this massive group of nearly 80 percent of folks who aren’t getting out of the middle area. And so that’s what we really wanted to focus on this year.”
“[None] of these [challenges] are technical problems…the things inhibiting large organizations [are] the people interactions and the team interactions and communication and all of [these] socio-technical systems. And when we make changes to the people side, it has an influence on the technology and vice versa…and it’s really inhibited…a lot of progress for people.”
Stevens-Hall points out that the current complexity faced by many organizations adds to that challenge:
“[To] get people out of that stuck-in-the middle state…the technology department in the enterprise can [ensure] that we work with that tool chain and the owners of that tool chain. [We] can also [bring] this into the service management system [and] people from the service organization can be working with people from the DevOps team and…they can be sharing information, creating knowledge,” he says.
Management must get involved, too
Kersten has tips for management, too. “From the…operations perspective…if you’re in management, the most important thing you can do is make sure your teams have an obvious purpose and identity [and] have well-defined interactions between them,” he explains.
“It’s all about collaborating across functional boundaries, but it turns out collaboration is actually a very time-intensive and inefficient way to operate at scale. And if you are a relatively small service management team [or] operations team with hundreds or thousands of developers there, you can’t possibly collaborate with them all.”
“The more evolved organizations are the ones that share practices and create spaces for it. As long as you don’t call it a center of excellence, which tends to be the death knell for actual progress for most environments.”
Kersten recommends some actionable tips:
- “[Call] it something else [like a] community of practice…something a little less hierarchical and autonomous and authoritarian.
- [Create] spaces where people can just listen and learn and share. I think people are genuinely pretty smart and generally pretty engaged.
- And if you give them the space to find out about solutions, they’ll implement them.”
DevOps Institute
Next, let’s take a look at the DevOps Institute Upskilling report. The DevOps Institute conducted the study researching the necessary skills a DevOps human must possess for a second year in a row. The following report goes into detail sharing the result of more than 1,260 individuals globally surveyed who identified which skills are considered critical to DevOps and digital transformation.
Some key findings include:
- DevOps topologies primarily used today are a huge challenge.
- DevOps transformation journey is still very difficult for more than 50%.
- Agile, DevOps, and ITIL are getting strong competition from SRE.
- The DevOps human as a hybrid job and role.
2021 Accelerate State of DevOps Report
Finally, we have the Accelerate State of DevOps Report for 2021, which comes from Google Cloud’s DevOps Research and Assessment team—aka DORA. The survey has been conducted over seven years. In total, more than 32,000 technical professionals from around the world have taken part.
Historically, their research looks at four key software delivery metrics to determine how well DevOps teams are working. This year’s research, however, bore out an additional fifth metric for DevOps teamsto measure success:
- Deployment frequency
- Lead time for changes
- Time to restore service
- Change failure rate
- (New) From availability to reliability
Beyond delivery metrics, the report also found many new DevOps trends, and these are some highlights:
- A healthy team culture mitigates burnout during challenging times (as we’ve all experienced recently).
- The highest performers continue to raise the bar.
- SRE and DevOps complement each other. (Learn more: SRE vs DevOps)
- Cloud adoption drives performance.
- Securing your software supply chain is essential—and it drives performance.
- Good internal documentation is a springboard for successfully implementing DevOps.
Steps to successful DevOps adoption
Now, with all these takeaways from industry research, let’s turn to creating the right foundation for DevOps. Year after year, organizations use these steps as the foundation for their DevOps adoption.
Pre-Step: Build a foundation
It’s kismet that the transformation of DevOps starts with its own evolution, beginning with a foundation that can truly support DevOps through and through. In this pre-stage development, company officers and other stakeholders will:
- Realize a need for better communication and collaboration.
- Begin rapid fire approvals to implement various pieces of technology to fill the gap.
While this process may begin hastily, it’s not something that can be accomplished quickly. Over time, businesses begin to fine-tune their approaches and customize technology to meet their goals. Eventually, a platform results that can facilitate DevOps throughout the organization. When this occurs, organizations are at a stage where they share ideas, technology, knowledge, and metrics.
Indicators of this phase? Companies who are…
- Making large software purchases
- Implementing new practices quickly
Step 1: Normalize tech stacks
In the first formal step of DevOps evolution, companies begin to normalize their stacks of technology. As teams organically choose agile practices that suit them and begin considering new methods, you’ll be able to identify this step in the process.
An important tech marker of this step is version control. Teams implement it and other practices that are considered early stages of continuous integration. Normalizing tech stacks may also look like reducing redundancy in the infrastructure or refactoring applications. The need to reduce redundancy comes in the forefront in the next part of the process, but it begins in phase one.
Step 2: Standardize
As eliminating redundancy comes into focus, teams implement more practices geared at reducing variance in a tech stack and standardizing it. In this stage, DevOps teams will limit the number of OSes as a form of consolidation.
Here, teams independently consolidating have the opportunity to collaborate. The overarching goal is a standardized family of technologies that work hand in hand to a foster collaboration and development effort.
This phase results in less overall complexity, which gives teams a greater opportunity to work across multiple applications, making the best use of their expertise. Benefits of this phase include:
- Faster application deployment
- Error reduction
- Service quality improvement
For these reasons, standardization is a key figure in the state of DevOps.
Step 3: Try out DevOps practices
During the previous phase, standardization occurs while teams are exploring the inner-workings of their platform technology. Plus, they’re really getting to know the system. In the DevOps practices phase, teams transition from simply exploring the system to being able to make recommendations and exploiting it for DevOps.
This is a good time to evaluate pain points. Typically, this deep-dive approach requires organizations to acknowledge any struggles at the deployment phase. Deployment can be tricky for many organizations that haven’t flushed out DevOps best practices.
A common issue is that the improvements made in previous steps of the evolution have caused developers to move toward deployment faster than the deployment process can support. This causes operations to bottleneck. Addressing it swiftly by implementing additional DevOps practices at deployment is critical. That includes reusing the same deployment patterns over and over, and testing deployment prior to reaching this stage.
Step 4: Automate infrastructure
With high-priority outcomes identified, automation helps teams become even more successful.
The high ticket items most often include provisioning and systems configuration. When teams automate infrastructure delivery, they solve the problem that occurs when developer throughput outpaces deployment. That’s why this is a crucial next step after DevOps practices. Moreover, automating configuration helps teams deploy software faster.
Automation is an important precursor to self-service. It’s the catalyst for the final step in the DevOps process. It stimulates self-service, leading to greater efficiency across an organization.
Step 5: Self-service
In the self-service phase, IT practice occurs throughout an organization and may not be limited to one cost center. This allows for advances within the organization that streamline self-service.
Collaboration in this final phase multiplies the benefits in previous steps. The most significant advances are actualized when application automation transcends standardization and evolves to include cloud migration and other higher level processes. Security also evolves from simply meeting the immediate needs of the team to create a baseline for compliance throughout the organization.
Making sense of DevOps
Keeping track of research on the “State of DevOps” can help IT organizations benchmark themselves and look for opportunities for improvement based on the feedback of others. The outlook for DevOps (and SRE) in 2022 is bright as application delivery and release automation becomes increasingly vital to a successful organization.
Related reading
- BMC DevOps Blog
- BMC Workload Automation Blog
- DevOps Guide, with 30+ articles
- The Complete DevOps Certifications Guide 2021-2022
- Workload Automation (WLA) vs Robotic Process Automation (RPA): What’s The Difference?
- How To Set Up a Continuous Integration & Delivery (CI/CD) Pipeline
These postings are my own and do not necessarily represent BMC's position, strategies, or opinion.
See an error or have a suggestion? Please let us know by emailing blogs@bmc.com.
