“Maximizing performance” is a phrase you hear get slung around a lot. It’s often spoken of in a way that implies maximized performance is some end goal only a few steps away. The reality is that maximizing the performance of anything, be it an Olympic gold medalist sprinter or an IT business system, is an ongoing process that can never be truly finished. The moment you stop trying to improve is the moment you lose the race.
The pursuit of optimization in the IT world is also an ongoing process that makes use of a vast variety of tools. Automation, Big Data, and machine learning are just a few of the spanners at hand that have proven to be incredibly powerful over the past decade. These are tools that can be applied to any number of systems and situations. These tools are used as a foundation for the creation of more specialized solutions.
One of the most recent solutions created using modern technologies is known as collective intelligence benchmarking.
What is Collective Intelligence Benchmarking?
If we break down the term “collective intelligence benchmarking”, it becomes much easier to grok. The idea of “collective intelligence” is a fairly simple concept: gather intelligence into collections of related data from multiple organizations or parties throughout the industry. “Benchmarking” is the act of comparing the real-time performance of a system with a preset baseline of expected performance.
Combining data from multiple sources creates a richer dataset that can then be used for establishing a baseline “benchmark” that takes more factors into account to be more reliable. Applying this collective intelligence approach, a benchmark can be established, weighing your live performance against it. This allows you to keep a finger on the pulse of all your IT environments by giving you real-time, actionable data that tells you whether everything is working as intended and meeting the benchmark or when processes are starting to fall behind.
According to Gartner, the originator of the idea,
Collective intelligence benchmarking (CIB) is an emerging and increasingly popular approach being adopted by monitoring tool vendors to ascertain a baseline of performance for a given service or application, based upon aggregated (often network) data from hundreds/thousands of end users. The aggregated data can be internal or external, although shared datasets from third-party end users is anonymized.
Internal vs External CIB
CIB lets you compare your live environment performance with baseline expectations (benchmarks) that are created from collections of historic performance data gathered either from your own organization (internal) or from other companies (external). With internal CIB, your organization’s historic performance data is aggregated into a collection used to create a realistic performance benchmark for insightful comparisons.
The other type of CIB, external benchmarking, uses aggregate data from participating organizations across the globe. This collective intelligence pulls from a much larger pool of knowledge to create even more accurate and realistic benchmarks. The data is anonymized to prevent any protected information or trade secrets from getting out while still providing access to that organization’s metrics while giving them access to the network of other organizations’ datasets.
External CIB provides a clear advantage for general performance comparison in terms of total data points from a variety of situations and environments. From a data analysis point of view, the most effective benchmarks are ones that most closely match real-world expectations of performance. External benchmarking looks at historic performance data from around the world that is relevant to specific services and applications so you know you are always comparing apples to apples.
Conversely, internal CIB can be used for comparing the performance of systems within your organization to each other. The benchmarks derived from your organization’s historic data can be applied in a more precise way to help you determine if certain departments or specific live environments are falling behind in some way.
Uses for Collective Intelligence
Both internal CIB and external CIB can be used for a multitude of purposes related to benchmarking or otherwise:
- Resource Usage Comparisons – Collective intelligence data can be used to compare more than uptimes and transfer speeds, but also whether the performances offered by your IT environments are being provided at the expense of excess resources due to poor optimization or implementation.
- Event Correlation – CIB data can be used to track and analyze things like service outages across your organization or even throughout the industry to provide insight into performance issues of a shared Internet infrastructure or commonly used applications.
- Network Monitoring – Perhaps the most obvious use of CIB, benchmarks from aggregated datasets provide an excellent baseline comparison that can be used for creating automated alerts when performance dips below a realistic performance expectation.
- Unified Communications – CIB can be used to analyze UC performance by comparing KPIs related to connection stability, voice quality, service availability, and even overall user satisfaction gathered through the use of quick surveys.
CIB is a method with a vast array of potential applications, many of which are still to be discovered, thanks to how nascent the technology is. Leveraging CIB allows organizations to gain improved visibility of their IT environments by providing them with alternative lenses through which they can view their systems. This new perspective can provide insights that would otherwise likely prove elusive.
CIB does come with its own set of obstacles that organizations looking to implement it should consider. One drawback of external CIB is that it is heavily dependent upon multiple organizations participating in the pooling of datasets. If the tool you use doesn’t have enough participating members in the external data collection process, the conclusions derived from that data may be easily skewed by outliers due to the relatively small size of the dataset.
Making the most of the still-emerging technology that is collective intelligence benchmarking will require skilled IT professionals and continued innovation. Additionally, not all CIB systems are created equal especially when considering external CIB. Participation is a vital part of creating more insightful and realistic datasets for real-time comparisons.
Pooling everyone’s data into a single, anonymized collection of datasets can provide everyone within the system with deeper insights into their own systems. This is the pursuit of CIB and why people are so excited by it.
