Data fabric refers to the unified data management architecture and the set of capabilities that provide consistent capabilities to conveniently connect data endpoints and enable end-to-end data management capabilities.
Let’s take a look at the data fabric architecture.
What is data fabric?
Data assets are generated in silos and hidden across the hybrid mix of infrastructure environments. Data preparation cycles are long, and users need a broad set of data management capabilities to overcome the limitations facing complex multi-vendor, multi-cloud, and evolving data environments.
The data fabric architecture is designed specifically to address the challenges facing the complex hybrid data landscape.
Essentially, data fabric can be described as a converged platform supporting the diverse data management needs to deliver the right IT service levels across all disparate data sources and infrastructure types. It operates as a consolidated framework to manage, move, and protect data across multiple isolated and incompatible data center deployments.
As a result, organizations can invest in infrastructure solutions that align with their business requirements—without concerns surrounding data service levels, access, and security.
Data fabric capabilities & principles
While traditional data management concepts such as DataOps are focused on the operationalization of large and distributed data assets, the Data Fabric is focused on capabilities that unify diverse and distributed data assets.
In simple terms, most organizations adopt frameworks such as DataOps to design, implement, and maintain a distributed data architecture. It helps make sense of data that is generated and maintained in a highly distributed infrastructure environment. By introducing a unified data management platform architecture such as Data Fabric, the end-to-end data management processes are combined, specifically:
- Data integration
- Data discovery
- Data governance
- Data curation
- Data orchestration
(Learn about BMC’s approach to DataOps.)
All tasks are managed within a single platform architecture designed to simplify access, management, and control over distributed data assets. A Data Fabric can include an array of data management capabilities across the following logical domains:
Knowledge, insights & semantics
- Semantic layers of descriptions that enable users to discover and access relevant data
- Access to a vast pool of data assets in the form of a marketplace
- Continuous analytics over growing data assets
- Use of advanced AI systems to connect business relationships between data across disparate applications.
- End-to-end data management visibility to measure various attributes and risk associated with data
Unified governance & compliance
- Local management of metadata in compliance with global organizational policies that can be applied to all data assets
- Automation makes it easier to apply policies, audit compliance, and identify potential breaches in systems
- Automation and AI capabilities augment data tracing and route querying
- The overall data governance and security process is centralized and consistent across all environments
Intelligent integration
- The design, deployment, and utilization are integrated across distributed data and infrastructure environments
- Automated flow and pipeline creation for the siloed data environments
- Optimal workload distribution and correction of schema drifts
- Self-service ingestion of new data assets within predefined policies
- Future proofs infrastructure; agnostic to platform and applications
Orchestration & lifecycle
- Self-service orchestration of disparate data sources using advanced AI systems, data lakes, and other platforms and technologies that ensure a comprehensive view of the data pipeline across all data environments.
- Unified data lifecycle to configure and manage all aspects of the data including development, operations, testing, and production release of data-driven applications.
Data Fabric: The value proposition in multi-cloud environment
Data Fabric architecture is particularly useful in IT environments that involve dynamic data workloads distributed across geographically distributed infrastructure systems. Here’s how the Data Fabric architecture delivers value in the modern cloud-based enterprise IT setting:
Operate a true hybrid cloud
When organizations invest in cloud storage and infrastructure solutions, their choice is based on cost, security, availability, scalability and service requirements. These requirements evolve over time, forcing them to either:
- Switch vendors
- Pursue different cloud models as viable alternatives
Cloud vendors, on the other hand, tend to lock customers into their service, making data migration a costly and challenging endeavor for their customers.
Data Fabric allows organizations to overcome the technical challenges in maintaining a diverse portfolio of data storage and infrastructure deployments. Customers can leverage the freedom to operate mission-critical data-driven IT services, apps, storage, and access from a range of hybrid IT infrastructure resources based on changing technical and business requirements.
Seamless cloud compute transitions
Data Fabric is designed to mitigate disruptions from switching between cloud vendors and compute resources to process data stored in disparate locations. As a result, Data Fabric reduces time to insights dramatically. With faster insights, organizations can:
- Identify patterns within data
- Understand trends
- Make decisions proactively
Better decisions through improved compute performance across all channels of data ensure that businesses leapfrog market competition while making the most of their data investments.
High performance & optimized data investments
Organizations invest significant resources and efforts into delivering the best performance for their apps and services. This is especially true for mission-critical apps that may be required to process a growing volume of data as the user base grows or to accommodate unpredictable peak usage demands.
In order to address these demands, organizations must also invest in cloud storage solutions that promise the desired performance levels. Similarly, the app or service may evolve into a legacy solution of tomorrow and the usage requirement may reduce accordingly. Either way, the app should be capable of delivering predictable performance whether the data is available at either:
- A highly available storage location
- A low-cost economy storage infrastructure
With Data Fabric, organizations can realize this capability and optimize their data investments based on evolving app usage requirements.
(See how usage requirements determine cold & hot storage.)
Future proofing & the flexibility to evolve
Data Fabric allows organizations the flexibility to adapt their infrastructure based on changing technology needs. Data Fabric makes it easy to connect various infrastructure endpoints to the consolidated and unified data management framework. Organizations don’t have to worry about the explicit location of the data.
If a startup firm invests in low-cost cloud storage solution until the company gains traction and needs highly available storage capacity at a later stage, it can leverage the flexibility that Data Fabric offers to choose infrastructure environments that best fit the nature of data and business requirements of the organization on an ongoing basis. The data management functionality will apply across all infrastructure deployments and organizations can future proof their data investments accordingly.
Benefits from data fabric
The Forrester New Technology: Projected Total Economic Impact 2020 study reveals the following business value of capabilities that make a unified Data Fabric architecture:
- 459% increase in Returns on Investment
- $5.8 Million business benefits on average
- 60X accelerated data delivery time
- 20X faster customer affinity analyses
While these numbers are specific to the case example of the IBM Cloud Pak for Data, Data Fabric capabilities are relevant to all organizations running multi-cloud environments. Data Fabric offers organizations a range of business value propositions by addressing the technical challenges of operating data services in a multi-cloud and hybrid IT environment.
