There is a lot of optimism about the ability of numbers to drive businesses. However, even the most digitally savvy businesses may find that this optimism is tempered by the data integration challenges that are essential to the decision-making process.
Even with the rise of cloud capabilities and toolsets, enterprises are still struggling to find a balance with data integration. Outdated tools and technology make it difficult for enterprise users to get timely information, according to a study by TDWI. "Without good data integration, it is difficult for businesses to make informed decisions, achieve smarter customer engagement, and enjoy agile and efficient operations," said DavidStodder, director of research at TDWI and author of the study. Data integration technology is critical to the speed with which companies can respond to customer and market needs, but the survey shows that many are not yet there."
Traditional ETL remained the most commonly used data integration method, with 73% citing it. Another 35% moved to live replication and/or change data capture. Less than one in five (18%) are using data virtualization or data federation. There are still too many spreadsheets being used to try to piece together information. As a result, 57% of enterprises say their data integration efforts are too slow to keep up with business priorities.
The shift to cloud-based infrastructure has seen an emphasis on data integration, the survey added. The majority of respondents (79%) plan to scale up the data curation currently used for business intelligence and analytics on cloud platforms. Only 1 percent plan to reduce this use. "Many enterprises are shifting the focus of data integration to quickly deliver data from new sources and legacy systems to cloud platforms," Stodder noted. Increasingly, the cloud supports faster and more comprehensive integration approaches, including automated data pipelines, real-time data streams, and data lake libraries that integrate cloud data warehouses and data lakes.
The results of this survey point to some general trends related to migration to the cloud:
There is a strong need for a centralized data integration architecture. Having a single data architecture was a goal supported by 72%, who agreed that a centralized integration platform offered the opportunity to "provide more choices for managing increasingly diverse data structures, end-user types, and business use cases." Only 11% felt that a centralized environment would create more problems "because of the complexity of the final architecture and the work required to build a data lake and/or a modern data warehouse."
Cloud-based data integration means more than analytics. Stodder noted, "For some enterprises, moving part of the digital transformation of business applications and operations to the cloud is at least as important as analytics. Digital transformation often increases and enhances the role of data; Both users and automation capabilities need continuous and timely data to drive more informed decisions. Data integration must be agile to meet day-to-day operational data needs, as well as data science and analytics needs."
Some organizations still prefer to do their analysis locally. Security and availability remain the top concerns when it comes to critical data assets. As a result, Stodder said, many executives are concerned about hacking, unauthorized access, and potential service outages on cloud provider platforms, even as these platforms show increasingly reliable security and availability. If these concerned organizations use the cloud, they often prefer a private cloud arrangement.
Many organizations have made a big commitment to cloud technology, thinking that analytics is the most important driver. Of these respondents, four out of five (80%) owned cloud data warehouses and 65% owned cloud data lakes. Nearly three in five (59%) expect their cloud-based data management to grow significantly over the next 12 months.
No comments