Silos Getting In the Way of Big Data Utilization
Silos are standing in the way of big data. Whether in data sources or organizational structures, silos lead to complicated analytics and lackluster results on big data initiatives, according to a recent Capgemini Consulting report.
The report, Cracking the Data Conundrum: How Successful Companies Make Big Data Operational, is based on a November 2014 global survey of senior executives. Respondents said they felt that big data was important, with nearly 60 percent of them saying they believe that big data will disrupt their industry within the next three years.
“But putting big data to work and delivering clear-cut results is no simple task,” writes CIO Insight. “In all, 35 percent of the companies surveyed have integrated big data and predictive insights into some business operations, while 29 percent are still in proof-of-concept for selected use cases, and 19 percent have gotten to the budgeting stage but identified focus areas. Only 13 percent have integrated big data and predictive insights extensively into business operations, and 5 percent haven’t implemented the technology nor have they allocated a budget.”
Respondents also said that the projects that did implement big data didn’t deliver returns. “Only 27 percent of respondents described their big data initiatives as ‘successful’ and only 8 percent described them as ‘very successful,’” Capgemini writes. “In fact, organizations were found to be struggling even with their Proof-of-Concepts (PoCs), with an average success rate of only 38 percent.”
These results are in line with other surveys and reports, noting that big data has failed to live up to expectations. Some are using those results to condemn big data as a concept, but given the hype around big data, one can hardly be surprised.
To be fair, mixed results and doubt aren’t unusual with new technology, and big data isn’t all that different from other lauded technologies that were thought to fall short. “If we look at it in the analogy of other technologies that have come along the way—a website, then digital presence, digital ecommerce, digital store, payments, and so forth—we saw the same type of errors in the beginning of those technology trends,” Jeff Hunter, Capgemini’s vice president of North American business information management, told InformationWeek.
The biggest problem organizations have with making effective use of big data is, “Scattered data lying in silos across various teams,” the survey found, with 46 percent of respondents citing it as a problem. And certainly the silo problem has been cited before as an issue. Capgemini reports that 79 percent of organizations have not fully integrated their data sources across the organization. “This means decision-makers lack a unified view of data, which prevents them from taking accurate and timely decisions,” notes the report.
But silos within the organizational structure themselves are also an issue, due to the nature of big data, which by definition is broad and doesn’t fit nicely into company departments. If the divisions themselves aren’t communicating with each other, it’s going to be difficult to develop a big data project that can. “Big data initiatives are rarely, if ever, division-centric,” Capgemini writes. “They often cut across various departments in an organization and consequently, coordination and governance are usually significant implementation challenges.”
Because of that, many organizations have scattered pockets of analytics resources or decentralized teams without any central planning and oversight. “As a result, best practices from successful implementations are not shared across the organization, initiatives are not prioritized, and resources are not deployed in the most effective ways,” Capgemini warns.
Well, that explains why companies continue to stumble on big data initiatives but how do companies overcome the silo challenge?
Organizations that have clear organizational structures for managing big data project rollout can help deal with the silo issue, Capgemini notes. “The success rates of big data initiatives are a direct function of the structural cohesion of the lead unit.” In particular, organizations with an analytics business unit are nearly 2.5 times more successful than those that have ad hoc, isolated teams for big data projects, according to the report.
An important factor for success with such a defined business unit is its leadership, Capgemini continues, noting that only 34 percent of companies have a Chief Data Officer, or an equivalent role. “Leadership is also crucial to foster a culture of data-driven decision-making within the organization,” notes the report. Skilled workers are also difficult to find, meaning that companies may need to use nontraditional methods for hiring people in the field.
Ultimately, reducing technological and organizational silos has its own value, even beyond what big data can provide.
