Data Silo: Data on a 360° Turn

Data on a 360° turn: Has it transitioned from being a business enabler to a silo aggravator?

Industry’s commitment to analytics stopped being a fleeting thing at least a decade ago. The economics of implementing big data were compelling. In fact, as the bedrock technology for the new era of ‘doing business’, it was rightly predicted as ‘definitely here to stay, and only going to get bigger’.

Initially, many organisations wrote off big data as an effective tool as they believed that managing large volumes of data, integrating it into the database systems, and building an analytical capacity to filter data could be a daunting task. Industry woke up to the caffeinated smell of big data, and finally started considering it a business enabler. They started leveraging the aggregated new information sources in a united platform to harness abundant knowledge about customer needs.

However, those organisations have now realised that the initial speculation of ‘data becoming difficult to handle’, i.e., data silo has become a reality. Newly created worldwide digital data is expected to grow at 30% or more annually reaching 163 zettabytes (1×10 bytes) by 2025, according to IDC. It is estimated that by the end of 2025, only 15% of the data in the global data-sphere of 163 zettabytes will be tagged and only 20% of that will be analysed. Therefore, if 80% of the data created is analysed, that data will likely reach archival status upon creation. This clearly reflects the magnitude and impact of data-related problems that need solutions in enterprises.

Organisations are investing in the development of technologies, systems, practices, and applications to analyse critical business data so as to gain new insights about business and markets. This represents the section of data that needs constant access and organisations cannot downplay the role of this important chunk of data, which provides a valuable string of insights ranging from customer purchasing habits to inventory status.

On the other hand, most organisations today also have information that might never become business insight. Thus, data backup and archiving have become the topmost priority nowadays. Faced with the challenge of making sense of voluminous amounts of data and to speed up decision-making, investments in analytics has paved the way to the growth of data back-up and archiving sectors. However, recent IDC analyst surveys indicate that less than 40% of organisation have a dedicated archiving strategy in place.

Let’s look at the top reasons for organisations to have a dedicated data archiving strategy:

  • To efficiently respond to legal discovery requests and compliance audits
  • To ensure compliance with government and internal record retention policies
  • To support business intelligence and knowledge management initiatives
  • To improve primary application and storage performance

The enterprise information archiving market size is expected to grow from USD 4.31 Billion in 2015 to USD 7.55 Billion in 2020, at a Compound Annual Growth Rate (CAGR) of 11.8%. Meanwhile, the Global Data Backup and Recovery Market size is expected to reach $12.9 billion by 2023, rising at a market growth of 10.9% CAGR.

The Verdict…

Data exist and will keep growing every day. Organisations have started feeling the heat of stockpiled data, and are not in a position to write off the need for continuous data backup and archiving as a “passing fad”. Backup and archiving data is an apt way to classify, nurture, and handle the huge magnitude of data that becomes difficult with the use of traditional tools. They are the tools that would help make sense of the data, eliminate the chaos, and harvest and integrate them when needed. Data Silo is pushing the enterprises to reform their respective industries. It is evident that the organisations need to be receptive to change and learn how to handle and access required information by classifying and appropriately archiving their data. The businesses who are willing to ride on the wave of this change will survive and the others will perish under the weight of data silo.