By Jon Horden, CEO, iKVA 

Global data creation is projected to increase to more than 180 zettabytes by 2025 – equivalent to 23 terabytes of data for every person alive today. 

In the commercial world, organisations have seen a huge increase in the use of email, video chat, messenger services and other non-traditional channels for dispensing information, which is not readily accessible for collation and indexing. Between 80% to 90% of data generated and collected by organisations is unstructured, and its volumes are growing rapidly, much faster than the rate of growth for structured databases, yet unstructured data stores contain a wealth of information that can be used to guide decision-making. 

The financial industry has never been short of data but, until recently, much of the information generated was too complex to be meaningful and the inability to connect data across organisational and departmental siloes was a major challenge for businesses operating in the sector. 

With tougher economic conditions, the ongoing energy crisis, and a continued focus on developing sustainable and environmentally friendly business practices, Artificial Intelligence (AI) and Machine Learning (ML) will play an increasingly important role in how the financial industry develops. 

Breaking down data siloes 

The trend for hybrid and remote working, accelerated by the pandemic, has resulted in senior leaders and decision-makers using MS Teams, Zoom and other platforms to communicate, hold meetings, and make decisions. Accessing knowledge created in MS Teams, for example, is challenging, especially since one meeting can cover multiple topics.  All the tools used – iManage, email, MS Teams, Sharepoint – also have different search interfaces that require multiple and repeated searches to find information. 

Data classification has historically relied on labour-intensive, costly methods of indexing to create retrieval systems. Most current systems rely on Boolean searching, which enables users to combine keywords and modifiers to retrieve relevant information, but these often yield irrelevant results as there are large search parameters combined with a lack of meaningful context. 

AI technology can overcome this by indexing and segmenting the knowledge created and allowing it to be discoverable, providing more accurate results, increasing business compliance, and helping to reduce business risk. So, inevitably, there will be an increase in the number of organisations integrating AI-enabled data discovery solutions into their workflows to enable employees to quickly and easily discover important insights to improve business decision-making. 

Saving money and the environment 

Around 90% of the unstructured data generated by a company is never analysed and may be classified as ‘dark data’; common examples of dark data include old versions of documents, analytics reports, and transaction histories. Dark data is hidden within an organization’s internal networks and represents a significant volume of knowledge that could be harnessed to provide high-value results. With the growing popularity of cloud storage, it is easy to continue to generate and store data that is then disregarded. 

However, the cost of storing this unused data is immense – both environmentally and financially. Storing data in vast server firms uses a tremendous amount of power and energy, contributing to excess carbon dioxide (CO2) emissions. It has been estimated that 6.4m tonnes of CO2 were released into the atmosphere in 2020 to power the storage of dark data, producing more carbon dioxide than 80 different countries do individually. Using AI and ML technology to discover and categorise available data for analysis will enable organisations to retain what is necessary, provide more visibility of the knowledge within the company, and reduce the need for energy-intensive storage systems. By reducing the amount of unused data being stored, an organisation can reduce the amount of carbon it is generating for data storage. 

Managing data to reduce excessive energy consumption will become a moral imperative for businesses across the globe and more firms will take steps to implement solutions to reduce emissions and improve operational sustainability. 

In addition to the environmental benefit, there is a clear financial value to reducing the amount of data being stored by a business – the more energy required to power storage systems, the higher the electricity bill. Navigating an evolving financial landscape, and weathering tougher financial conditions, will incentivise businesses to implement AI-enabled data discovery tools for a three-fold financial benefit: decreasing energy costs, reducing business risk by unlocking insight from unused data, and improving operational efficiencies. 

Increasing human understanding

The volume of knowledge being generated and stored as unstructured dark data is beyond human capabilities to digest and discover. With the global economy facing recession, it is clear that AI technologies can help organisations to drive cost and operational efficiencies that will help their businesses to weather the economic downturn.