By Junta Nakai, Global Industry Leader, Financial Services and Sustainability at Databricks

Historically, competitive advantage in the financial services industry has been obtained through scale and capital i.e. how many branches you have and how much money you have. However, this notion is becoming ever more outdated as we transition into an increasingly digital world, further accelerated by the pandemic. Data underpins digital transformation and when addressed effectively, improves business performance. Data is now a business’ greatest asset, with around 70% of all financial services firms globally already using data-intensive machine learning (ML) to predict cash flow events or detect fraud. As data becomes more prolific across financial services, those organisations that survive, thrive and innovate  will be those that create a sound “Data Culture. This means rethinking the kind of talent you need to attract and retain, and providing them with advanced tools that bring data and people together.

Switching from “data-supported” to “data-driven”

Without simple, open, and easily accessible data, financial organisations are going to be left behind. However, embedding a data culture with these core principles in mind is by no means an easy feat, given how financial firms have been run historically. Start-ups are commonly well versed in dealing with risk and have an increased appetite to experiment. It’s the safe space for iteration and improvements that enable them to innovate. Larger banks and financial organisations on the other hand are designed to not fail and avoid high risk situations. Driving a new kind of ‘start-up like’ data-driven culture at an incumbent institution is a big departure from how financial organisations have been run in the past, but a necessary change.

Many larger banks could look to challenger companies, like Tide, for example, who quickly and efficiently built a data culture from the ground up. The UK-based financial services firm scaled its team from 0 to 35 in two years, having had no artificial intelligence (AI) systems previously. The team created an AI platform using Databricks’ Lakehouse architecture to make it simple for its data scientists to collaborate, as well as make use of streaming capabilities that allow near-real time applications. The lakehouse architecture facilitates cross-functionality with ML engineers and data scientists, allowing them to work autonomously on models from concept to production, as well as enabling them to access any data sources that are necessary and relevant to their job. This is a prime example of how the right talent coupled with the right data architecture is rapidly scaling AI across the business.

So, how do leaders in the finance space learn from examples like the above and how do they even start when it comes to creating a data culture? Firstly, they must consolidate their data into a modern architecture such as a data lakehouse. The lakehouse is currently the most effective data architecture, efficient at storing, cleaning and analysing data. Unlike other architectures, a data lakehouse allows data teams to collaborate across the entire data and AI workflows from one platform. Here, the data will be open, easily contextualised and accessible. Secondly, leaders must define who has access to that data, enhancing data fluidity across the company. In many situations, data can be isolated from many people within a company and controlled by a select few, but different users require access to different data points and it’s important to do that correctly to not fragment data or cause data drift (unexpected or undocumented changes to data structure). Lastly, they need to align the data strategy to the business, all while following security and privacy protocols. Leaders that actively encourage employees to embrace a data culture, and give them the tools to do so, will ultimately inspire collective innovation.

These changes need to come from the top. Leadership must prioritise investment into, and the use of, the most powerful IT tools available today, namely, the cloud, data analytics and open source. Investment into these fundamental building blocks will allow financial firms to bring people and data together in the most productive way. Data culture isn’t about hiring that one person that will revolutionise the company, it’s about enabling new types of data roles to flourish across the organisation. To accelerate this, leaders should look at peers that are more advanced in their data and AI journeys and learn from them. For example, it’s worth carefully reviewing the job postings on the sites of innovators in the space, as well as the number and types of jobs that they recruit for. What insights can be drawn from the detailed skills, culture and other aspects important to the organisation?

What would a data-driven financial organisation look like?

Data has already revolutionised the way banks are detecting fraud, for example.Most financial organisations today employ historical analysis on transactional datasets to find patterns to weed out fraud. It works, but it’s  backwards looking, rigid and relies on narrow data sets. Fraudsters are constantly innovating and finding new ways to commit fraud. Therefore, financial organisations need to be just as agile, and must use real-time analytics and AI to stay a step ahead. The most advanced firms today are bringing in alternative data, such as geospatial data, to augment fraud detection in real-time. Instead of using blunt instruments like hard-coded fraud algorithms,  financial organisations can combine real-time transactional data analytics with geospatial data. Afterall, identifying abnormal patterns can only be achieved with the ability to first understand what normal behaviour is, and doing so for millions of customers is a challenge that requires data and AI combined into one platform.

Not only this, the right architecture allows for truly personalised AI. As in the example above, data and AI enable organisations to better understand customers spending behaviours in terms of who they are and how they manage money. There is no ‘standard procedure’, but rather an individual experience for each customer, born from their own behaviour. This framework can be used in all sectors of the business, from fraud detection, to risk management, to environmental and social governance (ESG).

Conclusion

Many parallels can be drawn between a bank with a strong data culture and a bank with a strong corporate culture. AI and corporate culture are similar in the sense that everyone talks about it, its virtues are written up everywhere, but no one really knows what it means unless there is clear action and leadership behind them. The difference between a bank with and without a data culture is the same thing as the difference between banks with or without a strong corporate culture. One bank lives its values. One bank does not. The same goes for AI. In the long run, that has significant implications to every part of a bank’s operations from recruiting to risk management. When you look around the world today at financial organisations, banks with strong corporate cultures continue to recruit the best people and make better decisions on average. Combining this with strong data cultures means organisations can both find new ways to innovate using its most powerful assets (data) and recruit and retain the best tech talent (people) to enjoy continued growth.