By Junta Nakai, Global Industry Leader,  Financial Services and Sustainability at Databricks

We are experiencing a fundamental shift in how financial institutions are thinking about interacting with and understanding their customers. Gone are the days of purely relying on focus groups and surveys to get a pulse check on customer needs.  According to a 2020 PwC study, ‘customer intelligence’ is set to be the biggest predictor of revenue growth and profitability for financial institutions over the coming years. What underpins this intelligence? Data and artificial intelligence (AI). Having the ability to analyse exponentially more data about what customers do and don’t want, will be the difference between those financial organisations that survive and those that don’t.

But where can financial services organisations start when it comes to harnessing data and AI? The key lies in embracing “Openness” and “Simplicity”.  By embracing open source software, organisations can achieve both. They equip themselves with the best of what modern technology has to offer. The open banking movement will be a pivotal moment in this regard. Specifically, open banking will impact financial firms in the same way open source impacted software. This shift will mean more rapid acceleration of innovation, faster launches of new business models and the creation of massive value for disruptors and adapters. It will likely bring the same dynamism of the software industry to financial services.

So, how do you create this openness and simplicity? It all starts with building a collaborative data platform.

The data building blocks

Many financial organisations are currently able to store and clean data, pull off reports and ad hoc queries, giving them insights from historical information. However, as data volumes increase, data can get stored in the wrong places and not utilised in the right way. This will prohibit financial firms from advancing their data analytics and AI to capture important insights, such as fraud patterns, customer behaviour and investment intelligence. To accelerate innovation and transformation, organisations should be looking at the wider potential of data. Analytical approaches that are barely used by organisations currently but yield tremendous value include data exploration – why did something happen, predictive modelling – what will happen, and prescriptive analytics – how can we make something happen. These types of advanced analytical approaches must be underpinned by a robust data architecture and will become even more important as open banking increases options for consumers.

Currently there are multiple architecture options for efficiently storing, cleaning and analysing data. There’s the data warehouse, the data lake and the data lakehouse. The data warehouse and data lake have their own strengths and weaknesses when it comes to what data can be stored and how the data can be analysed. The data lakehouse takes the best of both the data warehouse and data lake, emerging as the necessary data structure for organisations to draw out crucial insights.

AI making an impact

Getting the right data architecture in place, such as a lakehouse, is the crucial first step for any organisation looking to reap the rewards of using data and AI. Here are four key areas where data and AI can transform financial organisations for the better:

1.Personalisation

Data and AI have a crucial role to play in helping to create a more personalised customer experience and help financial firms move away from product centricity towards customer centricity. Continuous intelligence – the marriage of event-driven decision making and historical context – ensures completely personalised interactions with customers based on the analysis of millions of unique data points every second from multiple sources. For example, real-time payment information is analysed in real-time against contextual data points to drive the customer experience. This isn’t about forgetting the products altogether, but innovating looking at customer insight first, so that products align with real-time behaviours and needs.

2.Fraud detection

Fraud detection at scale is no easy feat, particularly as data volumes increase and online fraudsters switch-up behaviour to avoid detection. Having data in one place helps with scale and visibility. Organisations can build a fraud detection data pipeline to visualise the data in real-time. This allows more flexibility than setting rules on how fraudsters behave and mapping this against a subset of data to detect possible fraud cases. A modern fraud prevention strategy must be agile at its core and combine a collaborative data-centered operating model with an established delivery strategy of code, data and machine learning.

3.Risk management

It has become increasingly complex to manage risk within financial services, especially in banking. In addition to the new kinds of risks that open banking may create, other risk frameworks, like the Fundamental Review of the Trading Book (FRTB), require more computing power and analysis of historical data going back years, as regulators demand more transparency and explainability from the banks they oversee. Legacy, on-premise systems, no longer meet these needs. A modern, agile risk management practice is the way forward to manage and respond to market and economic volatility through data and analytics. As new threats emerge, historical data and aggregated risk models can lose their predictive values fast, so insights in real-time and at scale become increasingly important.

4.Environmental, social and governance

To remain competitive, organisations in the financial services industry are becoming increasingly focused on their environmental, social and governance (ESG) messaging while also innovating new products to meet the demands for sustainability from customers. The challenge with productising ESG is that the vast majority of ESG data is unstructured and thus difficult to handle without AI. So where does data analytics come in? For example, organisations can combine natural language processing (NLP) techniques and graph analytics on textual data from consumer facing businesses to extract and calculate the sustainability impact of an individual’s spending decisions. This helps financial services adapt to the expectations of the modern customer and innovate new customer experiences.

The future is open

An open, simple and collaborative approach to data and AI will propel the financial services industry forward in many ways, and accelerate innovation. It may be a heavily regulated industry, but the richness of customer data and its velocity, brings many opportunities for positive change and disruption, all the while keeping customer convenience and security at the heart of business growth. Ultimately, embracing the “openness” and “simplicity” may help the sector to finally achieve what it so desperately wants – to become more like a tech company.