Covering Disruptive Technology Powering Business in The Digital Age

Financial Services Need Data Agility for Growth
August 13, 2021 News


Authored By: Joe Ong, Vice President and General Manager – Hitachi Vantara ASEAN at Hitachi Vantara


Singapore’s Infocomm Media Development Authority (IMDA) signed on July an agreement with a group of banks, ports, shipping companies and commodity exporters to launch a new trade data exchange, which it says will unlock more than $150 million of value annually. According to the IDMA, the Singapore Trade Data Exchange, or SGTraDex, is designed to be a neutral and open digital infrastructure through a public-private partnership to provide a centralised gateway that will allow companies to share trade data securely through API integration.

The SGTraDex initiative, which aims to promote digitalisation of supply chains, underscores why many call data the oil of the digital economy. Both pools are immense, vast and untapped, and therefore a valuable asset, a tradable commodity.

In the financial services industry, data paves the way for every strategic move. Whether it is to create a new service, comply with regulations or overhaul and reengineer legacy operations, it is data that is central to the effort. For businesses in this sector, the pace at which they can reshape and repurpose data is key to determining their ability to predict market trends and meet client expectations.

The main issue for established players is that the speed at which they can get the data they need to support the development of advanced analytics, artificial intelligence (AI) and machine learning (ML) projects, or research and ultimately, the development of innovative new products, takes far too long to fulfill. That amount of time acutely restricts the ability of these institutions to grow.

The most effective solution is to immediately, if not gradually, move towards creating an agile data infrastructure. Sometimes called a data fabric, this approach affordably supports creating and maintaining the huge number of data pipelines needed to compete in today’s financial services marketplace.


Real-Time Everything

The finance industry is being driven towards agile data by the twin demands for instantaneous knowledge and the anticipation of constant change. Hyper-personalisation, data-led listening and real-time everything are the new normal for financial services. The standard against which financial services are being judged has been set by digital consumer businesses such as Grab, Razor, SEA, and those rapidly rising competitors, Fintechs that are digital natives and setting up neobanks.

Whether a person is making a payment, sending money across borders or anticipating market data for an investment decision, anything slower than “right now” would cause inconvenience. Slow processes are the visible reflections of cumbersome systems that are incapable of meeting modern requirements.


DataOps Can Save the Day

Firms in financial services are masters of data capture and creation and storage techniques essential for storing product information, capturing customer details, processing transactions and keeping records of accounts. But that bedrock of most financial institutions, the data platform, was not built for a real-time world that feeds huge rivers of data to analytics, AI or ML apps. Existing data infrastructure is either static and fragmented across data silos or stuck on legacy platforms incompatible with today’s requirements, undesirably creating mismatches between products and underlying data and thus between expectations and deliverables.

Instead, financial service firms need to embrace technological evolution and graduate from DevOps to DataOps, which offers real-time delivery and rapid change. The organisation can adopt an agile data platform which supports DataOps and thus enables rapid change and stable operations.

DataOps is an approach to data analytics that deploys automated, process-oriented methodology to improve the quality of, and reduce the cycle time of, data analytics. It is how you manage every step from the source of the data source to the consumer of that data, in a manner that improves efficiency, ensures the data’s security and reduces costs. DataOps combines the agile development principles we learned from DevOps with operations management.

To help an organisation transition from its legacy infrastructure to an agile data-driven environment, the agile data platform is built on three layers:

  • Agile data flows. These enable the operationalisation of analytical and ML models without significant development while maintaining the agility of data pipelines. It moves the organisation away from static, batched and siloed data pipelines to allow data to flow freely through the organisation.
  • Converged analytics and ML. Operationalising AI, or AIOps, creates a powerful, unified, easy-to-use data modelling and analytical environment. This environment can be deployed in a wide range of scenarios, including quantitative research, business analyst functions and operations activities, enabling data to drive agility at all levels within the organisation.
  • High-speed data. Transitioning from data warehouses and data lakes to hybrid data architectures able to leverage high-speed data substrates allows raw data to be ingested and passed directly to analytic data engines and ML functions in near real time. An effective data fabric is important for managing the distributed environments of your enterprise, ecosystem and supporting navigation, from the edge to the core, to the cloud.


Figure 1. An Agile Data Platform


To remain competitive, the minimum effort required would be to adopt the foundational layers shown in Figure 1.

Agile Data Platforms power modern experiences and meet business requirements, thus they offer substantial rewards. Some use cases that reflect both new and enhanced products, as well as modernizations to operations, could be:

  • Enabling more accurate credit decisions by incorporating advanced ML algorithms across a very large dataset in near real time.
  • Automation of mass wealth client advisor and investment planning that tailors advice and automates custom portfolio construction and management activities.
  • Development and maintenance of ML training pipelines, ensuring that models are updated with the latest data and are accurate and auditable.
  • Providing a high-performance computing (HPC) data platform for computationally intensive risk calculations and trading models.
  • Analysing customer interactions, creating intelligent products and customer journeys that can be optimised in real-time based on streaming interaction data.


Financial Institutions Need DataOps to Thrive in the Digital Age

By virtue of the pain of adapting data to meet the needs of new applications and the rising costs incurred to support bespoke solutions on legacy infrastructure, financial services firms will be driven to implement agile data platforms. The benefit of creating an agile data platform based on DataOps is that it offers the kind of accelerated innovation that your competitors will be powerless to match.