Making 300+ Data Champions the owners of the fastest growing fintech unicorn in South East Asia

Context and background 

Fintech in South East Asia often draw on proven business models from the US and Europe, bringing these new products and services to the consumers and businesses across the region. Many blend these innovations with the traditional conglomerate business structures of Asia, developing sprawling enterprises that touch many aspects of their customers’ financial lives. Our client was no exception, leveraging both international best practice and local market nuance to become one of the fastest growing fintech startups globally.   

The business expanded to offer a diverse range of products to both business and consumer customers—including payments, lending and banking services—across five countries in South East Asia, facilitating tens of billions of USD in financial transactions annually.  Not only was it necessary for our client to understand their business as it grew upwards of 25% monthly but data were also an integral part of their products and regulatory compliance. 

Problem statement 

The primary issue our client faced was ensuring their data platform could handle the rapid scaling of volumes from existing business lines, as well as new innovations across the group.  The data platform informed automated and human decisions internally, was provided to customers as a data product, and fed external reporting: 

• Platform interaction data was critical in mapping the behavior of fraudulent actors, triggering automated protections and informing fraud risk teams’ investigations. 

• Metadata on customers and their activities created features for credit risk models determining whether to advance loans to the individual or business. 

• Payment system partners and customers demanded prompt and reliable daily reports to enable reconciliation, transaction investigation and billing. 

• Financial system regulators required accurate periodic reporting of payments processed and loans disbursed as part of their oversight and enforcement of anti–money laundering legislation, payments system operation and credit reporting, helping to ensure that the financial systems remained stable. 

The technical challenges of scaling data processing were becoming exponentially more challenging as data volume doubled every quarter, requiring a future-proofed solution to data pipeline architecture, data transformations and query engines.  At the same time, the data platform needed to present a simple interface for non-technical risk team members and government relations staff to conduct their work. 

Our solution 

Understand how people work with data 

Cruxdata was bought in to help ensure that the data platform was up to the challenges being faced.  We started with the foundations—people—gaining a good understanding of how the organization wanted to work with data, through a combination of data maturity surveys and stakeholder interviews. We conducted these quarterly throughout our engagement to ensure that the solutions we were implementing were fit for purpose and delivering on the pain points raised by staff throughout the organization. 

Team structure was the next question to solve. We brought together the data engineering and analyst teams to provide better coordination between the infrastructure provision and the major data models being built on top of that infrastructure.  Outside of the central data teams, we implemented a new Data Operating Model, codifying the roles and responsibilities of Data Owners and Champions within distributed product, engineering and business teams.  

An operating model is only worth the paper it is written on if the people named in it use it, and this one named a lot of people: more than 300 Data Champions across an organization of over a thousand. A Champion was not a title granted at a workshop and then forgotten. Each was accountable for the data their team produced and consumed—the definitions behind it, the quality checks on it, and its fitness for the decisions being made on it—and each had the standing to change those models directly rather than raise a ticket and wait on the central team.

That accountability is also what made the federated model affordable. Centrally owning every dataset in a business growing 25% monthly would have required the central team to acquire domain expertise faster than the business created domains, which is not a race anyone wins. Distributing ownership to the teams who already held the context removed that transfer instead of trying to win it. The quarterly maturity surveys and stakeholder interviews were how we checked the model was working rather than assumed it: they showed us where Champions were blocked, which parts of the platform they were avoiding, and what to change before the next quarter.

Create infrastructure that works in harmony 

Cruxdata then designed and implemented a federated data model informed by organization structure, allowing for central control of mission critical, shared metrics in a governed semantic layer, while providing freedom for Data Champions to tailor team specific models to their individual needs.  Central models aggregating across products were synchronized with team-owned and certified datasets, minimizing time-consuming domain expertise transfer from owners to the central team.  Data quality checks and design standards ensured accuracy on definition updates and regular data refreshes.   

Data pipeline infrastructure was configurable by workload, allowing for critical or high-availability data products to be built on top of dedicated compute, while general purpose workloads could leverage the cost efficiency of shared resources. 

The results 

Our client began seeing benefits from implementation incrementally as components of the solution were released throughout our engagement.  Timeliness of major centrally owned datasets improved, allowing users to confidently schedule next-day reporting for customers and partners.  Reconciliation issues between team-specific datasets and centrally owned aggregations all but disappeared, making decisions on next actions to take more efficient.  Drill-through from centrally owned datasets to detailed team-specific datasets was enabled, improving analyst productivity throughout the organization.  High quality regulatory reporting was enabled, for example, CTR and STR reports to ALMC in the Philippines and LTDBB to BI in Indonesia, supported by failure handling and alerting mechanisms should the reports fail to meet customized tests.  More efficient and reliable data processing saved our client time and money.

The clearest signal, though, was not a pipeline metric. The Data Operating Model only pays for itself if the Champions it names keep showing up, and they did: more than 80% of the 300+ Data Champions were active on the data platform every month, in an organization of over a thousand people. Distributed ownership held rather than decaying back to a queue of tickets at the central team's door, which is the way most of these models quietly end. 

• >80% of 300+ Data Champions active on the data platform monthly, in a 1,000+ person organization

• >40% reduction in data refresh time, from up to 16 hours to less than 9 

• >99.8% transaction reconciliation success rate over 200+ million records 

• >70% improvement in query speeds for common queries