ABN AMRO
A metadata-driven approach to data engineering
Summary
Building a MetaLake architecture as a transformative framework for data storage, management, and utilization, enabling real-time analytics, advanced machine learning, and stronger data governance.
We partnered with ABN AMRO's Finance and Risk division to design and implement a metadata-driven, cohesive, and scalable data framework. The solution seamlessly integrates diverse data sources, enabling rapid and accurate data retrieval for actionable insights across the organization.
Industry
Services
The Challenge
Fragmented data ecosystems, slow insight generation, and limited scalability.
ABN AMRO's Finance and Risk division faced exactly this challenge: fragmented data ecosystems made it difficult to leverage data for real-time decision-making and innovation. The existing infrastructure couldn't keep pace with the organization's need for rapid, accurate data retrieval. Insight generation was slow, scalability was limited, and there was no unified approach to data governance and security across sources.
Problem 01
Fragmented data with no integration layer
- ▪Multiple disconnected data sources with no cohesive integration, making cross-functional analysis difficult
- ▪Manual processes and siloed systems preventing timely, actionable insights
- ▪No unified governance structure across data sources
- ▪Information assets underutilized across the organization
Problem 02
Limited scalability for advanced workloads
- ▪Existing infrastructure unable to scale for real-time analytics
- ▪No support for advanced machine learning workloads
- ▪Slow insight generation constraining decision-making
- ▪No standard API layer for downstream integration
The Solution
We applied a combination of solution design, data mesh principles, systems thinking, and clean architecture to build a future-ready data platform, guided by a data-as-product mindset.
01
Architecture design and data landscape mapping
Mapped the existing data landscape, identified integration gaps, and designed a metadata-driven architecture that unifies diverse data sources into a single, cohesive framework with built-in governance.
02
Platform implementation and data catalog
Implemented the MetaLake platform with seamless integration of diverse data sources, a comprehensive data catalog for discoverability, and automated pipelines for rapid and accurate data retrieval.
03
Real-time analytics and ML enablement
Enabled real-time analytics and advanced machine learning workloads, supported by a robust API layer offering REST, GraphQL, batch processing, and streaming capabilities.
Impact
From fragmented data ecosystems to a unified, metadata-driven platform unlocking new opportunities for innovation, growth, and operational excellence.
Consistent, Reliable Data
A single source of truth with built-in governance, quality checks, and traceability across all data domains, ensuring metadata-driven fast and accurate access.
Real-Time Analytics
Support for advanced ML workloads and real-time analytics across growing data volumes, with multiple API types enabling seamless downstream integration.
Future-Ready Platform
The MetaLake architecture positions the Finance and Risk division for increased agility, stronger governance, and the ability to unlock new opportunities for innovation.
“DEUS brought deep expertise in data architecture and a pragmatic approach to solving our data challenges. The MetaLake framework has given us a scalable foundation that we can build on confidently.”
Senior Data Engineering Lead, ABN AMRO
Methods
A structured data engineering approach combining data mesh principles, systems thinking, and clean architecture.
Solution design
Systems thinking
User story mapping
Data catalog
Data mesh
Clean architecture
Medallion layers
Automated pipelines
REST
GraphQL
Batch processing
Streaming
Cloud lakehouse
Data quality
Orchestration