Shell
A global data marketplace for trading and supply
Summary
A data mesh platform that transforms enterprise data into discoverable, trustworthy, and reusable assets, enabling democratized data access across the organization.
Together with Shell Trading and Supply, we designed and built their first global data marketplace leveraging a data mesh paradigm. The platform provides democratized data access paired with seamless discovery, utilization, and reusability, transforming how thousands of employees discover, trust, and use data across the organization.
Services
Industry
The Challenge
Multiple apps, fragmented data storage, and an absence of governance, making complex models impossible to reuse.
Shell Trading and Supply faced challenges when dealing with multiple apps and tools, including fragmented data storage and an absence of governance across business units. This resulted in duplicated efforts when building data services and the inability to reuse highly complex models, incurring cost and efficiency issues across the organization.
Problem 01
Fragmented data storage and absent governance
- ▪Multiple disconnected apps and tools across the business
- ▪No unified governance structure or data ownership
- ▪Information assets underutilized and siloed
- ▪Complex models impossible to share or reuse across teams
Problem 02
Duplicated effort and efficiency losses
- ▪Teams rebuilding the same data services independently
- ▪High cost of building new data products from scratch
- ▪No standard way to discover, understand, or trust data
- ▪Significant friction incurred across the organization
The Solution
We designed and built a data marketplace built on four interconnected platform components, giving Shell's business units self-service access to data products while enforcing federated governance and security at every layer.
01
Domain-driven and decentralized
A stream data product managed by domains, which also handle access management. Users contact focal points to obtain credentials, keeping ownership close to the source. Domains create API data products using the platform UI.
02
Self-service data products
Domains create API data products using the platform interface. Users consume these products by following documentation on the platform, applying the same self-service principle across the full data catalog without needing human intervention.
03
Federated governance and security
API data products can have global policies and security checks through a federated committee, while implementing local policies by selecting its own identity provider, ensuring compliance and control at every layer.
This diagram is best viewed on a larger screen. Please rotate your device or view on desktop.
Impact
A paradigm shift in how a global enterprise approaches its data, with 70+ new services created in two months and 11 product teams building on the platform's patterns.
Data Quality and Utilization
By treating data as a product, information assets are no longer underutilized, promoting a culture of data excellence and innovation across the organization.
User Experience
Service registration reduced by 95%, from 10 hours to 30 minutes, with a focus on treating data consumers as valued customers.
Interoperability
The platform addresses the challenges of integrating distinct infrastructure sets, offering a unified platform that caters to the diverse needs of big data and operational technologies.
“DEUS led the architectural design, the governance framework, and they delivered the final solution.”
Stuart Blyde, Global Head of Agile Hub, Shell Worldwide
Methods
A modern data engineering approach combining data mesh principles, containerized deployment, and federated governance.
Data mesh
Data catalog
Solution design
Systems thinking
Containerization
Distributed compute
ML serving
API design
REST
GraphQL
Streaming
Batch processing
Federated governance
Operating model
Engineering