8 learnings from building a Data Platform

DIGITAL INTERNATIONAL

Goal

Of this talk

About

The org - me

Why

Did we embark on this anyway?

First, take a step back

Vision

Mission

Objectives

Key Results

We have a dream: Vision

It's our vision to be the engine of customer obsessed

intelligence, continuously driving agile fact based decision making

Tangible: Objectives

We have a flat file data lake updated in daily batch with at least 1 data source

Tangible: Key results

Tables ABC are from source X offloaded in landing zone

Staging, transformation and harmonisation are automated

1 2

Learning #1

Laser focus

Learn by trying

Learn by testing

Set criteria BEFORE

Maintainability Scalability Reliability Security Cost(ability?)

etc...

Learning #2

Do POCs

Are you ready to roll?

Right skills Budget Data access Infra freedom Sponsor

Learning #3

Remove dependencies

Hindsight engineering = Pain

GDPR

Learning #4

Actively involve privsec

Design Principles

● PaaS● Automate all things● Separate compute from storage● Event driven● Loosely coupled

Learning #5

Acceptance criteria

With the business for the business

Learning #6

Include your customer

Don’t make it a big project

Don’t make it perfect

Your goal

Learning #7

Be Agile - adopt

Data is an intangible asset - make it real

Learning #8

Share!

Learnings

- Learning 1: laser focus

- Learning 2: do POCs

- Learning 3: be independent

- Learning 4: involve privsec

- Learning 5: design principles

- Learning 6: include your customer

- Learning 7: be Agile

- Learning 8: share!

Thank you