data modeling for communication

A German, a Frenchman and an Italian walk into a bar: A case study of data modeling as communication tool across different culturesRICHARD FREGGI

SENIOR SUPPLY CHAIN ARCHITECT

HP

DMZ AUSTRALIA 2015

About this case study

Experience from many years working in China…◦ Communicate difficult concepts across cultural and language gaps

◦ Invent new business processes to meet complex and unfamiliar requirements

◦ Develop solutions that support the processes

◦ Train users and gain stakeholder support

… tells me that data is “everywhere”◦ Great benefits from using data correctly

◦ But little understanding on how to do it

◦ This is a seldom explored “no man’s land” – neither users nor technologists are comfortable with it

I’d like to share and learn◦ New roles for the Data Modeler

◦ Methodology, approaches, lessons learned

◦ Who else has similar experiences and can share insights?

DMZ AUSTRALIA 2015

A German, a Frenchman and an Italian walk into a bar…

They have a pet panda bear. The panda orders a hamburger and eats it.

Then the panda pulls out a gun, shoots the table and walks out.

The three friends follow the panda, and the bartender stops them at the door:

“What the hell was that about?”

The German answers: “Just Google it”

So the bartender goes online:

"Panda: a black and white bear from China.

Eats shoots and leaves."

DMZ AUSTRALIA 2015

Why this matters to Data Modelers

This joke has some resonance to doing business in unfamiliar environments◦ We can’t assume that our understanding is the same as everyone else’s

◦ Differing points of view and mismatched expectations are common

◦ Assumptions and past experience can lead us in the wrong direction

◦ We can end up here… although our goal is here

DMZ AUSTRALIA 2015

The Data Model can help us reach the right goal!

Setting up operations in China

Many different players◦ Prime contractors◦ Suppliers, sub-suppliers◦ Service providers◦ Several local, provincial, central agencies

Language gaps

Each has their own culture, jargon, assumptions and expectations

Many regulations◦ Local laws, foreign government and international regulations

Time pressure

Lack of precedents… most stakeholders have never done this before

DMZ AUSTRALIA 2015

We need to:◦ Agree

◦ Understand before we agree

◦ Invent new solutions, understand them and then agree

◦ Make sure that the solutions are realistic and implementable

◦ Communicate the agreement across organizational, cultural and language barriers

Progress can be slow…

DMZ AUSTRALIA 2015

We cannot tell how the individual Processes

fit together

ConfirmResponsibilities, SOPs

Define goals Form Team List high levelProcesses

Define detailedProcesses


Argue

Pull in new Stakeholders


We use a Data Model to communicate complex concepts across cultural and language gaps

• Agree• Understand before we agree• Invent new solutions, understand them and then agree•Make sure that the solutions are realistic and implementable• Communicate the agreement across barriers

DMZ AUSTRALIA 2015

1 to 2 hours per Stakeholder

Semantic DataDictionary

Based on facts, not assumptions or

opinions

Form Team, explainData Dictionary

Agree on how to use theinformation to reach the Goals

(UML Sequence Diagrams)

We can easily tell

if the Information generated from one Process

is enough to feed the next Process

Remove ambiguity of different

terminologies, assumptions, practices

ConfirmResponsibilities,

SOPs

Approx. a dozen SOPs

agreed among ~20 Stakeholders

StakeholderWorkshop

StakeholderWorkshop

StakeholderWorkshop

StakeholderWorkshop

StakeholderWorkshop

StakeholderWorkshop

Model Fragment

Model Fragment

Model Fragment

Model Fragment

Model Fragment

Model Fragment

Factualvalidation

Factualvalidation

Factualvalidation

Factualvalidation

Factualvalidation

Factualvalidation

Consolidate many different views

into a consistent whole

that explains the subject area in question

Great benefits from using data correctly

Long discussions about how to obtain a critical document◦ Who wants to do it?◦ Who can do it?◦ Who should do it?◦ Very hard to answer without a very clear understanding of what is “it”!

Change discussion from “How” it should be done into “What” this document is◦ What information it contains (Attributes)◦ How to identify the information (Primary Key)◦ Where the information comes from (Foreign Keys)

When we agree what the document is, it’s clear who and how should do it!

Then we can progress quickly through complex discussions

DMZ AUSTRALIA 2015

AGENCYHEADER

loadno

loaddate

periodfrom

periodto

OPSHEADER

opsheaderid

opsname

generationdate

periodfrom

periodto

AGENCYDOCUMENT

transactnumber

loadno (FK)

agentname

processdate

agdestinationarea

agtransactmode

OPSDOCUMENT

transactrecordid

issuingopsname

opsheaderid (FK)

opsname (FK)

casenumber

origin

opsdestination

opsdestinationarea

issuedate

opsagentname

opstransactmode

TRANSACTIONTITEM

itemid

itemrecordid

transactrecordid (FK)

issuingopsname (FK)

transactnumber (FK)

itemparentid

itemparentrecordid

itemquantity

itemtype

itemproperty

itemrecordsource

TRANSACTIONCONTENT

transactcode

itemid (FK)

itemrecordid (FK)

contenttype

contentdescription

contentquantity

◦ Entities defined through stakeholder workshops+ validation by reviewing actual documents and data extracts

◦ Attributes and PK/FK help stakeholders keep track of how each Entity can/cannot be used in real life processes

◦ The Semantic Data Model provides a common language to understand each other and talk about new ideas

◦ This fragment shows how to reconciledifferent interpretations of the same transaction as seen by different functional silos

◦ This fragment leads to significant simplification in business process and R&R

The Data Model helps us understand, invent, agree

Semantic Data Model fragment

DMZ AUSTRALIA 2015

The Data Model helps us define processesand communicate the agreement across barriers - 1

Semantic Data DictionaryAGENCYHEADER

loadno

loaddate

periodfrom

periodto

OPSHEADER

opsheaderid

opsname

generationdate

periodfrom

periodto

AGENCYDOCUMENT

transactnumber

loadno (FK)

agentname

processdate

agdestinationarea

agtransactmode

OPSDOCUMENT

transactrecordid

issuingopsname

opsheaderid (FK)

opsname (FK)

casenumber

origin

opsdestination

opsdestinationarea

issuedate

opsagentname

opstransactmode

TRANSACTIONTITEM

itemid

itemrecordid


issuingopsname (FK)

transactnumber (FK)

itemparentid

itemparentrecordid

itemquantity

itemtype

itemproperty

itemrecordsource

TRANSACTIONCONTENT

transactcode

itemid (FK)

itemrecordid (FK)

contenttype

contentdescription

contentquantitySemantic Data Model fragment

DMZ AUSTRALIA 2015

Responsibilities and Operating Procedures

UML Sequence Diagram (Process and R&R)

Check Recordsseq

: Ops TransactTeam

: Agency B : Agency A : Ops

Document :OpsDocument

Document :AgencyDocument

1 [Trigger date 1] : Request

2 : Get()

3

4 : Request

5 : Get()

6

7 : Confirm()

8

9 : Update()

10 [Trigger date 2] : Get()

The Data Model helps us define processesand communicate the agreement across barriers - 2◦ Entities from the Semantic Data Model relate to

Objects in UML Sequence Diagrams

◦ Since the Entities represent the business view of information, the transformation is easy

◦ The Data Model helps us keep track of what each Object can and cannot do

◦ This helps us develop new processes very quickly

◦ Sequence Diagrams are documented as SOPs (Standard Operating Procedures) and communicated to all stakeholders

◦ Everyone can clearly see who does what: R&Rs, process flow and how each document is used

DMZ AUSTRALIA 2015

AGENCYHEADER

loadno

loaddate

periodfrom

periodto

OPSHEADER

opsheaderid

opsname

generationdate

periodfrom

periodto

AGENCYDOCUMENT

transactnumber

loadno (FK)

agentname

processdate

agdestinationarea

agtransactmode

OPSDOCUMENT

transactrecordid

issuingopsname

opsheaderid (FK)

opsname (FK)

casenumber

origin

opsdestination

opsdestinationarea

issuedate

opsagentname

opstransactmode

TRANSACTIONTITEM

itemid

itemrecordid


issuingopsname (FK)

transactnumber (FK)

itemparentid

itemparentrecordid

itemquantity

itemtype

itemproperty

itemrecordsource

TRANSACTIONCONTENT

transactcode

itemid (FK)

itemrecordid (FK)

contenttype

contentdescription

contentquantity

Check Recordsseq

: Ops TransactTeam

: Agency B : Agency A : Ops

Document :OpsDocument

Document :AgencyDocument

1 [Trigger date 1] : Request

2 : Get()

3

4 : Request

5 : Get()

6

7 : Confirm()

8

9 : Update()

10 [Trigger date 2] : Get()

The Data Model helps us implement technology solutions to support the agreed processes

Very difficult to manually reconcile tens of thousands of different documents from multiple incompatible sources

Use the Data and Process Models to understand reconciliation issues

Derive Physical Data Model and implement in analytical database◦ >20 queries help us reconcile

different interpretations of the same transaction

◦ Agreement on Data and Process Model is a great help in obtaining right data feeds

◦ The queries enable us to perform the agreed processes

DMZ AUSTRALIA 2015

AGENCYHEADER

loadno

loaddate

periodfrom

periodto

OPSHEADER

opsheaderid

opsname

generationdate

periodfrom

periodto

AGENCYDOCUMENT

transactnumber

loadno (FK)

agentname

processdate

agdestinationarea

agtransactmode

OPSDOCUMENT

transactrecordid

issuingopsname

opsheaderid (FK)

opsname (FK)

casenumber

origin

opsdestination

opsdestinationarea

issuedate

opsagentname

opstransactmode

TRANSACTIONTITEM

itemid

itemrecordid


issuingopsname (FK)

transactnumber (FK)

itemparentid

itemparentrecordid

itemquantity

itemtype

itemproperty

itemrecordsource

TRANSACTIONCONTENT

transactcode

itemid (FK)

itemrecordid (FK)

contenttype

contentdescription

contentquantity

Conclusion

Data Modeling helps to communicate across cultural and language gaps, invent new business processes and solutions

Who has similar experiences?

Different experiences?

Feedback?

DMZ AUSTRALIA 2015

data modeling for communication

Documents