methods & algorithms at the un global platform filemethods & algorithms at the un global...

Methods & Algorithms at the

UN Global Platform Dr Joni Karanka

Lead service designer - UN Global Platform

daWos - 10 September 2018 t: #UNGlobalPlatform / @UNBigData

UNITED NATIONS GLOBAL PLATFORM

FOR OFFICIAL STATISTICS

Trusted Data - Trusted Solutions

TRUSTED

DATA

PARTNERS

METHODS

LEARNING

Assets

- Budgets

- Parking spaces

- Publications

- Data

- Processes

…

- Algorithms

They have:

- Owners

- Policies

- Definitions

They are:

- Actionable

- Quantifiable

- Monitored

Algorithms

The main assets of statistical

offices are data & algorithms

But we don’t have an exclusive entitlement to them

Global Challenges

- Knowledge-loss: we are hemorrhaging our intellectual

property in methods & algorithms

- Rigidity: we are not adapting fast enough to new data

opportunities

- Inefficiency: as a collective, our investments are

redundant and rarely shared

Methods & Algorithms in UNGP

Exploration & Data Science Method Library Orchestration

- Create experimental statistical products

- Research and explore data - Develop methods and

algorithms - Collaborate

- Publish trusted methods - Store, manage and execute

methods - Deploy machine learning &

AI models

- Create robust statistical products

- Consume data and methods

- Automate processes

methods.officialstatistics.org

Join us:

● I want your algorithms & methods

● I want you as a user

email methods@officialstatistics.org for access

Perfect technology storm

APIs

Interoperable Accessible

Portable

Scalable

Shareable

Available on

demand

Reproducible

methods.officialstatistics.org

● Partnering with Algorithmia

● Users contribute algorithms, methods, functions…

● … and they are published with an accessible API

● All hosted on public cloud

● Algorithmia automates all the stuff you don’t want to do:

testing, versioning, building examples, etc.

Code

Container

API

Compile

Publish

Commit

Consume

Container Container Container

Consume from anywhere with

any client

Algorithmia manages:

- Compilation

- Dependencies

- Containers

- Testing

- Documentation

- Versioning

- API publishing

- Code store

- Accreditation

- Scaling

No devops needed

Algorithm microservices - Q&A

Is sending data to the algorithm inefficient vs in-

house processing?

- Remember the principle of ‘leave no-one behind’ -

shared microservices mean you do not need

infrastructure so they are truly accessible

- Also, do not assume your data will live in your

machine or ‘on-premises’ (it’s likely to also be in the

cloud)

Yes, but what if I need higher performance? - The data can be hosted closer to our platform, for

example, in the same physical region at a cloud

provider

OK, what if I have tons of data that needs the

highest performance?

- All the algorithms are already modular, with explicit

APIs, open code and containers. They are ready to

redeploy in a different environment.

I like high performance, but it sounds fiddly. Can

you make it easier?

- Over time the technology to redeploy containers in

hybrid cloud / multi cloud will mature and this will

become automated rather than a hands-on

redeployment.

UN Global Working Group on Big

Data for Official Statistics October 21, 2018 - Dubai - events@officialstatistics.org

Join us at Dubai!

Thank you

Contact:

methods@officialstatistics.org