methods & algorithms at the un global platform filemethods & algorithms at the un global...
TRANSCRIPT
Methods & Algorithms at the
UN Global Platform Dr Joni Karanka
Lead service designer - UN Global Platform
daWos - 10 September 2018 t: #UNGlobalPlatform / @UNBigData
UNITED NATIONS GLOBAL PLATFORM
FOR OFFICIAL STATISTICS
Trusted Data - Trusted Solutions
TRUSTED
DATA
PARTNERS
METHODS
LEARNING
Assets
- Budgets
- Parking spaces
- Publications
- Data
- Processes
…
- Algorithms
They have:
- Owners
- Policies
- Definitions
They are:
- Actionable
- Quantifiable
- Monitored
Algorithms
The main assets of statistical
offices are data & algorithms
But we don’t have an exclusive entitlement to them
Global Challenges
- Knowledge-loss: we are hemorrhaging our intellectual
property in methods & algorithms
- Rigidity: we are not adapting fast enough to new data
opportunities
- Inefficiency: as a collective, our investments are
redundant and rarely shared
Methods & Algorithms in UNGP
Exploration & Data Science Method Library Orchestration
- Create experimental statistical products
- Research and explore data - Develop methods and
algorithms - Collaborate
- Publish trusted methods - Store, manage and execute
methods - Deploy machine learning &
AI models
- Create robust statistical products
- Consume data and methods
- Automate processes
methods.officialstatistics.org
Join us:
● I want your algorithms & methods
● I want you as a user
email [email protected] for access
Perfect technology storm
APIs
Interoperable Accessible
Portable
Scalable
Shareable
Available on
demand
Reproducible
methods.officialstatistics.org
● Partnering with Algorithmia
● Users contribute algorithms, methods, functions…
● … and they are published with an accessible API
● All hosted on public cloud
● Algorithmia automates all the stuff you don’t want to do:
testing, versioning, building examples, etc.
Code
Container
API
Compile
Publish
Commit
Consume
Container Container Container
Consume from anywhere with
any client
Algorithmia manages:
- Compilation
- Dependencies
- Containers
- Testing
- Documentation
- Versioning
- API publishing
- Code store
- Accreditation
- Scaling
No devops needed
Algorithm microservices - Q&A
Is sending data to the algorithm inefficient vs in-
house processing?
- Remember the principle of ‘leave no-one behind’ -
shared microservices mean you do not need
infrastructure so they are truly accessible
- Also, do not assume your data will live in your
machine or ‘on-premises’ (it’s likely to also be in the
cloud)
Yes, but what if I need higher performance? - The data can be hosted closer to our platform, for
example, in the same physical region at a cloud
provider
OK, what if I have tons of data that needs the
highest performance?
- All the algorithms are already modular, with explicit
APIs, open code and containers. They are ready to
redeploy in a different environment.
I like high performance, but it sounds fiddly. Can
you make it easier?
- Over time the technology to redeploy containers in
hybrid cloud / multi cloud will mature and this will
become automated rather than a hands-on
redeployment.
UN Global Working Group on Big
Data for Official Statistics October 21, 2018 - Dubai - [email protected]
Join us at Dubai!
