edf2012 nigel shadbolt - transparency and open data
TRANSCRIPT
![Page 1: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/1.jpg)
Transparency and Open Data: Why Bother? Being also a discourse on some lessons learnt
Professor Nigel Shadbolt FREng
7th June 2012
European Data Forum, Copenhagen
Nigel Shadbolt
![Page 2: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/2.jpg)
open the data…
![Page 3: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/3.jpg)
and the applications follow…
![Page 4: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/4.jpg)
at all scales…
![Page 5: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/5.jpg)
around the world…
![Page 6: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/6.jpg)
leading to…
• Improved transparency & accountability
• Better Public Services
• Participation… people enhance, improve, capture the data
• Improved efficiency
• Creation of social value
• Economic value, innovation and growth
![Page 7: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/7.jpg)
Lesson: Open Data…should be non-partisan
![Page 8: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/8.jpg)
“We are determined to go further in breaking down the walled garden of government, using technology and information to provide greater transparency…”
“Greater transparency across Government is at the heart of our shared commitment to enable the public to hold politicians and public bodies to account
“Our ambition is to become the world leader in open data, and accelerate the accountability revolution that the internet age has made possible..
“My aim is clear: to make the provision of data (with reasonable formats and licenses) a routine task of public administrations.”
![Page 9: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/9.jpg)
Lesson: Open Data…needs a steady stream of
success
![Page 10: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/10.jpg)
![Page 11: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/11.jpg)
![Page 12: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/12.jpg)
Lesson: Open Data…is not a technical problem
![Page 13: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/13.jpg)
UK Public Data Principles
It is about changing behaviour
![Page 14: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/14.jpg)
However…
★ Put your data on the Web with an Open Licence
★★ Make it available as structured data★★★ Use open, standard formats★★★★ Use URIs to identify things★★★★★ Link your data to other people’s data
![Page 15: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/15.jpg)
• National digital infrastructure being built
• URIs for schools, roads, bus stops, post codes, admin boundaries...
• Some of the data links across and connects other data together
• Key data link points exist
Create, Use and Share URIs
![Page 16: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/16.jpg)
• National digital infrastructure being built
• URIs for schools, roads, bus stops, post codes, admin boundaries...
• Some of the data links across and connects other data together
• Key data link points exist
Create, Use and Share URIs
![Page 17: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/17.jpg)
Create, Use and Share URIs
• Schools
• Post-codes
• Government posts
• Ministers
• Bus stops
• :
• :
![Page 18: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/18.jpg)
Lesson: Open Data…if it doesn’t scale it will fail
![Page 19: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/19.jpg)
Datasets on data.gov.uk over time
![Page 20: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/20.jpg)
• Network effects
– Accessibilitly
– Popularisation
– Centreless
– Timeliness
– Convergence
Open Data – the network effect
Courtesy Mathew Hurst
![Page 21: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/21.jpg)
Lesson: Open Data…needs location, location, location
![Page 22: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/22.jpg)
www.ordnancesurvey.co.uk/opendata
See also: Linked data http://www.ordnancesurvey.co.uk/opendata/linkeddata.html
![Page 23: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/23.jpg)
the connective tissue of open data
![Page 24: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/24.jpg)
24
![Page 25: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/25.jpg)
![Page 26: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/26.jpg)
Lesson: Open Data…release data that matters
![Page 27: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/27.jpg)
UK Reported Crime Data• A tough data set to
release
• All reported crime each month
• By crime type
• At the street level
• Police now use the apps themselves
![Page 28: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/28.jpg)
![Page 29: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/29.jpg)
![Page 30: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/30.jpg)
![Page 31: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/31.jpg)
From Schools to Arterial Surgery
![Page 32: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/32.jpg)
Lesson: Open Data…must have a balanced ecosystem
![Page 33: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/33.jpg)
Source Deloitte LLP
![Page 34: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/34.jpg)
• The Network Effect
• for data, people and organisations• A Focal Point
• coordinate, convene, link and integrate
• The Power of Open
• Standards, Data, Licences, SW• Open Data as a Platform for
• Government• Business• Research
![Page 35: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/35.jpg)
“The vision is to establish the Open Data Institute as a world leading centre to innovate, exploit and research the opportunities for the UK
created by the Government’s Open Data policy.”
• Business Innovation
• Public Sector Innovation
• Training the Open Data Generation
• Developing and Researching Open Data
• Developing Open Data Standards and Policies
• Advisor to Government & International Collaboration
![Page 36: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/36.jpg)
OGD 2.0 • We haven’t worked out
how to close the feedback loops
• Publish out
• Acquire or Enhance
• Write Back
![Page 37: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/37.jpg)
Lesson: Open Data…needs Business
![Page 38: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/38.jpg)
Economic Value, Innovation and Growth
![Page 39: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/39.jpg)
39
![Page 40: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/40.jpg)
40
![Page 41: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/41.jpg)
Lesson: Open Data…isn’t everything
![Page 42: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/42.jpg)
© 2012 Deloitte Global Services Limited
• Big - Small• Public – Private• Open - Closed• Personal - Non-Personal• Anonymous - Identified• Aggregate - Individual
Varieties of Data
![Page 43: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/43.jpg)
midata
Nigel Shadbolt
![Page 44: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/44.jpg)
Personal Data Asset Revolution…
![Page 45: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/45.jpg)
Lesson: Open Data… will need defending
![Page 46: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/46.jpg)
Challenges
• Infrastructure
• Quality
• Interpretation
• Security and Privacy
• Evidence of impact
• Sir Humphrey 2.0
![Page 47: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/47.jpg)
A sample of “data hugging” excuses• It’s held separately by n different organisations, and we can’t join it up
• It will make people angry and scared without helping them
• It is technically impossible
• We do not own the data
• The data is just too large to be published and used
• Our website cannot hold files this large
• We know the data is wrong
• We know the data is wrong, and people will tell us where it is wrong
• We know the data is wrong, and we will waste valuable resources inputting the corrections people send us
• People will draw superficial conclusions from the data without understanding the wider picture
• People will construct league tables from it
• It will generate more Freedom of Information requests
• It might be combined with other data to identify individuals/sensitive information
• It will cost too much to put it into a standard format
• Our IT suppliers will charge us a fortune to do an ad hoc extract
![Page 48: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/48.jpg)
A sample of “data hugging” excuses• It’s held separately by n different organisations, and we can’t join it up
• It will make people angry and scared without helping them
• It is technically impossible
• We do not own the data
• The data is just too large to be published and used
• Our website cannot hold files this large
• We know the data is wrong
• We know the data is wrong, and people will tell us where it is wrong
• We know the data is wrong, and we will waste valuable resources inputting the corrections people send us
• People will draw superficial conclusions from the data without understanding the wider picture
• People will construct league tables from it
• It will generate more Freedom of Information requests
• It might be combined with other data to identify individuals/sensitive information
• It will cost too much to put it into a standard format
• Our IT suppliers will charge us a fortune to do an ad hoc extract
• All these sometimes have some truth in them• Often they rationalise the official fear of the unknown• Data owners need to be helped through • Example & precedent are your friends• For first phase, compromise on data not licence
![Page 49: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/49.jpg)
Lesson: Open Data…keep reminding them
![Page 50: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/50.jpg)
![Page 51: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/51.jpg)
![Page 52: EDF2012 Nigel Shadbolt - Transparency and Open Data](https://reader030.vdocuments.us/reader030/viewer/2022012813/543887ad8d7f72c05a8b80f4/html5/thumbnails/52.jpg)
Open Data and the Power of Opennever one driver nor one reason
•Open Licences
•Open Standards
•Open Data
•Open Data Principles
•Open Participation
•Open Minds