An open approach to data analytics

Sometimes you just have to think about your data

Tim Willoughby

CTO

LGMA

Local Government Management Agency

• Shared Services to the Local Government Sector in Ireland

• Procurement

• ICT

• HR and IR

• Business Technical support and shared services – Finance, Payroll, Planning, Housing, Libraries etc

• Shared service bureaus

• Office for Local Authority Management (OLAM)

2

3 Pillars

Making sense of it all...

References

www.localgov.ie

Data.localgov.ie

Building Control

Elections

www.localgov.ie

• Service Catalogue

• Local Government Services

• Shared Registration

• Shared Forms

• Shared Payments

• Shared Document Management

• Initial candidate service – Building Control

Data.Localgov.ie

• Service Catalogue

• Local Government Services

• Shared Directory

• National Datasets

• Initial candidate service – Election

So.. What about analytics

• When you have a central service – you have central data..

• Toolsets used – none of them harmed for this presentation…• Drupal, SugarCRM

• Excel (PowerQuery, PowerMap, PowerPivot), Notepad

• Dropbox, Amazon, Google, Twitter

• Talend, Hadoop, Elastic Search, SOLr, Kibana

• With help from AnnerTech, ProvidentCRM, Ostia, SindiceTech

BCMS Timeline

• Minister Decision on new Legislation October 2013• New Legislation to be in place (live) 1st March 2014• LGMA told to do development Mid October 2013

• 2 Tenders – Front end / Back End• Front end has to be ready for 1st March – Registration / Forms /Payments• Back end has to appear to be ready 1st March – not scale required• Single Infrastructure / Data Infrastructure / ePayments in place• Legislation still being finalised on 28th February 2014• System Live on March 1st “2014

• To date 115,000 Documents, 6400+ Commencements, €1,300,000 fees, 20,000 registered users

Public InterfaceDrupal Based

Use Existing Local Government PortalReusable Components

RegistrationForms

Document UploadePayments

LA Process ManagementSugarCRM Based

Household Charge InfrastructureTakes Data from DrupalBack Office Functions

Shared Services

10

DRUPAL Portal

SugarCRM backend

What can you learn from a year of registering all building works in the state…• Inputs

• 6500 Commencements

• 115000 Documents (500 or more a day)

• 1.3m in fees

• 22000 registered users (Builders and Architects)

• Outputs

• ePayments

• Building Register

• DATA• Fancy Maps

• NLP Searches

• Open

• ANALYTICS• Biggest, Most, Least, Average

0

100

200

300

400

500

600

700

800

900

1000

09

/01

/20

14

27

/01

/20

14

06

/02

/20

14

19

/02

/20

14

28

/02

/20

14

09

/03

/20

14

18

/03

/20

14

27

/03

/20

14

05

/04

/20

14

12

/04

/20

14

22

/04

/20

14

30

/04

/20

14

08

/05

/20

14

15

/05

/20

14

22

/05

/20

14

29

/05

/20

14

05

/06

/20

14

12

/06

/20

14

19

/06

/20

14

26

/06

/20

14

03

/07

/20

14

10

/07

/20

14

17

/07

/20

14

24

/07

/20

14

31

/07

/20

14

07

/08

/20

14

14

/08

/20

14

21

/08

/20

14

28

/08

/20

14

04

/09

/20

14

11

/09

/20

14

18

/09

/20

14

25

/09

/20

14

02

/10

/20

14

09

/10

/20

14

16

/10

/20

14

23

/10

/20

14

30

/10

/20

14

06

/11

/20

14

13

/11

/20

14

20

/11

/20

14

27

/11

/20

14

04

/12

/20

14

11

/12

/20

14

18

/12

/20

14

28

/12

/20

14

04

/01

/20

15

11

/01

/20

15

18

/01

/20

15

25

/01

/20

15

01

/02

/20

15

08

/02

/20

15

15

/02

/20

15

22

/02

/20

15

01

/03

/20

15

08

/03

/20

15

15

/03

/20

15

Total

Total

Documents Uploads per day / 115000

Time of day / week

0

2

4

6

8

10

12

14

16

18

00

:00

:12

07

:05

:18

08

:07

:03

08

:34

:06

08

:56

:25

09

:16

:37

09

:35

:19

09

:53

:25

10

:11

:13

10

:29

:00

10

:46

:29

11

:04

:11

11

:21

:49

11

:39

:10

11

:56

:46

12

:14

:19

12

:32

:22

12

:50

:48

13

:09

:17

13

:27

:27

13

:45

:30

14

:03

:22

14

:20

:34

14

:37

:59

14

:55

:09

15

:12

:06

15

:29

:15

15

:46

:10

16

:03

:07

16

:20

:31

16

:37

:57

16

:55

:35

17

:13

:48

17

:32

:28

17

:52

:44

18

:17

:07

18

:42

:53

19

:10

:50

19

:44

:56

20

:23

:23

20

:56

:39

21

:29

:30

22

:05

:57

22

:56

:36

Total

Total

0

5000

10000

15000

20000

25000

Mon Tue Wed Thu Fri Sat Sun

Total

Total

Dashboard

Demo

If you think there is no building going on..

• (Link) Mapped Building Projects - 2014/2015

Money..

Local Elections - Analytics in action

ProblemOver 400 Local Electoral Areas / LEA34 Local Authorities38 Count Centres3.2m Voters

DesireTo have all of the Count Data available as open data as the local counts happen…

Solution?

1. Set up a Dropbox Account for each County 2. Each LEA set up as Separate Folders

3. Agree and Set up Standards for data 4. Set up way of Publishing Data…..

Data collated Data Stored Data Mapped Data Socialised

Demo

What has changed to make all of these things easier for us… for consumers• Open Data

• Open Standards

• Open Source

Thinking Matters More!

What has Analytics ever done for us?

31

Apart from Decisions, Scale, Speed, Agility, Low Cost, Enterprise Adoption, Mapping of issues, Data Standards, API’s, etcClarity, repositories of information, etc etc

GEO - More than just Location, Spatial Analysis and wider predictions possible

Now that we are thinking this way…

• Micro formats for all published sites

• Open Data for Builders / Architects

• Alerts / Decisions

• GEO Alerts

Conclusions

• If you don’t like our Analytics or Visualisations… take the data – make your own…

• Start somewhere• Understand your own data

• Use the tools you have first

• Communicate