SDI Evaluation Frameworks

• Presented to: Pan Canadian SDI Summit

• Presented by: Gordon Plunkett

November 24, 2015

What is an SDI Evaluation Framework?

• An SDI is made up of a number of diverse components that when fitted together and when working properly, provide data and services to end users.

• A Framework is the articulation of the diverse components required to make an SDI work.

• An SDI Evaluation Framework is a way of assessing each individual component including how these components work together.

Three SDI Evaluation Frameworks

• Modified GSDI Evaluation Framework (2010)• German Marine SDI Evaluation Framework (2012)

• Report Card on the U.S. NSDI (2015)• GeoAlliance Canada (2015)

Modified GSDI Evaluation (2010)

• Feasibility Study for a National Spatial Data Infrastructure in Jordan

• https://www.infodev.org/infodev-files/resource/InfodevDocuments_1119.pdf

• Feasibility Study for a National Spatial Data Infrastructure in Uganda

• https://www.infodev.org/infodev-files/resource/InfodevDocuments_1134.pdf

Evaluation FrameworkFactor Decision Criteria1. Organizational • Political Vision

• Institutional leadership• Umbrella legal Agreements

2. Data availability • Digital Cartography Available• Metadata available

3. People • Human Capital• SDI culture-education• Individual leadership

4. Access Network • Web connectivity• Telecom infrastructure• Geospatial s/w available

5. Financial Resources • Government central funding• Policy for ROI• Private Sector activity

$

German Marine SDI Framework (2012)

• A framework for the evaluation of marine spatial data infrastructures accompanied by an international case-study.

• http://www.agile-online.org/Conference_Paper/CDs/agile_2012/proceedings/posters/Poster_Rueh_A_framework_for_the_evaluation_of_marine_spatial_data_infrastructures_Accompanied_by_an_internal_case-study_2012.pdf

Evaluation Framework1. Policy Level

• Policy (government SDI policy, IP rights, privacy, pricing, acquisition and use of spatial data)

2. Management Level• Standards (technical, institutional arrangements, core datasets)• Access Network (pricing, delivery mechanisms, access, value added

arrangements)

3. Operational Level• Access Network (type, data volume, response time)• Data (format, capture, core datasets, maintenance, quality, accuracy)

4. Other Influencing Factors• People (# organizations, # people, training, market situation)

5. Performance Assessment• Meeting objectives, satisfaction, diffusion & use of data

Evaluation Indicators & AssessmentArea Technical indicator Organization indicatorData Core datasets {VG} Degree of organizational

involvement {VG}Coordinate reference system {VG}

Metadata Catalogue availability {G} Coordination {VG}

Data quality and accuracy {G}

Services & interfaces

Availability {G} Access, custodianship {N/A}

Performance {N/A}

Clearinghouse & portal {G}

Standards Interoperability {VG}

Modelling (Policy)

Government SDI policy, architecture, business models {VG}

US NSDI Evaluation Framework (2015)

• Report Card on the U.S. National Spatial Data Infrastructure

• http://www.cogo.pro/uploads/COGO-Report_Card_on_NSDI.pdf

• Esri Canada blog post• http://www.esri.ca/en/blog/do-we-need-report-card-performance-canadian-geospatial-data-infrastructure

Framework Data Evaluation Framework

Subject

1. Cadastral Data

2. Geodetic Control

3. Elevation Data

4. Hydrography Data

5. Orthoimagery Data

6. Government Units Data7. Transportation Data

8. Overall Data Grade

Grade Data Evaluation CriteriaA Fit for the FutureB Adequate for nowC Requires attentionD At riskF Unfit for purpose

Evaluation criteriaTheme and descriptionLead agencyCollaboration and partneringStandardsEstimate of completenessAccessibilityAuthority, governance and management

Comprehensive Evaluation Framework Category Description

Capacity The Framework’s capacity to meet current and future demands

Condition The existing or near term condition of the Framework themes as an integrated whole.

Funding The funding capability of the Framework.

Future Need Whether future funding prospects will be able to meet the need.

Operation and Maintenance

The ability of key lead organizations to develop and maintain the Framework and to adopt new technology, procedures, and standards.

Public Use The Framework’s ability to provide data resources that meet the everyday needs of organizations and the general public, and to provide data resources that meet the need to respond to public safety incidents, natural disasters, and other Emergencies

Resilience The ability of the geospatial community to participate in development of the Framework and to contribute to its sustainability as a long term asset of value for the nation

Framework Data AssessmentSubject NSDI CGDI Comments on CGDI data gradeCadastral Data D+ D- In many provinces, cadastral data is only

available as a fee-based serviceGeodetic Control B+ A- Geodetic control data is readily available and

generally accurate on a national basisElevation Data C+ C- CDED data has some deficiencies in quality and

resolution, but has national coverageHydrography Data C C+ The National Hydrographic Network is quite

good, but often has deficiencies in data currency

Orthoimagery Data C+ D There is no national imagery strategy resulting in a patchwork of out-of-date imagery

Government Units Data

C C+ Administrative boundaries data is generally of good quality and currency

Transportation Data D D+ The National Road Network generally suffers from data currency issues

Overall Data Grade C C While several data layers are currently useable, overall CGDI framework data requires attention

Comprehensive AssessmentSubject NSDI CGDI Comments on CGDI comprehensive gradeCapacity C C Many CGDI framework data themes do not meet

current or future needsCondition D D Many CGDI framework data themes are out of date

Funding D D CGDI and framework data funding is low and continues to decline

Future Need D C Future funding could help some data layers, but cadastral data improvements require new policies

Operation and Maintenance

C D+ Many key lead organizations are no longer able to maintain and develop their framework data

Public Use C D Most framework data is not accessible in a consistent manner by the general public

Resilience C B- The geospatial community has several methods of participating and providing input

Overall Grade C- C- The overall comprehensive grade indicates that the CGDI still requires attention and funding

GeoAlliance Canada (2015)

• Work originally started by NRCan ~2013

• Results used by the Canadian Geomatics Community Roundtable

• Currently being updated by source organizations on behalf of GeoAlliance Canada

Evaluation FrameworkFactor Criteria1. Initiative • SDI Initiative present

• Name of Initiative• Governance

2. Technology • Portals and Registries• Web Services• Tools and Applications• Atlases

3. Data • Themes of data• Fee or Free

4. Standards • Data / Metadata / Web Services / Other5. Policies • Data licenses

• Data Access6. Miscellaneous • Interesting and important information

• Contacts

Thank You

Gordon Plunkettgplunkett@esri.ca