U.S. Department of the InteriorU.S. Geological Survey

USGS GIS 2006 Workshop

Denver, Colorado

Overview of the USGS Plan for Quality Assurance of Digital Aerial Imagery

2

OutlineOutline

Introduction Background USGS Plan for Quality Assurance of Digital Aerial

Imagery Why is this important to you? Do’s and don’ts

3

The major players:

Manufacturers

Data Providers

Procurement officials

End-users

IntroductionIntroduction

4

Why calibration at USGS?

Photogrammetric methods for map generation Map production primarily done in-house Quality assurance measure for aerial photography

from aerial contractors

Leadership role in standards development Unbiased, independent agency with technical

expertise Quality assurance for The National Map

BackgroundBackground

5

USGS Camera Calibration HistoryUSGS Camera Calibration History

USGS responsible for calibration services for film camera in United States since 1973

USGS operates Optical Sciences Lab (OSL) in Reston, VA with a custom-built calibration instrument

Current policy requires current (within 3 years) camera calibration report on file before award of contract

The “Catch-22” problem for digital sensors

6

The major players:

Manufacturers

Data Providers

Procurement officials

End-users

Who are you?Who are you?

7

Will digital imagery meet my needs? How do I specify it in a RFP? Who can provide what I need and how good are they? What is it going to cost? Is it better, faster, and cheaper?

Does the product meet specifications?Accuracy, spatial resolution, spectral resolutionImage qualityTimelinessMetadata

End-User PerspectiveEnd-User Perspective

8

End-User from a USGS PerspectiveEnd-User from a USGS Perspective

Image products and services for customers Contracts with Data Providers Occasional procurement of sensor systems Specialized geospatial data for research projects End products for The National Map

Virtually no need for products for in-house production

9

USGS Plan for Quality AssuranceUSGS Plan for Quality Assurance

Four major parts covering two major processes:

Data Production: Manufacturers Certification Data Providers (flyers) Certification

Data Purchasing & Acceptance Contracting Guidelines Data Acceptance Standards

Education and training also a major component

10

The USGS PlanThe USGS Plan

Data Procurement:

Contracting Guidelines & Boilerplate Tool

Data Users and Inspectors:

Acceptance Standards

Sensor Manufacturers:

Manufacturers Certification

Data Providers:

Data Providers Certification

User Needs

Data Procurement Domain

Data Generation Domain

Final Product

11

Manufacturers CertificationManufacturers Certification Aerial Digital Imaging is in its “Wild West” phase

Anything & everything being tried Some metric-quality systems Many “other” systems

How does the customer know which can produce mapping-quality data?

USGS to offer “type certification” of mapping-quality digital aerial sensors

Must be stable, well-quantified, repeatable Able to routinely generate mapping-quality data

When operated properly!

12

Communicates specifications Provides evidence of system performance Independent certification helps to promote sensor

systems Supports verification of Data Provider’s system Type certification eliminates burden of calibration for

each sensor sold in the United States (1 time vs. n times)

Eliminates need for USGS to have custom-built calibration instrument for calibration purposes

Benefits of Manufacturers CertificationBenefits of Manufacturers Certification

13

Data Providers CertificationData Providers Certification

Second half of data generation is the flyers/Data Providers’ data processing

USGS to provide Data Providers Certification

Focused on processes and process control Ensures that Data Providers are operating sensors in

accordance with manufacturer’s instructions and limitations Ensures that Data Providers follow quality procedures

Desire to ensure reliability, repeatability, and trust

14

Provides evidence of performance of products Independent certification helps to promote product

specifications and Data Provider’s capabilities Documents Data Provider’s quality assurance plan

and “best practices” One certification for Data Provider and not for each

camera Data Providers no longer have to send cameras to

OSL for calibration, reducing down-time and shipping expenses

Benefits of Data Providers CertificationBenefits of Data Providers Certification

15

Contracting GuidelinesContracting Guidelines

User community is not sure how to contract for digital imagery

New terms, capabilities, standards, lexicon Inhibits digital contracting Addresses boilerplate requiring “USGS Certificate” Goal is to remove barriers to digital aerial contracts Encourage digital imaging

Created Federal Digital Imagery General Contract Guideline

16

Standardized terms and descriptions make the contracting process easier and more uniform among agencies

Guidelines help acceptance of digital sensors and educate end-users on benefits of digital technology

Standardized terms and guidelines help contracting officers describe their users needs

Standardized performance measures USGS certifications provide a priori acceptance of

systems and Data Provider’s “best practices” Manufacturer and Data Providers Certification reduces

necessary documentation in the RFP process

Benefits of Contracting GuidelinesBenefits of Contracting Guidelines

17

Digital Data Acceptance StandardsDigital Data Acceptance Standards

End-users unsure of how to judge digital aerial data quality

New terms & capabilities (resolution, spectral, etc.) Each customer understands things differently

There is a need for common, uniform definitions and methods for evaluating quality of image data

USGS to work with Inter-Agency Digital Image Working Group to develop these standards

Goal is a Web-based tool illustrating quality problems, measurement techniques, and standards

18

Data consumers have common standards to evaluate data products

More consistent acceptance/rejection criteria among contracting agencies

Clearer standards and guidelines helps to eliminate false expectations

Ensures high quality products Increases customer satisfaction

Benefits of Acceptance StandardsBenefits of Acceptance Standards

19

Manufacturers Certification Guidelines in work now Up to 4 manufacturers to be certified this fiscal year Two factory visits completed; reports pending Four Data Providers have expressed interest working

on the initial round of Data Providers Certification First draft of Federal Digital Imagery General Contract

Guidelines completed and reviewed by limited group A Web-based tool to help generate contracting language is

being developed

StatusStatus

20

USGS Plan for Quality Assurance of Digital Aerial Imagery briefed during ASPRS panel session

Overall USGS plan – Finalize and obtain ASPRS approval of the plan - June 06

ScheduleSchedule

21

Know your requirements and be honest with yourself Leverage resources and requirements with others Understand what advantages the technology offers Use contracting guideline Ask for sample products early in the project Review deliverables Insist on metadata

Don’t ask for camera calibration report for digital sensors

Do’s and Don’tsDo’s and Don’ts

22

Four phases of technology assimilation:

Phase 1: Identification and investment Phase 2: Learning and adaptation Phase 3: Rationalization and management control Phase 4: Maturity and widespread acceptance

Closing ThoughtClosing Thought

23

To implement a comprehensive, meaningful process that ensures the quality of data products and services

To cooperatively develop the plan with all elements of the geospatial community

Good for one is good for all

To establish a model to support new technologies

SummarySummary

24

IADIWG Web site at: http://calval.cr.usgs.gov/

For more informationFor more information

25

Contact:

Gregory L. StensaasRemote Sensing Systems Characterization ManagerUSGS EROS Data Center47914 252nd StreetSioux Falls, SD 57198605-594-2569stensaas@usgs.gov

For issues or commentsFor issues or comments

26

Questions or comments?

The USGS PlanThe USGS Plan