white paper semi-automated historical airphoto processingmetadata, large positional and orientation...

White Paper

Semi-Automated Historical Airphoto

Processingby Shawn Melamed, Applications Specialist

PCI Geomatics

Semi-Automated Historical Airphoto Processing - 2 -

Semi-Automated Historical Airphoto

ProcessingGeometric Correction of historical imagery is a complicated, time consuming and expensive operation. Due to a lack of development related to streamlining these operations, a significant amount of manual processing time must be invested into current projects. A system capable of achieving reliable accuracy that considerably reduces project turnaround time is required to help manage processing costs and improve the operating capacity.

BackgroundIn 2011, Blackbridge Geomatics, formerly Iunctus Geomatics, began a project with the goal of processing large volumes of historical aerial images. Initially, the organization implemented a manual workflow that provided reliable spatial accuracy, but required a significant amount of time and manual processing. This led to a workflow that had limited throughput capacity and high per unit costs. Acknowledging that the current method was less than optimal, Blackbridge (at the time Iunctus) worked with PCI Geomatics to develop a workflow that could reduce processing time and costs, while maintaining moderate to high accuracy.As a result of the combined experience and expertise of PCI’s photogrammetric scientists, as well as the company’s engineering successes in automated and high speed geospatial image processing, the first version of the semi-automated workflow was developed. The historical airphoto processing production workflow has undergone major revisions since the initial development, and has been proven to provide the intiial customer as well as other clients great success in reducing processing time, minimizing manual efforts and has reliably meeting accuracy requirements.


Challenges

This production workflow was designed to consider the various problems that are often encountered with historical imagery. Several factors make historical airphotos more difficult to process than standard airphotos, which include: limited or incorrect metadata, large positional and orientation errors, poor fiducial marks, poor quality imagery as well as changes in land cover over time.

MetadataMost historical images are accompanied by only limited metadata information. This is often limited to: Camera focal length, frame size, flying height and approximate corner coordinates.

Initial Positional AccuracyGeocoding information provided for historical airphotos are commonly limited to approximate corner coordinates and or scene centers where initial positional displacements can often exceed 1km. Furthermore, image orientation (Kappa) is often unknown, which makes it more difficult to automatically acquire feature matches.

Fiducial MarksHistorical images come with a variety of different types of fiducial marks, some of which are not well suited for automated pattern recognition. Furthermore, many images encountered have missing fiducial marks, as they can be cut-off during the scanning process (digitizing).

Image QualityImages are often faded, scratched or incompletely scanned. This can hinder the automated feature matching algorithms and make it very difficult to manually acquire matches.

Ground FeaturesSignificant land cover changes between the raw imagery and reference imagery can make it difficult and sometimes, impossible to collect GCPs.


Historical Airphoto Processing Workflow

PCI’s semi-automated historical airphoto processing workflow was developed to improve automation, project turnaround time and maintain processing accuracy. As such, automated processes were developed in areas that have been identified as significant time investments, such as Ground Control Point (GCP) and Tie Point (TP) collection. The workflow consists of three main processing steps (Data Ingest, Coarse Alignment and Fine Alignment) required to build the geometric models used to orthorectify the imagery. Each of the three steps is followed by a manual quality assurance step that is optional, but recommended and in some cases required.

Figure 1: Overview of Historical Airphoto Processing Production Workflow

Step 1) Data IngestIn this first step of the workflow, the goal is to provide approximate positioning to historical images so that they are close enough for automated feature matching with accurate geocoded reference layers (images or vectors).

Input Requirements• Raw images in supported raster

format (e.g. TIFF)• Reference data (i.e. geocoded

reference image, road vector layer)• DEM (DTM)• Minimum Metadata

o Image nameo Focal Lengtho Image dimensionso Flying Heighto Nominal Center Coordinate

Operations1. Run a custom python script that ingeststhe images and metadata information in a batch process2. Perform semi-automated (interactive)fiducial mark measurements3. Run python script to generate initialmodel4. Review scene orientation andapproximate positioning


Data Ingest Result/OutputThe result from this processing step is the calculation of an initial model that is used as a starting position for automated feature matching. At the end of this step, images should be properly oriented and at minimum, have a positional accuracy that is improved.

Figure 2: Data Ingest Step Workflow

Step 2) Coarse AlignmentIn the second step of the production workflow, the goal is to improve positional accuracy of the historical airphotos, reaching an approximate accuracy of 200-300 pixels (from reference imagery).

Input Requirements• Output from previous step (Ingest)

is input into this step (CoarseAlignment)

• User can specify multiple projectsto run in a sequence (ProjectBatching)

• Custom strategy file can beprovided to customize parametersto improve efficiency

Operations1. Optionally, setup a custom strategyfile that defines parameters such as GCP search radius, number of GCPs, GCP/TP refinement2. Run Coarse Alignment Script3. Review results and performnecessary cleanup (Number of GCPs and TPs, Distribution, RMS and residual Error, Manually identified feature matching errors, if required, manually remove or add GCPs/TPs


Coarse Alignment Result/OutputThe result from this processing stage is the calculation of a new (coarse) model that is used as a starting position for the next stage of feature matching (fine alignment). At the end of this stage, images will have improved orientation and positional accuracy.

Figure 3: Results from coarse alignment displayed in Geomatica Focus Alberta, Canada

Step 3) Fine AlignmentIn the final step of the production workflow, the goal is to provide final positioning to historical images so that they achieve the stated spatial accuracy requirements.

Input Requirements• Output from previous step (Coarse

Alignment) is input into this step(Fine Alignment)

• User can specify multiple projectsto run in a sequence (ProjectBatching)

• Custom strategy file can beprovided to customize parametersto improve efficiency

Operations1. Optionally, setup a custom strategyfile that defines parameters such as GCP search radius, number of GCPs, GCP/TP refinement2. Run Fine Alignment Script3. Review results and performnecessary cleanup (Number of GCPs and TPs, Distribution, RMS and residual Error, Manually identified feature matching errors, if required, manually remove or add GCPs/TPs

Fine Alignment Result/OutputThe result from this processing stage is the calculation of a final model that achieves the final accuracy requirements for the historical images. The results from this stage are then used to generate the orthorectified images.


Figure 4: Results from fine alignment displayed in Geomatica Focus. Road vectors (overlayed) provide a reference layer to visually assess accuracy.

Further Processing (Ortho/Mosaic)With the alignment of the historical images complete, the refined models can be used to perform further processing of the imagery. In order to fully utilize the historical airphotos, producing a seamless image from the hundreds/thousands of individual airphotos is highly desirable. Standard Geomatica tools and/or scripts can be used to generate the orthorectified images and mosaics with high speed and automation.

Using Geomatica OrthoEngine, the Fine Alignment results are loaded along with a suitable Digital Elevation Model (DEM) to perform the orthorectification. Automated tools for generating and selecting cutlines ensure high quality results are achieved quickly. Quality Assurance steps can also be implemented through interactive cutline and colour balancing adjustment tools available in OrthoEngine.

Figure 5: Ortho / Mosaic Workflow implementation in OrthoEngine


Figure 6: Tools to adjust automatically generated cutlines and colour balancing

Quality AssessmentIn order to assess the quality of the results, two separate datasets of similar sizes (data volume, resolution, etc) were chosen and processed using the Historical Airphoto Processing system. The following characteristics describe the different datasets:Dataset #1 (good quality)• 266 scenes• 0.75m ground resolution• 155m initial RMS error (~206px)• Identifiable Fiducials• Similar to reference imagery (temporal

discontinuity)• Good image quality

Dataset #2 (poor quality)• 278 scenes• 0.6m ground resolution• 254m initial RMS error (~403px)• Poor quality fiducials marks• Significant differences with reference

imagery• Varying image quality (scratches, etc.)

Accuracy AssessmentReported accuracy based on control point verification:Dataset 1 (good): RMSE initial: 155m, RMSE Final 5.7mDataset 2: (poor): RMSE initial: 254m, RMSE Final 9.3m


Performance AssessmentThe processing time required for each of these project varied considerably, and it is important to better understand the processing bottlenecks - as processing strategies and approaches can be implemented to overcome the processing workflow challenges. The performance was measured in terms of operator time and computing resources to determine areas of bottlenecks.

Operator TimeThe number of hours allocated to the operator for the two datasets varied considerably. This can mainly be attributed to the poor quality of Dataset 1 which required setting fiducials using manual methods and manual inspection and editing of GCPs and TPs.

Computer TimeIn terms of computing resources, the two datasets varied considerably - the greatest processing bottleneck being the Fine Align, as a result of requiring a larger search radius to find suitable GCPs and TPs. Overall, the better quality dataset ran almost three times faster.

Quality and Performance DiscussionAs can be observed from the performance metrics acquired from the two datasets, the project turnaround time can vary significantly. The most influential factors are initial quality of the input imagery, the initial positional error and the degree of land cover change. Dataset 1 included relatively high quality input imagery that is well suited for automated processing. The total amount of operator time was approximately 2 hours and the automated processing of this dataset (computer time) took 16 hours. Dataset 2 included considerably worse input imagery and metadata information. The fiducial marks had to be measured manually, there was a higher degree of land cover change and the initial RMSE was 67% larger than the RMSE in the first dataset. With these additional processing obstacles, the project took up 15 hours of manual processing and 45 hours of automated (computer time) processing. The performance metrics obtained from the datasets processing performed by PCI are consistent with performance information reported by PCI’s customers, including Blackbridge.


he following benefits have been highlighted by users of the system:• The processing time, on average, was two times faster than conventional,

manual intensive approaches• Dataset can be run using a fully automated approach - once initial data has

been ingested, the rest of the processing can be completely automated• The ideal size / number of images to work with in a specific project is typically

between 300-600 images (although project of 1,000 images are not uncommon)• The use of the semi-automated HAP system significantly lowers operator time

(almost completely removes the need for intensive manual labour)• Quality specifications can be achieved - results of processing are of very

good quality in terms of accuracy, alignment, and colour balancingOperationalization and ScalabilityThe processing time savings as demonstrated in this study significantly reduce manual labour and shorten processing time. Implementation of the Historical Airphoto Processing Production Workflow on an operational level would further benefit from:

Staggered Processing - A processing computer and quality assurance computer, connected through a network would allow a single operator to stagger projects so that they can perform manual quality assurance on the QA machine without interruption to the processing.

Scalability - The system can be scaled up to include more processing systems, quality assurance computers and operators in order to achieve the throughput requirements of larger projects. The optimal ratio of processing systems / QA machines to operators is determined on a case by case basis.

About the AuthorShawn Melamed is an Application Specialist, and has been with PCI Geomatics since 2009. Shawn has worked in PCI’s Quality Assurance department, where he designed and tested new capability for PCI’s Geomatica software suite. Currently he is acting as a Applications Specialist for Pre-Sales Engineering, working with prospective customers to fit their operational requirements with PCI’s software solutions. Mr. Melamed received his Bachelor of Arts degree in Geomatics and Spatial Analysis from the University of Ottawa in 2008.

white paper semi-automated historical airphoto processingmetadata, large positional and orientation...

Documents