Field Validation

Background:

When creating complex GIS databases, it is essential that all or a significant portion of the digitized (or

acquired) data should to be validated; this is done with various methods but the most common is “field

validation”, visiting the location and comparing GIS data to your observations. Remote Sensing often

calls this process “ground-truthing”. To accomplish verification you have to have some very detailed

version (copy) of your data available in the field to compare with your observations. You also need a

process to record the results of your observations.

Initial concept:

Prepare your data within a GIS to use in the field. This

could be a very detailed printed copy of a map of your

study area or some electronic version (hand-held device).

The key issue is that all of the data you need to verify

should to be “OBVIOUS” on the map or easily found on the

handheld device. This means that “scale” is very

important. You must print a very detailed large-scale map

or have the ability to change the scale on your hand-held

device. (zoom-in).

The common issue with exporting data

is to render your data at a level of

detail to support the required scale

change (zoom-in/out).

1st step:

Prepare a map of your study area (or

the area in which you will perform the

validation).

Create a PDF of your area. PDF’s are

useful as you can provide details of

many relevant layers. Use of aerial

imagery is also common but often you

need to limit the amount of

information to just what is needed for

validation and only enough related layers to maintain your orientation.

Export the PDF in the highest

resolution possible at the largest

scale possible.

In this example, I used a “tabloid

size” at 600 DPI.

You should also ensure that the

Advanced Tab (Export Map as

PDF) is set to “Export Map

Georeference Information”

2nd Step: Move the PDF to the hand-held device.

There are several methods to move a PDF (in this case a geo-

spatially enabled or GeoPDF) to a hand-held. A common approach

is to email the PDF as an attachment. For this example we use and

“App” that transfers files over a FTP WiFi connection.

Using an Android based device, Nexus 7, start the installed “App” ES

File Explorer.

If you do not see a screen that looks like the one to the right, swipe

across the screen to display the “desktop” with ES File Explorer.

Below (left), find the “down arrow” in the middle of the screen and

it will bring up a full screen; like the one on the right. Select the

“View on PC” tool.

Select the “View on PC” tool.

On the left screen (below) tap the blue button at the bottom “Turn On”

Then note the ftp address. Shown

to the right is an EXAMPLE, your

numbers will be similar but not

exactly the same.

This is an FTP address.

On your computer (on the same

WiFi as the hand-held) navigate to

File Explorer (not an internet

browser).

Type in the search bar EXACTLY what you see on the hand-held eg: ftp://10.132.109.73:3721/

You will now see the contents of the hand-held (as shown above right).

Copy your geo PDF map of your area to be verified to the PDFMaps folder on the hand-held device.

Remember the ES File Explorer connection process as you will use the “opposite” to copy your field

verified data from the hand-held to your computer.

On the handheld, open PDF Maps and add the PDF map you just transferred (see next page).

Import your PDF map from the PDFMaps folder on the hand-held unit.

When finished importing, open the PDF and explore the map.

An example of a geospatially enabled PDF map

in Avenza Maps (PDF Maps)

Zoom in or pan around.

Find the features you wish

to verify.

Notice the cursor (above) is colored in “orange”. The cursor color is user selectable. Use your finger to

move the map around “under” the cursor.

Verifying points (pins) will only be dropped at the cursor location. If you drop a pin right now, it will be

located at your cursor location. Pins are dropped by tapping the “pin” icon at the top left.

When you have an active GNSS position, and you are “on the map”, it will display as a “Blue Dot”

If you walk around on the map the “Blue Dot” moves displaying

your current position. If you wish to drop a pin at your current

position you need to center the cursor (now shown in BLACK)

over the “Blue Dot”.

You do this by pressing the

“target” icon.

Then drop the pin

Practice dropping a pin, You

should erase practice work

before field verifying your

features.

As you drop a “pin” select the right arrow to open the detail screen

This detail screen the “generic”

Placemark screen.

We will use a “custom” screen with

attribute appropriate for verification

To delete a “pin” aka Placemark, select the “trash” icon in

the upper right of the Edit Map Feature, when Placemark detail is displayed

Custom Attributes:

For our work we will be collection verification details for several “different” types of features.

For each type we should first create a “folder” and then add unique detailed attributes to the features

within that folder.

For example: Folder: Canopy

Attributes: Type, string Choices: Conifer, Broadleaf

Height, decimal

Verified, Boolean ( this means Yes or No)

Date, string

Folder: SewerGrate

Attributes: Verified, Boolean

Date, string

Folder: Building

Attributes: Roof Type, string Choices: Flat, Pitched, Steep Pitched

Height, decimal

Verified, Boolean

Date, string

You can create these folders and add the attributes yourself or load the .kml file provided

Data Verify Exercise Tabloid.kml

To add your own: select the Pin list from the top menu

and then select

Then “hamburger” button at the bottom right.

Then select “Add Folder” (not shown here); remember to select the

check mark to complete each step.

Add as many folders and need. Having separate folders will help

you later just export the Pins (Placemarks) you want.

Now you will select each folder (one at a time) and add the attributes. To Select a folder from the list,

do a LONG press, that will turn the folder title YELLOW. Then select the PENCIL icon in the upper

toolbar to add your attributes.

My empty folder: Here, I long pressed and selected the Pencil icon

Now press the Plus by Attribute Schema.

Add your attributes (if you wish)

Then add a new folder and add attributes to that new folder. Repeat as often as needed.

Here is my completed: Folder List

Remember to LONG press to enter the folder and select the pencil

icon for edit.

Here is my Buildings folder

attributes:

Notice that the Choices for the Roof Type or the Yes/No of the

Verified does not show here. The choices only come up

when you need them at entry time.

(if you want to you can touch the entry for Roof Type and see or

change the details).

Next validation data in the field.

Remember you have folders for each type of verification. You will only be able to drop a pin in to one

folder at a time. This means you must REMEMBER WHICH FOLDER IS CURRENT.

At the folder level there is a listing for SET TO CURRENT FOLDER,

Press that and it will CHANGE to CURRENT FOLDER

You can also change the colors of the

pins if you wish.

Now with your folder current, add

pins for that verification type. For

each pin complete all the attributes.

If you make a mistake, you can select the individual pin within a folder and LONG PRESS, then using eht

MOVE FOLDER icon, move it to another folder. If you do move points, the attributes might need to be

re-entered or corrected.

To send your data back to your GIS.

There are several steps:

1. At the Map Features (list of folders) select the “hamburger icon” and select Export Map

Features.

2. Now you have to select File Name for Export what you want your output to be called).

3. Select Export Format. You can send out KML but you lose your attribute data. To send out each

record with the attributes, select CSV

4. Select Send To: here you want Device Storage but you can “email it to yourself”

5. Now what to Export, if you select visible only you can control the grouping of the output. If you

export everything, it will send all the records for each folder, all together and you will have to

sort it out in ArcMap. The best approach is back a step. At the folder list LONG PRESS on each

folder and select the “eyeball” icon at the top to make all the records in that folder “NOT

VISIBLE”. If you leave one folder VISIBLE and the others NOT VISIBLE you can control the export

process. Remember to check the Export visible only box on the Export Map Features screen.

6. Repeat the connection with ES File Explorer and copy the exported files to your computer.

To Load CSV files into ArcMAP

In ArcMap tables in several formats can be loaded; CSV, Text, Excel, dBase, etc.

You add the your CSV table to the

ArcMap Table of Contents (TOC). Open

the table and make sure you understand

the headings and all the information is

present.

Then Use the FilleAdd X, Y data menu

selection to add the table to the map.

You have indicate which is the X value

and which is the Y value. Also, you have

to state the projection/datum that the

table is using. Most commonly external

CSV, KML & GPX files are in Geographic

(WGS84).

Select OK to the Object-ID warning p

Add X,Y data to ArcMap DOES NOT CREATE A

SHAPEFILE. THE LOADED TABLE IS A EVENT

You must select the event and EXPORT DATA to

a Shapefile.

Finally, you have many choices to choose how you add

(combined) the field data with your GIS data layers.

You can APPEND DATA or transfer the data manually, or do an

Identity, Spatial Join (recommended) and then use Field

Calculator to update fields.

Note: if you are attempting to perform a Spatial Join between

Sewer Grates (as Points) and Verified Sewer Grates (as Points),

you may need specify a Search Radius.