0110011010101011010010101111010010001001001011010

1101010101101011101010011001101010101101001010111010010110

011001101010101101001010101011110001010010101001000010001010001

Data ManagementChad A. Steed

Naval Research Laboratory

30 July 2004 USM 2004 Upper Pearl Survey 2

Discussion Topics

• Survey Logs

• Digital Data Administration

• Analog Data Management

• Primary Computer Systems

• Lessons Learned

30 July 2004 USM 2004 Upper Pearl Survey 3

Vessel Survey Logs

• 3 ring binder for all survey log sheets– Bertram log and Skiff log

• Vessel logs contain:– Daily survey crew assignments and date– Equipment setup and breakdown times and settings.

• Ancillary observations recorded in vessel logs– Skiff: Digibar sound speed profiles, Bar Check– Bertram: CTD cast

• Logs also contain time stamp for other significant events (e.g., bumping logs, equipment problems)

• Survey line start and end times.

30 July 2004 USM 2004 Upper Pearl Survey 4

Other Logs

• Data Processing Log– USM1 Base Station Setup / Breakdown– Simultaneous Tide Gauge Observations– Data Processing Notes– Side Scan / Contact Processing Notes

• Personal Logs– Personal notebooks for temporarily recording

events when the official log is not available– Significant events recorded in personal logs are

eventually transferred to the official log book

30 July 2004 USM 2004 Upper Pearl Survey 5

Digital Data Repository

• Digital data repository established on the \\Hydro\HydroData network drive.

• The major stages of the survey are represented by sub folders.

• Each stage folder is further sub-divided by the type of dataset (e.g., Bertram, Skiff, Geodetics, Tides).

• Data mining datasets and the final products are also organized in repository

• Digital data repository backup to \\Coral\SANS drive after significant changes.

30 July 2004 USM 2004 Upper Pearl Survey 6

Vessel Digital Data

• Skiff data was recorded on the HYPACK Max laptop.– The HYPACK RAW files were then

copied to the digital survey repository.

• Bertram data was recorded in ISS-60 and on EM-3000 computer.– At end of survey day a DVD was

burned with Bertram data and transported to the building 1206.

– The DVD was then copied to the digital survey repository and stored in the Bertram day folder.

30 July 2004 USM 2004 Upper Pearl Survey 7

Tides / GPS Digital Data

• Tides– Tide gauge data was downloaded and

copied to the survey repository.– Simultaneous observations were

transferred to an electronic format (Excel Spreadsheet) and archived in the repository.

• Geodetics– Rover receivers (Skiff, Bertram) cards

were purged to the repository each survey day.

– Base receiver (USM1) data card was purged to the repository each survey day.

30 July 2004 USM 2004 Upper Pearl Survey 8

Analog Data Management

• Log Binders hold all original log sheets.• Vessel Day Folders

– Organized by survey day– For collecting all of survey day analog (paper) records.– Skiff day folders include Knudsen paper trace (folded)

and copied of survey logs– Bertram day folders include a DVD with the survey

day’s MBES dataset (ISS60 and Simrad format) and CTD data files and copies of the official survey log sheets.

• All analog records are stored in a marked storage box in the USM Hydrographic Science Library (building 1206 Stennis Space Center).

30 July 2004 USM 2004 Upper Pearl Survey 9

Primary Computer Systems

• Bigmaus – CARIS HIPS, QTC bottom analysis, Fledermaus.

• HYDROWS13 – HPD• 1206classroom –

Contacts, SBES HIPS, Side Scan.

• Personal Laptops – MBES Line cleaning, HYPACK, Tides, misc. tasks

Bigmaus1206classroom

HYDROWS13Laptop

30 July 2004 USM 2004 Upper Pearl Survey 10

Lessons Learned

• Logs are invaluable tools for data processing.• Hard drives are cheap and facilitate easier backup

of critical datasets.• Entire survey required about 300 GB for all digital

data.• CARIS HIPS can be used to break up line cleaning

(kind of).– HIPS can reboot your laptop!

• Dave Dodd’s office phone is x87127• USM Hydro should consider designating a network

drive that is backed up regularly and automatically.• Having another high end workstation (like

Bigmaus) would have made processing easier.

Preparation Activities:Skiff Line Development

Rafael Ponce