cloudy with a chance of fish · cloudy with a chance of fish tiffany c. vance nmfs/alaska fisheries...
TRANSCRIPT
![Page 1: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/1.jpg)
Cloudy with a chance of fish
Tiffany C. Vance
NMFS/Alaska Fisheries Science Center Stephen Sontag and Kyle Wilcox RPS/Applied Science Associates
![Page 2: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/2.jpg)
We’re not in California anymore
![Page 3: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/3.jpg)
Participants and Research Subjects
• Fisheries biologists, biological and physical oceanographers studying fisheries recruitment in the Gulf of Alaska, Bering Sea and Arctic Ocean
• Walleye pollock and Pacific salmon • Early life history in relation to recruitment • Field and laboratory research, modeling • GIS and IT in the cloud/server
![Page 4: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/4.jpg)
![Page 5: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/5.jpg)
Sample and Data Types
• Oceanographic data – CTD, water chemistry, nutrients
• Biological data – zooplankton and ichthyo- plankton samples, age-0 and age-1 fish, diet studies
• Modeling – currents, temperature, salinity, particle-tracking
• Ancillary data – station location, weather conditions, sample information
![Page 6: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/6.jpg)
Walleye Pollock and Pacific Salmon
![Page 7: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/7.jpg)
EcoDAAT and LarvaMap
• EcoDAAT – ArcServer/Oracle based data discovery and display
• LarvaMap – Amazon cloud based modeling of larval dissemination
![Page 8: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/8.jpg)
EcoDAAT Goals and Process
• Design single Oracle database • Load data from existing Access databases • Prototype front-end created in ArcServer • Flex viewer, user authentication • Link to ArcGIS for further analyses • Tiered data access • Test deployment on an Amazon cloud hosted
instance of ArcServer
![Page 9: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/9.jpg)
EcoDAAT Architecture
![Page 10: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/10.jpg)
Main Interface
• LDAP authentication • ArcServer security as an option
![Page 11: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/11.jpg)
Forms-based Data Selection
![Page 12: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/12.jpg)
Map-based Data Selection
![Page 13: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/13.jpg)
Results
![Page 14: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/14.jpg)
Querying and Exporting Data
![Page 15: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/15.jpg)
Spatial Analyses
![Page 16: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/16.jpg)
LarvaMap: Particle Tracking 101 • Atmospheric and oceanographic applications § Volcanic plumes § Hazmat releases § Sediment § Larval fish
• Circulation model output • Particle tracking model • Visualization of results
![Page 17: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/17.jpg)
• Circulation model output, e.g. ROMS • 4D - velocities at each grid point • May also have water properties such as temperature • DAP server – THREDDS, Hyrax
Particle Tracking Model • Behaviors of particles
• Particle transport model – frequently parallelized
• Run locally or remotely
• Data management
![Page 18: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/18.jpg)
Complete System • Behavior library to create, catalog, and share larva
behaviors: http://bit.ly/WFiZjK
• Fate and transport model written as a python library: http://bit.ly/WFjz17
• LarvaService implementing the transport library in a cloud architecture
• Web client interacting with LarvaService through REST API: http://bit.ly/14aZinD
![Page 19: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/19.jpg)
LarvaMap Web Client
LarvaMap Web Service
Run Queue
Transport Model Transport Model
Transport Model Transport Model
Transport Model Transport Model Remote Datasets
(THREDDS)
Workflow
1.) User configures run
2.) Run request sent to service and validated
3. ) Run request is put onto queue
4. ) Configured Amazon instance picks up run from queue
5. ) Raw data are returned to service
6. ) Service generates output formats and visualizations
7. ) User downloads results
![Page 20: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/20.jpg)
Remote dataset Local “cached” copy
Data Extent Data Extent
• Data requested “as needed” from remote DAP endpoint • Particles use local cached copy in forcing algorithms
Remote Data Request
![Page 21: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/21.jpg)
Remote dataset Local “cached” copy
Data Extent
Particle reaches extent of cache and requests new data be collected
“DataController” gathers remote data and adds to local cache
Remote Data Request
![Page 22: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/22.jpg)
Remote dataset Local “cached” copy
Data Extent
Particles continue to run and the process is repeated
Remote Data Request
![Page 23: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/23.jpg)
Remote dataset Local “cached” copy
Data Extent Data Extent
Particle reaches extent of cache and requests new data be collected
“DataController” gathers remote data and adds to local cache
Remote Data Request
![Page 24: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/24.jpg)
Remote dataset Local “cached” copy
Data Extent Data Extent
Particles continue to run and the process is repeated
Remote Data Request
![Page 25: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/25.jpg)
Remote dataset Local “cached” copy
Data Extent
Eventually…
Remote Data Request
![Page 26: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/26.jpg)
Net CDF and Shapefile Particle Output
• Direct import into ArcGIS • Foundation for 2D mapping visualizations • netCDF complies with CF1.6 Discrete Sampling Geometries • Use of compliance checker for CF • Compliant units and standard names where applicable • Foundation for movie/animation visualizations
![Page 27: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/27.jpg)
Case Study – Pollock Larval Transport
• Field data suggest transport both into Shelikof Strait and along the shelf • Testing release (spawning) off Kenai Peninsula • 10 – 100 particles • Released May 15th
• Model run for 100 days • Took 1:32 to run
![Page 28: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/28.jpg)
Results
![Page 29: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/29.jpg)
Stress Testing LarvaMap as a teaching tool – WHOI class using it this fall – excellent stress testing
![Page 30: Cloudy with a chance of fish · Cloudy with a chance of fish Tiffany C. Vance NMFS/Alaska Fisheries Science Center . Stephen Sontag and Kyle Wilcox . RPS/Applied Science Associates](https://reader031.vdocuments.us/reader031/viewer/2022011900/5f0563df7e708231d412b930/html5/thumbnails/30.jpg)
Future Possibilities
• Linking LarvaMap and EcoDAAT – start a model run and automatically gather any associated in situ data for display with model results
• Store model runs in EcoDAAT • Very large scale model outputs (100,000
particles, 1 year, 300GB) and how to display and visualize in ArcGIS