Intersite Coordination (ISC)Intersite Coordination (ISC)
Mark Mathewson
Sept 24-27, 2002 Intersite Coordination 2
OverviewOverview
Motivation for ISC How ISC Works GFESuite ISC Capabilities
Including several lab exercises Diagnosing ISC Problems Discussion
Sept 24-27, 2002 Intersite Coordination 3
Motivation for ISCMotivation for ISC
Sept 24-27, 2002 Intersite Coordination 4
Motivation for ISCMotivation for ISC
Obvious – want your forecast to be consistent with your neighbors
But previously, it wasn’t obvious if your forecast was different. Hard to compare words
NDFD (National Digital Forecast Database) changes all of this.
Sept 24-27, 2002 Intersite Coordination 5
Motivation Motivation for ISCfor ISC
Sept 24-27, 2002 Intersite Coordination 6
How ISC WorksHow ISC Works
Sept 24-27, 2002 Intersite Coordination 7
How ISC WorksHow ISC Works
Sending ISC Grids is triggered by the forecaster in a variety of ways.
Grids are sent over the WAN to your neighbors.
Grids are mosaic’d with other WFO ISC grids and stored in the ISC database.
Sept 24-27, 2002 Intersite Coordination 8
ISC InternalsISC Internals
Sept 24-27, 2002 Intersite Coordination 9
Spatial MosaicSpatial Mosaic
Sept 24-27, 2002 Intersite Coordination 10
Temporal MosaicTemporal Mosaic
Sept 24-27, 2002 Intersite Coordination 11
Sample ISCSample ISC Database Database
Central Cluster NDFD
EAX Perspective
Sept 24-27, 2002 Intersite Coordination 12
Mosaic Technique Mosaic Technique CharacteristicsCharacteristics
There may be more than one ISC grid for one of your Fcst grids. Max/MinT across time zones are interesting.
There may be “blank” areas if not all adjacent WFO grids are available.
Sept 24-27, 2002 Intersite Coordination 13
NDFD and Intersite CoordinationNDFD and Intersite Coordination Related – but not the same.
WFO
WFO
WFO
Fcst
Fcst
Fcst
WAN
WAN
WAN
Intersite Coordination Grids
Official
Official
Official
NDFD CentralServer
NDFD
Customers
WAN
WAN
WAN
NDFD Grids
Sept 24-27, 2002 Intersite Coordination 14
GFESuite ISC CapabilitiesGFESuite ISC Capabilities
Sept 24-27, 2002 Intersite Coordination 15
GFESuite ISC CapabilitiesGFESuite ISC Capabilities Sending Grids Displaying ISC Mosaic Displaying ISC and Fcst grids together Discrepancy Tools
IFPS10 ISC and IFPS11 ISC is different. Improved dialogs and use. IFPS12 has more changes, including ISC for
combined public and marine areas
Sept 24-27, 2002 Intersite Coordination 16
Send ISC DialogSend ISC Dialog Select which weather elements Select time period -- allows single grid Select which sites to transmit your grids to
Sept 24-27, 2002 Intersite Coordination 17
Publish DialogPublish Dialog
Can select weather element
Can select time period
Can choose to send ISC or not All sites always
Sept 24-27, 2002 Intersite Coordination 18
Save Weather Element DialogSave Weather Element Dialog
Sends to all sites Sends just the grids that
are to be saved In IFPS12 In IFPS11, all grids
from weather element sent
Sept 24-27, 2002 Intersite Coordination 19
sendISCsendISC
sendISC program can be used to send ISC grids. Can define: Time range over which grids are sent Weather Elements Sites, including sites that are NOT defined in
ISC_Sites. Official database or Fcst database
Sept 24-27, 2002 Intersite Coordination 20
Displaying ISC GridsDisplaying ISC Grids
Load ISC database from Weather Element Browser
Sept 24-27, 2002 Intersite Coordination 21
Lab Exercise 1Lab Exercise 1
Edit today’s Maximum Temperature grid. Send it to all your neighbors. View the ISC mosaic by loading the ISC
database through Weather Element Browser Watch as other sites appear/change Status bar indicates receipt of data Your data will also appear
Sept 24-27, 2002 Intersite Coordination 22
GFESuite – “Show ISC Mode”GFESuite – “Show ISC Mode”
Available from the toolbar
Merges Fcst database over your CWA with ISC database over remainder of grid
Sept 24-27, 2002 Intersite Coordination 23
GFESuite – “Show ISC Mode”GFESuite – “Show ISC Mode”
Entire grid from EAX
OAX
EAX
SGF
TOP
ICT
Sept 24-27, 2002 Intersite Coordination 24
GFESuite – “Show ISC Mode”GFESuite – “Show ISC Mode”
Samples show: ISC Update Time ISC Site Identifier “P” for data from published database Data values
Markers show: Similar to samples, but no data values Useful for anchoring to monitor ISC
Sept 24-27, 2002 Intersite Coordination 25
GFESuite – “Show ISC Mode”GFESuite – “Show ISC Mode”
Data Value
Site IDUpdate Time
Official DB
SAMPLEs
MARKERs
Sept 24-27, 2002 Intersite Coordination 26
GFESuite – “Show ISC Mode”GFESuite – “Show ISC Mode”
Samples/Markers Site ID, UpdateTime, “Publish” flag can be
turned off Markers
Set are calculated with ifpServer edit areas on startup
Suggest you remove extra ones, move existing ones the way you like as “SITE”.
Sept 24-27, 2002 Intersite Coordination 27
Grid Data HistoryGrid Data History
Sept 24-27, 2002 Intersite Coordination 28
Lab Exercise - 2Lab Exercise - 2
Use tomorrow’s Maximum Temperature Modify it, choose whether to publish or just send
ISC Experiment with “Show ISC Mode”
Set up “markers” – MB3 with sample tool Explore turning on/off sample modes and marker
modes through Maps->Samples and Maps->Markers menus
Try Time-Stepping – observe behavior with/without “Show ISC Mode” on.
Sept 24-27, 2002 Intersite Coordination 29
““Show ISC Mode” effect on Show ISC Mode” effect on toolstools
You already see the change in the Sample Tool.
When “Show ISC Mode” is on: Pickup value tool pickups Fcst or ISC data Move/Copy pickups Fcst or ISC data Contour Tool pickups Fcst or ISC data Pencil Tool pickups Fcst or ISC data
Sept 24-27, 2002 Intersite Coordination 30
““Show ISC Mode”…Show ISC Mode”…
Smart Tools are not affected by ISC mode. If you want to access ISC data from smart
tools, “T” is always Fcst, no ISC data. Use getGrids() to access ISC data.
Temporal Editor – ignores ISC mode Grid Manager – doesn’t show ISC mode Spatial Editor Legends – do change
Sept 24-27, 2002 Intersite Coordination 31
Lab Exercise - 3Lab Exercise - 3
Experiment with “Show ISC Mode” on and off with: Sample Tool Move/Copy Tool Pencil Tool Contour Tool Smart Tools – such as Assign Value and
Smooth
Sept 24-27, 2002 Intersite Coordination 32
Discrepancy ToolsDiscrepancy Tools
Discrepancy Tools are simply smart tools. You can extend and write your own.
Allow you to analyze all grids quickly. Highlights grids in grid manager if thresholds
exceeded Can look at difference grid Can look at edit area that exceeds thresholds Uses discrepancy value, or discrepancy grid.
Sept 24-27, 2002 Intersite Coordination 33
Discrepancy ToolsDiscrepancy Tools
Procedure ISC_Discrepancies
Tools Show_ISC_Highlights Show_ISC_Grid Show_ISC_Area
Edit Area ISC_Tool_Area
Sept 24-27, 2002 Intersite Coordination 34
ISC_Tool_AreaISC_Tool_Area Calculated when ifpServer starts. Tools use this area for calculations.
CWA boundary
Sept 24-27, 2002 Intersite Coordination 35
Discrepancy AlgorithmsDiscrepancy Algorithms
Four neighboring points are examined
Discrepancy Value or Grid used to determine significant discrepancies
Uses “Composite” Grid (Fcst + ISC)
Neighboring Points
Sept 24-27, 2002 Intersite Coordination 36
Discrepancy AlgorithmsDiscrepancy Algorithms ISC-Fcst Difference Subtracts two fields Discrepancy Value or Grid used to determine
significant discrepancies
Fcst ISC
EAXOAX EAXOAX
Sept 24-27, 2002 Intersite Coordination 37
Discrepancy Value and GridDiscrepancy Value and Grid
Can set discrepancy value for each weather element.
Sept 24-27, 2002 Intersite Coordination 38
Discrepancy GridDiscrepancy Grid Can create discrepancy grids.
Allows varying values per gridpoint.
Use normal editing tools to create it.
Sept 24-27, 2002 Intersite Coordination 39
ISC_Discrepancies DialogISC_Discrepancies DialogTime Period
Weather Elements
Sept 24-27, 2002 Intersite Coordination 40
Discrepancy HighlightingDiscrepancy HighlightingGrids highlighted in red indicate that thresholds were exceeded.
Sept 24-27, 2002 Intersite Coordination 41
Difference GridDifference Grid
Original Fcst/ISC Grids
83
56
Difference Grid
Fcst – ISC grid = TDiff
Sept 24-27, 2002 Intersite Coordination 42
Discrepancy Edit AreaDiscrepancy Edit Area
Edit Area, Exceeding thresholds
Sept 24-27, 2002 Intersite Coordination 43
Lab Exercise - 4Lab Exercise - 4
Edit MaxT, Save it Send grids Run ISC_Discrepancy tool
Grids are highlighted Look at discrepancy edit area Look at discrepancy difference grid Reset the highlights
Experiment with various options. Try both algorithms.
Sept 24-27, 2002 Intersite Coordination 44
ISC ISC (current)(current) Limitations Limitations
Supplied tools don’t handle: Wind direction Weather
ISC area based on shapefiles CWAs only through IFPS11 Marine and CWAs IFPS12+
Sept 24-27, 2002 Intersite Coordination 45
ISC ISC (current)(current) Limitations Limitations
Remember, ISC display and editing tools (non-smart), are hard-coded. Need your input to improve them.
However, there really are no other tool limitations, since smart tools are used. You can write your own algorithms/techniques.
Sept 24-27, 2002 Intersite Coordination 46
Diagnosing ISC ProblemsDiagnosing ISC Problems
Sept 24-27, 2002 Intersite Coordination 47
Diagnosing ISC ProblemsDiagnosing ISC Problems
ISC has a fairly complicated processing path.
Numerous log files are used to diagnose problems.
Log Files are documented.
Sept 24-27, 2002 Intersite Coordination 48
Common ISC ProblemsCommon ISC Problems
I can’t see other site’s ISC grids.
I can’t see my own ISC grids.
Other sites can’t see my ISC grids.
Sept 24-27, 2002 Intersite Coordination 49
Log Files – the key to diagnosing problemsLog Files – the key to diagnosing problems
Sept 24-27, 2002 Intersite Coordination 50
Log Files – the key to diagnosing problemsLog Files – the key to diagnosing problems
Receiving Grids
for iscDataRec
17:13:52.991 EVENT: received: /data/adapt/GFESuite/iscDataRec/FSLC-339357.001 859317:13:53.165 EVENT: Sent to: rpp-fslc 98000000
for iscMosaic
00:20:00.348 iscMosaic.py 791 EVENT: iscMosaic Starting00:20:00.528 iscMosaic.py 665 EVENT: Processing file=/home/awipsusr/release/products/ISC/iscWO7Sgf00:20:00.533 iscMosaic.py 665 EVENT: Input file size: 275092iscMosaic.py 665 VERBOSE: Processing T_SFC #Grids= 23 Site= GID00:20:56.413 iscMosaic.py 791 EVENT: iscMosaic Finished
Traceback with problem if fails, starting in IFPS11
Sept 24-27, 2002 Intersite Coordination 51
Log Files – the key to diagnosing problemsLog Files – the key to diagnosing problems
Sending Grids
for ifpnetCDF:
18:36:38.191 ifpnetCDF.py 918 EVENT: ifpnetCDF Starting18:36:38.734 ifpnetCDF.py 659 EVENT: Saved 10 T grids18:36:39.234 ifpnetCDF.py 659 EVENT: Saved 9 Td grids18:36:44.632 ifpnetCDF.py 983 EVENT: ifpnetCDF Finished
For iscExtract:
16:53:50.778 EVENT: Sent: size: 8594 ,wctime: 15.47704494 ,cputime: 0.14 ,sites: ['FSLC'] ,time: -s 19700101_0000 -e 20380119_0314 ,parms: ['T']
For iscd:
17:15:02.814 EVENT: sending: fname: @7955.1.isc.gz ,size: 8591 ,to: ['FSLC']
Sept 24-27, 2002 Intersite Coordination 52
Collaboration TechniquesCollaboration Techniques
Is ISC really enough? Coordination of grids may come too late in the
forecast process. NDFD Clusters have been experimenting
with: FX/C or FX/Collaborate
Sept 24-27, 2002 Intersite Coordination 53
DiscussionDiscussion
Sept 24-27, 2002 Intersite Coordination 54
DiscussionDiscussion
Shortcomings of current tools/methods Tools which could help?
ISC Consensus Tool Methodology which could help?