cause v transition - nisc · cause v transition - the data lifecycle 4 for near-realtime...

CAUSE V TransitionThe Data LifecycleJanuary 29, 2018

Table of Contents

Purpose.....................................................................................................1Introduction.............................................................................................. 2Setting the Stage........................................................................................6Data Collection Forms.............................................................................11Dashboards for Visualizing Data............................................................16Deploy!....................................................................................................21

PurposeThis document will provide guidance for setting up the end-to-end process for situational aware-ness, from field reporting tools to dashboards and common operating pictures. The intended audi-ence is an ArcGIS Online user with Administrator privileges for an organization, or other ArcGIS Online users, with the capability to create and publish feature layers and applications for use by their agency or company.

The guidance in this document may be used for your own exercises or actual incidents, where useful data could be collected by field personnel, and consumed by decision-makers.

CAUSE V Transition - The Data Lifecycle 1

The Canada U.S. Enhanced Resilience Experiment (CAUSE) series is a collaborative effort be-tween Defence Research and Development Canada’s – Centre for Security Science (DRDC CSS), Public Safety Canada (PS), and the U.S. Department of Homeland Security (DHS) Science and Technology (S&T) Directorate First Responders Group.

CAUSE V, the fifth experiment in the Canada-U.S. Enhanced Resiliency Experiment series, was held on November 15-16 in the state of Washington and the Lower Mainland of British Columbia. The experiment provided an opportunity for first responders and emergency managers to leverage emerging interoperable communications and response technologies and test whether these lead to better response and recovery outcomes for a cross-border incident.

Many different components were used to create a cloud-based situational awareness capability for CAUSE V. The use case called for personnel to be in the field, with tools that enabled them to efficiently relay relevant, coherent observations back to decision-makers back at the EOC.

Specifically, CAUSE V demonstrated a need for:

• Simple, form-based input via a variety of devices, either natively or by using web-based forms.• Shareable, cloud-based data models for layer data with a variety of options for filtering, sym-

bology, and analysis.• Powerful tools for creating dashboards and other visualizations, which can be secured, yet

accessible from a web browser by authenticated users.


Introduction

CAUSE V made use of this “End-To-End” workflow, beginning with off-the-shelf tools made available to field personnel and digital volunteers, flowing into a secure but shareable cloud GIS platform (ArcGIS Online), and made available to observers and decision-makers via configurable dashboards and applications.

Use cases for this data collection and visual-ization workflow during the CAUSE V exper-iment included the Incident Log, and the Dig-ital Volunteer Reports, which are described below.

Use CasesIncident LogThe Incident Log form is meant to capture in-cident information that is flowing in via com-munications personnel, email, or other sourc-es. It begins by allowing the user to categorize the new data, using a dropdown menu which is prepopulated with numerous sectors, Incident Log


such as Transportation, Public Safety, or En-ergy. The user is able to stipulate whether or not the location in question is operational or not, and can also indicate if follow up action is needed.

The Incident Log form was developed using Esri’s GeoForm application template.

Digital Volunteer ReportsThe Digital Volunteer report forms were used to collect input from the teams of volunteers which monitored simulated social media, based upon the scenario. Participants iden-tified trends in the information being posted and shared this information with the relevant decision makers to augment information from current channels. The teams also assisted with the verification of the social media data, by gathering additional, relevant metadata, such as hashtags and geographic location.

Survey123, which was used to create the re-port forms, is a data-gathering solution, which provides for both web-based and native form applications.

The instructions in this document will step through replicating the CAUSE V data work-flow, from entry forms, to cloud-based storage and customization, and finally to Esri’s Oper-ations Dashboard. These tools and data mod-els will be created in the user’s own ArcGIS Online organizational account. Visit Esri’s on-line documentation to learn more about Arc-GIS Online.

Digital Volunteer Reports


For near-realtime visualization, CAUSE V participants made use of Esri’s Operations Dashboard, which is a configurable app template, designed to provide location-aware data visualization and analysis.

Operations DashboardAn Operations Dashboard is configured around an ArcGIS Online web map, and may be set up to display features on the map, numerical metrics, bar graphs, lists, and rich text. Once published, the Operations Dashboard is available to decision-makers and observers via their web browser.

Operations Dashboard

The remainder of this document will walk through how to set up this data collection/visualization workflow in any ArcGIS Online organization.


Setting the StageCreating a Folder for ContentGet started by signing in into your organiza-tion’s ArcGIS Online account. You should be taken directly to your Content page. (It’s also possible that your account is configured to take you to a different page when you sign in. Sim-ply click on the Content link at the top of the web page to be taken to your Content page.)

Any folders that you have created will be list-ed in a column at the left side of the Content page. Click the “New” button near the top of that column to start the process of creating a new folder. A popup window will appear, prompting you for a Folder Name. Enter a sensible name, such as “CAUSE V Data”, and press the OK button to continue. A new folder with this name will appear in the left column, available for use.


Feature Layer SetupNow let’s create a feature layer in ArcGIS Online to hold incoming Incident Log data:

Proceed to your Contents page, and ensure that you are working in your newly-created folder (last chapter) by clicking on it. Above the Folders column, click the Create button:

Select the option for Feature Layer in the dropdown menu. A popup window labeled “Create a Feature Layer” will display. Three options will be available at the top of the popup: From Tem-plate, From Existing Layer, and From URL. Choose From URL to continue.


Copy and paste the following web link into the provided space for the URL:

https://services1.arcgis.com/Nt1b6lBKgKX3wost/ArcGIS/rest/services/CAUSEV_IncidentLog/FeatureServer/0

Press the Next button at the bottom of the popup window to continue to the layer selection step. The IncidentLog layer should be selected by default, but if it is not, check the box beside it.

Press the Next button again to continue to the map extent step. Drag the map, and zoom in and out, using the zoom buttons, to specifiy an area of interest for your new feature layer. (You can also double click on the map to zoom in.)

Press the Next button again to continue to the final step, which is where metadata for the new layer is provided. Give the layer an appropriate Title (e.g. “Charlotte NC Incident Log”), and at least one Tag. You may optionally provide Summary information, and select a different folder in which to create the layer. It should default to the layer in which you began this process.

Press the Done button to finish the configuration of the new layer. ArcGIS Online will begin the process of generating the new layer. When complete, you will be taken to it’s item page. Edit metadata such as the Description, Summary, or Terms of Use to adequately describe and docu-ment your new information source.


Creating a Web MapAn ArcGIS Online web map allows users to display geographic information in a configurable and interactive fashion. It can include a basemap, layers such as our new feature layer, and tools for analysis. Most importantly, it is the starting point for creating applications, with which users can visualize and contribute to your layer data.

Requirement: A web map requires an ArcGIS Online feature layer.

The best place to start (for the purposes of this walk-through) is the feature layer item page, which is where we left off in the last chapter.

Press the button at the top right of the page labeled Open in Map Viewer. ArcGIS Online will take you to a map viewer page, with your new feature layer loaded, as well as a basemap. The viewer will likely open in “Change Style” mode, and will show the color coded icons that the CAUSE V Incident Log template used. You can change this symbology if you like, or you can simply press the Done button at the bottom of the page to continue.

Once you have exited the Change Style screen, you should see the map on the right side of the page, and a simple list of layers in a column to the left. (Illustration below)


The map viewer page allows you to change the presentation of your layer data in many ways, such as:

• Altering the style (symbols, icons)• Altering the map extent (geographic area of interest)• Changing the basemap• Adding other data sources• Performing analyses• Adding bookmarks

It is outside of the scope of this walk-through to try to cover all of these topics. Visit here to learn more about ArcGIS Online web maps.

To save your new, configured web map, press the Save button at the top of the screen. You will be prompted for a Title (e.g. Charlotte NC Incident Log Map) and Tags for the web map, and an optional Summary Description. Press the Save Map button when you’re ready to save your new web map.

Note: A best practice for naming web maps is to make it a recognizable name for the current in-cident or exercise, so that users can easily find them in ArcGIS Online.


Data Collection FormsThe ArcGIS Online platform provides some standard applications for use by field personnel and other users that want to contribute data to be shown and analyzed in a data lifecycle:

GeoFormGeoform is a configurable app template for reporting data. This application allows users to enter data through a form instead of a map’s pop-up while leveraging the power of the Web Map and editable Feature Services. This app geo-enables data and workflows by lowering the barrier of entry for completing simple tasks.

Requirement: A GeoForm requires an ArcGIS Online web map.

For more information about Esri’s GeoForm application template, go here.

Survey123Survey123 is a digital solution for creating form-based surveys, and publishing them as either web-based forms or as native applications. The solution supports more sophisticated functional-ity such as XLSForm-based logic and calculations, offline data caching, and editing of existing data records. It’s also possible to download the Survey123 source code and use Esri’s AppStudio development environment to customize the applications capabilities and user experience.

Requirement: A Survey123 application requires an ArcGIS Online web map. (The web map may be created by the application workflow itself.)For more information about Esri’s Survey123 digital solution, go here.

For simplicity, this document will walk through the process of setting up the CAUSE V-style In-cident Log using GeoForm.


Creating the Incident Log GeoFormThe most common way to create a GeoForm is within the ArcGIS Online map viewer. Conve-niently, the last chapter (Creating a Web Map) left off in the map viewer for our newly-created Incident Log web map. If you are not still in the map viewer, proceed to your ArcGIS Online Contents page, locate the Incident Log web map, and select Open in Map Viewer. Before pro-ceeding, make sure that you have saved your web map.

To start the process of creating the GeoForm application, press the Share button at the top of the map viewer page. The Share popup window will open, presenting options for sharing within your organization, within groups, or with Everyone. (The public) Sharing will discussed in detail later; for now, proceed farther down the popup window to the button labeled Create a Web App.

Press the Create a Web App button to continue. You will be presented with another popup win-dow, titled Create a New Web App. A search box will be on the right side of the popup. Type in the word GeoForm to filter the applications that are available. The GeoForm tile should appear in the popup.


Press the GeoForm tile, and then press the Create Web App button that appears.

Similar to the previously-mentioned process for creating a web map, you will be prompted to enter metadata such as the Title (e.g. Charlotte NC Incident Log Form), tags, and an optional summary description. Press the Done button to continue. This will begin the step-by-step process of generating a GeoForm application in ArcGIS Online.

GeoForm Steps There are 10 steps in the process of creating the GeoForm, at the time of this writing. For the purposes of this tutorial, you may simply press the Next button for the first 2 steps, Start and Webmap.

Step 3 - LayerIf you added more layers to your web map during its creation, you can use this step to focus the GeoForm on the specific layer about which you’re gathering data. If you did not add more layers, then you can simply press the Next button.


Step 4 - DetailsThe Details configuration allows you to set several things that will appear on your published GeoForm, such as the Title, Logo Image, and custom Description, which may include helpful information and links. Press the Next button when satisfied with the configuration.

Step 5 - FieldsThe Fields configuration allows you to select which fields will be presented to users of the form. You may also make your form more user-friendly by changing the labels that are shown, and even add help text and hints for the user Press Next to continue.

Step 6 - StyleThe Style picker presents options for the look and feel that the user will see when they use the form. Select the option that is preferred, and press Next.

Step 7 - ViewerThis step presents the option for enabling the Viewer, which will show submissions to the Geo-Form in a web mapping application, should the user of the form desire it. Press Next to continue.

Step 8 - OptionsChoose what options, such as social media sharing, pushpin customization etc, to include in your form. Press Next to continue.

Step 9 - PreviewThe Preview step allows you to see what the GeoForm will look like prior to actually publishing it. This is an opportune time to make changes, by stepping back through the configuration process and altering your choices.

Step 10 - PublishThe Publish step is the final stop prior to publishing the GeoForm. Press Save and Exit to finish the process. A popup should appear to notify you that you are publishing the form, and then you will be taken to the new application’s item page in ArcGIS Online.

From the new GeoForm’s item page, you may press the View Application button at the right side of the page to try out the new form. The chosen fields will be presented to the surveyed user, with dropdown options where applicable. A map will be displayed so that the user may find the loca-tion of the incident.

When the Submit button at the bottom of the form is pressed, a new data entry will be added to the Incident Log feature layer.


GeoForm Application


DashboardsFor Visualizing Data

Operations Dashboard for ArcGIS is a configurable web application that provides location-aware data visualization and analytics for a real-time operational view of people, services, assets, and events. During the CAUSE V Experiment, emergency managers used this Dashboard to monitor crowd-sourced data, damage assessments, input from social media, and logged incidents.

Requirement: An Operations Dashboard requires an ArcGIS Online web map.

Creating the DashboardAs with the data collection apps described earlier, the logical place to begin the configuration of the Operations Dashboard is in the ArcGIS Online map viewer. If you are not still in the map viewer, proceed to your ArcGIS Online contents, locate the Incident Log web map, and select Open in Map Viewer. Before proceeding, make sure that you have saved your web map.

To start the process of creating the GeoForm application, press the Share button at the top of the map viewer page. The Share popup window will appear. Press the Create a Web App button near the middle of the popup window to continue. The Create a New Web App popup window that appears will present 3 options: Configurable Apps, Web AppBuilder, and Operations Dash-board. Select the Operations Dashboard option to continue.

Fill in metadata for the new Dashboard, such as a Title (e.g. Charlotte NC Incident Log Dash-board), Tags, and an optional Summary Description. Press Done to continue.

ArcGIS Online will take you to the screen where you can configure your new Dashboard. To be-gin with, you will only have a map, featuring the web map you created.


At the top of the Operations Dashboard con-figuration screen, you will find a dropdown menu with a “plus sign” (+). Press the plus sign to see the options which may be added.

Click on Header to add that option. You will be taken to the page to configure its appear-ance. Feel free to change such items as Title, Logo, and Background. Press Done when ready to continue.

Next, press the + button and click on Indi-cator. On the configuration screen, select the layer to show in a number. (There may be only one option.) Use the next screen that appears to adjust the data that is shown, and the ap-pearance of the Indicator. You may add a filter, based upon a field value, show different statis-tics, or multiple the value by a factor.

Using the Indicator tab on the left side of the screen, you can add text to the Indicator, or include an icon. The General tab allows you to add metadata to the Indicator such as a Title or Descrip-tion, and to adjust colors. Press the Done button at the bottom right of the screen when you are ready to continue.

At this point, your Operations Dashboard may look something like this:


By default, the Dashboard configuration has made the map and the Indicator the same size. It’s likely that you want the Indicator to be much smaller. Note the gray line that runs down the mid-dle of the screen. You can click and drag that line to the left to make the Indicator much smaller, and allow your map to have more screen real estate.

Click the highlighted, blue disk button at the top right to save your Dashboard’s configuration at any time during this process.

Click the plus button again to show options, and press the List button. Select the desired layer (again, there may only be one) on the next screen, and proceed to the List configuration screen. From this screen, you may filter or sort by a field in the data, or select how many features to show in the list. As with the Indicator configuration, use the List and General tabs at the left side of the screen to add text or style the List. When adding Line Item Text, you may show a field value by putting it in squiggly brackets:

{Category}{MessageText}

This would configure the Line Item for each feature to show something like:


Press the Done button when you are ready to continue.

When you return to your Dashboard, you will see that the configurator has made the List very large, and placed it to the left of the rest of your Dashboard items. To create a column with the Indicator and the List, do the follow-ing:

1) Hover your mouse (cursor) over the List. A small blue box will appear at the top right of it. Now hover over that blue box. A number of buttons will appear. The top one is labeled Drag Item.

2) While hovering your mouse over the Drag Item button, click down on your mouse, and begin to drag the item over the previously-created Indicator. It will show options for where to drop the List. Go to the bottom square and drop the List there.


Your Operations Dashboard should now look similar to this:

Using a similar workflow, you may add Serial Charts, Pie Charts, Gauges, etc, which can be tied to the values that are stored in your web map’s feature data.


Deploy!Sharing Within Your OrganizationNow that you’ve created your information products (the Incident Log form and Operations Dash-board), it’s time to make them available throughout your ArcGIS Online organization. The mech-anism for making data and applications available is the sharing group. A group with a meaning-ful name can be created, and items such as GeoForms and Operations Dashboards can be shared into it. This makes those applications available to any users that are then populated into the group.

To share our new GeoForm and Dashboard (and corresponding layer and web map), following these steps:

1) Create a sharing group in ArcGIS OnlineClick the Groups link at the top of the page, while in ArcGIS Online. On the Groups page, click the button labeled Create New Group.


On the Group Details screen, specify a meaningful Group Name (e.g. Incident Log Group), Tags, and an optional Summary Description.

The next four radio button options determine the extent to which users can view, join, contribute, and update items in the new group. Select the options that fit your use case.

Note: The last option, “What items in the group can its members update?”, should be considered carefully. If the default option, “Only their own items”, is changed to “All items”, it will alter the group such that only members of your ArcGIS Online organization can be members. This could present a problem, if you ultimately want to share outside of your organization.

Press the Create Group button at the bottom of the screen when ready.

2) Share your newly-created information products to the groupClick on the Content link at the top of the page to return to your Content page. Find your appli-cations, web map, and feature layer, which were probably created inside of the folder that you created at the beginning of this tutorial. (Note that the GeoForm shows up in the list as a “Web Mapping Application”, and the Operations Dashboard is simply shown as a “Dashboard”)

Click on the checkboxes to the left of these items to select them.


Click the Share button, above the selected items. A Share popup window will appear, present-ing options for sharing the items at different levels. Under “These groups”, look for the group which you created in the last steps. In this example, the group is called Incident Log Group. Check that box, and click OK to continue.

Next, add some users to the group. Click on the Groups link at the top of the page, and then click on the group that you created. It should appear in a list on the right side of the page. (Again, it is “Incident Log Group” in this example.)

Once the group page is opened, click on Invite Users to add users to your group. The Invite Users popup window will appear. Search for the user names of users from within your organization, to make them appear in the left column of the popup. When the names appear, click on them to add them to the right column. (Under Invitation List.) Click the Send Invitation button when you have searched-for and added the desired users for your group.

Note: Depending upon your privileges in ArcGIS Online, you may have a checkbox option la-beled “Add members of the organization immediately, without requiring confirmation”. If this box is checked, it will expedite the process, since requested users will immediately be in the group. If this is not done, then the requested users must proactively confirm their membership in the group, when they are logged into ArcGIS Online. They will not actually be in the group until this step is done.

At this point, you will have a group, which is populated by user(s). In the Group page, you can click on the Content tab to view your items. We have added a Feature Layer, Web Map, GeoForm (Web Mapping Application), and a Dashboard. By clicking on the title for the Dashboard or Geo-Form, you can view those applications. When they are open, you can copy their URL’s from your browser’s URL bar, and send them directly to the users in your group. They should be able to use the applications immediately.


CAUSE V demonstrated the need to share information products and data to experiment partici-pants that are outside of your ArcGIS Online organization. Since sharing within this online en-vironment is accomplished through the use of sharing groups, it is necessary for these people to have some way to log in via a named user in that group.

For participants that do not have an ArcGIS Online account, there are 2 options:

• Acquire a user account from the ArcGIS Online organization in which the information prod-ucts were developed.

• Acquire an account in another ArcGIS Online organization

If you are using either option to create a user account on behalf of an exercise participant, it is important to follow up with an email, notifying them of their new ArcGIS Online credentials, and any other helpful information about the login process.

Considerations to Each ApproachAcquiring a User Account to the Organization Where the Information Products ResideThis is the most straight-forward approach. An Administrator for the ArcGIS Online organization can simply add a new user, and then use the process outlined in the last chapter to add the user to the sharing group.

A disadvantage to this approach would be that it requires another user for which the organization is licensed. If the organization is entitled to up to 50 users, they will not be able to license 100 exercise participants (or users during an actual disaster), without some kind of action being taken. (i.e. upgrading the licensing for the organization, or getting permission from Esri to expand their user base for the during of the event.)

Acquiring an Account in Another ArcGIS Online OrganizationIf an exercise participant works for/in conjunction with an agency which happens to have their own ArcGIS Online account, they may already have a user account, or be able to get one by work-ing with the organization’s ArcGIS Online administrator. This user account name could then be added to the sharing group in the same way that was outlined in the last chapter.


Sharing Outside of Your Organization

Note: When adding group members that are from another organization, the search will be done by unchecking the box labeled “Only search for members in [organization name]”, as shown below.


As with the first approach, there may be limits to how many users the organization has available for use.

Other Considerations for Sharing

The group sharing approach outlined above is one simple approach, but it may be that an agen-cy or company wants to segregate access to their data collection and visualization tools. Instead of having a single sharing group, it is possible to create one group for data collection, and an-other for dashboards and other products that present the data.

In the case of this walk-through, the GeoForm could be shared to the data collection group, and then it would be populated by field personnel who would be using that information product.

Similarly, the Dashboard would be shared to a “Dashboard Group”, and shared to executives and operations users.

It’s also worth noting that if an actual disaster strikes, and you find yourself in need of more users than your ArcGIS Online organization can accommodate, you can request assistance from the Esri Disaster Response Program.

cause v transition - nisc · cause v transition - the data lifecycle 4 for near-realtime...

Documents