exor corporation limited · 2011-11-24 · 4 nov 2010 ah product options bypassctl, bypasshac have...
TRANSCRIPT
Exor Corporation Limited r
The Global Leader in
Infrastructure Asset Management
Street Gazetteer Manager System Admin V4.5
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd, England, 2008 All rights reserved
This page intentionally blank
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
i
Quality Assurance Statement
Document Details File:
Street Gazetteer Manager System Admin V4.5.doc
Prepared by: T.C. Stewart
Manual Name: Street Gazetteer Manager System Admin V4.5
Reviewed by: P.Rogers
Version: 1.0
Approved for issue by: T.C Stewart
Date of Issue: Nov 2010
Product Manager: P.Rogers
File Name: Document Centre/Exor/Product Manuals/4.5.0.0 Product Manuals/4.5.0.0 completed
documentation/Street Gazetteer Manager Admin v4.5.docx
Document Version Control Revision Date By Chapter
/ Page Description
1 30-Jan-2008 CS 6 The following new or previously undocumented Product Options have been added: ESULIMIT, FORCEVERS, LOADHIST, NSGEXPACC,NSGEXPHIST, NSGSPECIAL,STRLIMIT,XREFLIMIT
1 30-Jan-08 CS 6 The following product Option has been removed: NSGNUMERIC. It has been superseded by NSGSPECIAL
1 30-Jan-08 CS 6 Highway Authority Code removed from NSG0060
1 31-Jan-08 CS New Modules added
16 NSG0070 – Maintain Highway Authorities
18 NSG0080 – Maintain Naming Authorities
19 NSG0090 - Configure ASD Record Type
20 NSG0110 – Organisations and Districts
2 Sep-2008 CS Released with 4.0.5
3 Jan 2010 AH All Released with 4.2
4 Nov 2010 AH Product Options BYPASSCTL, BYPASSHAC have been added.
4 Nov 2010 AH NSG0040 – NSG Loader Batches
4 Nov 2010 AH NSG0045 – Monitor NSG Loader
5 May 2011 IS None No changes for v4.4
6 Nov 2011 IS P33 NSG metadata refresh note added
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
ii
Contents Quality Assurance Statement ......................................................................................................... i Product Options .............................................................................................................................. 7 This value is shipped by Exor and must not be changed. ........................................................... 9 Adding New Towns and Localities – NSG0060 ........................................................................... 12
Language ................................................................................................................... 13 Counties ..................................................................................................................... 13 Towns ......................................................................................................................... 14 Localities .................................................................................................................... 14
Maintaining Highway Authorities – NSG0070 ............................................................................. 16 Highways Authority ..................................................................................................... 16 County ........................................................................................................................ 17 Refresh ....................................................................................................................... 17
Maintaining Naming Authorities – NSG0080 ............................................................................... 18 Configure ASD Record Types – NSG0090 .................................................................................. 19
Classification .............................................................................................................. 19 Record Type ............................................................................................................... 19
Maintaining Organisations and District Information – NSG0110 ............................................... 20 EToN .......................................................................................................................... 21 FPN (Fixed Penalty Notice) Contact ........................................................................... 21 FPN Payment ............................................................................................................. 22 Validation Warnings .................................................................................................... 23
Reset Street Coordinates – NSG0015 .......................................................................................... 24 Generate ASD Placements – NSG0025........................................................................................ 25
Common Errors .......................................................................................................... 26 Importing NSG and ASD Data ...................................................................................................... 27 NSG Load Batches - NSG0040 ..................................................................................................... 28
Batches tab ................................................................................................................ 29 Files in Batch tab ........................................................................................................ 32 Batch Schedule tab .................................................................................................... 32 Records tab ................................................................................................................ 35 Batch Log tab ............................................................................................................. 36 Floating Toolbar .......................................................................................................... 38
Monitor NSG Loader - NSG0045 .................................................................................................. 41 Administer User Districts – NSG0120 ......................................................................................... 45
General Information .................................................................................................... 45 User Districts .............................................................................................................. 46
My Districts - NSG0130 ................................................................................................................. 47 General Information .................................................................................................... 47 User Districts .............................................................................................................. 47
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
iii
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
4
CHAPTER
1 Introduction
Exor Street Gazetteer Manager is a BS7666 compliant application for creating and maintaining a level 3 Street Gazetteer and Associated Street Data (ASD). Developed using Oracle Spatial technology, it provides an open and interoperable database solution that can be accessed by any other System in use across the Organization. Street Gazetteer Manager provides an easy to use, map driven interface that provides an Authority with industry standard, accurate methods of creating and managing Street and Associated Street Data which underpins Exor’s solution for the Traffic Management Act. Under the TMA, one of the key responsibilities of the Network Manager is to coordinate works and other activities occurring on the Network. Accurate, easily understood Associated Street Data is essential to this task. Street Gazetteer Manager assists in this process by automatically creating a ‘linear event’ representing the location of the ASD record. This can then be viewed on the map without the need to ‘hand digitise’ the ASD record manually. All changes to Street information and Associated Street Data are audited and ‘closed’ Streets and ASD are retained for historical purposes. An ‘effective date’ may be set which allows the User to view the Street Gazetteer and any Associated Street Data that were open at the specified effective date. This is essential for dispute resolution when an undertaker may be subject to a Fixed Penalty Notice for an invalid submission. Street Gazetteer Manager is suitable for all scales of implementation and can be deployed on a single Desktop with the data being available to the entire organization via Exor’s web mapping functionality. Street Gazetteer Manager has been developed as an extension to Spatial Manager which is based upon ESRI® ArcGIS® 9.x - ArcMap application, utilizing the advanced graphic editing functionality and user interface that ArcMap offers, including many tools to manage the way that information is displayed on maps in the on-screen map view and in printed map layouts. To find out more about this functionality see ArcMap documentation. This document describes the functions and options available to the Systems Administrator to assist in the maintenance and definition of the Local Street Gazetteer. The options described are normally carried out by the Systems Administrator and not the end user, however this may vary from site to site.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
5
Refer to the Street Gazetteer User Guide for information on the how to manage and maintain your LSG.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
6
CHAPTER
2 Using Street Gazetteer Manager
The following functions are included within this document:
SGM related Product Options
Adding new Towns and Localities – NSG0060
Maintaining Highway Authorities – NSG0070
Maintaining Naming Authorities – NSG0080
Configure ASD Record Types – NSG0090
Maintaining Organisation and District Information – NSG0110
Reset Street Coordinates – NSG0015
Generate ASD Placements – NSG0025
NSG Load Batches – NSG0040
Monitor NSG Loader – NSG0045
Administer User Districts – NSG01020
My User Districts – NSG0130
Note Before using Street Gazetteer Manager ensure that all necessary Product Options are set to an appropriate value for your Organisation. Product Option Details can be found on page 6.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
7
Product Options
Figure 1 Product Options Menu
Product Options allow ‘switches’ or default values to be set, which defines how certain functionality operates within the Exor suite of Products. The following Street Gazetteer Manager related Product Options should be set to suit your Organisational requirements. Product Options are set using HIG9130 – Product Options. Product Options marked with a ‘*’ are also User Options (see the Street Gazetteer Manager User Guide for more information).
Product Code
Option Id Option Name Sample Value
Remarks
HIG NSGDATA System uses NSG data
Y This MUST be set to Y if the Organization is a Highway Authority who maintains their LSG within Exor.
NSG BYPASSCTL Bypass CTL refresh
N Bypass refresh of County/Town/Locality during the load of NSG data
NSG BYPASSHAC Bypass HA County refresh
N Bypass refresh of Highway Authority counties during a load of NSG data.
NSG CHECKODXML Check OD XML Content on Export
Y If set to Y when OD XML files are produced they will be checked for compliancy against EtoN Organisation/ District XSD
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
8
Product Code
Option Id Option Name Sample Value
Remarks
NSG CREATE_CTL Create counties, towns and loc
Y This option works in conjunction with the NSG Loader. If set to Y any New Counties, Towns or Localities detected during a data load will be automatically created. Counties, Towns and Localities can be viewed and maintained using NSG0060 (see page 6).
NSG DFLTDMODE Default Loader Mode
FULL Default mode to use when submitting NSG Loader batches
NSG DFLTSUBMIT Default Loader Submission Time
18:00 Default time to use when submitting NSG Loader batches HH24:MM
NSG* DFLT_AU Default Admin Unit Code
DCC This is the Admin Unit Code assigned to a Street. This is not used with BS7666 , but is used as part of the overall Exor security mechanism to restrict access to certain Streets to certain Users.
NSG* DFLT_CNTY Default County DORSET This is the default value for the ‘County Name’ field. The ‘Highway Authority’ code will be derived from this.
NSG* DFLT_NAUTH Default Naming Authority Code
1235 This is the default value for the ‘Naming Authority’.
NSG* DFLT_TOWN Default Town TURTON This is the default value for the Street Town name. The List of Towns will only contain valid Towns within the selected ‘County’. (See page 6 for details on maintaining a List of Towns).
NSG* DFLT_TRNCE Default Tolerance
10 This is the default value for the Street Tolerance (in metres).
NSG* DFLT_TYPE Default Record Type
1 This is the default NSG Record Type. The available options are 1,2,3 or 4.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
9
Product Code
Option Id Option Name Sample Value
Remarks
NSG DFTVERSION DFT spec version number
5.3 This value is shipped by Exor and must not be changed.
NSG FORCEVER Force Version for Export
Y This option is used when exporting the LSG for submission to the NSG Hub. If set to ‘Y’, any Streets where new ESU’s have been added or ESU’s have been removed since the date of the last Street version will be automatically versioned.
NSG LOADSTATS Gather Stats as data is loaded
Y If set to Y the system will gather statistics on tables as data is loaded i.e. with the aim of improving performances of the load process
NSG NSGDISP NSG ESU Displacement value
1 This value is used when the ESU mid point is being calculated to ensure uniqueness. It should only be changed if instructed to do so by Exor support staff.
NSG* NSGDISTREF NSG Extract District Ref
001 This is the District ref used when creating the Organisation and District extract file for submission to the Hub. This value should not be changed.
NSG NSGEXPACC NSG Export Accuracy
3 This value determines the number of decimal places used for Street, ESU and ASD coordinates. When exporting the LSG to the NSG Hub this must be set to ‘0’.
NSG NSGEXPSIZE NSG Export Record Size
500 This value is used in conjunction with the NSG exporter. It is the number of records retrieved per fetch. This value should only be changed if instructed to do so by Exor support staff.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
10
Product Code
Option Id Option Name Sample Value
Remarks
NSG NSGLANG1 Primary Language
ENG Primary language to attribute to descriptions LG submissions/to attribute to streets when loading gazetteer files.
NSG NSGORGREF NSG Extract Organisation Ref
1245 This is the Organisation ref used when creating the Organisation and District extract file for submission to the Hub.
NSG NSGSPECIAL Allow special characters
N If this is set to ‘Y’, characters defined within the ‘NSGSPECIAL’ character set may be used within the Street record. This should only include Characters other than those defined by the Custodian for Character Set 2. Special Characters are maintained using HIG9150 – Maintain Character Sets. Refer to the General System Admin Guide for further details.
NSG USEALIASES Aliases are in use?
N This flag should only be set to ‘Y’ for Welsh Authorities. It is used to enable the use of Street Aliases
NSG VERSIONING Versioning in use
Y This option must be set to Y if the Organisation maintains its LSG data within Exor, i.e. a Highway Authority. For Utilities Customers this should be set to N.
NSG NSGIMPMSG NSG Import message Level
S This option should be set to D or S. If set to 'D' - all messages that the NSG loader produces will be logged If set to 'S' - just the key messages that the NSG loader produces will be logged.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
11
Product Code
Option Id Option Name Sample Value
Remarks
NSG ESULIMIT Limit for ESU loading
5000 This value is used in conjunction with the ‘NSG File Loader’. It determines the commit points during a load. The default value should not be amended unless recommended by Exor Support.
NSG LOADHIST Load historical data
Y If this value is set to Y, then all versions of Streets and ESU’s will be processed when loading data using the ‘NSG File Loader’. If set to ‘N’ only the most recent versions are processed. This should be set to ‘Y’ after initial data take on.
NSG NSGEXPHIST Export Full NSG History
N If set to ‘Y’ all versions of Streets and ESU’s are exported. When exporting the LSG to the NSG Hub for all TMA compliant submissions, this must be set to ‘N’
NSG STRLIMIT Limit for Street loading
5000 These values are used in conjunction with the ‘NSG File Loader’. It determines the commit points during a load. The default value should not be amended unless recommended by Exor Support.
NSG XREFLIMIT Limit for XREF loading
10000
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
12
Adding New Towns and Localities – NSG0060
Figure 2
Street Gazetteer Manager provides validation when creating new Streets to ensure that valid County, Town and Locality values are added. These ‘locations’ exist within a hierarchical order with the County (Highway Authority) being at the top of the hierarchy. Each County will have many Towns and each Town may have many Localities. When adding these details for a Street, only those Towns that exist within the selected County may be chosen. Similarly, only Localities that exist within the selected Town may be chosen. This ensures that data integrity and quality is maintained. The initial hierarchy of County/Town and Localities is established from existing Street Data when Street Gazetteer is implemented. If new Gazetteers are loaded using the Street Gazetteer Loaders the new hierarchies can be created and existing hierarchies maintained automatically if Product Option CREATE_CTL is set to ‘Y’.
Note: this option would normally be set to ‘N’ for non Highway Authority customers.
Locations are maintained within the main Exor Application using the Maintain Locations – NSG0060 module.
Figure 3
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
13
The Locations module is made up of 4 separate areas, one each for maintaining Language, Counties, Towns and Localities and. Any changes made in this form are immediately available with the Street Gazetteer form. The multi language functionality allows counties/towns and localities to be grouped by language code.
Figure 4
Language
The Language panel will display the available languages. These are set up using Domains – HIG9120 under the option NSG_STREET_LANG. Name Display Only
The code for the language will be displayed. Language Display Only
A description of the language will be displayed.
Figure 5
Counties
This panel is used to maintain the List of Counties managed within Street Gazetteer Manager. Name Required
This is the Name of the County that will appear in the Highway Authority Description field and the County field within the Street Gazetteer form.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
14
Figure 6
Towns
This panel is used to add and maintain a valid list of Towns within the selected County (Highway Authority). To add a new Town ensure the cursor focus is set within the Town panel and press [Create Record] button on the menu toolbar. Town
This is the Town Name. A maximum of 30 characters is allowed
Figure 7
Localities
This panel is used to add and maintain a valid list of Localities within the selected Town. To add a new Locality ensure the correct Town is selected and that the cursor focus is set within the Locality panel. Press [Create
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
15
Record] button on the menu toolbar. Locality
This is the Town Name. A maximum of 35 characters is allowed
Figure 8
The [Regenerate Data] button when pressed looks at all of the streets that exist and the attributes of the street and builds up a definitive list of counties/towns/localities.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
16
Maintaining Highway Authorities – NSG0070
Figure 9
Street Gazetteer Manager is shipped with a complete set of UK Highways Authorities and Codes and should not normally need to be amended. Each Authority may be associated with one or more ‘Counties’. Under normal circumstances this should be a one to one relationship, however, several different spellings of the same ‘COUNTY’ value may exist within some Local Street Gazetteers. During the loader stage new ‘Counties’ are automatically created (only if Product Option CREATE_CTL is set to Y) if they do not already exist within Street Gazetteer Manager. This facility prevents failure of the loading process if the data is of poor quality. Highways Authorities are maintained using NSG0070 – Maintain Highways Authorities. To ensure that any new ‘Counties’ create during a ‘Gazetteer Refresh’ are correctly associated with their appropriate Highway Authority the County List should be refreshed using the [Refresh] button at the bottom of the form.
Highways Authority
The top block displays the Highway Authority Code and Name. A Flag may also be set to determine if the Authority Name is displayed in the relevant List of values in the main Streets Form.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
17
Code Required
This is the Code for the Highway Authority Name Required
This is the Highway Authority Name. Visible Checkbox
If the Visible flag is set, the Authority will be included in the relevant List of values within the Streets form both with Spatial Manager and the Forms module. Authorities not flagged as Visible are still valid and may be queried, but will not appear within the LOV. Figure 10 shows an example where 2 Authorities have been set as Visible and the ‘Highway Authority’ List of Values within the Streets Form.
Figure 10
Figure 11
County
This panel allows a ‘County’ to be associated with a Highway Authority. This should normally be the same as the Highway Authority but in some cases several spellings of the same ‘County’ may exist with a Local Street Gazetteer that has been loaded from a 3rd party application. Where this is the case a flag may be set to define the ‘Preferred’ County. Only the preferred value will be displayed within the List of Values for the County attribute within the main Streets Form.
Refresh
To ensure that any new ‘Counties’ create during a ‘Gazetteer Refresh’ are correctly associated with their appropriate Highway Authority the County List should be refreshed using the [Refresh] button at the bottom of the form.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
18
Maintaining Naming Authorities – NSG0080
Figure 12
A full set of Naming Authorities along with their respective Codes and USRN Prefixes is shipped as standard within Street Gazetteer Manager and should not normally need to be maintained. If however changes are required, this is done using NSG0080 – Maintain Naming Authorities.
Figure 13
The List of Values for the Naming Authority field within the Street Form can be restricted to those Naming Authorities used within a Highways Authority by checking the ‘Visible’ flag for only those values required. If a change is made within the form press the [Save] button on the menu toolbar. Figure 14 shows the Naming Authority LOV for the ‘values flagged as Visible above.
Figure 14
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
19
Configure ASD Record Types – NSG0090
Figure 15
Street Gazetteer Manager is designed to enable the management of ASD data in England/Wales, Scotland or both simultaneously. The Description of the ASD record Types, displayed within all relevant screens, can be defined as required. ASD classifications and Record Type descriptions are defined using NSG0090 – Configure ASD.
Figure 16
Classification
This panel is used to indicate the ASD regime(s) required. Customers working in the proximity to the England/Scotland border may require both regimes to be in place.
Record Type
The Description for an ASD record type can be amended as required. The value entered is used within all relevant screens when displaying ASD records. Figure 17 shows an example where both the English/Welsh and Scottish ASD regimes are in use.
Figure 17
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
20
Maintaining Organisations and District Information – NSG0110
Figure 18
The Organisations and Districts module is used to identify all organisations that are involved within the street works arena. This includes Utilities, Highways Authorities and Promoters. This form is pre-populated by the loading of an organisations OD XML file (Operational District File). Organisations can be split into as many districts as required to simplify the day to day running of their business.
Figure 19
Note that the District Works Ref prefix column is only displayed if the TMA product is licenced.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
21
On the right of the form there are a number of tabs with details of Eton requirements, FPN Contact details, FPN Delivery Address and FPN Payment Details.
Figure 20
EToN
EtoN (Electronic transfer of notices) details are used to send and receive files electronically. These details get transmitted within the OD files to other Organisations to allow this electronic communication.
Figure 21
FPN (Fixed Penalty Notice) Contact
These are the contact details of the person issuing the FPN’s. An undertaker may need to contact this person in the event of Fines being issues. These details also get sent in the OD file.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
22
Figure 22
This is the Delivery address that the fines will go to. This can be either a postal address or a URL address.
Figure 23
FPN Payment
The FPN Payment panel contains the postal address information and or the bank details that the fines will be paid into. The Organisation and District details can be filled in manually to enable the production of your own OD batch file in either XML or the old standard text file. These may be produced using either the [Produce OD Batch File] or [Produce OD XML File] depending on the post or pre TMA regime in use. These files are also produced when the LSG is extracted for submission to the National Hub.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
23
Figure 24
Validation Warnings
If for any reason any Mandatory attributes values are missing the [Validation Warnings] button will be made available. The number of warnings will also be displayed. To view the list of issues press the button.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
24
Reset Street Coordinates – NSG0015
Figure 25
The Reset Street Coordinates – NSG0015 module is an Administrators module and allows the Start and End Coordinates for every Level 3 street to be checked and reset where appropriate, using the Street’s constituent ESU’s. Any Streets which have their Start and/or End Coordinates updated will be versioned. The Start and/or end Coordinates can be checked and reset for individual Streets using the [Calc Coords] button (Spatial Manager) or [Reset Nodes] button (Web Forms) on the Maintain Streets form. During the process an indicator will be displayed showing the progress of the job.
Figure 26
Note that running this process may take several hours
Tip
If this is being done for error correction purposes and Street Versioning is not required, set product option ‘VERSIONING’ (page 10) to N. On completion of the process this Product Option MUST be reset to Y.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
25
Generate ASD Placements – NSG0025
Figure 27
The Generate ASD Placements – NSG0025 module is an administrator’s module that generates ASD membership records for all ASD items flagged as not ‘Whole Road. Within the form each column heading can be used as a sort button. Select the heading once to sort ascending (^) and again to sort descending (v). For Exor customers who create and manage their ASD within Street Gazetteer Manager, running this module is a one-off task that should be run before ASD Spatial Layers are initially created. The creation of subsequent ‘partial’ ASD member records is automated when new ASD records are created or existing records are amended. For Exor Customers who load their ASD data using the NSG Loader – NSG0040, this process must be repeated after each successful data load.
Figure 28
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
26
To create the ‘Partial’ ASD members press the [Perform Operation] button. If ASD data from more than one Highway Authority has been loaded, the process can be restricted to a specific Authority by selected the required Authority from the available List. The default is to generate ASD placements for All Authorities. When the operation is performed, all Level 3 Streets (i.e. Streets with constituent ESU’s) with no ASD member records are displayed. The process cycles through each Street in turn and creates the ‘Partial’ ASD members for each ASD record Type. To view all errors at the end of the process press the [Enter Query] button, tick the ‘Errors’ field then press the [Execute Query] button.
Common Errors
There are several common errors that can occur during the generation of ASD placement records.
Error Message Meaning
ORA-20001: NO PATH ORA-06512: AT "DCCNSG.NM_CNCTS", LINE 625 ORA-06512: AT "DCCNSG.NM_CNCTS", LINE 1
This error means that the ESU’s between the Start and End Coordinates of the ‘Partial ASD’ record are not connected, i.e. there is no connected path between the start and end of the ASD record.
ORA-20001: NO STREET ELEMENTS CLOSE ENOUGH TO THE XY CO-ORDINATES ORA-06512: AT "DCCNSG.NM_CNCTS"
This error means that the Start and/or End Coordinates of the ‘Partial ASD’ record are not within the tolerance distance of the Street. This is generally due to poor quality ASD data
Refer to the Spatial Server Admin Guide for details on the actual Layer Creation process.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
27
Importing NSG and ASD Data
Figure 29
Street Gazetteer Manager allows the Import of a Level 3 Gazetteer complete with Associated Street Data (types 21, 22 and 23) and an Organisation and District file. Importing data is done using the NSG Load Batches – NSG0040 module. The progress of the load can be viewed using the Monitor NSG Loader – NSG0045.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
28
NSG Load Batches - NSG0040
Figure 30
Figure 31
The NSG Loader Batches – NSG0040 module is used to load the gazetteer files into the database. Multiple files can be added to any given batch. Multiple batches can be set up and then loaded sequentially or at a defined time. For example, gazetteer loads can be set up during the working day but then be submitted to run overnight when no users are on the system. A batch can be composed of a mixture of files of any data transfer format and from a mixture of suppliers.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
29
As a load progresses, full details of the activities undertaken are written to a log. There is one log per batch. This is similar to the functionality of the previous loader but is less verbose. It is designed to complement the higher level information shown in the batch schedule. The file column headings can be used as sort buttons. Press once to sort in ascending order (^). Press again to sort in descending order (v).
Figure 32
Batches tab
The Batches tab is used to identify the batch name when creating a new batch. Once the batch has been selected for processing further details regarding the batch will be shown automatically. Batch Default
The batch number will be added by the system when the record is saved. Name Required
Enter the name of the batch. Available Date Display Only
The available date will display the date and time that the batch is to be processed once it has been selected. Load Started Display Only
The date and time the load process started will be displayed. Load Ended Display Only
The date and time the load process finished will be displayed. Successful Display Only
The number of files that have been loaded successfully will be displayed. Unsuccessful Display Only
The number of files that have not been loaded successfully will be displayed. Completion Message Display Only
Once the batch loading has finished this will display the message ‘Processing Complete’.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
30
[Files Selection] The [Files Selection] button is used to select the files to be placed in the batch. Pressing this button will call the File Selection window.
Figure 33
Directory Default
This will display the filepath set in the Directories – HIG1895 module under the option NSG_DATA. This can be amended by pressing the [Set] button to call form Directories – HIG1895. .csv Radio Button
Select this radio button to display only csv files in the directory. .xml Radio Button
Select this radio button to display all xml files in the directory. It may be that the user wishes to load all the organisation and district files first. This can be done easily be selecting the .xml button and then the [Select All] button. [Refresh Directory] button refreshes the directory listing to pick up files deposited in or removed from the NSG_DATA directory since the last listing was produced. [Copy file to Directory] button allows a file held locally to be copied down to the NSG_DATA directory. Details of the files available will include:
File name
Organisation code
Organisation name
Date
Format
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
31
DTF version number Files are selected by checking the box against the required record. [Auto Pick] will apply a filter on filenames in the directory that have previously been attributed to a batch and tick the file checkbox for inclusion in the batch. [Create a Batch per Supplier] will list all files in the directory (not just those queried back in the form) and create a batch per distinct supplier and read the files in for each batch. It does not automatically submit the resultant batch. [Add Selected File(s) to Batch] will add the selected file to the batch. Status Display Only
Initially the status will refer to the file highlighted and give a status message of, for example, ‘File appears valid’. Once the files have been selected and the [Add Selected File(s) to Batch] pressed the status message will change.
Figure 34
Once the files have been read and pre-processed with no errors the File Selection window will close. The files will have been moved from the NSG_DATA directory to the NSG_PROCESSED directory.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
32
Figure 35
Files in Batch tab
This tab will display the files selected during the File Selection process and display:
File name
Organisation code
Organisation name
Date
Format
DTF version number
Read error If there is an error in reading the file it can be removed from the batch using the [Delete Record] button. Status Message Display Only
Initially the status message will read ‘File ready for processing’.
Figure 36
Batch Schedule tab
This tab will track the progress of the batch load process. A number of operations must be performed, depending on the records read from files associated to the batch. At the time of submission these operations are identified and built into the batch schedule. This information is presented to the end user to gauge progress, success and performance of the load.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
33
Each operation attributed to a batch schedule is referred to as a stage. Once a stage has been processed it will be attributed with a status depending on the outcome of the processing. Dropping of Spatial Indexes If a batch contains street or ESU or cross references or ASD records then the very first stage of the batch schedule will be to drop the spatial indexes. This aids performance when processing data. When indexes are dropped a warning message is shown against the stage within the batch schedule tab. This is to re-iterate to the user the fact that the indexes have been dropped and that they will need to be recreated before the application can be used.
Figure 37 Warning
Street Spatial Consolidation If a batch contains street or ESU or cross references records then the last but one stage of the batch schedule will be a ‘Street Spatial Consolidation’. The processing carried out at this stage ensures that the spatial data for streets within the batch is kept up to date.
Where a street has been closed, existing street shapes will be end dated.
Where an esu member has been removed from a street the effected shape will be end dated.
Where an esu member has been added to a street a new shape will be created.
Figure 38 Spatial Consolidation
Note: The NSG metadata will need refreshing after loading the data because the area covered by the gazetteer could have increased so it will need re-assessing. This is carried out using the GIS Layer Tool – GIS0020 (Spatial Server Admin Guide – Creating Street Gazetteer manager Layers, Page 53).
Dropping Spatial Indexes
Street Spatial Consolidation
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
34
ASD Spatial Consolidation If a batch contains street or ESU or cross references or ASD records then the last stage of the batch schedule will be a ‘ASD Spatial Consolidation’. The processing carried out at this stage ensures that the spatial data for ASD against streets within the batch is kept up to date.
Firstly, the partial ASD memberships (also referred to as ASD placements) need to be kept in line with the existing streets and esus for streets in the batch. Therefore existing partial ASD members are dropped.
Partial ASD memberships are then re-created on a street by street basis.
Note: if a single ASD record fails, perhaps because of xy coordinates of the ASD record are nowhere near the street that the ASD is attributed to, then ALL partial ASD memberships upon that street will not be created.
Next existing ASD shapes for whole road and partial ASD on the effected streets in the batch are removed.
Finally ASD shapes are re-generated.
Figure 39 ASD Spatial Consolidation
The Schedule will display:
Sequence number
Processing stage The Record Counts will display the number of records:
Unprocessed
With error
With no error
A count for each The Timings will display the time taken for each record type processed:
Days
Hours
Minutes
Seconds
A count for each
ASD Spatial Consolidation
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
35
The Outcome will show whether the record type has been successful, has errors or has warning. Errors will be highlighted in red. Warnings will be highlighted in yellow.
Figure 40
Records tab
This tab will list records that either have errors or have been unprocessed. The Filter can be used to display only Unprocessed records, only Failure records or both by selecting the relevant radio button. The file column headings can be used as sort buttons. Press once to sort in ascending order (^). Press again to sort in descending order (v). Details shown for each record are:
Type of record
File name
Organisation name
Record identifier
Status The Notes field will display an explanation of for each record with a status or Error. Common errors can include:
ESU does not have a valid geometry
Interested Authority <number> does not exist as an Organisation
Maintainer district <district number> does not exist as a district for <district number>
Street <number> does not exist (this note may be shown when loading asd data)
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
36
Figure 41
Batch Log tab
This tab shows a breakdown of the batch process. During a load batch it will be continually updated. As a load progresses, full details of the activities undertaken are written to a log. There is one log per batch. It is designed to complement the higher level information shown in the batch schedule. Now, after a batch has been processed, four log files are automatically created in the NSG_PROCESSED directory. These can be opened in a text editor or Excel.
batch<batch_id>_schedule.csv lists the processing stages and outcome of each stage.
Figure 42
batch<batch_id>_file_summary.csv lists the files contained within the batch and a count of records which were not successfully processed.
Figure 43
batch<batch_id>_problem_recs.csv lists records within a batch which were not successfully processed.
Figure 44
batch<batch_id>_log.csv lists the full batch log.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
37
Figure 45
There are a number of buttons on the bottom of the form. [Submit Now] Select the [Submit Now] button to submit the selected batch. The batch will be submitted using the system date and time. [Submit Later]
Figure 46
Date Default
The date will default to the system date. Time Default
The time will default to the time defined in module Product Options – HIG9130 under option Id DFLTSUBMIT. [Withdraw] Selecting the [Withdraw] button will withdraw the submission of the batch. This will remove the ‘Available Date’. The batch can be resubmitted when convenient. If a batch is withdrawn which is already being processed the load process will finish at the earliest convenient opportunity. The processing of a batch that has failed and is re-submitted OR has been withdrawn and then re-submitted will pick up where it left off.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
38
Floating Toolbar
The Floating Toolbar contains functions that can be performed in the NSG Loader Batches form. To call the toolbar press the Toolbar icon on the main toolbar. The functionality behind each button is as follows: [Spatial Indexes] The [Spatial Indexes] button can be used to re-create spatial indexes on NSG tables that the loader works with. Certain tables that the NSG Loader interacts will have a spatial index which allows that data in that table to be retrieved efficiently. A consequence of having such an index in place is that as data is added to/removed from these tables it takes time to keep the index in line. Therefore, these indexes are now dropped as part and parcel of NSG loader processing to aid performance. As and when all batches have been processed this button can be used to reinstate them.
Note: Without spatial indexes in place the relevant spatial layer(s) will not be visible in Locator.
[Gather Statistics] The [Gather Statistics] button allows statistics on necessary tables that the NSG Loader works with to be refreshed. With up-to-date statistics the loader can make the correct choices when deciding how to process data. Pressing the [Gather Statistics] button after preparing all of batches (but before submitting them for processing) should make sure the loader is primed to process data efficiently. Pressing the button again after all batches have been process should make sure that the Street Gazetteer Manager application is primed to work with data efficiently as well. If necessary you can still opt to gather statistics on tables whilst the loader is running. However, the result of taking this approach to keeping statistics up-
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
39
to-date is that it may take longer to gather the statistics than the performance benefit gained by having then in place. [Refresh CTL] The [Refresh CTL] button can be pressed once all batches have been processed to allow the list of Counties, Towns and Localities to be refreshed. The NSG loader will always attempt to refresh Counties, Towns and Localities whenever streets have been included in the batch that is being processed. However, if Product Option – BYPASSCTL is set to Y then this part of the load will be bypassed. [Refresh HA] The [Refresh HA] button can be pressed once all batches have been processed to allow the list of Highways Authorities Counties to be refreshed The NSG loader will always attempt to refresh HA Counties whenever streets have been included in the batch that is being processed. However, if Product Option – BYPASSHA is set to Y then this part of the load will be bypassed. [Purge Batch] The [Purge Batch] button will clear down records from database that are related to the current batch but which are no longer required. This will aid performance when processing other batches and it is advised that you purge on a regular basis. [Purge Successful Batches] The [Purge Successful Batches] will perform the same functionality as the [Purge Batch] button, but for all batches that have been processed without error. [Shape Validation] The functionality behind the [Shape Validation] button will validate the shapes of all ESUS and allow removal of those shapes that are invalid and cannot be repaired. One common example of an invalid ESU shape is one that is actually a point, rather than a line. It’s expected that this will be a one-off exercise and should be done as soon as upgrading to release 4.3.0.0.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
40
[Shapeless ESU’s] Street Gazetteer Manager relies on all ESUs having a shape. The [Shapeless ESU’s] button gives access to a screen which allows ESU’s without a shape to be removed from the database. Such ESUs could have been created by the loader prior to release 4.2.0.0, or be shapeless as a result of using the aforementioned ‘Shape Validation’ functionality. It’s expected that this will be a one-off exercise and should be done as soon as upgrading to release 4.3.0.0 but after the [Shape Validation] has been carried out.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
41
Monitor NSG Loader - NSG0045
Figure 47
Figure 48
This module is used to monitor the progress of files that have been submitted to the queue and processing activity. A database job runs constantly which looks for available batches to process.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
42
[Withdraw Batch] Selecting the [Withdraw Batch] button will withdraw the submission of the batch. This will remove the ‘Available Date’. The batch can be resubmitted when convenient. If a batch is withdrawn which is already being processed the load process will finish at the earliest convenient opportunity. The processing of a batch that has failed and is re-submitted OR has been withdrawn and then re-submitted will pick up where it left off. The screen is split into three areas:
the batch queue
batch schedule of the current batch in the queue
progress information relating to the batch currently being processed
The Batch Queue will display:
Batch Id
Batch name
Available date – the date and time the batch is to be processed
Load started date The progress information relating to the batch currently being processed will display:
The batch schedule stage being processed highlighted in green
Any batch schedule stage with a status of Error highlighted in red
Any batch schedule stage with a status of Warning highlighted in amber
Any batch schedule stage with a status of Success highlighted in grey
A stage can be broken down into several operations. Where possible, when an operation is expected to take some time to complete a progress bar indicating a percentage complete and estimated end time will be shown. Errors and warnings can be checked by using module NSG Loader Batches – NSG0040.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
43
[Options] Latest Activity Display Only
The date and time of the last job run through the NSG Loader Batch will be displayed. Session Display Only
The session number of the last job run will be displayed. Job Display Only
The job number of the latest activity will be displayed. Watch Checkbox
This checkbox allows the user to switch on/off the constant monitoring of progress. When exiting the form the checkbox is automatically unchecked. If the form is then re-opened and the box was previously checked it will be re-checked automatically and monitoring of progress will resume. Gather Statistics Whilst Loading Checkbox
In order for Oracle to determine the most efficient approach to manipulate data, it requires up-to-date statistics against tables in the database. Due to the nature of any loader, data volumes can increase/decrease during the load process and therefore statistics can become out of date (stale) and potentially have an adverse effect on load performance. Therefore the end user has the option to use this checkbox to re-gather statistics as and when they are needed by the load process. It is advised that this option is always taken. However it is recognised that there is always an offset of the time taken to gather statistics VS the performance benefit the statistics offer. Statistics will now only be gathered if the existing statistics on the database table in question are out of date i.e. there is potential benefit in collating them.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
44
The default value will have been defined in module Product Options – HIG9130 under option Id LOADSTATS. Bypass Count/Town/Locality Refresh
If this box is ticked the County/Town/Locality refresh will be skipped when processing data.
Note: This can be done manually in form NSG Loader Batches - NSG0040 using the [Refresh CTL] button on the toolbar.
Bypass HA Refresh
If this box is ticked the Highways Authority refresh will be skipped when processing data.
Note: This can be done manually in form NSG Loader Batches - NSG0040 using the [Refresh HA] button on the toolbar.
The [Recreate Job] button can be used to restart the job if possible. The [Drop Job] button can be used to drop the job if, for example, it doesn’t exist. The [Close Window] button closes the Options window and returns to the front screen.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
45
Administer User Districts – NSG0120
Figure 49
Figure 50
General Information
The Administer User Districts form enables the System Administrator to define which operational districts are available to a specific user. For example: a user may perform the function of a street authority and monitor notices coming in from a works promoter. The same user may also perform the function of a works promoter and enter notices on behalf of the highway authority. The user then has the option of swapping from one district function to another by using the My Districts – NSG0130 form.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
46
The Administer User Districts form is available from either the Street Gazetteer Manager menu or the TMA Manager menu.
Figure 51
User Districts
Organisation Optional List
Enter the organisation to be available to the selected user. District Optional List
Enter the organisational district to be available to the selected user. District Function Display Only
The function of the selected district will be displayed. Current Checkbox
The current district assigned to the user will be checked.
Use the [Set as Current] button to change the current district.
Note: the new district must be selected before the [Set as Current] button is available.
Note: all forms should be closed before changing the district of the user.
The following alert prevents the district being changed for a user.
Figure 52
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
47
My Districts - NSG0130
Figure 53
Figure 54
General Information
The My Districts enables the user to define which operational district they are currently using. The user can swap between the district functions assigned to the user by the System Administrator. For example: a user may perform the function of a street authority and monitor notices coming in from a works promoter. The same user may also perform the function of a works promoter and enter notices on behalf of the highway authority.
User Districts
Organisation Display Only
The organisation set as available to the selected user will be displayed. District Display Only
The organisational district set as available to the selected user will be displayed.
Exor Leading the way in Infrastructure Asset Management Solutions...
© Copyright Exor Corporation Ltd., England, 2008. All rights reserved
48
District Function Display Only
The function of the selected district will be displayed. Current Checkbox
The current district assigned to the user will be checked.
Use the [Set as Current] button to change the current district.
Note: the new district must be selected before the [Set as Current] button is available.
Note: all forms should be closed before changing the district.
The following alert prevents the district being changed for a user.
Figure 55