360 degree conops · executive summary this report details the design, deployment, and evaluation...

Implementation and Evaluation of the

360 Degree Radar Information System

Prepared By:

Daniel M. Nelson, AICP

AECOM

July 2016

Published By:

Minnesota Department of Transportation

1500 West County Road B2

Roseville, MN 55113

This report represents the results of research conducted by the authors and does not necessarily represent the

views or policies of the Minnesota Department of Transportation or AECOM or RhiZone. This report does not

contain a standard or specified technique.

The authors, the Minnesota Department of Transportation, and AECOM do not endorse products or

manufacturers. Any trade or manufacturers’ names that may appear herein do so solely because they are

considered essential to this report.

Table of Contents

Executive Summary .................................................................................................................. 1 Chapter 1 Introduction ......................................................................................................... 1

1.1. Project Purpose ........................................................................................................... 1 1.2. Project Background .................................................................................................... 1 1.3. Operational Needs ...................................................................................................... 2

1.4. Project Description ..................................................................................................... 2 1.5. Report Organization ................................................................................................... 3

Chapter 2 Systems Engineering for ITS Process ................................................................. 4 2.1. MnDOT Statewide Regional ITS Architecture and ITS Development Objectives ... 5 2.2. Systems Engineering Documents ............................................................................... 7

Chapter 3 360 Radar Information System Equipment......................................................... 8 3.1. 360 Degree Radar and CCTV Camera Equipment .................................................... 8

3.2. FCC License for 360 Radar Unit.............................................................................. 10 3.3. Central Office Equipment ........................................................................................ 10 3.4. Central Software Calibration .................................................................................... 12 3.5. Overall System Installation and Adjustment Timeline ............................................ 14

Chapter 4 360 Degree Radar System Evaluation .............................................................. 16 4.1. System Validation Process ....................................................................................... 16 4.2. System Validation Results ....................................................................................... 17

4.3. System Evaluation Process....................................................................................... 27 4.4. System Evaluation Results ....................................................................................... 28

Chapter 5 Conclusions and Recommendations ................................................................. 57 5.1. Conclusions .............................................................................................................. 57 5.2. Considerations for Future Installation ...................................................................... 59

5.3. Recommendations for Future Installation ................................................................ 63

Appendix A – System Component Data Sheets ..................................................................... 65

Acknowledgements

The authors of this report wish to acknowledge the very helpful efforts of several MnDOT

staff at the Regional Transportation Management Center (RTMC) that helped make this

Innovative Idea project a successful project, including Project Manager Rashmi Brewer,

RTMC Operations staff Brian Kary, Terry Haukom, Ralph Adair, and Jesse Larson, as well

as technical support received from MN.IT technician Tim Johnson.

Figures in Document

Figure 1-1 – General Problems and Operational Needs ........................................................... 2 Figure 1-2 – 360 Degree Radar Information System Overview ............................................... 3 Figure 2-1 – Systems Engineering V-Diagram ......................................................................... 4 Figure 3-1 – 360 Degree Radar and CCTV Cameras ............................................................... 9 Figure 3-2 – Diagram of 360 Radar Equipment and Locations on Pole ................................... 9

Figure 3-3 –Central Office Components in RTMC Server Room .......................................... 10 Figure 3-4 – Witness Software Interface of Corridor Sections and Vehicles ......................... 11 (July through September 2015) ............................................................................................... 11 Figure 3-5 – Witness Software Interface of Corridor Sections............................................... 11 (December 2015 through March 2016) ................................................................................... 11

Figure 3-6 – NVR Interface with Camera Views of Corridor ................................................ 12 Figure 3-7 – Current Settings with Central Software ............................................................. 13

Figure 4-1 – MnDOT Computer Aided Dispatch Summary of Dec. 17th Event .................... 18 Figure 4-2 – MnDOT Camera Record of Dec. 17th Event for Confirmation with ................. 18 360 Degree Radar System ....................................................................................................... 18 Figure 4-3 – 360 Degree Radar System Record of Dec. 17th Event ....................................... 19

Figure 4-4 – Interface of NVR with Video Record Dec. 17th Event ..................................... 20 Figure 4-5 – Navtech and Wavetronix WB and EB Speed Comparisons on Nov. 20th .......... 30 Figure 4-6 – Navtech and Wavetronix WB and EB Speed Comparisons on Nov. 20th .......... 33

By Lane for all Time Periods .................................................................................................. 33 Figure 4-7 – Navtech, Wavetronix, and Manual WB and EB Count Comparisons on Nov.

19th .......................................................................................................................................... 34 Figure 4-8 – Navtech and Wavetronix WB and EB Count Comparisons on Nov. 19th By

Lane for all Time Periods ....................................................................................................... 38

Figure 4-9 – Navtech and Wavetronix WB and EB Speed Comparisons on Jan. 17th, 2016 . 39

Figure 4-10 – Navtech and Wavetronix WB and EB Speed Comparisons on Jan. 17th By

Lane for all Time Periods ....................................................................................................... 42 Figure 4-11 – Navtech and Wavetronix WB and EB Count Comparisons on Jan. 17th ......... 43

Figure 4-12 – Navtech and Wavetronix WB and EB Count Comparisons on Jan. 17th By


Figure 4-13 – Navtech and Wavetronix WB and EB Speed Comparisons on Feb. 2nd, 2016 48 Figure 4-14 – Navtech and Wavetronix WB and EB Speed Comparisons on Feb. 2nd By

Lane for all Time Periods ....................................................................................................... 51 Figure 4-15 – Navtech and Wavetronix WB and EB Count Comparisons on Feb. 2nd .......... 52

Figure 4-16 – Navtech and Wavetronix WB and EB Count Comparisons on Feb. 2nd By


Tables in Document

Table 2-1 – Rule 940 Requirements ......................................................................................... 5 Table 2-2 – Minnesota ITS Development Objectives Related to 360 Degree Radar

Information System ................................................................................................................... 6

Table 2-2 – Systems Engineering Documents .......................................................................... 7 Table 3-1 – System Component Model Numbers and Detail ................................................... 8 Table 3-2 – High-Level System Installation and Adjustment Timeline ................................. 14 Table 4-1 – Validation Plan Measures of Effectiveness ......................................................... 16 Table 4-2 – Summary of Events Logged by 360 Degree Radar and MnDOT CAD System

(August – September 2015) .................................................................................................... 22 Table 4-3 – Summary of Events Logged by 360 Degree Radar and MnDOT CAD System

(February - March 2016) ......................................................................................................... 23 Table 4-4 – Summary of Events Logged by 360 Degree Radar and MnDOT CAD System . 24

Table 4-5 – Summary of False Alarms Logged June 20th to June 26th ................................... 25 Table 4-6 – Summary of False Alarms Logged March 2nd to March 9th ................................ 26

Table 4-7 – Summary of Person and Reverse Vehicle Alarms Verified June 20th to June 26th

................................................................................................................................................. 27

Table 4-8 – Dates of Vehicle Speed and Count Comparisons with Wavetronix Data ........... 27 Table 5-1 – Summary of Conclusions on 360 Degree Radar Information System ................ 57 Table 5-2 – Summary of Key Findings from Evaluation of Incident Detection Capabilities

the 360 Degree Radar Information System ............................................................................. 58 Table 5-3 – Summary of Operational Needs Addressed by the .............................................. 58

360 Degree Radar Information System ................................................................................... 58

Executive Summary

This report details the design, deployment, and evaluation of the 360 Degree Radar Information

System that has been installed along the side of Interstate 94 in Minneapolis between the 3rd Ave.

S. and MN-65 overpasses. The purpose of the project is to evaluate the system for MnDOT and

determine whether the system could reliably be used by operators at the MnDOT Regional

Transportation Management Center (RTMC) for automated incident detection and dispatch of

emergency response personnel.

MnDOT selected AECOM and RhiZone under the Innovative Idea Program in June 2014 to plan

for and install the 360 Degree Radar Information System. AECOM utilized the Systems

Engineering process to guide the project through the concept, design, installation, and evaluation

stages. RhiZone coordinated with the system manufacturer – Navtech Radar Surveillance

(Navtech) – to procure, configure and install the system which consisted of a single 360 Degree

Radar unit as field equipment, as well as a central software package that operated on a server at

the MnDOT RTMC in Roseville, MN.

The system was initially installed on June 2nd, 2015 and initial configurations to field equipment

along with central software were then completed by June 19th, 2015 to allow for the operational

test to begin. Subsequent configuration adjustments were made in the following months to

improve the accuracy of incident detection and measurements of vehicle counts and speeds.

The purpose of this project was to evaluate the system operation for MnDOT after system

installation and determine whether the system could reliably be used by operators at the MnDOT

RTMC for automated incident detection and dispatch of emergency response personnel.

AECOM performed the evaluation between July 2015 and March 2016 and CCTV cameras were

installed for the purpose of recording incidents detected by the 360 Degree Radar Information

System to measure the amount of false alarms generated by the system.

Three key findings of the evaluation related to automated incident detection are summarized

below:

1. The 360 Degree Radar Information System would have provided an average of 10.5

minutes of advance notification to RTMC operators of incidents in the project area (based

on 11 total events logged in the project for comparison with the MnDOT CAD system).

This shorter notification time would help accomplish a MNDOT safety objective of

advanced notification of incidents.

2. The false alarm rate of incident notification decreased from 0.71 at the beginning of the

project to 0.28 near the end of the project, resulting in a positive sign that the 360 Degree

Radar Information System can reduce its false alarm rate over the course of its operation.

3. Between August and September 2015, the 360 Degree Radar Information System

detected 11 events that were not logged by RTMC operators as traffic incidents. A

notification to RTMC operators of these events would also greatly help accomplish the

higher level safety objectives for MnDOT of reducing injuries, fatalities, and the potential

for secondary crashes on the corridor.

AECOM also performed an evaluation between July 2015 and March 2016 of the accuracy of

vehicle counts and speeds as measured by the 360 Degree Radar Information System. While the

ability to collect accurate vehicle counts and speeds was not the primary objective of the project,

Navtech Radar was able to upgrade the central software to allow for the system to

simultaneously perform this task alongside the incident detection features of the overall system.

Two key findings of the evaluation related to vehicle counts and speeds are summarized below:

1. The vehicle speeds recorded by the system were within 10% of the vehicle speeds

measured by the Wavetronix unit for all lanes of traffic measured in November 2015.

2. The vehicle counts recorded by the system were within 10% of the vehicle speeds

measured by the Wavetronix unit at the field site for the right lane of traffic in each

direction in November 2015. System vehicle counts were consistently lower than the

Wavetronix measured data by greater than 10% for the middle and fast lanes of traffic.

3. Software configurations performed in January 2015 impacted the ability to accurately

compare vehicle counts and speeds measured by the system in January and February

2016 with Wavetronix data.

4. The accuracy of the counts and speeds in the westbound lanes was about 15-20% more

accurate when compared with Wavetronix data than count and speed measurements of

the eastbound lanes, and this was expected due to the greater distance of those eastbound

lanes from the radar unit.

After the evaluation of the incident detection capabilities and the accuracy of the vehicle counts /

speeds were performed, MnDOT allowed for a limited trial period of the system within the

MnDOT RTMC for two weeks in late May and early June. The system was monitored by a

MnDOT traffic operations supervisor who noted that many stopped vehicle alarms were

generated during periods of heavy traffic congestion on the corridor. As a result, the value of

alarms for incident detection was limited given the frequency of traffic congestion in the area.

As a lesson learned for this traffic environment, Navtech proposed a modification to the software

that could limit the number of alarms generated in heavy traffic congestion.

The biggest challenge encountered on the project by the AECOM team was properly configuring

both the radar unit in the field and the central software installed at the MnDOT RTMC to

accurately perform the simultaneous tasks of incident detection and measurements of vehicle

counts and speeds on the corridor. The proper tilt of the radar toward the roadway was initially

established but then modified slightly in July in an effort to improve the accuracy of vehicle

detection in the project area. This configuration of the radar unit slightly limited the radar’s

capability to detect stopped vehicles on the corridor.

In addition, the presence of the MN-65 bridge overpass and the 3rd Avenue South bridge

overpass presented an environment that Navtech had not previously tested with the 360 Degree

Radar Information System. It was also learned through the course of the project that heavy

traffic congestion in the project area caused the central software to ignore multiple alarms of

stopped vehicles at the same time. This had the impact of causing the software to ignore stopped

vehicle events reported by the radar unit that were actual incidents logged by the RTMC

operators. Navtech Radar learned of this issue through this project and is actively developing a

software upgrade to address this issue in future deployments.

Configurations to the central software were also performed in October in an effort to improve the

accuracy of vehicle counts and speeds on the corridor, while also simultaneously performing the

main task of incident detection. This configuration required software upgrades from Navtech

Radar to perform these simultaneous tasks of incident detection and vehicle counts, which

impacted the ability to perform an evaluation of the system operations during the period of the

software updates.

Through the overall project, the following advantages and limitations were observed:

System Advantages System Limitations

1. Speed of detection – Ability to

quickly detect objects and track the

progress of movement through

project area

1. Occlusion – Radar requires line-of-sight with

object for detection. Large semi-trucks can

block smaller vehicles from being detected

2. Non-intrusive detection – Detection

of objects does not require in-

pavement loops or other hardware

for operations

2. Configuration – Software configuration of

detection zones and rules for detecting objects

requires careful monitoring in the initial period

of the system installation. Changes to system

settings also require time to confirm that

accurate detection is being performed.

3. Maintenance – Low amount of

hardware maintenance;

recommended to change an internal

belt once every three years.

3. System Cost – Cost of the field equipment and

software for the one location in this project was

approx. $50,000, which could be used for other

safety countermeasures such as more Service

Patrol vehicles. Navtech has been made aware

of the cost concerns and has noted future

deployments would need to have reduced costs,

as well as a small monthly fee to make the

radar cost effective at detecting incidents.

Given the overall performance of the 360 Degree Radar Information system on the project, the

following next steps are envisioned for future system installations:

1. Operation at Signalized Intersections – The system could be used to monitor signalized

intersections given its noted accuracy in detecting specific events and vehicle presence.

The system could replace in-pavement loops or other technology currently installed to

actuate signal operations. In terms of safety, the system’s ability to detect pedestrians

could help reduce pedestrian-vehicle crashes and injuries at those intersections.

2. ICWS Integration – The system could also be integrated with Intersection Conflict

Warning System (ICWS) installed by MnDOT at rural locations. The proven ability of

the system to detect vehicles traveling at high speeds from similar distances as the current

ICWS installations could prove to be more cost effective than previous installations while

maintaining or improving the safety of travel at uncontrolled intersections with a previous

history of vehicle incidents

1

Chapter 1 Introduction

This final report details the design, deployment, and evaluation of the 360 Degree Radar

Information System that has been installed along the side of Interstate 94 in Minneapolis

between the 3rd Ave. S. and MN-65 overpasses.

The 360 Degree Radar Information System is designed to generate automated alerts of traffic

incidents to Traffic Management Center (TMC) operators, who could then verify the incident

through use of CCTV cameras and, in turn, dispatch emergency response personnel in a more

timely and efficient manner than waiting for a 911 emergency call from those involved in the

incident.

1.1. Project Purpose

The purpose of this project is to install and evaluate the operation of the 360 Degree Radar

Information System for MnDOT over a 6-month period and determine whether the system could

reliably be used by operators at the MnDOT Regional Transportation Management Center

(RTMC) for timely incident detection and dispatch of emergency response personnel.

The area of Interstate 94 where this system has been installed experiences very high volumes of

vehicular traffic and the highest amount of vehicle crashes and incidents in the state of

Minnesota, given its convergence with Interstate 35W and proximity to the Lowry Hill tunnel.

The ability to quickly dispatch emergency response personnel based on timely automated alerts

from the 360 Radar Information System could reduce the number of vehicular fatalities, injuries,

and property damage that may result from the primary crash and potential secondary crashes as

well.

1.2. Project Background

The 360 Degree Radar Information System was submitted as a proposal in response to a MnDOT

solicitation under the Innovative Idea program. A detailed proposal was submitted by AECOM

(and sub-contractor RhiZone) to MnDOT on June 3rd, 2014 as part of Stage II of the program

that illustrated the plan to install the 360 Degree Radar Information System. The use of the

Systems Engineering (SE) process for Intelligent Transportation Systems (ITS) was also

proposed to guide the project through the concept, design, installation, and evaluation stages of

the project.

An optional application of installing a second 360 Radar Information System at a signalized

intersection was also proposed, however, only as an option for MnDOT to consider. The current

system design does not anticipate installing the second system at an intersection for traffic

monitoring, although, this could be an option for MnDOT to consider in future testing of the

system.

The use of 360 Degree Radar Systems have been tested and are operational in Europe, the

Middle East, and Australia. The manufacturer of the equipment and software is known as

Navtech, and this project is the first demonstration of the radar unit in the United States. An

2

experimental license from the Federal Communications Commission (FCC) was obtained locally

by RhiZone prior to system installation.

1.3. Operational Needs

Current methods of traffic and vehicle detection have various limitations that are summarized in

the figure below. The operational needs for the 360 Degree Radar Information System are also

summarized with respect to each of the general problems noted in Figure 1-1.

Figure 1-1 – General Problems and Operational Needs

General Problems Operational Needs

1. Limited zone coverage – Current methods of

loop detection are spot-based with loops installed

at intervals in lanes of traffic.

1. Deploy a traffic detection system

that can monitor multiple lanes of

traffic from one single location.

2. Understanding of latency in incident detection –

It is difficult to understand the latency in time

between when traffic incidents occur on a corridor

and when a formal incident notification is received

by emergency responders

2. Gather incident detection

information quickly and accurately in

the project area and provide

information to operational staff for

emergency response

3. Inability to measure driver behavior; individual

vehicle movements – Loop detection and other

non-intrusive detection methods measure traffic

volumes as a whole along the roadway, and do not

allow the capability for measuring driver

behaviors to potentially understand how

congestion forms on a roadway.

3. Gather traffic congestion information

quickly and accurately in the project

area

4. Inability to perform multiple real-time data

collection tasks over a wide area – Loop detection

and other non-intrusive detection methods only

allow for vehicle count measurements at those

locations.

4. Understand the amount of lane-

weaving occurring within the project

area

5. Difficult to gather performance metrics in a cost

effective manner

5. Gather performance metrics that can

be reported regarding traffic congestion

and incidents in the project area in a

readable and usable format

1.4. Project Description

Field equipment includes the 360 Degree radar unit and independent CCTV cameras to be used

for verifying incidents detected by the radar unit along the side of I-94 between the 3rd Ave. S.

and MN-65 overpasses. This equipment utilizes MnDOT fiber-optic cable to communicate data

to a System Server installed within the MnDOT RTMC Server Room. Data from the radar and

video from the CCTV cameras are recorded on a Network Video Recorder (NVR) for the

purposes of recording information communicated from the 360 Degree Radar Information

System in the field. In addition, data and video can be viewed locally within the MnDOT RTMC

Server Room via an existing MnDOT workstation in the server room, as well as through remote

3

internet access Figure 1-2 contains an illustration of the field and central office elements within

the system.

Figure 1-2 – 360 Degree Radar Information System Overview

1.5. Report Organization

This report is organized into the following Chapters:

Chapter 2: Systems Engineering for ITS Process

Chapter 3: 360 Degree Radar Information System Equipment

Chapter 4: 360 Degree Radar System Evaluation

Chapter 5: Conclusions and Recommendations

4

Chapter 2 Systems Engineering for ITS Process

This Chapter describes the Systems Engineering for ITS process that was followed in planning,

designing, installing, and testing the 360 Degree Radar Information System. This process is

highly recommended by the Federal Highway Administration (FHWA), which helps to guide

ITS implementations, such as the 360 Degree Radar Information System, by involving

stakeholders early and developing their ideas before high-level and detailed design activities are

conducted. This interdisciplinary approach assures that the ultimate design and implementation

of the system reflects this early input. The Systems Engineering “V” Diagram is provided in

Figure 2-1 below.

Figure 2-1 – Systems Engineering V-Diagram

Systems Engineering integrates all the disciplines and specialty groups into a team effort forming

a structured development process that proceeds from system concept to implementation and

operation. Systems Engineering considers both the business and the technical needs of all

customers with the goal of providing a quality product that meets the user needs.

The Systems Engineering process is used to identify a project’s needs and constraints and lay out

the activities, resources, budget, and timeline for the project. A critical part of the process is to

build consensus among the stakeholders of the project. The process is applicable at all stages of a

project, from initial system planning through final operations and maintenance of the system.

FHWA Federal Rule 940, Intelligent Transportation Systems Architecture and Standards, which

implements Section 5206 (e) of the Transportation Equity Act of the 21st Century (TEA – 21),

requires agencies implementing projects with ITS elements utilizing federal funds to develop

5

regional architectures and adopt a Systems Engineering approach for project deployments in

order to qualify for ITS grants. Table 2-1 illustrates the relationship between the various steps of

the SE process and Rule 940.

Table 2-1 – Rule 940 Requirements

Systems Engineering

Process Steps

Corresponding Rule 940 Requirements

Concept of Operations

Identification of participating agencies roles and

responsibilities

Procedures and resources necessary for operations and

management of the system

System Requirements:

High-Level and Detailed Requirements definitions

Design:

High-Level and Detailed

Identification of portions of the regional ITS architecture

being implemented

Analysis of alternative system configurations and

technology options to meet requirements

Procurement options

Identification of applicable ITS standards and testing

procedures

2.1. MnDOT Statewide Regional ITS Architecture and ITS Development Objectives

MnDOT has developed a Statewide Regional ITS Architecture that provides the base for all ITS

projects illustrated in the Systems Engineering diagram in Figure 2-1. From the MnDOT

Statewide Regional ITS Architecture in 2014, MnDOT’s ITS Development Objectives were

refined to better align with Minnesota’s ITS needs and be consistent with the National ITS

Architecture.

The goal of the Minnesota ITS Development Objectives is to enhance transportation through the

safe and efficient movement of people, goods, and information, with greater mobility, fuel

efficiency, less pollution and increased operating efficiency statewide. These ITS Development

Objectives are rooted from the overall Minnesota GO Transportation Plan that defines MnDOT’s

vision for transportation and guiding principles and outlines strategies to satisfy its vision and

mission. The 20-Year Statewide Multimodal Transportation Plan (SMTP) further clarifies these

strategies and lays out actions to implement the strategies. The Minnesota ITS Development

Objectives presented in Table 2-2, establishes the specific objectives that can be achieved

through implementing ITS countermeasures, such as the 360 Degree Radar Information System.

6

Table 2-2 – Minnesota ITS Development Objectives Related to 360 Degree Radar

Information System A. Improve the Safety of the State's Transportation System

A-1 – Reduce crash frequency

A-1-01 Reduce number of vehicle crashes

A-1-02 Reduce number of vehicle crashes per VMT

A-1-04 Reduce number of crashes due to unexpected congestion

A-1-05 Reduce number of crashes due to red-light running

A-1-08 Reduce number of crashes due to inappropriate lane departure, crossing and merging

A-1-10 Reduce number of crashes at signalized intersections

A-1-11 Reduce number of crashes at un-signalized intersections

A-1-15 Reduce number of crashes involving pedestrians and non-motorized vehicles

A-1-19 Reduce number of all secondary crashes

A-2 – Reduce fatalities and life changing injuries

A-2-01 Reduce number of roadway fatalities

A-2-02 Reduce number of roadway fatalities per VMT

A-2-04 Reduce number of fatalities due to unexpected congestion

A-2-05 Reduce number of fatalities due to red-light running

A-2-09 Reduce number of fatalities due to inappropriate lane departure, crossing and merging

A-2-11 Reduce number of fatalities at signalized intersections

A-2-12 Reduce number of fatalities at un-signalized intersections

A-2-16 Reduce number of fatalities involving pedestrians and non-motorized vehicles

A-2-22 Reduce number of roadway injuries

A-2-23 Reduce number of roadway injuries per VMT

A-2-25 Reduce number of injuries due to unexpected congestion

A-2-30 Reduce number of injuries due to inappropriate lane departure, crossing and merging

A-2-26 Reduce number of injuries due to red-light running

A-2-32 Reduce number of injuries at signalized intersections

A-2-33 Reduce number of injuries at un-signalized intersections

A-2-37 Reduce number of injuries involving pedestrians and non-motorized vehicles

B. Increase Operational Efficiency and Reliability of the Transportation System

B-1 – Reduce overall delay associated with congestion

B-1-01 Reduce the percentage of facility miles (highway, arterial, rail, etc.) experiencing recurring

congestion during the peak period

B-1-02 Reduce the percentage of Twin Cities freeway miles congested in weekday peak periods

B-1-03 Reduce the share of major intersections operating at LOS F

B-1-06 Reduce the number of hours per day that the top 20 most congested roadways experience

recurring congestion

B-1-10 Reduce hours of delay per capita

B-1-11 Reduce hours of delay per driver

B-1-16 Reduce mean time for needed responders to arrive on-scene after notification

B-1-17 Reduce mean incident clearance time per incident (Defined as the time between awareness of an

incident and the time the last responder has left the scene.)

B-1-18 Reduce mean incident clearance time for Twin Cities urban freeway incidents (Defined as the

time between awareness of an incident and the time the last responder has left the scene.)

C. Enhance Mobility, Convenience, and Comfort for Transportation System Users

C-1 – Reduce congestion and incident-related delay for travelers

C-1-09 Increase number of regional road miles covered by ITS-related assets (e.g., roadside cameras,

dynamic message signs, vehicle speed detectors) in use for incident detection / response

7

2.2. Systems Engineering Documents

The documents developed by the AECOM team for MnDOT are listed and described in Table 2-

2 below. Documents were shared with MnDOT team members and updated based on input and

feedback from team members. Final versions of the documents have been produced and shared

with MnDOT.

Table 2-2 – Systems Engineering Documents

Systems Engineering

Documents Purpose/Description Deliverables

Project Plan Describes all of the tasks that need to be performed

to accomplish the project.

Mid-December

2015

System Engineering

Management Plan

(SEMP)

Addresses project controls that are to be developed

and implemented throughout the project and

documents the technical processes to be used.

Mid-January 2015

Concept of Operations Describes how the system will operate and outlines

the roles and responsibilities of each stakeholder. Mid-January 2015

Verification and

Validation Plan

Defines the step by step procedures to conduct

verifications that components meet system

requirements. Describes how the goals and

objectives of the system (identified in the ConOps)

will be measured

Mid-January 2015

System Requirements

and Design

Describes what the system will do and how its

components will function at a high level. Also

includes the development of high-level design

documents and detailed design plans including shop

drawings and specifications of the proposed system

System

Requirements in

February 2015;

System Design in

Mid-April 2015

Acceptance Testing

and Evaluation Plans

Documents the detailed procedures to be used to

verify and validate the deployed 360 Degree Radar

Information System.

June 2015

Final Project Report

Provides overall summary of activity and the

recommendations for MnDOT on next steps that

could be taken. Includes results measured during

the Acceptance Testing and Evaluation stages of the

project, as well as results from the Validation stage

of the project.

April 2016

8

Chapter 3 360 Radar Information System Equipment

A summary of the system components and quantities to be provided by the AECOM Team and

installed is presented in Table 3-1. Refer to Appendix A of this document for datasheets on the

various system components with more detailed information.

Table 3-1 – System Component Model Numbers and Detail

System

Components Unit Type Quantity Notes

360 Degree Radar CTS350-X 1 Refer to Appendix A for datasheet with

detailed component data

CCTV Cameras Axis Q1614-E 2 Current lens offered only provides 80

degree horizontal field of view

System Server Navtech Server 1

To be provided by Navtech with radar

unit and cameras as one system. Refer to

Appendix A for more detailed data on

server.

System NVR Unit Samsung NVR 1

To be implemented by Navtech on the

System Server for video recording. Refer

to Appendix A for more information.

Recording

Trigger Module Adams 1

Implemented to trigger a video recording

of an incident detected by the system

software

System Software Witness

Software 1

To be implemented by Navtech on the

System Server with graphical user

interface for viewing of radar and

cameras.

3.1. 360 Degree Radar and CCTV Camera Equipment

The 360 Degree Radar unit and CCTV cameras are pictured in Figure 3-1 as they have been

installed in the field. The radar and cameras connect via Ethernet cables into a MnDOT operated

Cisco switch in a cabinet on the pole. Figure 3-2 also illustrates the general height of the

equipment in relation to other equipment previously installed on the pole. The optimal height for

the radar to operate in the field is 12 to 15 feet above the highest point of the lanes of travel on

the I-94 corridor. Since the highest point of the roadway is at the same level as the ground at the

base of the pole, the 360 degree radar unit was installed on a bracket just above the hinge on the

pole which has been measured at about 13 feet above ground level.

Once the radar unit on the pole is properly leveled on its mounting bracket, it requires very low

maintenance to continue to operate. The manufacturer of the radar recommends that a rubber

belt used to spin the radar device 4 times per second be replaced once every three years. This

process of replacing the belt could likely be performed in the field after removing the radar from

the bracket. The re-installation process of the radar after the belt replacement would also take

9

less time, since the upper plate of the bracket on which the radar rests would not need to be re-

leveled.

The University of Minnesota Center for Transportation Studies (CTS) has installed additional

radar equipment on the pole for research and data collection purposes. During the course of time

when the University’s equipment was installed on the pole, there were no reported issues in the

operation of the two radar units operating in close proximity given their different operating

frequencies. The University’s equipment was removed from the pole in July 2015.

CCTV Cameras

(Ports 1+2)

360 Degree

Radar Unit

(Port 3)

Figure 3-1 – 360 Degree Radar and CCTV Cameras

Figure 3-2 – Diagram of 360 Radar Equipment and Locations on Pole

10

3.2. FCC License for 360 Radar Unit

As noted earlier, the 360 Degree Radar Information System has not been tested or deployed in

the United States, and thus required approval from the FCC prior to field installation and

operation. The application was formally submitted on February 10th, 2015 and approved by the

FCC on March 10th, 2015 under the call sign WH2XQL.

The license only allows for the use of the radar at the current field location. Additional

modifications can be proposed to the license that would allow for the movement of the radar to

an additional location, such as an intersection which had been proposed as an optional work task.

The current expiration date of the license is March 1st, 2017, but could be extended through a

renewal application.

3.3. Central Office Equipment

The equipment installed at the MnDOT RTMC Server Room is displayed in Figure 3-3 below.

Field equipment communicates via Ethernet cables to a MnDOT operated Cisco switch which is

also connected to MnDOT fiber-optic cable leading to the MnDOT RTMC.

Incident Trigger Module

Network Video Recorder

Navtech Server with Software

Figure 3-3 –Central Office Components in RTMC Server Room

The Navtech Server houses the Witness Software program that receives and processes data from

the radar unit on the pole. The software program determines when alarms should be generated to

alert a TMC operator of the following types of events:

Stopped vehicles on the road for a period of 10 seconds or greater

Slow-moving vehicles on the road at 15 MPH or less

Vehicle moving in Reverse on the road

People detected walking on the road

11

These alarms can be configured so as not to set off multiple alarms for similar events in

succession, and they can also be delayed. Alarms are also segmented on the corridor as shown in

Figure 3-4. This alerts an operator to focus on a specific area of the corridor where an event is

detected and determine what type of emergency response is needed. For this project, the CCTV

cameras have the best view of sections 2 and 3 in the westbound direction of I-94 and sections 6

and 7 in the eastbound direction of I-94. The circular objects in each section represent vehicles

detected by the radar. The red circular object represents the location of the radar unit on the

corridor. It should be noted that data from these sections were collected between July and

September 2015, after which time a different set of sections were implemented following updates

to the system software to improve the accuracy of vehicle counts and speeds. The updated

sections are presented in Figure 3-5.

Figure 3-4 – Witness Software Interface of Corridor Sections and Vehicles

(July through September 2015)

Figure 3-5 – Witness Software Interface of Corridor Sections

(December 2015 through March 2016)

12

When the software detects one of the four events as noted earlier, it sends a message to the

Incident Trigger Module, which in turn, instructs the Network Video Recorder (NVR) to begin a

recording of the event that was detected. The recording captures 15 seconds of video before the

event was detected, and 30 seconds of time after the event was detected. While the NVR is

always operating to record video, it only saves videos when instructed by the trigger module and

it buffers the remaining video to conserve storage space on the NVR.

The purpose of recording videos of events detected by the radar unit and Witness software is to

verify for MnDOT that the alarms presented to RTMC operators are reliable to act upon, and that

only a very low number of “false alarms” are created. This will help to create trust in the

reliability of the software and, in turn, reduce incident notification times for incidents detected on

the corridor if the equipment were deployed over a longer term. An image of the interface

available with the NVR unit is presented in Figure 3-6. Each of the two cameras present fixed

views of the corridor with date / time stamps that can be verified against the date / time stamps of

events logged by the radar unit and software.

Figure 3-6 – NVR Interface with Camera Views of Corridor

3.4. Central Software Calibration

The Witness software that analyzes the radar data communicated from the field requires a lot of

fine calibration during the initial setup of the system as shown in the screenshot taken below in

Figure 3-7. The settings displayed in the Figure represent the most recent settings implemented

for the project. The challenge is in setting the “Track Density for Queue”, the “Slow Density for

Queue”, and the “Minimum Speed for Queue” to detect traffic incidents while trying not to

detect general traffic congestion as much.

It should be noted that Navtech Radar is currently developing on a software upgrade to better

detect traffic incidents during periods of heavy traffic congestion, which came as a direct result

from their experience on this project.

13

Westbound Traffic Eastbound Traffic

Figure 3-7 – Current Settings with Central Software

14

3.5. Overall System Installation and Adjustment Timeline

The biggest challenge encountered on the project by the AECOM team was properly configuring

both the radar unit in the field and the central software installed at the MnDOT RTMC to

accurately perform the simultaneous tasks of incident detection and measurements of vehicle

counts and speeds on the corridor. The proper tilt of the radar toward the roadway was initially

established but then modified slightly in July in an effort to improve the accuracy of vehicle

detection in the project area. This configuration slightly limited the radar’s capability to detect

stopped vehicles on the corridor. Configurations to the central software were also performed in

October in an effort to improve the accuracy of vehicle counts and speeds on the corridor. This

configuration required software upgrades from Navtech Radar to perform these simultaneous

tasks of incident detection and vehicle counts.

A summary of installation and adjustment activities is presented in Table 3-2. A key event in the

timeline of activity was the adjustments made to the central software to improve the radar’s

ability to measure vehicle counts and speeds in early October 2015. This created a level of

uncertainty in the radar’s ability to accurately measure incident detection through alarms on the

corridor. As a result, the evaluation of the radar’s ability to detect incidents on the corridor was

paused to allow for the adjustments to be made on the central software.

Subsequent adjustments were made to the central software in the months of October and

November 2015 to improve the accuracy of measuring vehicle counts and speeds on the I-94

corridor. These adjustments were completed in December and the evaluation of the radar

resumed in late December. To allow for further data collection on the project, the end date for

the radar’s operation was extended 3 months from Dec. 31st, 2015 out to March 31st, 2016.

While the system’s accuracy at detecting and measuring vehicle speeds and counts was

significantly increased along the I-94 corridor, it is recommended from the radar unit’s

manufacturer (Navtech) to install a different model of radar equipment that can better measure

vehicle speeds and counts on an arterial type of roadway without losing the capability of

detecting stopped vehicles for incident detection purposes.

Table 3-2 – High-Level System Installation and Adjustment Timeline

Date Activity Notes

June 2nd, 2015

Radar and cameras installed on the

pole; equipment placed into cabinet

for verifying connection to MnDOT

RTMC

Configuration of IP address for the

radar required a second trip to the

corridor; leveling of the radar with

an inclinometer took longer than

planned.

June 8th, 2015 Radar IP address configured at the

cabinet

Able to view the radar data through

central software. Central software

calibration activity performed by

Navtech staff.

15

Table 3-2 – High-Level System Installation and Adjustment Timeline

Date Activity Notes

June 17th, 2015

Radar equipment on the pole was

adjusted to allow radar to pick up

more vehicles on the east half of the

corridor. North-south tilt was

increased from 1 to 7 degrees

Adjustments were made manually

without use of inclinometer.

June 19th, 2015

Initial system configurations were

completed through central software

after visit from Navtech staff

Initial date for confirming system

operations and beginning evaluation

of radar

July 20th, 2015

Radar equipment was adjusted on

the pole to allow radar to pick up

more vehicles on the west edge of

the corridor

Adjustment with the inclinometer to

decrease the north-south tilt of the

radar that was made on June 17th

(from 7 degrees to 1 degree)

October 5th,

2015

Adjustments to rules within the

software about vehicle detection

were made to improve vehicle count

/ speed detection

Adjustments negatively impacted

the ability of the software to issue

alerts for stopped vehicles on the

corridor; evaluation of radar paused

October 23rd

through Dec. 7th

Navtech radar staff dedicated time to

improving the capability of the radar

to measure vehicle counts and

speeds at an accurate level while

maintaining the ability of the radar

to detect incidents

Multiple meetings held with

Navtech staff to understand progress

being made on the creation of new

zones on the corridor for vehicle

counts and speeds

Dec. 18th, 2015 Date set for resuming evaluation of

the radar per system requirements

Small adjustments made to zones to

improve detection of vehicles on

shoulder lanes. New set of sections

were implemented on the corridor.

January 15th,

2016

Software upgrade made. Adjustment

made to system parameters by

Navtech that negatively impacted

the ability for the software to detect

stopped vehicles measured by radar

Result of changes made to system

parameters was a sharp decrease in

the number of incidents detected by

the radar and impact to accuracy of

speed measurements.

March 2nd, 2016

Correction implemented on system

parameters to increase the ability of

the software to pick up vehicles and

incidents more accurately

Result of correction to parameters

was an increase in the number of

incidents detected and the accuracy

of incident detection and vehicle

counts

March 31st,

2016 End of operational test for project.

End date of system operations for

evaluation purposes.

16

Chapter 4 360 Degree Radar System Evaluation

This section contains the results of the evaluation of the 360 Degree Radar Information System.

In following with the Systems Engineering process, the system will be evaluated against the

goals and objectives of the project, in addition to being evaluated against the accuracy of the data

collected by the radar and processed by the software.

4.1. System Validation Process

The intent of the System Validation process is to measure the goals and objectives that were

documented within the Concept of Operations for the project. Measures of Effectiveness

(MOEs) have been proposed to quantifiably demonstrate that the system is meeting the goals and

objectives that were defined at the beginning of the project. Table 4-1 below contains a

summary of the proposed MOEs that trace back to the goals and objectives of the 360 Degree

Radar Information System previously identified in the Concept of Operations. Given that the

system was operational across multiple seasons, system operations in all types of weather

conditions can be evaluated.

Table 4-1 – Validation Plan Measures of Effectiveness

Goals and Objectives

from Concept of Operations Validation Plan MOEs

Goal

Perform traffic / incident detection and

data collection activities accurately

using the Navtech 360 Degree Radar

Information System

System positively identifies all

desired types of measures through

alerts

Objectives

1. Test the accuracy of traffic

congestion detected against camera

images monitoring traffic along the

corridor and against other sources

of data (i.e. Wavetronix data)

System positively identifies traffic

congestion through accurate

vehicle counts and alerts when

speeds are detected below pre-

determined MPH thresholds

2. Test the accuracy of incident

detection alerts recorded by the

system against other sources of

incident data (i.e. CAD system)

System positively identifies a

traffic incident through a pre-

configured alert threshold

3. Measure the efficiency of the

system’s operations in all weather

conditions to verify reliability of

system alerts generated

Determine percentage variance

between “false positive” alerts in

various weather conditions

4. Test the accuracy of all other alerts

detected against camera images

monitoring traffic along the

corridor and against other sources

of data (i.e. CAD System)

System positively identifies all

other conditions through pre-

configured alert thresholds

17

4.2. System Validation Results

A summary of the four MOEs listed in Table 4-1 is presented in the following sub-sections.

These sections were included within the Validation Plan document developed for the project and

have been modified to reflect actual conditions encountered in the project.

4.2.1. MOE #1 – Speed Threshold Alerts and Vehicle Counts

This MOE pertains to the accuracy of alerts generated when vehicle speeds are detected

below a pre-determined speed threshold. The speed threshold has been set at 15 miles

per hour (MPH). An alert is generated in the system and logged when a vehicle’s speed

is detected below this threshold.

The speed threshold data generated by the 360 degree radar system has been validated

against lane speeds measured from the Wavetronix detector installed within the project

area. Detector data has also been gathered from the MnDOT DataExtract Tool at 1-

minute intervals that indicate the measured travel speeds. DataExtract provides lane-

specific travel speeds have been used to validate the speed threshold alerts generated by

the 360 Degree Radar System.

DataExtract has also been used in a similar manner to validate the accuracy of vehicle

counts by the 360 Degree Radar Information System. DataExtract provides lane-specific

vehicle counts that are also used to validate the accuracy of vehicle counts down to a 1-

minute interval. DataExtract exports the vehicle count data into an MS Excel file for ease

of comparing count data in MS Excel.

During the course of the demonstration, there were a large amount of speed threshold

alerts generated from slow moving travel in the rush hour periods. The alerts that could

be presented for vehicle speeds less than 15 MPH was not found to be as valuable as

stopped vehicles in the corridor. In the second period of the demonstration, a sample of

speed threshold alerts was extracted from the central software for comparison with

Wavetronix data at the same time period.

The MOE was validated as the system positively identified events when speeds are

detected below the pre-determined 15 MPH threshold for all lanes of traffic in the

westbound and eastbound directions, as compared against the MnDOT DataExtract that

was also gathered.

Regarding vehicle counts, this MOE was validated as part of the evaluation of the vehicle

counts measured against Wavetronix data which is presented in Section 4.3.

4.2.2. MOE #2 – Traffic Incident Alerts

This MOE pertains to the accuracy of alerts generated when stopped vehicles are detected

within the project area, which may include traffic incidents involving one or multiple

vehicles that require emergency response.

The process of validating traffic incident alerts involves verifying those incidents against

incident logs made by RTMC operations staff. Figure 4-1 below presents a screenshot of

18

an incident from Dec. 17th, 2015 in the westbound direction of the corridor limits. The

incident notification time was logged as 12:57 pm. These incidents are reviewed with

MnDOT cameras to confirm that the events occurred within the project limits of the

corridor. A screenshot of the camera confirmation is presented in Figure 4-2, with the

timestamp and incident also highlighted.

Figure 4-1 – MnDOT Computer Aided Dispatch Summary of Dec. 17th Event

Figure 4-2 – MnDOT Camera Record of Dec. 17th Event for Confirmation with

360 Degree Radar System

Location of stopped vehicles

Timestamp of incident in video playback

Timestamp of incident notification in MnDOT CAD

MnDOT CAD EID 7750376

19

The 360 Degree Radar system recorded the event as shown in Figure 4-3. The

timestamp that aligns best with the MnDOT incident notification is circled and

highlighted in blue. At this point in time, the vehicles shown in Figure 4-2 first

proceeded in Section 0 that represents the emergency shoulder area installed in this part

of the corridor. However, the 360 radar first recorded the stopped vehicles in Section 12

at 12:50 pm, approximately 7 minutes before received a notification from the motorists

via cell phone. This event is also circled in Figure 4-3.

Figure 4-3 – 360 Degree Radar System Record of Dec. 17th Event

The notification of the incident at 12:50pm is also recorded by the CCTV cameras in the

NVR. This is shown in Figure 4-4. The location of the incident in recorded by the radar

in Section 12 is circled in the still image. Note that the incident was recorded at 12:50:32

in the timestamp of the camera. The 360 Degree Radar central software was configured

to record an event after a vehicle was stopped for a period of 10 seconds in the corridor.

The timestamp of the alert in Figure 4-3 is recorded about 10 seconds later at 12:50:41.

The two vehicles involved in the incident then pulled ahead into the emergency shoulder

area at about 12:58 with the assistance of a MnDOT FIRST vehicle that arrived on the

scene. As the vehicles stopped in that area, an additional alert was created in the

software, which can also be seen in Figure 4-3.

Timestamp of stopped vehicles match with MnDOT incident notification

Timestamp of stopped vehicle alert in 360 radar system – 7 minutes prior to

MnDOT notification in CAD system

20

Incident recorded on camera by NVR

Camera timestamp NVR timestamp

Figure 4-4 – Interface of NVR with Video Record Dec. 17th Event

This MOE has been considered as validated through the comparison of 360 Degree Radar

system events, as compared against CCTV camera footage recorded on the NVR unit and

against MnDOT CAD events logged throughout the project. A summary of the events

compared against the MnDOT CAD system is presented on the following pages.

Table 4-2 presents a summary of events from the August to September period of incident

data collection. Table 4-3 presents a summary of events from the February to March

period of incident data collection.

It should be noted that various updates to the central software occurred during the periods

of October and November that limited the ability of the system to accurately detect

incidents in the project area. In addition, it was learned through the course of the project

that heavy traffic congestion in the project area caused the software to ignore multiple

alarms of stopped vehicles at the same time. This had the impact of causing the software

to ignore stopped vehicle events reported by the radar unit that were actual incidents

logged by the RTMC operators. Navtech Radar has learned of this issue through this

project and is actively developing a software upgrade to address this issue in future

deployments. These events of congestion (i.e. stopped vehicles) masking the traffic

incidents are noted in Tables 4-2 and 4-3 below under the Description column.

21

Table 4-4 presents a summary of 11 events from the August to September period that

were un-detected by RTMC operators, but were recorded the 360 Degree Radar

Information System. This finding for the two month period of time indicates that other

events may have gone un-detected by RTMC operators throughout the course of the

project because those events were not called in to 911 operators as quickly as they could

have been by those involved in the incidents.

22

Table 4-2 – Summary of Events Logged by 360 Degree Radar and MnDOT CAD System (August – September 2015)

Date Radar Alarm

Radar Time

Section CAD Event CAD Time Radar/CAD Time Diff.

CAD EID CAD

Response Response

Time Description

8/9/2015 Stopped vehicle

15:03 2 Crash 3:03 PM 0 min 7479318


13:15 10 Crash 1:22 PM +7 min 7482760 FIRST 1:34 PM Tow dispatched at 1:34 pm. Trooper dispatched at 1:44 pm.

8/12/2015 None Stall 4:59 PM 7485217 none -- General congestion that masked an incident by radar.


13:00 2 Crash 1:03 PM +3 min 7499194 FIRST 1:09 PM

Crash in section 2, vehicles stopped in section 3. Tow vehicle dispatched at 1:23pm as well.


14:27 10 Stall 2:44 PM +17 min 7512696 FIRST 2:44 PM Trucked pulled over and stopped.


17:56 7 Stall 5:59 PM +3 min 7537516 none Bus pulled over on shoulder.


18:50 10 Crash 6:53 PM +3 min 7537624 FIRST 6:56 PM Two cars involved in crash.

9/9/2015 None Stall 9:40 AM 7547724 FIRST 9:40 AM General congestion that masked an incident by radar.


19:19 7 Stall 7:19 PM 0 min 7559448 FIRST 7:19 PM

9/15/2015 None Stall 4:40 PM 7561197 FIRST 5:27 PM General congestion that masked an incident by radar.


12:19 10 Crash 12:29 PM +10 min 7568883 Trooper 12:53 PM Two cars involved in crash.

Average Difference of Notification Time (Based on 8 events in bold) + 5.4 min

23

Table 4-3 – Summary of Events Logged by 360 Degree Radar and MnDOT CAD System (February - March 2016)

Date Radar Alarm

Radar Time

Section CAD

Event CAD Time

Radar/CAD Time Diff.

CAD EID CAD

Response Response

Time Description


10:30 2 Stall 10:38 AM +8 Min. 7750107 none none Tractor trailer stalled.


12:50 10 Crash 12:57 PM +7 Min. 7750376 FIRST 12:57 PM

Actual crash recorded. Drivers got out and stood on highway. Image shown in Figure 4-4.


7:22 0 Crash 8:20 AM +58 Min. 7892574 FIRST +

Tow Truck

8:27 AM + 8:40AM

3/1/2016 None n/a 3 Crash 7:39 AM n/a 7894412

Trooper + FIRST +

Tow Truck

7:50 AM + 7:56 AM + 8:04 AM

Eastbound in constant queue.


16:11 3 Stall 4:01 PM n/a 7901374 FIRST 4:10 PM

General congestion that masked an incident by radar. Detected MnDOT vehicle @ 4:11pm.


17:29 0 Crash 5:12 PM n/a 7938652 FIRST + Trooper

5:23 PM + 5:26 PM

Congestion Queue activated during this time

3/25/2016 None n/a 2 Crash 4:25 PM n/a 7945331 FIRST 4:25 PM General congestion that masked an incident by radar.

Average Difference of Notification Time (Based on 3 events in bold) + 24.3 min

Average Difference of Notification Time for all Incidents in Project (Based on 11 total events in bold in Table 4-2 and Table 4-3)

+ 10.54 minutes

24

Table 4-4 – Summary of Events Logged by 360 Degree Radar and MnDOT CAD System

Radar Events Not On CAD with Minimum of 4 Minute Alarm (Total of 12 events listed)

Date Radar Alarm Radar Time Section

CAD Event Description

8/9/2015 Stopped vehicle 10:23 10 None Vehicle in pulloff for 8 minutes

8/10/2015 Stopped vehicle 15:58 2 None 2 vehicles pulled onto shoulder, got out of vehicles, walked around

8/11/2015 Stopped vehicle 4:59 7 None Vehicle stopped on shoulder for a minimum of 5 minutes.

8/12/2015 Stopped vehicle 10:30 7 None Pickup truck stopped on shoulder for a minimum of 5 minutes.

8/12/2015 Stopped vehicle 14:05 10 None Truck pulled over and stopped for a minimum of 7 minutes.

8/15/2015 Stopped vehicle 17:45 10 None Trucked pulled over and stopped for a minimum of 22 minutes, possibly 1 hour.

8/19/2015 Stopped vehicle 1:30 10 None 2 cars pulled over and stopped for a minimum of 5 minutes.

8/20/2015 Stopped vehicle 11:25 10 None 2 cars pulled over and stopped for a minimum of 4 minutes.



9/1/2015 Stopped vehicle 9:56 10 None Pickup truck stopped on shoulder for a minimum of 8 minutes.

12/18/2015 Stopped vehicle 9:58 0 None Vehicle pulled off for over 10 minutes, person got out and walked around.

Notes: 1) There were 10 to 15 more alarms ranging from 1 to 2 minutes where the vehicles were most likely stopped for a longer period of time.

2) A stopped vehicle will only alarm for 1 to 2 minutes then "blend" it into the landscape even if it is stopped for +2 minutes. This helps to eliminate

"blobs" on the radar detection. We are working on trying to track stopped vehicles for a longer period of time. Alarms for large trucks last longer as they

take longer to "blend" into the landscape. This is one reason we see more alarms for trucks.

25

4.2.3. MOE #3 – Measure of “False Positive” Alerts

This MOE pertains to the number of “false positive” alerts that may be generated by

the system for all types of pre-configured alerts.

A “false positive” alert is defined as an alert that cannot be verified against a camera

recording on the NVR unit of an event for which the alert was created, or against data

gathered from the MnDOT DataExtract for the purposes of verifying the alert was

accurate.

Based on prior project deployments, the manufacturer of the radar has noted that false

positives are likely to occur in the initial stages of the radar’s operation. As these

false positive alerts are reviewed, the occurrence can be greatly reduced by modifying

the rules within the software that are set to define when alerts should be generated.

Table 4-5 illustrates the false alarms observed for the period of June 20th to June 26th

after initial system configuration. A total of 5 false alarms were noted (out of a total

of 41 events for the period), with descriptions and possible causes listed. The average

false alarm rate over these 7 days was recorded 0.71, which would be less than one

per day.

Table 4-5 – Summary of False Alarms Logged June 20th to June 26th

Date Time Alarm Type

East / West

Section Alarm

(Verified / False)

Description Possible Cause

6/22/2015 16:53:38 Stopped vehicle

East-bound

6 False Shows tracks over all 3 lanes. 35W ramp moving slow with truck passing.


East-bound

7 False

Shows tracks on shoulder close to off-ramp. Other tracks shown around same time. 35W on-ramp stopped.

Per video truck stopped on 35W on-ramp, possibly causing false alarms.


East-bound

6 False

Shows tracks on shoulder close to off-ramp. Other tracks shown around same time. 35W on-ramp stopped.

Per video 2 large semi's pass slowly during alarm on 35W on-ramp possibly causing false alarm.

6/26/2015 12:56:09 Reverse vehicle

East-bound

7 False No vehicle in reverse. Westbound traffic very slow.


East-bound

6 False No vehicle in reverse.

Westbound traffic very slow, truck passing slowly westbound.

26

Table 4-6 illustrates the false alarms observed for the period of March 2nd to March

9th, 2016 after central software updates were made later in the project. A total of 2

false alarms were noted (out of a total of 66 events for the period), with descriptions

and possible causes listed. The average false alarm rate over these 7 days was

recorded 0.28, which would be less than one false alarm per day. This decrease in the

false alarm rate observed from the beginning of the project to the end of the project

helps to demonstrate how the accuracy of incident detection by the 360 Degree Radar

Information System improves over time as the system is updated to better understand

how to ignore previous false alarms encountered in a project.

Table 4-6 – Summary of False Alarms Logged March 2nd to March 9th

Date Time Alarm Type

East / West

Section Alarm

(Verified / False)

Description Possible Cause


East-bound

3 False No stopped vehicle present


East-bound

3 False No stopped vehicle present

4.2.4. MOE #4 – All Other Condition Alerts

This MOE pertains to the accuracy of all other types alerts generated when pre-

determined conditions are detected within the project area. These conditions will

include the following:

1. Individual person(s) detected within the project area

2. Vehicles slowly moving in reverse on the corridor

For item #1, an alert is generated in the system and logged when a person is detected

within the project area. These alerts can be categorized in a separate Microsoft Excel

file for review in analysis of the system alerts. Video is recorded of the person(s) on

the NVR unit within the system server for verification purposes. The timestamp of

the video recording is then correlated with the timestamp of the alert recorded on the

system server.

The alerts are validated when the video recording shows the person(s) within the field

project area. This portion of the MOE has been validated based on events logged by

the radar compared against CCTV camera footage recorded on the NVR unit.

For item #2, an alert is generated in the system and logged when vehicles are detected

to be slowly moving in the reverse direction on any portion of the corridor. This

portion of the MOE has been validated based on the reverse movement of vehicles

detected on the corridor, as compared against CCTV camera footage recorded on the

NVR unit.

27

Table 4-7 illustrates the number of alarms that were “verified” for the period of June

20th to June 26th after initial system configuration. A total of 6 verified “person”

alarms and 2 “reverse vehicle” alarms were confirmed against CCTV recorded video,

with descriptions listed. Some of the verified alarms relate to the same event that

occurred within the field.

Table 4-7 – Summary of Person and Reverse Vehicle Alarms Verified June 20th to June 26th

Date Time Alarm Type East/West Section Description

6/20/2015 15:06:54 Person Westbound 10 4 people get out of 3 vehicles on pulloff.


Eastbound 6 Vehicle reversed on shoulder 300ft to get on 35W on-ramp.


Eastbound 6 MnDOT FIRST vehicle backing up on fast lane shoulder.

6/22/2015 11:55:30 Person Westbound 2 Driver and other vehicle came back to get vehicle.

6/22/2015 11:59:39 Person Westbound 2 Same driver causing another alarm.

6/22/2015 19:20:36 Person Westbound 2 Tow truck driver.

6/25/2015 0:53:59 Person Westbound 2 Road crew got out of truck.

6/26/2015 7:54:01 Person Westbound 10 Person walked around truck.

4.3. System Evaluation Process

For the more detailed System Evaluation, a subset of data was gathered at various 15-minute

time periods over the course of multiple dates. A summary of the time periods is presented

in Table 4-8.

Table 4-8 – Dates of Vehicle Speed and Count Comparisons with Wavetronix Data

Dates of Data Collection Time Periods Notes

Nov. 19th – 20th, 2015

2:30 am to 2:45 am

10:00 am to 10:15am

2:00 pm to 2:15 pm

5:00 pm to 5:15 pm

Initial comparison of vehicle

speeds and counts with the

Wavetronix Data. Manual

counts also performed.

Jan. 17th, 2016

3:30 am to 3:45 am

9:30 am to 9:45 am

1:30 pm to 1:45 pm

5:30 pm to 5:45 pm

High temperature of minus 8

degrees Fahrenheit.

Feb. 2nd, 2016

3:15 am to 3:30 am

11:15 am to 11:30 am

1:15 pm to 1:30 pm

5:15 pm to 5:30 pm

Heavy snowfall (10 inches)

from 12 noon to 12 midnight.

28

Radar data and CCTV camera video were recorded on the NVR unit for playback and

evaluation purposes against data gathered from the MnDOT DataExtract tool over an

identical period of time.

Vehicle speeds and vehicle counts measured by the 360 Degree Radar System were

compared against vehicle speeds estimated by the Wavetronix unit installed at the project

area measuring each lane of traffic. Westbound lanes are indicated as #356W1, #357W2,

and #358W3, while the eastbound lanes are indicated as #497E1, #498E2, and #499E3. For

the Nov. 20th date of data collection, visual counts of traffic using CCTV camera recordings

of traffic were performed to have an independent point of comparison with both the 360

Degree Radar and the Wavetronix data sets.

Data on each lane of travel was gathered from the DataExtract tool. Vehicle speeds

estimated by the Wavetronix unit were gathered at 1-minute intervals during the time periods

indicated in Table 4-8. Vehicle speed data for each lane of traffic, as well as an Aggregate

measure of all lanes, was gathered from the DataExtract tool and exported to a Microsoft

Excel file for ease of analysis.

The Excel files were imported into a separate Microsoft Excel file generated through use of

the 360 Degree Radar central software. Charts and graphs were created in Microsoft Excel to

compare radar data with the MnDOT data to determine how close the two sets of data were

to each other.

4.4. System Evaluation Results

The System Evaluation results are presented in the following Figures in this section.

Multiple dates are presented in the following figures that are summarized below. For data

collected on Thursday, Nov. 19th and Friday, Nov. 20th, 2015, the following figures have

been created on the following pages:

Figure 4-5 – Navtech and Wavetronix westbound (WB) and eastbound (EB) speed

comparisons for all lanes of traffic combined based on data collected on Nov. 20th,

2015 at three different time periods


comparisons for individual lanes of traffic based on data collected on Nov. 20th, 2015

at different time periods

Figures 4-7 – Navtech, Wavetronix, and Manual vehicle count comparisons for

westbound (WB) and eastbound (EB) directions based on data collected on Nov. 19th,

2015 at four different time periods

Figure 4-8 – Navtech and Wavetronix westbound (WB) and eastbound (EB) vehicle

count comparisons for individual lanes of traffic based on data collected on Nov. 19th,

2015 at different time periods

For data collected on Sunday, Jan. 17th, 2016, the figures listed have been created on the

following pages. It should be noted that this date was selected for the very low temperatures

observed. The high temperature recorded on this date was minus 2 degrees Fahrenheit, with

the low temperature at minus 13 degrees Fahrenheit. Winds between 10 and 20 miles per

29

hour were recorded, which created wind chill temperatures lower than the noted low

temperature of minus 13 degrees.


comparisons for all lanes of traffic combined based on data collected on Jan. 17th,



comparisons for individual lanes of traffic based on data collected on Jan. 17th, 2016


Figures 4-11 – Navtech and Wavetronix vehicle count comparisons for westbound

(WB) and eastbound (EB) directions based on data collected on Jan. 17th, 2016 at

different time periods


count comparisons for individual lanes of traffic based on data collected on Jan. 17th,


For data collected on Tuesday, Feb. 2nd, 2016, the figures listed have been created on the

following pages. It should be noted that this date was selected for the high amount of

snowfall throughout the day and the low temperatures observed. Approximately 10 inches of

snow fell on this date in the Minneapolis area between 12 noon and midnight.


comparisons for all lanes of traffic combined based on data collected on Feb. 2nd,



comparisons for individual lanes of traffic based on data collected on Feb. 2nd, 2016


Figures 4-15 – Navtech and Wavetronix vehicle count comparisons for westbound

(WB) and eastbound (EB) directions based on data collected on Feb. 2nd, 2016 at

different time periods


count comparisons for individual lanes of traffic based on data collected on Feb. 2nd,


For all three time periods observed, vehicle counts remained relatively close to the

Wavetronix unit on the pole at the location of the radar unit regardless of weather conditions

observed on these dates. The vehicle speed measurements were also relatively close for the

November time period, but were consistently lower than what the Wavetronix device had

reported in the January and February time periods. This is primarily due to changes that

were made to system parameters on Jan. 15th that negatively impacted the accuracy of speed

measurements, as well as a decrease in the number of incidents recorded after this date.

30

Figure 4-5 – Navtech and Wavetronix WB and EB Speed Comparisons on Nov. 20th

31

Figure 4-5 (continued) – Navtech and Wavetronix WB and EB Speed Comparisons on Nov. 20th

32

Figure 4-5 (continued) – Navtech and Wavetronix WB and EB Speed Comparisons on Nov. 20th

33

Figure 4-6 – Navtech and Wavetronix WB and EB Speed Comparisons on Nov. 20th

By Lane for all Time Periods

34

Figure 4-7 – Navtech, Wavetronix, and Manual WB and EB Count Comparisons on Nov. 19th

35

Figure 4-7 (continued) – Navtech, Wavetronix, and Manual WB and EB Count Comparisons on Nov. 19th

36


37


38

Figure 4-8 – Navtech and Wavetronix WB and EB Count Comparisons on Nov. 19th By

Lane for all Time Periods

39

Figure 4-9 – Navtech and Wavetronix WB and EB Speed Comparisons on Jan. 17th, 2016

0

10

20

30

40

50

60

70

80

9:30 AM 9:33 AM 9:36 AM 9:38 AM 9:41 AM 9:44 AM 9:47 AM

Spee

d (

MP

H)

Time

WB Speed Comparison (All Lanes) -- Jan. 17th, 2016 - 9:30 am to 9:45 am

Navtech

Wavetronix

0

10

20

30

40

50

60

70

80

9:30 AM 9:33 AM 9:36 AM 9:38 AM 9:41 AM 9:44 AM 9:47 AM

Spee

d (

MP

H)

Time

EB Speed Comparison (All Lanes) -- Jan. 17th, 2016 - 9:30 am to 9:45 am

Navtech

Wavetronix

40

Figure 4-9 (continued) – Navtech and Wavetronix WB and EB Speed Comparisons on Jan. 17th, 2016

0

10

20

30

40

50

60

70

1:29 PM 1:32 PM 1:35 PM 1:37 PM 1:40 PM 1:43 PM 1:46 PM

Spee

d (

MP

H)

Time

WB Speed Comparison (All Lanes) -- Jan. 17th, 2016 - 1:30 pm to 1:45 pm

Navtech

Wavetronix

0

10

20

30

40

50

60

70

1:29 PM 1:32 PM 1:35 PM 1:37 PM 1:40 PM 1:43 PM 1:46 PM

Spee

d (

MP

H)

Time

EB Speed Comparison (All Lanes) -- Jan. 17th, 2016 - 1:30 pm to 1:45 pm

Navtech

Wavetronix

41

Figure 4-9 (continued) – Navtech and Wavetronix WB and EB Speed Comparisons on Jan. 17th, 2016

0

10

20

30

40

50

60

70

5:28 PM 5:31 PM 5:34 PM 5:36 PM 5:39 PM 5:42 PM 5:45 PM 5:48 PM

Spee

d (

MP

H)

Time

WB Speed Comparison (All Lanes) -- Jan. 17th, 2016 - 5:30 pm to 5:45 pm

Navtech

Wavetronix

0

10

20

30

40

50

60

70


Spee

d (

MP

H)

Time

EB Speed Comparison (All Lanes) -- Jan. 17th, 2016 - 5:30 pm to 5:45 pm

Navtech

Wavetronix

42

Figure 4-10 – Navtech and Wavetronix WB and EB Speed Comparisons on Jan. 17th


-50

-45

-40

-35

-30

-25

-20

-15

-10

-5

0

0315-0330 0915-0930 1315-1330 1715-1730

Per

cen

tage

Dif

fere

nce

Time

Navtech WB Average Speed Comparison Per Lane with Wavetronix Data Jan. 17th, 2016

Slow Lane Middle Lane Fast Lane

-50

-45

-40

-35

-30

-25

-20

-15

-10

-5

0

0315-0330 0915-0930 1315-1330 1715-1730

Per

cen

tage

Dif

fere

nce

Time

Navtech EB Average Speed Comparison Per Lane with Wavetronix Data Jan. 17th, 2016


43

Figure 4-11 – Navtech and Wavetronix WB and EB Count Comparisons on Jan. 17th

0

2

4

6

8

10

12

14

16

3:30 AM 3:33 AM 3:36 AM 3:38 AM 3:41 AM 3:44 AM 3:47 AM

Co

un

t

Time

EB Count Comparison (All Lanes) -- Jan. 17th, 2016 - 3:30 am to 3:45am

Navtech

Wavetronix

0

10

20

30

40

50

60

9:30 AM 9:33 AM 9:36 AM 9:38 AM 9:41 AM 9:44 AM 9:47 AM

Co

un

t

Time

EB Count Comparison (All Lanes) -- Jan. 17th, 2016 - 9:30am to 9:45 am

Navtech

Wavetronix

44

Figure 4-11 (continued) – Navtech and Wavetronix WB and EB Count Comparisons on Jan. 17th

0

20

40

60

80

100

120

1:29 PM 1:32 PM 1:35 PM 1:37 PM 1:40 PM 1:43 PM 1:46 PM

Co

un

t

Time

EB Count Comparison (All Lanes) -- Jan. 17th, 2016 - 1:30 pm to 1:45 pm

Navtech

Wavetronix

0

10

20

30

40

50

60

70

80


Co

un

t

Time

EB Count Comparison (All Lanes) -- Jan. 17th, 2016 - 5:30 pm to 5:45 pm

Navtech

Wavetronix

45


0

2

4

6

8

10

12

14

3:30 AM 3:33 AM 3:36 AM 3:38 AM 3:41 AM 3:44 AM 3:47 AM

Co

un

t

Time

WB Count Comparison (All Lanes) -- Jan. 17th, 2016 - 3:30 am to 3:45 am

Navtech

Wavetronix

0

5

10

15

20

25

30

35

40

45

50

9:30 AM 9:33 AM 9:36 AM 9:38 AM 9:41 AM 9:44 AM 9:47 AM

Co

un

t

Time

WB Count Comparison (All Lanes) -- Jan. 17th, 2016 - 9:30 am to 9:45am

Navtech

Wavetronix

46


0

10

20

30

40

50

60

70

80

1:29 PM 1:32 PM 1:35 PM 1:37 PM 1:40 PM 1:43 PM 1:46 PM

Co

un

t

Time

WB Count Comparison (All Lanes) -- Jan. 17th, 2016 - 1:30 pm to 1:45 pm

Navtech

Wavetronix

0

10

20

30

40

50

60

70

80

90

100


Co

un

t

Time

WB Count Comparison (All Lanes) -- Jan. 17th, 2016 - 5:30 pm to 5:45 pm

Navtech

Wavetronix

47

Figure 4-12 – Navtech and Wavetronix WB and EB Count Comparisons on Jan. 17th


-40

-20

0

20

40

60

80

100

120

0315-0330 0915-0930 1315-1330 1715-1730

Per

cen

tage

Dif

fere

nce

Time

Navtech WB Count Comparison Per Lane with Wavetronix Data Jan. 17th, 2016


-150

-100

-50

0

50

100

150

0315-0330 0915-0930 1315-1330 1715-1730

Per

cen

tage

Dif

fere

nce

Time

Navtech EB Count Comparison Per Lane with Wavetronix Data Jan. 17th, 2016


48

Figure 4-13 – Navtech and Wavetronix WB and EB Speed Comparisons on Feb. 2nd, 2016

0

10

20

30

40

50

60

70

11:13 AM 11:16 AM 11:19 AM 11:22 AM 11:25 AM 11:28 AM 11:31 AM

Spee

d (

MP

H)

Time

WB Speed Comparison (All Lanes) -- Feb. 2nd, 2016 - 11:15 am to 11:30 am

Navtech

Wavetronix

0

10

20

30

40

50

60

70

11:13 AM 11:16 AM 11:19 AM 11:22 AM 11:25 AM 11:28 AM 11:31 AM

Spee

d (

MP

H)

Time

EB Speed Comparison (All Lanes) -- Feb. 2nd, 2016 - 11:15 am to 11:30 am

Navtech

Wavetronix

49

Figure 4-13 (continued) – Navtech and Wavetronix WB and EB Speed Comparisons on Feb. 2nd, 2016

0

2

4

6

8

10

12

14

16

18

1:14 PM 1:17 PM 1:20 PM 1:23 PM 1:26 PM 1:29 PM 1:32 PM

Spee

d (

MP

H)

Time

WB Speed Comparison (All Lanes) -- Feb. 2nd, 2016 - 1:15 pm to 1:30 pm

Navtech

Wavetronix

0

5

10

15

20

25

30

35

40

45

1:14 PM 1:17 PM 1:20 PM 1:23 PM 1:26 PM 1:29 PM 1:32 PM

Spee

d (

PM

H)

Time

EB Speed Comparison (All Lanes) -- Feb. 2nd, 2016 - 1:15 pm to 1:30 pm

Navtech

Wavetronix

50

Figure 4-13 (continued) – Navtech and Wavetronix WB and EB Speed Comparisons on Feb. 2nd, 2016

0

5

10

15

20

25

5:13 PM 5:16 PM 5:19 PM 5:22 PM 5:25 PM 5:28 PM 5:31 PM

Spee

d (

MP

H)

Time

WB Speed Comparison (All Lanes) -- Feb. 2nd, 2016 - 5:15 pm to 5:30 pm

Navtech

Wavetronix

0

5

10

15

20

25

30

35

40

45

5:13 PM 5:16 PM 5:19 PM 5:22 PM 5:25 PM 5:28 PM 5:31 PM

Spee

d (

MP

H)

Time

EB Speed Comparison (All Lanes) -- Feb. 2nd, 2016 - 5:15 pm to 5:30 pm

Navtech

Wavetronix

51

Figure 4-14 – Navtech and Wavetronix WB and EB Speed Comparisons on Feb. 2nd


-40

-35

-30

-25

-20

-15

-10

-5

0

0230-0245 1000-1015 1400-1415 1700-1715

Per

cen

tage

Dif

fere

nce

Time

Navtech WB Average Speed Comparison Per Lane with Wavetronix Data Feb. 2nd, 2016


-80

-70

-60

-50

-40

-30

-20

-10

0

0230-0245 1000-1015 1400-1415 1700-1715

Per

cen

tage

Dif

fere

nce

Time

Navtech EB Average Speed Comparison Per Lane with Wavetronix Data Feb. 2nd, 2016


52

Figure 4-15 – Navtech and Wavetronix WB and EB Count Comparisons on Feb. 2nd

0

2

4

6

8

10

12

14

16

3:12 AM 3:15 AM 3:18 AM 3:21 AM 3:24 AM 3:27 AM 3:30 AM 3:33 AM

Co

un

t

Time

EB Count Comparison (All Lanes) -- Feb. 2nd, 2016 - 3:15 am to 3:30 am

Navtech

Wavetronix

0

10

20

30

40

50

60

70

80

90

11:13 AM 11:16 AM 11:19 AM 11:22 AM 11:25 AM 11:28 AM 11:31 AM

Co

un

t

Time

EB Count Comparison (All Lanes) -- Feb. 2nd, 2016 - 11:15 am to 11:30 am

Navtech

Wavetronix

53

Figure 4-15 (Continued) – Navtech and Wavetronix WB and EB Count Comparisons on Feb. 2nd

0

10

20

30

40

50

60

70

1:14 PM 1:17 PM 1:20 PM 1:23 PM 1:26 PM 1:29 PM 1:32 PM

Co

un

t

Time

EB Count Comparison (All Lanes) -- Feb. 2nd, 2016 - 1:15pm to 1:30 pm

Navtech

Wavetronix

0

10

20

30

40

50

60

70

5:13 PM 5:16 PM 5:19 PM 5:22 PM 5:25 PM 5:28 PM 5:31 PM

Co

un

t

Time

EB Count Comparison (All Lanes) -- Feb. 2nd, 2016 - 5:15 pm to 5:30 pm

Navtech

Wavetronix

54


0

1

2

3

4

5

6

7

8

3:12 AM 3:15 AM 3:18 AM 3:21 AM 3:24 AM 3:27 AM 3:30 AM 3:33 AM

Co

un

t

Time

WB Count Comparison (All Lanes) -- Feb. 2nd, 2016 - 3:15 am to 3:30 am

Navtech

Wavetronix

0

10

20

30

40

50

60

70

80

90

100

11:13 AM 11:16 AM 11:19 AM 11:22 AM 11:25 AM 11:28 AM 11:31 AM

Co

un

t

Time

WB Count Comparison (All Lanes) -- Feb. 2nd, 2016 - 11:15 am to 11:30 am

Navtech

Wavetronix

55


0

10

20

30

40

50

60

70

80

1:14 PM 1:17 PM 1:20 PM 1:23 PM 1:26 PM 1:29 PM 1:32 PM

Co

un

t

Time

WB Count Comparison (All Lanes) -- Feb. 2nd, 2016 - 1:15 pm to 1:30 pm

Navtech

Wavetronix

0

10

20

30

40

50

60

70

5:13 PM 5:16 PM 5:19 PM 5:22 PM 5:25 PM 5:28 PM 5:31 PM

Co

un

t

Time

WB Count Comparison (All Lanes) -- Feb. 2nd, 2016 - 5:15 pm to 5:30 pm

Navtech

Wavetronix

56

Figure 4-16 – Navtech and Wavetronix WB and EB Count Comparisons on Feb. 2nd


-20

-10

0

10

20

30

40

50

60

70

0230-0245 1000-1015 1400-1415 1700-1715

Per

cen

tage

Dif

fere

nce

Time

Navtech WB Count Comparison Per Lane with Wavetronix Data Feb. 2nd, 2016


-40

-20

0

20

40

60

80

0230-0245 1000-1015 1400-1415 1700-1715Per

cen

tage

Dif

fere

nce

Time

Navtech EB Count Comparison Per Lane with Wavetronix Data Feb. 2nd, 2016


57

Chapter 5 Conclusions and Recommendations

This section contains conclusions on the overall 360 Radar Information System and

recommendations for MnDOT to consider in deploying further systems at other locations

along corridors and at intersections as well.

5.1. Conclusions

A summary of the goals and objectives as presented in the Concept of Operations in addition

to general conclusions on each of these objectives are provided in Table 5-1. A summary of

key findings from the evaluation are presented in Table 5-2. A summary of the operational

needs addressed by the system (as noted in Chapter 1 of this report) are presented in Table 5-

3.

Table 5-1 – Summary of Conclusions on 360 Degree Radar Information System

Goals and Objectives

from Concept of Operations Conclusions

Goal

Perform traffic / incident detection

and data collection activities

accurately using the Navtech 360

Degree Radar Information System

Low number of incident false alarms

observed at beginning of project

decreased to a smaller number near

the end of the project

Objectives

1. Test the accuracy of traffic

congestion detected against

camera images monitoring traffic

along the corridor and against

other sources of data (i.e.

Wavetronix data)

Vehicle counts and speeds from radar

unit were found to be consistent with

Wavetronix counts and speeds, as

well as manual counts from

November 2015

2. Test the accuracy of incident

detection alerts recorded by the

system against other sources of

incident data (i.e. CAD system)

Traffic incidents positively identified

through stopped vehicle alerts and

compared with MnDOT CAD logs of

the same events

3. Measure the efficiency of the

system’s operations in all

weather conditions to verify

reliability of system alerts

generated

Vehicle counts and speeds from radar

unit were found to be consistent with

Wavetronix counts and speeds in

November period; Low number of

incident false alarms observed at

beginning and end of the project

4. Test the accuracy of all other

alerts detected against camera

images monitoring traffic along

the corridor and against other

sources of data (i.e. CAD

System)

Alerts generated by 360 Degree

Radar were validated alongside those

events from the MnDOT CAD

system; Low number of incident false

alarms observed at beginning and end

of the project

58

Table 5-2 – Summary of Key Findings from Evaluation of Incident Detection

Capabilities the 360 Degree Radar Information System

Measures Results Notes

Advance Notification Time for

Incidents 10.5 minutes

Represents the average amount of time

that MnDOT RTMC operators would

have had on a total of 11 events

monitored during the project.

Additional Incidents Confirmed

by 360 Degree Radar System 11 events

Represents the number of incidents

detected by the 360 Degree Radar

System that were not detected by

RTMC operators

Incident False Alarm Rates 0.71 / 0.28

Represents the incident false alarm

rates monitored for one week at the

beginning and end of the project.

Table 5-3 – Summary of Operational Needs Addressed by the

360 Degree Radar Information System

General Problems Operational Needs Needs Addressed by System

1. Limited zone

coverage

1. Deploy a traffic detection

system that can monitor multiple

lanes of traffic from one single

location.

System detected vehicles and

incidents in three lanes of WB

and three lanes of EB traffic

from single location

2. Understanding of

latency in incident

detection

2. Gather incident detection

information quickly and accurately

in the project area and provide

information to operational staff for

emergency response

Incidents were detected in a

quick manner by the system.

Alarms were not provided to

RTMC operators, given high

number of alarms created for

stopped traffic.

3. Inability to measure

driver behavior;

individual vehicle

movements

3. Gather traffic congestion

information quickly and accurately

in the project area

Slow moving traffic alarms

were detected, but not found to

useful given traffic congestion

in the area.

4. Inability to perform

multiple real-time data

collection tasks over a

wide area

4. Understand the amount of lane-

weaving occurring within the

project area

Lane weaving was not

measured during the course of

the project.

5. Difficult to gather

performance metrics in a

cost effective manner

5. Gather performance metrics that

can be reported regarding traffic

congestion and incidents in the

project area in a readable and

usable format

Metrics of congestion and

incidents not prepared for use

in performance measure

reporting. Total incidents and

vehicle count summaries were

prepared for the project area.

59

5.2. Considerations for Future Installation

5.2.1. IT Environment Considerations

The 360 Degree Radar unit and associated software provided by Navtech have the

ability to work within existing computer / IT environments through use of existing

server hardware and communications infrastructure in a client-server type of IT

architecture. Users of the software can be established as clients and provided read-

only access to the software showing a live view of the corridor.

While the software can be viewed remotely, Navtech has not provided to training to

TMC operators on the various functions that area available with the Witness software

package. This is primarily because of the complexity involved with the software’s

functions, the firewall and network requirements, and the limited time that TMC

operators have to learn new software packages, in addition to their primary function

of responding to reported 911 incidents and monitoring traffic on a daily basis. Past

clients have simply desired the ability to have alarms generated by the software,

which operators can then act upon by clicking on the alarm, which can bring up a live

camera view of the event detected by the radar.

In the event that assistance is needed in modifying alarms or other features of the

system, Navtech has responded with remote or on-site assistance as necessary. It is

envisioned that a similar arrangement could be established with MnDOT through

agreement either with Navtech directly or through RhiZone, Inc. acting as the local

distributor of the radar unit.

5.2.2. Relocation to Signalized Intersections / DSRC Integration

Through modification to the FCC license previously approved for the radar, the 360

Degree Radar unit could be re-located from its current installation to monitor

signalized intersections given its noted accuracy in generating alerts upon detecting

specific events and in counting vehicles along the I-94 corridor. The central server

that hosts the software package would also need to be installed in the signal cabinet to

provide a connection with the traffic signal controller.

At signalized intersections, the radar and central software could replace in-pavement

loops or other technology currently installed to count vehicles. The radar unit could

be installed at an intersection for a temporary basis to provide counts where no loop

detectors exist to gather a representative sample of vehicle counts for use by traffic

engineers to create or adjust traffic signal timings as needed.

The radar unit and central software could also be used to improve the safety and

efficiency of signal operations under the following types of scenarios:

1. The detection of vehicles at certain distances from the intersection by the radar

and central software could be used to extend signal cycles on main or side street

60

approaches. This could improve the operational efficiency of the intersection

during peak or off-peak travel periods.

2. Stopped vehicles detected by the radar and software on the cross streets of an

intersection could instruct the signal controller to serve those vehicle waiting on

the side street, then return to serving the main approaches of the intersection. This

function could replace the need for in-pavement loops on side street approaches.

3. Vehicles detected by the radar and central software as traveling at high speeds

(greater than “X” mph) as they approach a red light could instruct the signal

controller to hold an all-red phase until the vehicle has either stopped or cleared

the intersection. This could improve the safety of the intersection by reducing

red-light running crashes at the intersection.

4. Pedestrians detected by the radar and central software as crossing the intersection

at a slower pace than allowed to with the given crosswalk time could instruct the

signal controller to hold the “Don’t Walk” until the pedestrian has cleared the

crosswalk. This could improve the safety of the intersection by reducing

pedestrian-vehicle incidents at the intersection.

To enable these types of scenarios, the Navtech server that hosts the software package

would be installed within the signal cabinet and a connection with the signal

controller could be established to allow for the controller to take action under these

scenarios. There are a few approaches that could be taken to enable this connection:

1. The signal controller manufacturer could be the primary lead (by working with

Navtech) to program the signal controller to understand how the inputs from the

radar’s central software can be processed by the signal controller to perform

certain functions.

2. Navtech could be the primary lead (by working with controller manufacturer) on

the development of a program written under an Application Programming

Interface (API), which would allow for the controller to receive inputs from the

radar’s central software and then perform certain functions.

Under either approach, it would be ideal for the traffic signal controller to be in

compliance with the most recent Advanced Traffic Controller (ATC) standard that

guides the development of ATC-model controllers. ATC model controllers operate

under a Linux-based operating system that allows for an API to host the type of

programming that would be required of either the signal controller manufacturer or

Navtech to allow for the radar to instruct the signal controller on how to react to the

types of scenarios noted above.

One consideration that Navtech and signal controller manufacturers could make in the

programming these scenarios is the future implementation of DSRC (Dedicated Short

Range Communications) protocols and the Connected Vehicles program developed

by the Federal Highway Administration. The radar and central software could

supplement this future vehicle-to-intersection (V-2-I) communication by detecting

other vehicles that are not communicating with the controller via DSRC protocols.

The messages sent from the radar central software to the signal controller regarding

61

detection of a moving vehicle can be similar to Basic Safety Messages (BSMs) sent

from Connected Vehicles to the signal controller.

Further investigation would need to be performed by Navtech and the signal

controller manufacturer prior to developing the programs that could enable successful

communication between the central software and the signal controller.

5.2.3. Relocation to Un-Signalized Intersections for ICWS Integration

At un-signalized intersections, the radar and central software could monitor for

vehicles approaching an un-controlled intersection at high speeds where vehicles on

minor side streets are waiting to cross the intersection. MnDOT currently utilizes an

Intersection Conflict Warning System (ICWS) in rural locations to alert vehicles that

may not be able to see vehicles on the main road approaching at high speeds, either

because of the roadway geometrics or a poor line-of-sight with the main roadway.

Similar to signalized intersections, the rural ICWS system utilizes a traffic signal

controller to activate flashing beacons as a warning to drivers when vehicles are

detected at intersection approaches. Since the radar and central software can detect

approaching vehicles from up to 500 meters (about 1,500 feet) away from the

intersection, the radar could successfully be utilized for detection of approaching

vehicles at distances of 1,000 feet from the un-signalized intersection. The central

server that hosts the software package would need to be installed in the nearby signal

cabinet to provide a connection with the existing traffic signal controller.

The radar unit and central software could also be used to improve the safety and

efficiency of ICWS operations under the following types of scenarios:

1. The detection of vehicles at 1,000 feet on the main approach from the intersection

by the radar and central software could be used to activate flashing beacons that

issue warnings to motorists on the cross street. This would operate similar to how

existing in-pavement loops activate these flashing beacons.

2. The detection of vehicles entering the intersection from the side street by the radar

and central software could be used to activate flashing beacons that issue

warnings to motorists on the main approach. This would operate similar to how

existing in-pavement loops activate these flashing beacons.

3. Pedestrians detected by the radar and central software as crossing the un-

signalized intersection could also trigger flashing beacons as a warning to

motorists on the main approach. This could improve the safety of the intersection

by reducing pedestrian-vehicle incidents at the intersection, and provides

additional functionality to the ICWS that is not available with the in-pavement

loops currently installed.

4. The radar and central software could also be used to monitor how vehicles are

currently reacting to the warnings presented to them on roadside signage that

illuminates as part of the ICWS installation. Recordings of vehicle speeds on

main approaches when vehicles enter an intersection could be established to

understand how vehicle speeds change with the flashing beacons activated at

62

various times of the day. This type of analysis could lead to potential adjustments

in how the ICWS is installed to improve the overall safety of the ICWS at the

intersection.

To enable these types of scenarios, the Navtech server that hosts the software package

would be installed within the signal cabinet and a connection with the signal

controller could be established to allow for the controller to take action under these

scenarios. There are a few approaches that could be taken to enable this connection:

1. The signal controller manufacturer could be the primary lead (by working with

Navtech) to program the signal controller to understand how the inputs from the

radar’s central software can be processed by the signal controller to activate the

flashing beacons.

2. Navtech could be the primary lead (by working with controller manufacturer) on

the development of a program written under an Application Programming

Interface (API), which would allow for the controller to receive inputs from the

radar’s central software and then perform certain functions.

Current installations of the ICWS utilize non-ATC model signal controllers since the

advanced functionalities on an ATC are not necessary for an un-signalized

intersection. It is likely that the signal controller manufacturer could program the

signal controller to respond to inputs from the central software by activating the

appropriate flashing beacons. Further investigation would likely be needed by

Navtech and the signal controller manufacturer prior to implementing this type of

application.

The radar unit could also be installed on a temporary basis to gather and record data

for a one week period of time, prior to being re-located to another ICWS installation

in other areas of the state as well. This work would involve installing the radar unit

and central software onto a mobile trailer where it could be moved to various

locations to collect data on incidents in specific areas, or perform incident detection

for a limited period of time as needed.

5.2.4. Use of Alarms with ATM Signage on Roadway

The alarms generated from the 360 Degree Radar Information System could be used

in multiple other manners by MnDOT or traffic operations staff. One example of the

use of slow-moving vehicle alarms could be to trigger messages to appear on

Dynamic Message Signs (DMS) that are installed as part of the Active Traffic

Management (ATM) systems installed on I-35W and I-94.

Currently, speeds displayed on the DMS that alert traffic to a recommended speed

based on the speed of traffic detected approximately one half-mile upstream from the

DMS. A slow-moving vehicle alarm from the 360 Degree Radar unit could trigger a

slower recommended speed, such as 15 or 20 MPH, that would reduce the potential

for crashes from traffic attempting to stop suddenly at unsafe speeds.

63

Other messages could be displayed on these DMS reading “Slow / Traffic / Ahead”

utilizing three lines of text on the DMS that are used as part of the ATM system. This

type of message is currently displayed on those DMS as heavy congestion is detected,

but these messages could be automated through use of the 360 Degree Radar unit.

This could require the development of an XML (Extensible Markup Language) feed

between the 360 Degree Radar central software that detects slow-moving and stopped

traffic alarms and the MnDOT IRIS system utilized to activate messages on DMS.

The XML feed would enable the alarms generated by the 360 Degree Radar central

software to allow for the activation of the messages on the DMS that are monitored

and controlled by MnDOT. Navtech Radar has developed similar types of XML

feeds for alarms for other clients and could do so for MnDOT after further discussion

of how the alarms could activate DMS messaging.

Further testing of the system and its ability to accurately report vehicle speeds during

cold weather periods would likely be required prior to implementing an XML feed as

described above. As noted earlier in this report, due to changes that were made to

system parameters within the software on Jan. 15th, the accuracy of speed

measurements in the winter period of time negatively impacted the ability of the

system to accurately measure speeds in this period of time.

5.3. Recommendations for Future Installation

This section contains next steps for MnDOT to consider taking with the 360 Degree Radar

Information System. Further discussions with the project team can take place in subsequent

meetings to determine how these steps can be taken to help improve the overall safety of the

transportation system.

5.3.1. Integrate into MnDOT RTMC Operations

This step could be taken to gather feedback from RTMC operators about the overall

operation and reliability of the system to assist them in detecting incidents in the

project area of I-94 between 3rd Avenue and the MN-65 overpasses.

It should be noted that MnDOT did allow for a limited trial period of the system

within the MnDOT RTMC for two weeks in late May and early June. A MnDOT

supervisor monitoring the system noted that many stopped vehicle alarms were

generated during periods of heavy traffic congestion on the corridor. While the

alarms were accurate for stopped traffic, the value of alarms for incident detection

was very limited given the frequency of traffic congestion in the area. As a lesson

learned for this traffic environment, Navtech proposed a modification to the software

that could limit the number of alarms generated in heavy traffic congestion.

RhiZone has also recommended an adjustment to the tilt of the radar to further

improve the detection of traffic incidents on the corridor. Adjustments to the central

software would then take approximately five days to account for the change in the

radar tilt toward the roadway. An XML feed could then be established on the project

server to send alerts directly to RTMC operators.

64

Navtech Radar is currently working on a software upgrade to better detect traffic

incidents when there is a high amount of congestion on the corridor. This upgrade

could be provided prior to alerting RTMC operators about incidents in the project

area.

5.3.2. Integrate System into a Signalized Intersection

A second step that could be taken in future years would be to move the 360 Degree

Radar Information System to a signalized intersection where it could perform the

simultaneous tasks of vehicle count / speed measurements, as well as incident

detection. This project could involve Navtech Radar working closely with a signal

controller manufacturer to determine how inputs from the radar in the field could feed

the signal controller (or the controller’s central software package) to actuate signal

operations in the field. Purpose of this project would be to see how well the radar

could replace existing in-pavement loops currently used for vehicle detection.

5.3.3. Integrate System onto a Mobile Trailer

A third step that could be taken in future years would be to move the 360 Degree

Radar Information System onto a mobile trailer where it could be moved to various

locations to collect data on incidents in specific areas, or perform incident detection

for a limited period of time as needed.

65

Appendix A – System Component Data Sheets

Pages 1-2 – 360 Degree Radar Specification

Pages 3-5 – Proposed Mounting Bracket and Detail for 360 Degree Radar

Pages 6-9 – Radar Power Supply Unit

Pages 10-13 -- Proposed CCTV Camera Brackets

Pages 14-17 – Proposed Axis 1614-E Camera Datasheet

Pages 18-19 – Proposed CCTV Camera Brackets

Pages 20-22 – Proposed Software Application for 360 Degree Radar System

Page 23 – Network Video Recorder Datasheet