copyright by kristie dorothy chin 2015
TRANSCRIPT
The Thesis Committee for Kristie Dorothy Chin
Certifies that this is the approved version of the following thesis:
Communicating Value to Stakeholders:
A Customer-Oriented KPI System for State DOTs
APPROVED BY
SUPERVISING COMMIDOTTEE:
Charles Michael Walton
Shelley Row
Supervisor:
Communicating Value to Stakeholders:
A Customer-Oriented KPI System for State DOTs
by
Kristie Dorothy Chin, AB HX Art & Arch; ScB & AB Eng; MArch
Thesis
Presented to the Faculty of the Graduate School of
The University of Texas at Austin
in Partial Fulfillment
of the Requirements
for the Degree of
Master of Science in Engineering
The University of Texas at Austin
December 2015
Dedication
To those who have made a world of difference in my life,
you inspire me to make a difference in the world.
v
Acknowledgements
For his continued support of my M.S./Ph.D. studies and related research for the
Texas Technology Task Force, I would like to express my sincere gratitude to my advisor
Dr. C. Michael Walton. His deep understanding of the relationships between policy and
transportation has underscored the importance of interdisciplinary work. I would also like
to recognize Shelley Row for guiding me throughout this process. Her expertise was key
to ensuring the translation of technical topics resonated with the respective stakeholders.
The insights she generously shared are deeply appreciated and have encouraged me to
pursue transportation in new contexts.
Furthermore, I would like to personally thank the following people: J.D. Stanley,
Commissioner Victor Vandergriff, Darran Anderson, Stacey Strittmatter, Kent Marquardt,
Yvette Flores, Amy Hawk, Kayleigh Axtell, Trent Thomas, Cory Henrickson, Bob
Kaufman, Jess Blackburn, Dana Glover, Dr. Randy Machemehl, and Andrea Gold for
sharing their time and perspectives. Special thanks to Vicki Simpson, who has diligently
provided assistance. Through the knowledge and expertise of this great team, this research
on developing a customer-oriented key performance indicator framework will serve as a
valuable reference document for the Texas Department of Transportation.
vi
Abstract
Communicating Value to Stakeholders:
A Customer-Oriented KPI System for State DOTs
Kristie Dorothy Chin, M.S.E.
The University of Texas at Austin, 2015
Supervisor: Charles Michael Walton
With the erosion of public trust, increasing shortfalls in funding, and continuous
forces of disruption, State Departments of Transportation (DOTs) can no longer afford to
rely upon traditional models of performance management that address all stakeholder
concerns with a single approach. A customer-oriented key performance indicator (KPI)
system offers a new paradigm and is based upon stakeholder segmentation and mapping.
The value of this system rests in its ability to communicate the right information to the
right decision-makers by aligning the customer’s needs with the organization’s mission
and other stakeholder priorities. This thesis synthesizes best practices used within and
outside the transportation industry in order to propose a recommended customer-oriented
KPI system for the Texas Department of Transportation (TxDOT). Feedback is collected
from each stakeholder group in an effort to validate the methodology. Results indicate the
customer-oriented KPI system is preferred over the current performance report for its
clarity, accessibility, and relevance.
vii
Table of Contents
List of Tables ...........................................................................................................x
List of Figures ........................................................................................................ xi
Chapter 1: Introduction ............................................................................................1
1.1 Problem Statement ....................................................................................2
1.2 Value Proposition......................................................................................3
1.3 Thesis Summary ........................................................................................3
Chapter 2: Background & Literature Review ..........................................................4
2.1 Motivation to Use Key Performance Indicators (KPIs) ............................4
2.1.1 Operational: Executing More Efficiently and Effectively ............5
2.1.2 Tactical: Improving Short-Range Decision Making .....................6
2.1.3 Strategic: Defining Agency Direction ..........................................8
2.1.4 Compliant: Demonstrating Transparency, Accountability, and Credibility ...................................................................................10
2.1.5 Influential: Encouraging Changes in Perception and Behavior ..12
2.2 Drivers of Change that Impact KPIs .......................................................14
2.2.1 Political .......................................................................................15
2.2.2 Institutional .................................................................................18
2.2.3 Cultural .......................................................................................19
2.2.4 Technological ..............................................................................20
2.2.5 Financial ......................................................................................21
2.3 Key Stakeholder Groups .........................................................................22
2.3.1 State DOT Executives .................................................................24
2.3.2 State Legislators and Governors .................................................24
2.3.3 Businesses ...................................................................................25
2.3.4 General Public .............................................................................26
2.4 First Principles of Performance Management .........................................28
2.4.1 What is “key”? ............................................................................28
2.4.2 Selecting Key Performance Indicators ........................................29
viii
2.4.2.1 Strategic ..........................................................................29
2.4.2.2 Measurable ......................................................................30
2.4.2.3 Accessible .......................................................................30
2.4.2.4 Relevant ..........................................................................31
2.4.2.5 Tangible ..........................................................................31
2.4.3 Communicating Key Performance Measures .............................32
2.4.3.1 Dashboards ......................................................................33
2.4.3.2 Reports and Scorecards ...................................................36
2.4.3.3 Traditional Media............................................................37
2.4.3.4 Websites ..........................................................................37
2.4.3.5 Open Data Portals ...........................................................39
2.4.3.6 Social Media ...................................................................39
2.5 The Evolution of Performance Management ..........................................40
2.5.1 First Generation: Traditional Infrastructure & Organizational Measurement ...............................................................................40
2.5.2 Second Generation: Hierarchy of Measurement .........................41
2.5.3 Third Generation: Catalyst-Driven Adaptation ..........................42
2.5.4 Fourth Generation: Customer-Oriented Paradigm ......................43
Chapter 3: Customer-Oriented KPI Development Approach ................................45
3.1 Performance Management Context in the State of Texas .......................46
3.1.1 Political Pressure .........................................................................47
3.1.2 Institutional Changes and Initiatives ...........................................49
3.1.3 Cultural Transformations and State of Public Trust ...................52
3.1.4 Technological Advancements .....................................................54
3.1.5 Financial Limitations ..................................................................56
3.2 Customer-Oriented KPI Framework Proposal for TxDOT ....................58
3.2.1 Stakeholder Profile: General Public ............................................59
3.2.1.1 Ask ..................................................................................59
3.2.1.2 Urban Customer Experience ...........................................59
3.2.1.3 Value Proposition............................................................60
ix
3.2.1.4 Proposed KPIs .................................................................61
3.2.2 Stakeholder Profile: State Legislator ..........................................62
3.2.2.1 Ask ..................................................................................62
3.2.2.2 Customer Experience ......................................................62
3.2.2.3 Value Proposition............................................................65
3.2.2.4 Proposed KPIs .................................................................66
3.2.3 Stakeholder Profile: Small Business ...........................................67
3.2.3.1 Ask ..................................................................................67
3.2.3.2 Customer Experience ......................................................67
3.2.2.3 Value Proposition............................................................68
3.2.3.4 Proposed KPIs .................................................................69
3.3 Analysis...................................................................................................70
3.3.1 TxDOT Leadership .....................................................................70
3.3.2 Public ..........................................................................................70
3.3.3 State Legislature ..........................................................................71
3.3.4 Businesses ...................................................................................71
Chapter 4: Recommendations ................................................................................72
4.1 Less is More ............................................................................................72
4.2 Design from the Outside In .....................................................................72
4.3 Power of Personalization ........................................................................73
4.4 Empower Stakeholders ...........................................................................73
4.5 Evolve or Risk Being Left Behind ..........................................................73
Chapter 5: Conclusion............................................................................................74
Appendix A: State DOT Scorecard Examples .......................................................75
Appendix B: TxDOT Stakeholder Profile Examples .............................................86
Bibliography ..........................................................................................................90
x
List of Tables
Table 1: GDOT’s West Point I-85 Interchange Quality Monitoring ......................5
Table 2: State DOT Primary Strategic Focus Areas ...............................................8
Table 3: Transportation Data Available through Maryland StateStat ...................11
Table 4: MAP-21 Performance Measures .............................................................16
Table 5: 2014 NDOT Performance Measures .......................................................41
Table 6: TxDOT KPIs reported to the Legislative Budget Board.........................48
Table 7: TxDOT Goals under Consideration by the Core Strategy Team ............51
Table 8: Proposed Customer-Oriented KPIs for the Texas Public ........................61
Table 9: Texas Legislature Composition ..............................................................62
Table 10: Proposed Customer-Oriented KPIs for the Texas State Legislature .....66
Table 11: Proposed Customer-Oriented KPIs for Texas Small Businesses ..........69
xi
List of Figures
Figure 1: Motivational Types of KPIs ....................................................................4
Figure 2: NCDOT Pre-Construction Project Status Dashboard KPIs ....................7
Figure 3: VDOT Trends for Transit Performance Indicators .................................9
Figure 4: ODOT KPI to Support Stewardship Goal .............................................10
Figure 5: Drivers of Change that Impact KPIs .....................................................14
Figure 6: WSDOT Performance Management Framework ..................................23
Figure 7: VDOT Performance Dashboard ............................................................33
Figure 8: UDOT Zero Fatalities Dashboard .........................................................34
Figure 9: Open Michigan Infrastructure Dashboard Sample ................................34
Figure 10: DOT&PF Dashboard Sample ..............................................................35
Figure 11: DelDOT Dashboard Sample ................................................................35
Figure 12: TxDOT Performance Results Summary Sample ................................38
Figure 13: Drivers of Change that Impact KPIs in Texas .....................................46
Figure 14: FDOT 2014 Performance Highlights ..................................................75
Figure 15A: CalTrans 2015 Q2 Mile Markers ......................................................76
Figure 15B: CalTrans 2015 Q2 Mile Markers ......................................................77
Figure 16A: MDOT 2015 MiScorecard Performance Summary ..........................78
Figure 16B: MDOT 2015 MiScorecard Performance Summary ..........................79
Figure 17A: MnDOT 2014 Transportation Results Scorecard .............................80
Figure 17B: MnDOT 2014 Transportation Results Scorecard .............................81
Figure 18A: ODOT 2015 Key Performance Measures ........................................82
Figure 18B: ODOT 2015 Key Performance Measures .........................................83
Figure 19A: VDOT 2015 Q4 Quarterly Report ....................................................84
xii
Figure 19B: VDOT 2015 Q4 Quarterly Report ....................................................85
Figure 20: Proposed TxDOT Stakeholder Profile for General Public ..................86
Figure 21: Proposed TxDOT Stakeholder Profile for State Legislature ...............87
Figure 22: Proposed TxDOT Stakeholder Profile for Small Business .................88
Figure 23: Proposed TxDOT KPI Report for Individual Commuter ....................89
1
Chapter 1: Introduction
With the erosion of public trust, increasing shortfalls in funding, and continuous
forces of disruption, State DOTs can no longer afford to rely upon traditional models of
performance management that address all stakeholder concerns with a single approach.
Conventional key performance indicator (KPI) systems are often complex, incoherent,
irrelevant, abstract, and disconnected from the organization’s strategic plan. The more
successful systems include an accompanying communications strategy, however it is rarely
tailored to the needs of specific stakeholder groups nor formatted to effectively influence
the respective stakeholder’s decision-making process. Furthermore, an infrastructure-
biased culture may be ingrained throughout the organization and in the public mindset to a
point where the agency struggles to overcome the inertia needed to advance multimodal
initiatives. Transportation agencies seeking to evolve their performance management
systems will need to overcome these challenges in order to prepare for the disruptions that
are taking place in the transportation sector.
Advances in technology enable transportation agencies to understand the customer
and the transportation experience at unprecedented levels of detail. By focusing on the
customer’s needs, State DOTs can develop a KPI system that aligns with the organization’s
mission and other stakeholder priorities. Customized communication strategies can build
awareness within each stakeholder group and constructively influence the decision-making
process. Potential benefits of a customer-oriented KPI system include improved
stakeholder engagement, budget justification, and the opportunity to incentivize changes
in travel behavior. This thesis contributes to the advancement of performance management
systems by demonstrating how customer segmentation and mapping may be applied as
effective strategies for communicating value to stakeholders.
2
1.1 PROBLEM STATEMENT
The conventional approach to performance management is ineffective – internal
and external priorities are often misaligned, measures are too complex, information is not
readily available, the data has limited meaning, and results are reported at a scale that is
considered too abstract. State DOTs have a tendency to concentrate on cascading KPIs
from strategic levels to internal operations of the organization; however, it is ultimately the
external stakeholders that provide trust, support, and funding. The states where funding
initiatives are defeated – such as the 2015 legislative plans in South Carolina, Tennessee,
and New Mexico – suffer from lack of trust, transparency, and accountability
(Transportation for America, 2015). Traditionally, performance management systems have
been designed from the inside out. This approach, however, has led to disconnects between
the agency and its stakeholders due to a lack of perspective and the inability to translate
technical needs into valuable, actionable decisions. As a result, a stakeholder is often
overwhelmed with an excess of irrelevant information – often paralyzing him from being
able to make the right decision within the right time frame (Turnbull, 2014).
3
1.2 VALUE PROPOSITION
Customer-oriented KPIs, on the other hand, empower the stakeholder to not only
make the right decision within the right time frame, but also to become an ambassador for
the agency. When properly designed, KPIs can be used to construct powerful narratives
that can influence stakeholders towards positive action. If steady progress is made towards
the stated goals, agencies will improve their public image and political position. The
integrity of the State DOT then becomes the basis for voters and legislators to approve
funding for transportation. The selection, organization, and presentation of KPIs can make
the difference between whether a State DOT is perceived as successful or incompetent. By
framing KPIs within the unique context of each stakeholder group, State DOTs can build
trust, garner support, and increase funding.
1.3 THESIS SUMMARY
This thesis synthesizes best practices from within and outside the transportation
industry based upon a series of key person interviews, analysis of existing State DOT
frameworks, and an assessment of stakeholder engagement tools. Chapter 2 describes the
principles of selecting and communicating KPIs as well as existing and proposed
methodologies. Chapter 3 describes the performance management context in the state of
Texas, generates three sample customer-oriented KPI frameworks for the Texas
Department of Transportation (TxDOT), and analyzes the feedback from each stakeholder
group to compare the proposed framework with the existing reporting techniques. Chapter
4 synthesizes the stakeholder feedback into a set of recommended best practices for State
DOTs seeking to implement customer-oriented KPI frameworks. Chapter 5 concludes by
summarizing the challenges and opportunities of performance management at the State
DOT level and suggests areas of future research.
4
Chapter 2: Background & Literature Review
2.1 MOTIVATION TO USE KEY PERFORMANCE INDICATORS (KPIS)
Motivations for applying key performance indicators (KPI) may be generally
categorized into those in which the motivation for an agency is internal or external
(Cambridge Systematics, 2000). In the first category, performance measures are used
primarily for decision making within an agency, including near-term, project-level
programming; system-wide prioritization; and long-range, organizational strategic
planning. Externally motivated applications of KPIs involve evaluation of a program or
agency by an outside entity as well as communication efforts designed to influence
stakeholders. In each case, there is a different audience and context for which the KPIs are
used and interpreted. A hierarchy of measures ensures that the objectives are detailed and
related to the responsibilities of the personnel at that level of the organization. It is
important to begin by understanding the five motivational factors so that State DOTs may
customize the KPI selection, language, and presentation according to each stakeholder’s
needs.
Figure 1: Motivational Types of KPIs
Operational
Tactical
Strategic
Compliant
Influential
5
2.1.1 Operational: Executing More Efficiently and Effectively
Operational KPIs include the most detail and focus on day-to-day improvement
opportunities in the field. Operational KPIs assist project managers and division heads in
monitoring immediate project-level and system-wide progress. Table 1 shows an example
of operational KPIs used to monitor the quality of Georgia DOT’s (GDOT) Interstate 85
Interchange project. The interface between the two levels must be aligned so that the
decisions at the site-specific level are consistent with those at the system-wide level. For
example, a decision to repair the pavement in an individual district should be consistent
with a system-wide priority program of preserving the state’s existing transportation
system. Factors that impact the selection of operational KPIs include standards and
specifications, budget limitations, and environmental constraints as well as agency
policies, financing, and human resource decisions.
Table 1: GDOT’s West Point I-85 Interchange Quality Monitoring
Performance Objective KPIs Targets 1. Promote Efficient System Management and Operation
Pavement Sound
Intensity Testing Results
HfL goal of 96.0 dB(A) or
less for all paved surfaces
2. Emphasize the Preservation of the Existing Transportation System
Pavement Smoothness
HfL goal of 43.8 in/mi (IRI) or less for all paved surfaces
Smoothness index of 900 or less for new construction
Smoothness index of 1,025 or less for corrective work
Profile index of 30 in/mi or less for bridge approaches
Hot-Mix Asphalt (HMA) Quality
HMA concrete thickness measurements
HMA concrete spread rates
IRI: International Roughness Index Source: Molenaar & Navarro, 2011
6
2.1.2 Tactical: Improving Short-Range Decision Making
Tactical KPIs focus on short-range planning and are designed to improve the
decision-making process of middle management. Short range is generally defined as a
period of time extending about one year or less in the future. Managers use tactical planning
to outline the actions various parts of the agency must do for the organization to be
successful. For example, the Office of Investment Management at Minnesota DOT
(MnDOT) develops plans and performance reports that reflect a performance-based
approach to planning and investment management (FHWA, 2011). This includes the
annual update of the four-year Statewide Transportation Improvement Program (STIP) and
capital program development processes. Transportation system and agency performance
reports are regularly reviewed by MnDOT management to actively guide decision-making.
Agencies are also integrating discussion of performance trends and targets with
information about funding resources to make more insightful decisions. An effective
example is the North Carolina Department of Transportation (NCDOT) that developed a
Pre-Construction Project Status Dashboard, a web-based application for senior executives
of NCDOT to manage the schedule and funding of Transportation Improvement Program
(TIP) Projects for Highway and Construction Maintenance (NASCIO, 2007). The
Executive Reporting Dashboard is designed to visualize the overall health of projects in
the Pre-Construction phase using KPIs for Schedule Status and Funding Status. The KPIs
are presented in a tachometer gauge and allow management to view the percentage of
projects that are on schedule and on budget. The application also allows the user to drill
down and view performance measures and supporting details. The tool has been successful
for effectively utilizing available transportation funds to complete transportation
improvement projects on schedule.
8
2.1.3 Strategic: Defining Agency Direction
Strategic KPIs enable agency leadership to set priorities and define direction. They
serve as a critical pivot point by connecting the internal agency needs with those of its
external stakeholders. Executive leaders also use performance management to focus their
staff and hold them accountable. Generally three to seven strategic focus areas are selected
from the themes in Table 2:
Table 2: State DOT Primary Strategic Focus Areas
Strategic Focus Area Sample DOTs 1. Safety & Security
Caltrans, FDOT, GDOT, MDOT, MoDOT, ODOT, TxDOT, UDOT, VDOT, WSDOT
2. Economic Vitality or Competitiveness
Caltrans, FDOT, MoDOT, ODOT, WSDOT
3. Fiscal Stewardship
Caltrans, MDOT, MoDOT, ODOT, VDOT, WSDOT
4. Environment & Sustainability
Caltrans, FDOT, ODOT, WSDOT
5. Livability, Health, & Quality of Life
Caltrans, FDOT, ODOT
6. Preservation & Maintenance
FDOT, GDOT, MDOT, MoDOT, ODOT, TxDOT, UDOT, VDOT, WSDOT
7. Planning, Construction, & Project Delivery
GDOT, MoDOT, TxDOT
8. Mobility, Operations, & System Performance
Caltrans, FDOT, MDOT, ODOT, TxDOT, UDOT, VDOT, WSDOT
9. Organizational Excellence & Management
Caltrans, VDOT
10. Customer Service MDOT, MoDOT. VDOT
9
The selection of strategic KPIs often includes input from internal and external
stakeholders. For example, Virginia Department of Rail and Public Transportation
(Virginia DRPT) uses performance indicators as a way to translate priorities set by the
governor to local agencies that operate public transportation in the state (Chisholm-Smith,
2011). Public transportation related goals are developed through a collaborative process
with Virginia DRPT’s stakeholders and performance measures are used to keep goals in
focus over time through regular reporting. This ensures that the agency’s priorities are
easily understood by the policymakers and the KPIs are capable of signaling which areas
are in need of investment.
Figure 3: VDOT Trends for Transit Performance Indicators (Source: Virginia DRPT, 2007)
10
2.1.4 Compliant: Demonstrating Transparency, Accountability, and Credibility
Compliant KPIs satisfy requirements imposed by external entities, such as state and
federal legislatures, federal agencies, and professional organizations. Legislative
requirements often prescribe categories of measures or specific measures that must be used,
as well as reporting methods and frequency. Such external evaluations are often linked to
peer comparisons, agency audits, and budget allocations. For example, Oregon DOT is
legislatively mandated to produce an Annual Performance Progress Report that features
“Key Performance Measures” specified by the legislature (Chisholm-Smith, 2011). Figure
3 shows an example of a KPI that supports ODOT’s “Stewardship” goal to “Maximize
Value from Transportation Investments.” “On-Time” and “On-Budget” KPIs are most
effective for demonstrating fiscal responsibility. In Minnesota, early efforts using public
transportation performance measures focused on demonstrating progress toward a
legislatively mandated performance target of public transportation service for all counties.
MnDOT cites increased trust as one of the primary benefits of its KPI program, stating that
its efforts help elected officials view MnDOT as a trusted partner and work together on
challenging issues (MnDOT, 2014). Although these processes are often compulsory, State
DOTs have the ability to frame compliant KPIs within a context that demonstrates
transparency and accountability, ultimately engendering trust.
Figure 4: ODOT KPI to Support Stewardship Goal
(Source: Oregon, 2014)
11
Some State DOTs have improved their performance management systems in
response to a dramatic loss in public credibility. For example, Washington State DOT
(WSDOT) first developed systematic performance reporting as part of a larger effort in the
late 1990s to address a credibility gap concerning project delivery and to repair an erosion
of trust between the department and the state legislature (Chisholm-Smith, 2011). Also, in
Maryland the governor requires all state agencies to participate in “Managing for Results,”
a strategic planning, performance measurement, and budgeting process that emphasizes
use of resources to achieve measurable results, accountability, efficiency, and continuous
improvement in State government programs (Maryland Department of Budget and
Management, 1996). Information from the Maryland Department of Transportation
(MDOT) feeds into the “StateStat” open data portal in order to make performance data
publicly available. The transparency enhancement has been well received and assists in
managing the state’s performance related to 16 wide-ranging, high-priority goals.
Table 3: Transportation Data Available through Maryland StateStat
Data Set Responsible Agency Popularity General Cargo by Month
Maryland Port Administration
7,288 views
Weekday Ridership by Month Maryland Transit Administration 3,255 views
Parking Garage Space Availability
Montgomery County 191,362 views
Bus Routes
Howard County 92 views
On Street Bike Facilities City of Baltimore 497 views Intercounty Connector (ICC) Vehicle Volume Data
Maryland Transportation Authority 9,260 views
Annual Vehicle Miles of Travel Maryland Transportation Authority 550 views
Source: Maryland Department of Information Technology, 2015
12
2.1.5 Influential: Encouraging Changes in Perception and Behavior
Influential KPIs are proactive in nature and encourage changes in the perceptions
and behaviors of external stakeholders. More than building awareness, influential KPIs
actively educate and engage with the stakeholder. For example, Google Now is capable of
learning about a user and his or her habits in order to provide relevant notifications (Google
Now, 2015). Based upon previous movements and searches, Google Now can inform the
user of his or her “Time to Work,” including real-time traffic and weather information.
This information provides value by empowering the user to make an informed decision
regarding his or her departure time, route selection, mode choice, or even telecommute
options.
When coupled with incentives, influential KPIS are capable of driving even greater
changes. Metropia is an app system that incentivizes commuters to choose departure times,
routes, and modes that help reduce and eliminate traffic congestion (Metropia, 2015).
Using an active demand management and data analytics platform, Metropia effectively
influences behavior through a social rewards-based ecosystem. Each user can track his or
her CO2 emissions, travel time saved, and driving score. Drivers earn points each time they
make smarter commute decisions based on Metropia’s suggestions, which can then be
redeemed for perks such as gift cards, discounts, and local retailer offers. During beta
testing in Austin with the Central Texas Regional Mobility Authority (CTRMA), Metropia
drivers saved 8,550 pounds of CO2, 211 hours in travel time, and planted 85 trees through
Metropia’s partnership with American Forests (Metropia, 2015). The company is also
partnering with local businesses to encourage flex scheduling among their employees.
Metropia helps employers measure the extent to which employees are meeting flex
schedule goals and alleviating peak hour congestion.
13
Influential KPIs may also be used to build the agency brand, which is important to
manage the perception of the State DOT and ultimately garner support. In particular, State
DOTs should seek to understand customer awareness, attitude, and habits. Knowing the
percentage and types of customers who recognize a brand enable the State DOT to focus
its campaign strategies. This becomes especially important when a State DOT wants to
differentiate itself from other state, regional, or local transportation agencies. By measuring
a customer’s belief or response to the agency brand, State DOTs can gauge the attitudes of
its customer base. If a State DOT wishes to be perceived as a leader, innovator, and good
steward of public funds, its KPI system should reflect these values. If customers, on the
other hand, consider the State DOT to be a follower, traditionalist, and fiscally
irresponsible, the State DOT should understand why there is a discrepancy between
perception and reality and work to bridge the gap in order to restore trust and integrity.
Analyzing customer habits can reveal patterns between a communications strategy
and responses in customer behavior. For example, the Ad Council’s “Buzzed Driving Is
Drunk Driving Campaign” that was launched in December 2005 successfully contributed
to saving lives (Ad Council, 2011). Performance metrics included: Opinion of the
Campaign, Importance of the Issue, Behaviors over the Holiday Season, and Ad
Effectiveness. In early January 2007, 17% of men age 21-34 said that in the past few weeks
they had decided to not drive after they had been drinking. Those who were aware of the
“Buzzed Driving” campaign were significantly more likely to report that they had recently
refrained from driving after drinking, stopped an impaired friend or family member from
driving, and discussed the risks of impaired driving with friends or family members. While
the public service announcements cannot claim to be the sole motivating factor behind
these behaviors, it is likely they played a large role.
14
2.2 DRIVERS OF CHANGE THAT IMPACT KPIS
State DOTs are operating in an environment of disruptive change, impacting how
agencies organize their performance measures and management processes. These include
leadership changes; legislative mandates; advances in technology; shifts in the customer
base; and funding stipulations. In order to develop a robust KPI system, it is imperative for
State DOTs to respond to, anticipate, and balance a variety of stakeholder priorities.
Figure 5: Drivers of Change that Impact KPIs
KPIs
Political
Institutional
CulturalTechnological
Financial
15
2.2.1 Political
Legislative oversight refers to the review and evaluation of selected executive
branch programs and activities, often creating political pressure to implement performance
reporting or enhance efforts in certain areas. Oversight takes place through many
mechanisms: legislative mandates, committee oversight, DOT leadership appointments,
review of administrative rules and regulations, program evaluation and sunset reviews,
reporting requirements, and requests for information (Rall et al., 2011). Most states use a
combination or all of these tools. The budget and appropriations process also includes
oversight activities, and in many cases is seen as the main forum for legislative oversight
of the DOT.
Legislative mandates for performance measurement and benchmarking can be
particularly challenging. When statutory requirements prescribe specific agency measures
or benchmarks, they may not be targeted to agency functions or management needs and are
difficult to refine or abandon once legislated (Bremmer et al., 2004). Data tracking and
reporting processes can consume valuable resources while the measures and its associated
data may lend little help to legislative bodies and the agency in managing programs more
efficiently. Agencies may not be able to influence such mandates, but legislative bodies
should be encouraged to give an agency flexibility in selecting measures or revising
measures if they do not prove useful. Agencies should also consider offering voluntary
performance reports to preempt such statutory requirements.
The Moving Ahead for Progress in the 21st Century Act (MAP-21) is the surface
transportation funding authorization. A key feature of MAP-21 is the establishment of a
performance- and outcome-based program for States to invest resources in projects that
collectively will make progress toward the achievement of the national goals (FHWA,
2013). National goal areas include: safety, infrastructure condition, congestion reduction,
16
system reliability, freight movement and economic vitality, environmental sustainability,
and reduced project delivery delays. Table 4 lists the specific performance measures that
are required. With the reauthorization of MAP-21 legislation under consideration, State
DOTs should be prepared to respond to and realign their own performance measures. By
remaining aware of national priorities, States can identify potential funding sources, more
efficiently collect data, and avoid duplicated efforts.
Table 4: MAP-21 Performance Measures
Metrics Annual Hours of Delay - NHS (millions) Annual Hours of Delay - Interstates (millions) Annual Hours of Delay - Non-Interstate NHS Reliability Index - NHS Reliability Index - Interstates Reliability Index - Non-Interstate NHS Annual Hours of Truck Delay - Interstates (millions) Truck Reliability Index Interstate Pavement in Good Condition (IRI <95) Interstate Pavement in Fair Condition (IRI 95 - 170) Interstate Pavement in Poor Condition (IRI >170) Non-Interstate NHS Pavement in Good Condition (IRI <95) Non-Interstate NHS Pavement in Fair Condition (IRI 95 - 170) Non-Interstate NHS Pavement in Poor Condition (IRI > 170) % Structurally Deficient Deck Area on NHS Bridges Based on Total NHS Deck Area % Structurally Deficient Deck Area on non-NHS Bridges Based on Total Non-NHS Deck Area Count of Bridges (Entire Inventory) with Cyclic Maintenance Needs % Bridges (Entire Inventory) by Deck Area with Cyclic Maintenance Needs Count of Bridges (Entire Inventory) with Preventative Maintenance Needs % Bridges (Entire Inventory) by Deck Area with Preventative Maintenance Needs Count of Bridges (Entire Inventory) with Rehabilitation or Replacement Needs % Bridges (Entire Inventory) by Deck Area with Rehabilitation or Replacement Needs Transit Fleet State of Good Repair (SGR) Average Condition Fatality Rate (5-year moving average) Number of Fatalities (5-year moving average) Serious Injury Rate (5-year moving average) Number of Serious Injuries (5-year moving average)
Source: FHWA, 2013
17
Performance audits are designed to promote confidence and accountability in state
government through professional and independent assessment of state agencies and
activities, consistent with legislative intent. State DOTs need to understand the
requirements of state performance audits in order to proactively address gaps in current
performance procedures. In Georgia, the Georgia State Department of Audits and Accounts
(DOAA) conducts evaluations of state funded programs and activities to answer such
questions as: 1) Is this program achieving its goals and objectives? 2) How well does this
program do what it is intended to do? and 3) Is this program complying with all applicable
laws and regulations? (Amekudzi, 2011). Other State DOTs that have had early
experiences with some type of performance audit or performance review include Texas,
Delaware, California, Washington, New Mexico, Michigan, Kentucky, and Colorado
(Bremmer et al., 2004).
Sunset reviews evaluate the functions of a state entity to assess whether it should
continue to exist, and if so, how the agency can work better to save taxpayer money. In a
true sunset process, an entity is automatically abolished unless the legislature or responsible
legislative committee chooses to affirmatively continue it. Arizona, Florida, Louisiana,
Tennessee, and Texas perform regular sunset reviews of the DOT (Rall et al., 2011). In
most cases when the sunset process is applied, it is used primarily to add another layer of
accountability rather than to seriously consider their discontinuation. State DOTs should
work closely with legislators considering new investments to instill confidence in agency
decision-processes and results. In states where funding increases or fiscal management and
credibility issues take center stage, strategic KPIs should transform to address specific
needs and compliant KPIs should assist in strengthening the State DOT’s political position.
18
2.2.2 Institutional
Leadership changes at the executive levels of a State DOT can significantly alter
the agency’s performance measurement process. Depending on the incoming leader’s
background, he or she will have different motivations for implementing KPIs. While some
leaders may prioritize compliant KPIs to build credibility, others may emphasize tactical
KPIs to improve management decision-making. Although this may lead to fundamental
changes in performance measurement, it is important that the agency’s KPIs match the
sensibilities and experiences of the leadership. State DOTs that undergo frequent
leadership turnover should anticipate such changes and be prepared to make adjustments
accordingly. The process and measurement framework should have the flexibility to
quickly adapt to or incorporate new requirements.
VDOT has a sophisticated performance measurement system that was put in place
by former Commissioner Philip A. Shucet (Padgette, 2006). Prior to his arrival at VDOT,
no evaluation existed to measure internal agency performance, nor was any effort made to
disseminate information to the public. In establishing VDOT’s Dashboard system, Shucet
sought a means to communicate information in a way that the public could understand and
would serve employees as they managed and executed project delivery. The Dashboard is
a successful top-down initiative that provides a platform to measure, monitor, evaluate, and
report performance to the public and external stakeholders.
KPIs may also be used to establish a performance-driven culture within the
organization. Missouri DOT (MoDOT) launched a program in 2006 that is designed to
reward employee achievements. The Performance Plus program is designed to compensate
MoDOT employees up to $2,000 annually for going above and beyond normal duty to
increase agency productivity (MoDOT, 2007). It began with a pilot project to compensate
construction project office employees for achieving a final construction cost of one percent
19
or less above the contract award amount on projects in the STIP. The pilot program’s
success paved the way for Performance Plus to become a permanent program beginning in
February of 2007. In addition to what is now called the Construction Cost Savings
Incentive, a Project Scoping and Estimating Incentive has been added that benefits
employees who accurately estimate project costs so that project award amounts are within
one percent over or five percent under the STIP construction budget. By encouraging
employees to participate in this innovative process, MoDOT links performance to tangible
results in order to save taxpayer dollars.
2.2.3 Cultural
The continuous growth of a State DOT is inextricably linked to the continuous
growth of the context in which it operates. It is important to develop KPIs that reflect the
cultural values of the respective stakeholders. By studying the customer profile, State
DOTs can learn how population growth, age, diversity, urbanization, and other trends will
impact the future transportation system. It will also assist in providing the right information
through the appropriate communication channels.
Customers are also increasingly expecting a unique experience tailored to their
needs and wants. Personalization is the foundation of the dynamic customer experience.
For example, Pandora generates personalized music streams, Flipboard individualizes
consumption of the news, and Amazon has customized online shopping. The customer-
oriented State DOT personalizes the travel experience, providing highly contextualized
information and adjusting its offerings to strengthen customer engagement.
This is the era of “NOW.” Customers expect services to be available 24x7 and want
answers to their queries delivered on-demand. RideScout is a mobile app that shows the
customer information about transportation options that are available right now (RideScout,
20
2015). Users can see transit, bus, bike, taxi, car share, rideshare, parking, and walking
direction in a single view. By transforming data into valuable information, State DOTs can
provide customers with the appropriate tools to make informed decisions.
Inconsistent, wrong, or outdated information is unacceptable to customers.
Standardizing information ensures that answers are delivered quickly and accurately. Also,
communicating at the scale of highest interest to the customer – for example, providing
information about a personal route to work – allows the message to resonate. Applying a
local approach demonstrates the State DOT is vested in its customers’ interests.
2.2.4 Technological
The nature of data collection, analysis methods, and visualization tools is also
changing and impacting KPIs. Organized by the Transportation Research Board of the
National Academies, the Peer Exchange on Aligning Data Systems to Communicate with
Decision Makers convened in April of 2014 to focus on communicating with decision
makers on transportation asset management, economic development opportunities, and
system performance. The ability to use advanced technologies to enhance data collection
for asset management, performance measurement, and real-time operations was noted as a
major advancement. Participants also recognized the challenges of maintaining new
technologies in addition to training and retaining staff with the technical skills to use the
tools and technologies. Furthermore, there is a need to integrate new data collection
methods with data from legacy systems, across different agency divisions, and between
agencies. The Utah DOT UPlan geographic database is one example of an interactive
planning and analysis tool that integrates a wide range of data from Utah DOT and other
agencies into one system. Using the latest technology, State DOTs can leverage relevant
data to provide customers with a personal and valuable experience.
21
2.2.5 Financial
KPIs may be used to demonstrate that the State DOT is a good steward of public
funds. In particular, metrics such as percent of projects completed on-time and on-budget
are the most common measures used for accountability. State legislatures exercise
significant power over state revenue sources and appropriations. The real power of
legislatures to allocate state funds is bounded by restrictions on the use of transportation
revenues. For example, 23 states have constitutional provisions – and three have statutory
provisions – that restrict use of state fuel tax revenues exclusively to highway and road
purposes (Rall et al., 2011). While it is encouraged to consider the transportation system
as a whole, it is also important for State DOTs to select KPIs that enable legislators to
easily identify the part of the transportation system that is being affected. This ensures that
funds are allocated towards areas in which the legislators and their constituents agree are
vital.
Funding shortfalls have resulted in many State DOTs creating system performance
measurement programs or revising existing programs as a means of allocating funds more
effectively. Transportation funding needs across the nation for highways, bridges, and
transit are projected to be $163 billion annually from 2015 through 2020 (AASHTO, 2014).
The highway and bridge backlog required to restore the system to the level of condition
and performance required to meet today’s demand is $740 billion. KPIs provide a base
level of understanding used to determine opportunities where increased funding could be
leveraged to improve system performance. Performance-based planning includes the
adoption of strategic KPIs linking performance evaluation with policy and investment
decisions.
22
2.3 KEY STAKEHOLDER GROUPS
By tailoring KPIs to different levels and stakeholder groups, State DOTs can
convey the right information to the right stakeholder, enabling them to make well-informed
decisions. There are two processes used to understand key stakeholder interests and
experiences: 1) Stakeholder segmentation organizes stakeholders into groups to understand
the specific nature of their priorities; and 2) Stakeholder mapping arranges the stakeholder
segments to clarify their relationships to one another and to the agency. Both of these tools
assist in identifying what KPIs will be considered most meaningful to which stakeholder
group.
Some State DOTs have already applied customer segmentation in their
communications strategies. Minnesota DOT (MnDOT) has identified the following
specific customer segments: emergency vehicle operators, farmers, intermodal shippers,
commuters, personal users, local communities, and neighbors (Stein and Sloane, 2001).
Florida DOT (FDOT) has identified a slightly different set of customers: residential
travelers, commercial users, elected/government officials, visitors, special need residents,
and property owners (Stein and Sloane, 2001). Both MnDOT and FDOT carry out surveys
to understand their customer needs and priorities.
23
Benefits of implementing a customer-oriented KPI framework include increased
stakeholder engagement, strengthened credibility, and additional investment in a trusted
organization. Washington State DOT (WSDOT) has transitioned its traditional model to a
three-prong approach with encouraging results. Figure 4 shows the three key audiences on
which WSDOT focuses: 1) legislative and other oversight bodies who make funding
decisions; 2) the opinion-makers and media; and 3) internal agency management (Bremmer
et al., 2004). The agency uses a self-created Performance Journalism method of reporting
in its quarterly performance report, the Gray Notebook, which is the agency’s central tool
for reporting (WSDOT, 2015). This approach has been well received by legislators and
media and it has also become an effective agency management tool.
Figure 6: WSDOT Performance Management Framework (Source: Bremmer et al., 2004)
24
2.3.1 State DOT Executives
State DOT executives are the champions for the organization and must be aware of
internal and external motivational factors. Effective leadership provides higher quality and
more efficient delivery of services. It also provides a sense of cohesiveness, an overarching
sense of direction and vision, a resource for invigorating the organizational culture, and a
mechanism for developing relationships with external stakeholders (Van Wart, 2003). An
effective KPI system will assist State DOT executives in managing their relationships with
legislators, businesses, and the public. KPIs communicate the strategic direction of the
agency, its current goals, its progress towards meeting those goals, and its areas of need.
The KPIs should assist in constructing a compelling narrative that will resonate with the
values of each stakeholder.
KPIs are an effective tool for empowering stakeholders to become ambassadors of
the State DOT. Armed with the right narrative, State DOT executives can serve as effective
spokespersons, or champions, for the organization when communicating with legislators,
businesses, and the public. If a State DOT executive is successful in activating the
stakeholder base through the use of meaningful KPIs, these ambassadors become brand
advocates for the agency. As a result, State DOTs extend their agency brand reach while
generating trust, support, and investment.
2.3.2 State Legislators and Governors
To improve the bottom line, effective communications with the state legislature are
essential to establish a stance from which to tell the DOT’s story. Legislators need to be
convinced through performance data that the projects selected are the right investments to
meet the state’s needs. The task for the DOT is to gain credibility and to inform legislators,
in appropriate detail, what is planned for their individual districts – and then to deliver
results that will support the promises to their constituents. An agency’s good performance
25
is an element that should be communicated and utilized to support the agency’s position
with political leaders. Aggressive outreach to the state legislature is essential for DOTs,
either directly or indirectly through other public agencies, customer comments, or ratings
as shown by surveys.
A new funding package typically moves an agency in a project performance
tracking direction. Legislative bodies increasingly make project-based funding decisions
and State DOTs should respond by communicating frequent and detailed information on
scope, time, and budget. Describing on-time and on-budget progress is critical for
legislators to track how state dollars are spent. Furthermore, it builds credibility with the
legislature by demonstrating fiscal responsibility.
Oregon DOT (ODOT) is an example of how a State DOT successfully engaged the
legislature using performance data. ODOT uses “one-pagers,” which are one-page
documents outlining the basics of what the legislators need to know in order to make
decisions (Kase, 2015). ODOT was seeking additional funding to employ railway track
inspection workers, since many of their employees were leaving to find better paying jobs
with the federal government. The one-pager explained how the funding was related to
maintaining the safety of the railroads. Due in part to the one-pager, the legislators decided
to increase funding.
2.3.3 Businesses
The debate on future investment would benefit from a better understanding of the
implications of a well-performing transportation system on business costs, productivity,
and profitability (Litman, 2010). If a state is unable to address worsening traffic congestion,
businesses may move away, go out of business, or adjust to smaller market areas for
workers, suppliers, and customers. On the other hand, State DOTs can foster economic
26
competitiveness by attracting capital into the state. KPIs that demonstrate business
sensitivities to congestion can encourage growth along certain corridors, improve access
to markets, increase worker productivity, and support company-led telecommuting
policies. Although many factors contribute to business decisions, it is the responsibility of
the State DOT to work with businesses and the State Department of Commerce to make a
compelling case for the economic benefits of transportation.
In response to former Governor Huntsman’s statewide energy reduction platform
in 2007, UDOT undertook a nationwide research review and local employer focus groups
in order to determine how to structure a traffic demand management (TDM) program in
Utah (Chisholm-Smith, 2010). Concurrently, UDOT consultants reviewed TDM best
practices nationwide and developed the TravelWise plan. The focus groups included two
groups of 18-20 company executives in the Wasatch Front metro area. It concentrated on
their opinions regarding transportation issues, the reduction of traffic congestion, air
quality improvement, and reductions in energy consumption. One element of was
employee hiring and retention, particularly related to the high costs of transportation for
those employees. They were also interested in a single contact point: they wanted to work
directly with UDOT as a leader, rather than having to coordinate amongst multiple
agencies. The focus group played a fundamental role in shaping UDOT’s TDM strategies
and served as a basis for notable partnerships throughout the state.
2.3.4 General Public
Successful State DOTs provide services reflecting the needs of their customers. The
average person travels 36 miles per day, spending about one hour in a vehicle (NHTS,
2009). Managing how that time is spent determines if the customer has a positive or
negative experience. By providing quality traveler information, roadway maintenance, and
27
roadside assistance services, transportation agencies increase customer satisfaction.
Proactively communicating important information to customers improves relationships and
reduces potential dissatisfaction. For example, notifying travelers of scheduled
construction or lane closures allows customers to identify alternative routes. Anticipating
customer concerns reduces support calls and empowers the customer to become a brand
advocate.
Customers need to be valued. By listening to concerns regarding brand, products,
and services, a transportation agency can learn new insights, gain perspective, and adapt
its strategies accordingly. Kentucky’s Transportation Cabinet undertakes annual customer
satisfaction surveys to find customer needs and wants (Stein and Sloane, 2001). The
Cabinet uses the results to fashion its requests for program approvals and funding from the
legislature. The VDOT Dashboard is another great example of a campaign used to
communicate performance measures to the public. When it was released, VDOT invited
the media to come in and review it first, working closely with Communications in
following up on initial press releases and media inquiries (FHWA, 2015). By monitoring
all communication channels, transportation agencies are able to quickly resolve problems
and improve customer relationships.
28
2.4 FIRST PRINCIPLES OF PERFORMANCE MANAGEMENT
Clearly KPIs are essential to successful management of a State DOT for effective
communication with a range of stakeholders. There is much to consider in establishing a
KPI framework, including the selection and presentation of KPIs through appropriately
designed tools. Strategic planning guidelines for CEOs and executives are focused around
the following four core functions associated with performance management (TransTech
Management, 2003):
1. Internal Communication – Enables leaders to communicate strategic priorities
to their employees
2. Business Management – Provides and organizing theme and focus point for
management frameworks
3. Decision Support – Informs decision-makers about emergent problems and
needs within the transportation system
4. External Communication – Enables the agency to communicate with
stakeholders and customers about its priorities, accomplishments, and goals
2.4.1 What is “key”?
The starting point for choosing which performance measures are considered ‘key’
to a particular State DOT should be those that will best equip the Executive Leadership to
manage and represent the agency. Sometimes referred to as ‘Critical Success Factors,’
KPIs should be narrowed down to the vital few of no more than five to eight
(PricewaterhouseCoopers, 2007). Many State DOT Executives receive percent of bridges
rated in good or higher condition, even though they may be communicating strategies such
as fostering economic competitiveness. A challenge is whether the KPIs currently
presented to the Executive leadership are those that allow external stakeholders to assess
progress against state strategies.
29
2.4.2 Selecting Key Performance Indicators
Indicators address the following four questions: 1) Where are we now? 2) Where
do we want to be? 3) How do we get there? 4) How do we measure progress? The literature
recommends KPIs adhere to SMART criteria to hold agency personnel accountable for
delivery of outcomes: Specific, Measurable, Achievable, Relevant, and Time-Bound
(Rozner, 2013). Different sources will offer different definitions or characteristics for the
“SMART” acronym because in reality there are more than five attributes to consider when
evaluating the merits of one KPI or another. A modified version is proposed below to
support a customer-oriented approach:
(S) Strategic
(M) Measurable
(A) Accessible
(R) Relevant
(T) Tangible
2.4.2.1 Strategic
Ideally, agencies will align performance measures in relation to achieving the
agency’s vision, mission, and goals. This enables stakeholders to assess the strategies
adopted by the agency and their potential to succeed. By cascading KPIs, day-to-day
activities are connected with the agency’s strategic goals. By extending KPIs, the agency’s
strategic goals are connected with the stakeholder’s desired outcomes. KPIs should reflect
and measure key drivers of value that guarantee future success, such as high customer
satisfaction or excellent service quality. The largest challenge in this attribute is the
evolving and dynamic nature of the institutional and political environments.
30
2.4.2.2 Measurable
KPIs express progress in an objective and quantifiable form. State DOTs should
use reliable and available data that the agency can collect and analyze without straining its
resources. Furthermore, it should clearly explain the limitations of the data and any
assumptions made in generating the KPI. If data is not being collected in an area, funds
should be allocated to capture new data or clean existing data. In the absence of standards
for the measurement of industry-specific indicators, an explanation of the components of
the indicator and how it is calculated are vital. Success is defined through appropriate target
and benchmark setting. These goals should be identified so they are recognized as
important by the stakeholder community.
2.4.2.3 Accessible
KPIs must be easy to comprehend. If there are too many or they are too complex,
KPIs will lose their power to grab the attention of stakeholders and influence behavior.
Given the rapidly increasing usage of industry-specific terminology, technical jargon
should be avoided so that the KPIs may be easily understood. Clear definitions of
performance indicators add greatly to the reader’s understanding of exactly what is being
measured and allows comparisons between agencies.
Providing access to the information is also critical to developing an open and
transparent process. Open Michigan was implemented by Governor Rick Snyder to provide
a quick assessment of the state’s performance in key areas, including economic strength;
health and education; value for money government; quality of life; infrastructure; and
public safety (Michigan, 2015). The infrastructure dashboard includes metrics describing
economic growth, safety, accountability, mobility, and conditions. The tool was found to
successful in situating transportation within a bigger picture and catalyzing a movement
towards data-driven government.
31
2.4.2.4 Relevant
KPIs should be meaningful when viewed from the stakeholder’s perspective.
Relevant KPIs as defined by the Conference Board of Canada, provide information that is:
1) significant and useful to the reporting organization’s stakeholders an 2) attributable to
its activities (Office of the Auditor General of British Columbia, 2010). By evaluating KPI
results, stakeholders can understand whether their expectations of performance
management have been met. Communicating the most important elements that will
effectively inform the decision-making process of the respective stakeholder are likely to
lead to positive action.
2.4.2.5 Tangible
Report the results at a scale that will resonate with the respective stakeholder. For
example, the respective district level is most appropriate for reporting current construction
projects to a state legislator. Texas DOT (TxDOT) has designed a project reporting system
that enables state legislators to view projects by highway, project ID, and county; monitor
its status from planning to design to construction; and identify the construction
cost/estimate (TxDOT, 2015). Residents are most likely to be concerned about their
specific commute that they experience on a regular basis. Utah DOT (UDOT) in
partnership with Cisco Systems, Inc. developed CommuterLink, an ITS network of
resources designed to maximize the efficiency of transportation in the state and help
residents “Know Before You Go.” The transportation services information program is
designed to keep travelers updated about road conditions and delays specific to the
commuter. CommuterLink integrates more than 800 traffic signals, 1,400 detector stations,
250 closed-circuit television cameras, and 70 variable message signs; saving Utah travelers
$179 million and 9.8 million hours annually; and preventing 948 traffic accidents and 3
traffic-related deaths each year (Cisco, 2004).
32
2.4.3 Communicating Key Performance Measures
Crafting messages that resonate with stakeholders is important to connecting with
diverse audiences. Presenting information visually and in a narrative format rather than just
listing facts and figures is a more effective way to communicate ideas to stakeholders who
are less familiar with the data. The charged political context around accountability in
transportation makes performance communication one of the most salient issues in the
performance measurement field (Bremmer et al, 2004). States that are the most advanced
in addressing issues of communications, are focused on improving both internal
communications with staff and external communications with the public, elected officials,
and businesses through media relations. Some states are even rebranding themselves and
identifying ways in which to clarify and improve their credibility with the public,
recognizing that image enhancement and improved stakeholder communications may lead
to an improved position for the agency, new resources, and a more supportive voter base
for the agency’s work. Increasingly, State DOTs report that proactive efforts to better
communicate and to position the agency positively with stakeholders have led to increased
public support and legislative funding (Stein and Sloane, 2001).
33
2.4.3.1 Dashboards
Several states have established or are moving toward “dashboard” style reporting
to communicate progress at a glance. This snapshot view generally uses red, green, or
yellow lights to describe results or simple arrows to describe directional trends. Virginia
DOT is setting the standard for the interactive, on-line dashboard approach. VDOT’s
Dashboard includes performance, safety, condition, and finance information in addition to
management, project, and citizen survey results. Other states that have implemented
publicly available dashboards are Utah, Michigan, Alaska, Georgia, and Delaware.
Figure 7: VDOT Performance Dashboard
(Source: http://dashboard.virginiadot.org/)
Performance Safety Condition Projects Citizen Survey Finances Management
34
Figure 8: UDOT Zero Fatalities Dashboard
(Source: http://dashboard.zerofatalities.com/?year=2014)
Figure 9: Open Michigan Infrastructure Dashboard Sample
(Source: https://midashboard.michigan.gov/infrastructure)
35
Figure 10: DOT&PF Dashboard Sample (Source: http://dot.alaska.gov/performance-dash/index.shtml)
Figure 11: DelDOT Dashboard Sample (Source: http://www.deldot.gov/dashboard2/condition.shtml?99)
36
2.4.3.2 Reports and Scorecards
Several states have an externally oriented, highly visible performance reporting
process. Scorecards offer a quick summary and include basic information such as policy
goals, performance measures, targets, results, trends, analytical notes, and sometimes short
narratives. Florida DOT (FDOT) publishes annual performance highlights that include
both system and organizational core measures in the following areas: Safety, Maintenance,
Environment, Mobility, and Economy (FDOT, 2014). The print version is easily accessible
on the website and is formatted as a brochure. VDOT’s Quarterly Report Card, also
available online, shows the agency’s performance on its “core business outcomes,”
highlighting the percent of construction and maintenance contracts completed on-time and
on-budget (VDOT, 2014). Appendix A includes examples of State DOT scorecards.
Typically lengthier than a scorecard, reports provide another opportunity to build a
story around the performance measures by adding detail and context. The Florida
Transportation Commission produces an annual report for the governor and legislature
called the Performance and Production Review. The report contains 17 primary measures
and detailed project delivery information (Florida Transportation Commission, 2014).
More robust than the “Performance Highlights” brochure, the Review offers additional
depth and insight. Another successful document is the Utah DOT (UDOT) Strategic
Direction and Performance Measures Report. The document presents emphasis areas,
agency core values, and the major performance accomplishments related to UDOT’s four
strategic goals: Zero Crashes, Injuries, and Fatalities; Preserve Infrastructure; Optimize
Mobility; and Delivering Results (UDOT, 2015). Distributed at the beginning of each year
before the legislative session, the Strategic Direction report provides a consistent means of
communicating Utah DOT’s priorities and performance to policy makers and the public
(Turnbull, 2014).
37
2.4.3.3 Traditional Media
Traditional media – newspapers, radio, and television – are important to DOT
image. Nearly every DOT tracks news clips and broadcast media reports to gather some
notion of how the agency is being presented, often on a daily basis. New Mexico DOT
(NMDOT) aggressively seeks out media contacts in an open, proactive way, resulting in
positive feedback. In tracking media coverage regularly, NMDOT reports its rate of
positive stories is around 80 percent (Stein and Sloane, 2001).
Management concerns about negative media coverage led the internal staff in the
Caltrans district around Los Angeles to analyze several thousand clips from a single year
to assess the positive and negative media coverage of the agency. The staff analyzed
newspaper coverage, along with tapes of radio and television reportage, in terms of
negative/positive/neutral stances taken, and then measured each in terms of column inches
and minutes of radio or TV coverage. Contrary to expectations, the finding was that media
coverage was overwhelmingly positive, by a 15 to 1 proportion of positive to negative
coverage. Furthermore, these positive media accounts would have cost Caltrans over $1.5
million if the agency had purchased such coverage at moderate-sized newspaper or
broadcast prices (Stein and Sloane, 2001).
2.4.3.4 Websites
Many state agencies make their performance data available through a dedicated
website. At a minimum, agencies will dedicate a site for uploading performance reports
and other materials for viewing and downloading. TxDOT goes a step beyond by
publishing a Performance Results Summary, which is a simplified dashboard in table
format that contains the TxDOT budgetary performance measures as reported to the
Legislative Budget Board and organized by the agency’s budget structure. The ease and
availability of these tools contribute to their effectiveness in disseminating information.
38
Figure 12: TxDOT Performance Results Summary Sample
(Source: http://www.txdot.gov/government/legislative/state-affairs/performance-results.html)
39
2.4.3.5 Open Data Portals
Open data portals are platforms that offer consumers a better way to access and use
public information. In an effort to improve transparency, accountability, and
interoperability, cities and states are releasing government-produced, machine-readable
data sets through centralized repositories. By engaging with the community, public
agencies are enhancing quality of life and the delivery of government services.
Local leadership is acting as a catalyst to jumpstart the open data movement. Led
by Mayor Greg Fischer and the chief of the Office of Performance Improvement, Theresa
Reo-Wever, Louisville has positioned itself to use data in transformative ways. Fischer
signed into law an executive order mandating that data is open by default and launched the
LouieStat open data portal in 2012. As a result of the program, drop-off policies were
revised and Emergency Management Services delivered 18,000 more patients to the
hospital between September and December of 2013 as compared to the same time period
in 2012 (Govloop, 2014).
2.4.3.6 Social Media
State DOTs are making increased use of social media platforms such as Facebook
and Twitter in order to communicate success stories and educate the public regarding the
urgency of transportation funding. Tennessee DOT (TDOT) offers Twitter feeds by
roadway and location to report out construction, traffic, and incident information. State
DOT Facebook pages are target to specific audiences, primarily teens and young drivers.
Texas finds that the majority of their “fans” are educated professionals, ages 25-44
(AASHTO, 2010). Arizona targets its messages to drivers 16-35 (AASHTO, 2010). These
tools are useful for communicating KPIs to customers who are inclined towards technology
and social networking. State DOTs that successfully build a social network can leverage
followers into becoming brand advocates.
40
2.5 THE EVOLUTION OF PERFORMANCE MANAGEMENT
Performance management is an evolving practice. A typology of performance
management frameworks was developed by Bremmer et al. (2004) and serves as a way to
classify different models according to their primary purpose. Citing Baird and Stammer
(2000) in addition to Poister (2004), Bremmer et al. emphasize that while State DOTs have
a strong tradition of using KPIs as part of the engineering and planning processes, it is only
since the 1990s that KPIs are being used as a business management and accountability tool
to inform decisions. Some agencies are developing more strategic systems, integrating
KPIs into vision frameworks to drive organizational outcomes. Still more recent, the need
to focus on serving the customer has become integral to the performance management
practice. This thesis reviews State DOTs according to the three previously identified stages
of development and proposes the emerging typology of customer-oriented paradigms.
2.5.1 First Generation: Traditional Infrastructure & Organizational Measurement
First generation frameworks define progress in meeting specific program or project
goals; however they typically lack strategic direction and are not meaningfully linked to
other agency processes (Bremmer et al., 2004). Agencies develop first generation
frameworks often in response to internal Total Quality Management initiatives or specific
legislative requirements. Standard measures are developed to track basic system
performance and organizational process improvement, useful for internal, project-specific
decision-making purposes but not directly valuable to the agency’s stakeholders.
Nevada DOT (NDOT) is an example of a first generation agency. NDOT has
established 15 performance measures to track, monitor, and report performance of the
major divisions and program areas (NDOT, 2014). Although NDOT has a strong awareness
of performance measure statuses in relationship to desired targets, there is a lack of
41
prioritization and focus. With such a multitude of metrics and lack of framework, NDOT
struggles to synthesize its performance measures into a unified vision.
Table 5: 2014 NDOT Performance Measures
Metrics 1. Reduce Work Place Accidents
6. Reduce and Maintain Traffic Congestion
11. Emergency Management, Security, and Continuity of Operations
2. Provide Employee Training 7. Streamline Project Delivery – Bidding to Construction
12. Reduce Fatal Crashes
3. Improve Employee Satisfaction
8. Maintain State Highway Pavement
13. Project Delivery – Schedule and Estimate for Bid Advertisement
4. Streamline Agreement Process
9. Maintain NDOT Fleet 14. Maintain State Bridges
5. Improve Customer and Public Outreach
10. Maintain NDOT Facilities 15. Streamline Permitting Process
Source: NDOT, 2014
2.5.2 Second Generation: Hierarchy of Measurement
Second generation frameworks connect measurement areas together in a strategic
orientation, yet are often too complex due to a proliferation of measures to easily
communicate results. Measures are usually based on a traditional planning framework and
are often long-range measurements linked to mid- or short-range strategic business plans.
Second generation frameworks enable State DOT leadership and managers to track
business functions and planning goals, and are accompanied by a public reporting tool to
meet legislative, public, or agency needs.
The Missouri DOT (MoDOT) is an example of a second-generation agency that has
established an overall framework and implemented a performance-based strategic plan.
MoDOT adopted a hierarchy that aligns agency goals, objectives, and performance
measures with the long-range transportation plan. Quarterly performance reports in the
42
MoDOT Tracker series are published to demonstrate progress toward the seven agency
priorities using 58 supporting metrics (MoDOT, 2015). Tracker thoroughly documents the
continued growth of the organization and appropriately addresses transportation funding
challenges, however, external stakeholders may feel overwhelmed or intimidated by the
130-page volume. Caltrans reports on 72 performance measures organized around five
goals, of which it elevates 26 as key (Caltrans, 2015). While the quantity and quality of the
goals are well focused, the documents are written from the DOT perspective, which has
led to equal weighting of internally and externally motivated KPIs.
2.5.3 Third Generation: Catalyst-Driven Adaptation
Third generation frameworks respond to changes in agency priorities and external
pressures to guide the strategic direction of the agency. KPIs are identified to focus on
building effective metrics and communication tools centered on agency responsibilities and
investment decision needs. To manage the complexity created within the second-
generation framework, agencies have explored alternative ways to measure and
communicate performance. This approach is dynamic and provides real-time information
responsive to the needs of agency leadership, the state’s political context, and public
opinion. Its greatest shortfall, however, is that third-generation frameworks are reactive.
State likes Ohio and Washington have responded to changing force by developing
third-generation frameworks. They have also tried to address the complexity of measures
created in their second-generation phase. The New Mexico DOT (NMDOT) retooled and
reorganized its previous performance report The Compass. The Compass was originally
designed to be an accountability tool, but was actually thought to be “data rich, but
information poor,” due to the sheer volume of measures and lack of a cohesive framework
dictating which measures were most important (Bremmer et al., 2004). It has since been
43
narrowed down to five strategic objectives supported by 40 measures, where each division
in the agency is responsible for developing a business plan to ensure success of the strategic
objectives.
2.5.4 Fourth Generation: Customer-Oriented Paradigm
Fourth-generation frameworks are proactive in nature and incentivize changes in
behavior. Similar to third generation models, fourth generation frameworks are dynamic
and readily adaptable; but rather than being catalyst-driven, fourth generation frameworks
actively influence change. Furthermore, these frameworks are accompanied by a well-
developed communications strategy that focuses its messaging on multiple customer
segments. The communications strategy is typically designed through stakeholder input
and focus groups. These frameworks are externally oriented and tend to prioritize strategic,
compliant, and influential KPIs. Agencies that implement fourth-generation frameworks
exhibit strong State DOT executive leadership who are cognizant of stakeholder interests
and forces of change. The customer-oriented KPIs become a tool that equips leaders with
the right information needed to construct appropriate narratives.
The online customer community, called Minnesota DOT Talk, is a randomly
selected market research panel of 450 Minnesota residents who participate in weekly online
surveys, discussion and brainstorming sessions, and chats on a range of transportation
topics (Turnbull, 2014). Members of the Online Customer Community are invited to serve
for one year and reflect the demographics of the 2010 Census. Questions can be targeted
toward subgroups—such as metropolitan area residents or workers who commute during
the peak hours—as needed. Information is used by groups throughout the agency, including
communications, planning, and operations. The ongoing feedback from the community
provides a deeper and richer understanding of transportation needs, preferences, and
44
reactions to proposed and new facilities, services, programs, and policies. Examples of
topics considered include active traffic management signs, approaches to improve work
zone safety, roundabouts, travel time information during freeway construction, financing
and funding alternatives, and financial accountability.
45
Chapter 3: Customer-Oriented KPI Development Approach
The background has thus far provided a solid foundation for understanding the
challenges and opportunities of developing highly impactful key performance indicators
(KPIs). Beginning with the motivation, this thesis demonstrated the power of using KPIs
to strengthen stakeholder relationships by building trust, garnering support, and
encouraging investment. Next, key stakeholder groups were identified along with their
respective priorities and relationships to the State DOT. Basic principles for selecting and
communicating KPIs were outlined to guide practitioners in marrying the right information
to the right stakeholder. Finally, a typology of KPI frameworks was described and extended
to include the emerging customer-oriented paradigm.
In order to apply the techniques previously described, a new customer-oriented KPI
framework will be developed for the Texas Department of Transportation (TxDOT). This
process begins with the analysis of the performance management context in the state of
Texas. Once grounded in the specific environment, three stakeholder profiles are developed
to equip the TxDOT leadership in effectively communicating the performance of the
agency and influencing stakeholders to take desired actions. Specific KPIs are proposed to
correspond with each stakeholder profile that may assist in generating a data-woven
narrative. In an effort to validate the success of the proposed framework, key person
interviews are conducted to gather feedback from each stakeholder group. The feedback is
then synthesized to produce a final set of key recommendations.
46
3.1 PERFORMANCE MANAGEMENT CONTEXT IN THE STATE OF TEXAS
While the principles outlined in the previous section are intended as general
guidelines, this section applies them specifically to Texas. TxDOT employs key
performance indicators (KPIs) to translate broad policy goals and objectives into actionable
programs, projects, and services. In particular, the Executive Administration and Texas
Transportation Commission use KPIs to build a unified vision, communicate the strategic
direction of the agency, promote TxDOT’s activities, justify project selection, and
influence investment decisions.
Figure 13: Drivers of Change that Impact KPIs in Texas
TxDOT KPIs
Cultural
Political
Financial
Institutional
Technological
SunsetReview PerformanceTargets HB20
LeadershipTurnover VisionReevaluation StrategicPlanning
DemographicChanges VehicleOwnership TrustErosion
Connectivity OmniChannel BigData
ConventionalFunding Proposition7 Proposition1
47
3.1.1 Political Pressure
A variety of political pressures are impacting the strategic direction of TxDOT,
leading to the most recent reevaluation of the Department’s vision, mission, values, and
goals, along with corresponding key performance metrics. As part of the 2010-2011 Sunset
Review Cycle of the 82nd Legislative Session Limited Scope Review, Texas SB 1420
continued TxDOT for four years to keep the agency under closer legislative scrutiny and
includes several changes to improve the transparency, accountability, and reliability of the
Department (Sunset Advisory Commission, 2011). The Legislature sought to increase
oversight to re-establish the Legislature’s and the public’s trust and confidence in the
Department. Furthermore, the Legislature modified the Chief Financial Officer’s duties to
include ensuring that the Department’s financial activities are conducted in a transparent
and reliable manner, and certifying that construction and maintenance contracts will not
create a state liability. SB 1420 also established the Unified Transportation Program (UTP)
covering a period of 10 years in statute to guide development and authorize construction
of transportation projects. The Commission must establish criteria for selecting projects;
definitions for program funding categories including safety, maintenance, and mobility;
and benchmarks for evaluating project progress and readiness to be implemented. The UTP
and summary documents must be published in appropriate media and on the agency’s
website. An online project information reporting system and a transportation expenditure
reporting system must also be developed as publicly available reporting tools. There are
additional provisions to improve TxDOT’s consistent, unbiased, and meaningful public
involvement. The bill requires the policy to include specific elements, such as using
techniques that target different groups and individuals, making efforts to clearly tie public
involvement to the decisions the Department makes, and applying the policy to all public
input with the Department.
48
Each biennium, the Texas Legislature establishes agency performance targets in the
General Appropriations Act. Table 6 outlines the TxDOT budgetary performance
measures, organized by the agency’s budget structure, for FY 2014-2015 biennium enacted
in HB 1 (83rd Regular Session, 2013) and reported to the Legislative Budget Board (LBB).
Table 6: TxDOT KPIs reported to the Legislative Budget Board
Metrics by Objective Provide Transportation Planning
Percent of Design Projects Delivered on Time Percent of Design Projects Delivered on Budget Number of Construction Project Preliminary Engineering Plans Completed Dollar Volume of Construction Contracts Awarded in Fiscal Year Number of Projects Awarded
Transportation Improvements
Percent of Construction Projects Completed on Budget Percent of Two-Lane Highways 26 Feet or Wider in Paved Width Percent of Construction Projects Completed on Time Percent of General Aviation Airport Pavement in Good or Excellent Condition Number of Grants Approved for Airports Selected for Financial Assistance
Preserve the Transportation System
Percent of Bridges Rated in Good Condition or Higher Percent of Highway Pavements in Good Condition or Better Statewide Maintenance Assessment Program Condition Score Statewide Traffic Assessment Program Condition Score Number of Lane Miles Contracted for Resurfacing Number of Highway Lane Miles Resurfaced by State Forces
Optimize Services and Systems
Percent Change in the Number of Small Urban and Rural Transit Trips Number of Fatalities per 100 Million Miles Traveled
Enhance Rail Transportation
Number of Federal Railroad Administration (FRA) Units Inspected
Source: TxDOT, 2015
49
Enacted in June of 2015, Texas HB 20 requires the commission by rule to develop
and implement performance metrics and performance measures as a part of the review of
strategic planning in the statewide transportation plan, rural transportation plans, and the
Unified Transportation Program (UTP); the evaluation of decision-making on projects
selected for funding in the UTP and Statewide Transportation Improvement Program
(STIP); and the evaluation of project delivery for projects in TxDOT’s letting schedule
(TxDOT, 2015). The bill also requires the commission by rule to adopt and periodically
review performance metrics and measures to assess how well the transportation system
performs and operates; provide TxDOT, the Legislature, stakeholders, and the public with
information to support decisions in a manner that is accessible and understandable to the
public; assess the effectiveness and efficiency of transportation projects and service;
demonstrate transparency and accountability; and address other issues the commission
considers necessary.
3.1.2 Institutional Changes and Initiatives
In 2015 TxDOT experienced an exceptionally high turnover in leadership, which
contributed to strategic ambiguity and a loss of institutional knowledge. In February, the
newly elected Governor appointed a Chairman and a Commissioner to the Texas
Transportation Commission. The Governor has declared reducing congestion to be a
priority of his administration and directed TxDOT to create a “focused initiative” to ensure
Texas’s most congested areas receive priority. In conjunction, the Chairman has charged
each of the commissioners with a specific customer concern to investigate and champion.
In particular, the five other commissioners are concentrating on 1) congestion relief
initiatives, 2) fiscal stewardship related to project selection, 3) performance measures, 4)
federal legislative affairs, and 5) implications of UAVs and other emerging technologies.
50
The diversification of priorities by the Commission has been successful in enabling them
to tackle a broad array of issues; the next step will be in synthesizing their initiatives into
a unified vision for the agency.
In addition to several on-boardings, there have been multiple resignations. In
August of 2015, the Deputy Executive Director submitted his resignation to end a nearly
30-year successful career with TxDOT. There was much concern that he would be the first
of many high-ranking officials to leave the state agency due to a provision in SB 20 (Texas
Government Insider, 2015). The bill stipulates that any state employee who was a decision-
maker in awarding a state contract to a vendor cannot be hired by that vendor until two
years after leaving the agency. Then in October of 2015, the Executive Director announced
just 18 months into his tenure that he would resign at the end of 2015. Weber’s resignation
means the Department will have its fourth Executive Director since 2011.
In the spring of 2015, TxDOT initiated a review of its vision, mission, values and
goals, and subsequently the initiatives, strategies, and associated key performance
measures used to guide the agency. A Core Strategy Team (CST), led by one of the
commissioners with Administration sponsorship, was formed to lead the effort. The team
is composed of 10 employees from various districts, divisions, and offices. The main
objective is to create a publicly available Commission Dashboard, designed to reflect the
agency’s progress toward departmental goals and strategic initiatives. The team sought
input throughout the process from employees who are closest to the work that TxDOT
performs as well as external stakeholders. The Core Strategy Team and the Office of
Strategic Planning conducted five focus groups around the state. The focus groups
consisted of district, division, and office personnel and were conducted in Austin, Bryan,
Paris, Lubbock, and Corpus Christi. The input is being used as input to enable the Core
51
Strategy Team to make a recommendation to Administration regarding the agency’s
Vision, Mission, and Values. Table 7 lists goals under consideration.
Table 7: TxDOT Goals under Consideration by the Core Strategy Team
Proposed Goals and Definitions
Focus on the Customer – People are at the center of everything we do
Deliver the Right Projects – Develop effective planning and forecasting processes
that deliver the right projects on-time and on-budget
Preserve our Assets – Deliver preventive care and maintenance for TxDOT’s
system and capital assets to optimize our investments
Promote Safety – Maintain a culture of safety first
Enhance System Performance – Develop and operate an interconnected
transportation system that provides reliable and accessible mobility enabling
economic growth
Value our Employees – Respect and care for the well-being and development of
our employees
Foster Stewardship – Ensure efficient use of state resources
Source: TxDOT, 2015
52
In the fall of 2015, the Office of Strategic Planning sought to develop an Integrated
Comprehensive Action Plan (ICAP) with the main objective of leveraging and identifying
collaborative activities to maximize asset use. Multiple workshops were held in order to
identify gaps and opportunities. The final proposed recommendations will go to the
Commission for approval in December 2015. During the workshops, two main points
arose: 1) the need for a unified, overarching communications strategy; and 2) the need for
a change in the funding structure to allow for holistic transportation projects beyond the
traditional segment-based highway and bridge projects. Customer-oriented KPIs can assist
with both of these.
3.1.3 Cultural Transformations and State of Public Trust
The population of Texas is growing rapidly – especially in a few already-congested
metropolitan and urban – thus taxing the existing transportation system. Population growth
is expected to continue outpacing the nation by a wide margin through 2040. The Texas
State Data Center projects that by 2040, Texas will add 13.5 million people to reach a total
population of 35.8 million (Potter and Hoque, 2014). Much of the population growth in
Texas is expected to come from the large urban counties of Harris, Dallas, Tarrant, Bexar,
and Travis, but the fastest growth will occur in the suburban rings surrounding these
counties. With such rapid growth, congestion is worsening and infrastructure maintenance
demands are increasing. In 1999, computer manufacturer Dell acknowledged that Central
Texas traffic contributed to its decision to expand in Tennessee rather than at its Round
Rock headquarters north of Austin (Batheja, 2014). Transportation and urban planners will
need to prepare for the state’s future to ensure that Texas continues to attract rather than
repel opportunities.
53
By 2050, one in five Texans is projected to be 65 and over (Potter and Hoque,
2014). More people will require quality public transit connections to medical and other
services. TxDOT will also need to be aware of cultural shifts, such as reduced vehicle
ownership and the shared economy, driven by the millennial generation. The sharing of
goods and services is disrupting traditional business models. Its rapid growth is rooted in
three factors:
Ability to leverage space capacity and expand systems without large investment
Exponential effect of collaborative experimentation and learning
Ubiquitous nature of distributed capacity
A typical car lies unused for approximately 23 hours a day (International
Transportation Forum 2015). Transportation network companies, such as Uber and Lyft,
are taking advantage of overcapacity to satisfy a demand for mobility. From 2007 to 2011,
vehicle ownership for people aged 18 to 34 fell almost 30% (NHTS, 2009). As a result, car
manufacturers and vehicle rental companies have introduced their own carsharing services,
including Daimler’s Car2Go, BMW’s DriveNow, and Avis’s ZipCar. Businesses are also
creating low-speed, shared vehicle fleets for employee use on campus and in neighborhood
communities. As cultural values continue to shift, it will be critical for TxDOT to design a
positive customer experience for all citizens.
Regarding the state of public trust in Texas, billing problems with TxDOT’s tolling
system operated by a private system integrator came to light in 2015. TxDOT will be
refunding about $1.7 million to tollway users who have been overbilled (Batheja, 2015).
The agency is continuing to find TxTag accounts that were improperly billed following
refunds to 31,000 accounts at the beginning of the year. In addition to billing issues, Texas
drivers have encountered trouble accessing accounts online and long wait times on
customer service calls. Although many of these problems have been blamed on the
54
company that the private system integrator was hired to replace, the public is still
attributing the errors to TxDOT. Due to this erosion in public trust, troubled state contracts
became a central topic of discussion with the House General Investigating Committee.
Consequently several bills were filed aimed at restricting TxDOT’s tolling operations, with
proposals ranging from restricting the agency’s ability to assess late fines to requiring it to
develop a plan to ultimately remove tolls from all Texas roads. As TxDOT develops its
new KPI framework, the agency should keep these concerns in mind.
Among the additional factors shaping the TxDOT environment are the troubled
economy, changing expectations regarding public sector transparency and accountability,
heightened sensitivity of government spending, and evolving expectations for increased
local control over transportation project selection and development. While the State faces
increasing needs for transportation infrastructure to serve its citizens, TxDOT is challenged
to obtain funds needed to maintain the existing assets, to continue to build new capability,
and to articulate what is required to fund transportation needs for the foreseeable future.
3.1.4 Technological Advancements
By 2020 the average person will be connected to 4.3 devices (Strategy Analytics,
2014). TxDOT will need to consider how to collect data from newly sources as well as
communicate with its customers through PCs, tablets, wearables, smartphones, televisions,
radio, and vehicle infotainment systems. Using the latest technology, TxDOT can leverage
relevant data to provide customers with a personal and valuable experience.
By viewing the transportation experience through the eyes of the customer, TxDOT
can orchestrate the customer experience across all layers so that it is seamless, integrated,
and consistent. OmniChannel anticipates that customers may start in one channel and move
to another as they progress to a resolution (Stocker, 2014). Facilitating the transitions so
55
that they are fluid for the customer develops continuity within the communications
strategy. The following outlines several of the most valuable channels:
Website: Developing an online presence through a mobile-optimized agency
website is critical as the basis for a, permanent digital relationship with the
customer.
Search: The next digital layer emerged from the development of powerful
search engines such as Google. Focusing on the customer’s discovery process,
transportation agencies can align customers to their desired services and
information.
Social Media: Social networks generate another layer of communications and
present a new way to engage with customer service. TxDOT can take advantage
of the opportunities to advertise safety campaigns, promote commerce, and
provide service options.
Mobile: Combined with social media, mobile presents an unprecedented
opportunity to focus on utility. By providing stakeholders with valuable
information in real time, customers can receive traffic alerts or evacuation
information and make effective decisions in a timely manner.
Wearables: Augmented reality, wearable technology, and advanced filters
increase the bond between customers and their technology. As individuals
become more health conscious, transportation agencies can leverage the trend
to promote pedestrian and bicycling initiatives.
Open communication channels form the basis of improved transportation planning
and resource allocation efforts. In a technology-advanced world, TxDOT can integrate its
KPI system to collect new information as well as engage with its customers on a more
56
personal level. A customer-oriented KPI framework can ensure that the right projects and
services are delivered to the right customers.
3.1.5 Financial Limitations
TxDOT has traditionally focused on the construction and maintenance of highways
and bridges. A primary reason for this focus is due to its restrictive funding structure as
stipulated by the state legislature. Policymakers in turn respond to the attitudes of their
constituents, who often view TxDOT’s sole function as the provider of the physical
infrastructure. As a result, the public is only willing to pay for traditional highway and
bridge projects; however, they still demand high levels of service and hold TxDOT
responsible for its inability to mitigate congestion. While the metrics required by the
Legislative Budge Board are heavily construction-biased, the public ultimately values
mobility. This discrepancy highlights multiple educational gaps: 1) the transportation
system must operate holistically, 2) congestion is a complex problem that requires a
comprehensive solution, and 3) a quality system requires a sustainable source of
investment. A customer-oriented KPI framework should strive to correct these
misperceptions and encourage voters to support comprehensive transportation investment.
Proposition 7 is the most recent example of Texas voters perpetuating the
traditional investment model. In February 2015, the Governor named transportation
funding one of five emergency items for the 84th Texas Legislature. To address this issue
and take steps toward closing the state’s transportation funding gap, the Texas Legislature
passed Senate Joint Resolution (SJR) 5 in May, placing Proposition 7 on the ballot on
November 3, 2015. Texans voted on the constitutional amendment to dedicate portions of
revenue from the state’s general sales and use tax as well as from the motor vehicle sales
57
and rental tax to the State Highway Fund for non-tolled projects. The amendment passed
and will allocate funds in the following ways:
Beginning in September 2017 (FY 2018), if state sales tax revenue exceeds $28
billion in a fiscal year, the next $2.5 billion of revenue will be directed to the
State Highway Fund.
Beginning in September 2019 (FY 2020), if state motor vehicle sales and rental
tax revenue exceeds $5 billion in a fiscal year, 35 percent of the amount in
excess of $5 billion will be directed to the State Highway Fund.
It is important to note that the funds may only be used to purchase right of way for,
build, maintain, and rehabilitate non-tolled public roads and to pay down certain
transportation-related debt. These restrictions were included in part due to the mistrust of
toll-road projects by the Texas public as well as hesitations towards non-traditional
infrastructure projects. A customer-oriented KPI framework could be used as an
educational tool to rebuild public trust, demonstrate thoughtful evaluation of diverse
options, and showcase TxDOT’s ability to deliver innovative solutions that improve overall
mobility.
Proposition 7 complements Proposition 1, which was passed in November 2014,
by tapping additional revenue sources to further enhance transportation funding.
Proposition 1 authorized a portion of the severance taxes on oil and natural gas to be
redirected to the State Highway Fund each year. For 2015, the first year in effect,
Proposition 1 resulted in an additional $1.7 billion going towards state highway funding.
Although these two amendments are an important step towards addressing highway
challenges, neither provision allows for investment in multimodal solutions. If TxDOT
envisions itself as a multimodal agency, then TxDOT should consider modifying its KPI
structure to support multimodal investment.
58
3.2 CUSTOMER-ORIENTED KPI FRAMEWORK PROPOSAL FOR TXDOT
Now that the stage is set, it is time to delve into the customer mindset. Three
example stakeholder profiles and a sample KPI report (see Appendix B) are designed to
equip the Texas Transportation Commission and TxDOT Executive Administration with
the right information in order to influence the respective stakeholder to take positive
actions. Each stakeholder profile begins with an “Ask” that aligns with TxDOT’s goals in
order to clearly identify the desired outcome. Next, a “Customer Experience Snapshot”
shifts the perspective from a traditional DOT-centric mentality to that of the stakeholder.
Then, a “Customized Value Proposition” is constructed to enable the TxDOT leadership to
deliver a data-woven narrative that effectively communicates the performance of the
agency. This format is considered more effective than overwhelming the stakeholder with
metrics that are overly complex or irrelevant, ensuring that the power of the KPIs is neither
diluted nor lost. Finally, a select few KPIs are proposed to support the Value Proposition,
allowing for it to be quickly adapted for its appropriate context and audience. Together
these four components constitute a Stakeholder Profile that prepare TxDOT leadership to
effectively communicate with a range of stakeholders, improving the position of the
agency, instilling trust, and encouraging investment.
The Stakeholder Profiles are specifically designed to use customer-oriented KPIs
to demonstrate how TxDOT is actively working to reduce congestion from multiple angles.
Congestion was selected as the focus since it is an everyday customer touch point, media
coverage is highly visible, and the Governor has explicitly directed TxDOT to address its
worsening conditions. Also, the KPIs are selected to broaden customer awareness
regarding non-construction, non-infrastructure solutions. The rationale behind these
decisions is to ultimately encourage stakeholders to perceive transportation as a holistic
system that deserves a corresponding holistic funding structure.
59
3.2.1 Stakeholder Profile: General Public
3.2.1.1 Ask
TxDOT should ask the Texas public to trust the agency to deliver quality solutions
to everyday congestion problems. TxDOT should earn the trust of the public by increasing
travel time reliability and evaluating all congestion mitigation measures in a transparent
manner.
3.2.1.2 Urban Customer Experience
Austin is the capital of Texas and the fourth largest city in the state with an
estimated population of 836,800 people. The average Austin resident spends 52 hours per
year in traffic congestion, typically commuting to and from work during peak hours in a
single-occupant vehicle. Austin traffic is concentrated along I-35 and Mopac, two of the
most congested and unreliable corridors in the state. Issues impacting public trust in the
transportation system include Mopac’s extensive construction delays, SH-130’s slow
traffic growth, and the incorrect TxTag billings. Austin residents – particularly the
Millennial generation – are receptive to bike, pedestrian, and certain public transit
initiatives. In November of 2014, however, Austin voters defeated Proposition 1, which
would have approved a $1 billion bond for city transportation with $600 million going
towards a 9.5-mile light-rail line and $400 million paying for various road improvements.
The urban rail initiative was considered an unwise investment in limited transportation
funds and voters doubted the design’s ability to meet ridership estimates needed to be
profitable. In transportation and land use, Austin residents are supporting walkable
downtown and neighborhood environments, and mixed-use development has increased to
meet growing demands. Special traffic events that resonate with the Austin public include
SXSW, ACL, and University of Texas at Austin athletic events.
60
3.2.1.3 Value Proposition
Today, the average commuter is dedicating more than 40 hours and $1,000 every
year towards sitting in traffic – time that could be better spent with family, money that
could be better spent on groceries or education. Furthermore due to the unreliability of the
roadways, Texans are leaving 20-30 minutes early to just to avoid being late for a doctor’s
appointment or facing hefty childcare penalties for picking the kids up late. With 157 new
people moving to the city every day, TxDOT understands that Austin residents cannot
afford the wasted time and money and has developed a suite of mobility initiatives to
address Austin’s everyday congestion problems.
TxDOT is dedicated to increasing travel time reliability and providing Texans the
choice and convenience of quality travel options. Using technology in order to generate
reliable travel time information, TxDOT enables people to plan their day without budgeting
an extra half-hour. Furthermore, the HERO program facilitates incident clearance when
unexpected accidents occur in order to keep Texans moving. Through a combination of
new lanes, enhanced operations, and traffic management solutions, TxDOT is committed
to reducing the hours Texans spend in traffic. TxDOT is also helping people save time and
money by exploring more cost-effective options to driving alone. Whether people are going
down the street or across the state, TxDOT wants everyone to arrive safely and on-time.
Through traditional and innovative solutions, TxDOT is dedicated to creating a positive
transportation experience every day for its customers.
61
3.2.1.4 Proposed KPIs
TxDOT should consider KPIs in the following five areas: 1) Reliability, 2)
Efficiency, 3) Choice & Convenience, 4) Safety, and 5) Quality. Table 8 shows proposed
customer-oriented KPIs for the Texas Public.
Table 8: Proposed Customer-Oriented KPIs for the Texas Public
Proposed KPIs
Reliability Average Incidence Clearance Time Buffer Time per Person*
Efficiency
Annual Hours of Delay per Person Annual Cost of Delay per Person
Choice & Convenience Time and Cost Savings from Non-SOV Trips per Person Time and Cost Savings from Telecommuting per Person
Safety Fatalities and Serious Injuries in Local Metropolitan Area
Quality Percent of State Highway Pavement in Good or Better Condition
*Buffer Time: Time added to the average travel time when planning trips to ensure on-time arrival
62
3.2.2 Stakeholder Profile: State Legislator
3.2.2.1 Ask
TxDOT should ask the Texas Legislature to trust the agency to be a good steward
of public funds. TxDOT should earn the trust of the legislature by evaluating all congestion
mitigation measures for best return on investment and responding to the customer demands
for increased mobility.
3.2.2.2 Customer Experience
The Texas Legislature is a bicameral body composed of a 31-member Senate and a
150-member House of Representatives. Regular session is limited to 140 calendar days,
beginning on the second Tuesday in January of each odd-numbered year. Table 9 shows
the current makeup of the legislature:
Table 9: Texas Legislature Composition
Affiliation Senate House Republican
20
98
Democrat
11 52
TOTAL 31 150
There are three primary committees that touch on transportation issues: 1) House
Committee on Transportation, 2) House Committee on Homeland Security and Public
Safety, and 3) House Committee on Government Transparency and Operations. The House
Committee on Transportation has jurisdiction over all matters related to the Texas highway
system; commercial motor vehicles, the licensing of private passenger vehicles; the Texas
Department of Motor Vehicles, the Texas Department of Transportation, and the Texas
Transportation Commission; the regulation and control of traffic on the Texas highways;
63
rail, water, and air transportation; the regulation of metropolitan transit; and a permanent
sub-committee that focuses on the transportation needs and funding mechanisms of the
state during the next ten years. In the 84th Session, the House Committee on Transportation
heard and considered 300 bills. Therefore, it is critical for TxDOT to organize its efforts
around its highest priorities; customer-oriented KPIs can focus the Legislature on the most
significant policymaking areas.
Based upon the customer experience of their constituents, legislators will
correspondingly exhibit a difference in priorities between urban and rural districts. Urban
communities face one set of unique challenges: growing populations, traffic congestion,
parking difficulties, limited public transit, bicycle and pedestrian connectivity, land
development, and freight distribution. On the other hand, rural communities face a different
set of challenges: declining populations, aging infrastructure, eminent domain threats,
limited access to services, environmental issues, agricultural changes, and unreliable
access to market. Generally, urban districts are focusing on reducing congestion while rural
districts are emphasizing roadway maintenance.
When developing their campaigns and careers, legislators typically build a platform
around a focal point. It is important to distinguish between the legislators who are closely
involved with transportation policymaking and those who are outside of the field. When
communicating with the latter group, it becomes especially important to link transportation
to their focal point so that it is relevant to their interest in either education, health care, or
the economy for example. By using customer-oriented KPIs to build a case for
transportation policy in the legislator’s mental space, TxDOT successfully translates
abstract concepts into a familiar language to become a trusted resource for information.
64
The following sample value proposition is designed for Republican State
Representative of House District 52, which includes the cities of Round Rock, Hutto,
Taylor, and part of Georgetown – suburbs of the greater Austin metropolitan area. The
Representative was selected for his innovative nature, communications background, and
engagement in transportation funding. The KPIs proposed in Section 3.2.2.4 are reflective
of HD52’s congestion priorities and the Representative’s focus on fiscal stewardship.
The Representative of HD52 is dedicated to developing solutions to transportation
challenges by finding new ways to fund critical infrastructure while seeking innovative
ways to make the most efficient use of the infrastructure already in place (Gonzales, 2015).
In the 84th Legislative Session, he authored HB 3690 relating to the operation of
autonomous motor vehicles by TxDOT to perform construction and maintenance work.
The bill has been referred to the House Transportation Committee. In addition to his public
service, he is owner of a communications and graphic design company that offers a wide
range of services including creative design and mediation. Combined with his education in
Government, Master’s in Public Administration, and almost 20 years of experience as a
Capitol staffer, he is aware of the power visualization tools have to communicate complex
issues to his constituents. Furthermore, he serves on the Appropriations committee and
chairs the Appropriations Subcommittee on Articles VI, VII, and VIII, which addresses
transportation funding. Appointed to serve for the first time on the 84th Legislature joint
House-Senate panel, he played a key role in negotiating the state budget. Conscious of the
need to achieve balance, he sought to “meet the needs of the state and the people of this
state while also being very mindful and a good steward of taxpayer dollars” (Collier, 2015).
Using customer-oriented KPIs, TxDOT can demonstrate its ability to be a good steward of
public funds in order to earn the trust and respect of key legislators such as the
Representative of HD52.
65
3.2.2.3 Value Proposition
Today, voters of Round Rock are dedicating more than 40 hours and $1,000 every
year towards sitting in traffic – time that could be better spent with family, money that
could be better spent on childcare or housing. Furthermore due to the unreliability of the
roadways, folks in Round Rock are leaving 30-40 minutes early just to avoid being late for
dinner with friends or risking the embarrassment of missing the start of a business meeting.
Thanks to the leadership in the Legislature and voter approval of Proposition 7, TxDOT is
actively working with the Governor to reduce congestion for HD52. Although the funds
will not be available until 2018, TxDOT is preparing now by evaluating several congestion
mitigation measures for best return on investment to ensure that the agency is meeting the
state’s future mobility demands in a transparent and responsible manner.
TxDOT is dedicated to being a good steward of public funds by investing in the
right projects. Through a thoughtful and transparent evaluation of the costs and benefits,
the Texas Transportation Commission selects congestion mitigation projects that will best
serve the residents and businesses of the state. Furthermore, TxDOT can be trusted to
deliver solutions on-time and on-budget, demonstrating accountability in the use of tax
dollars. Through a combination of new lanes, enhanced operations, and traffic management
solutions, TxDOT is committed to reducing the hours people spend in traffic and
maximizing the value of goods moved in the state. In partnership with the City and other
transportation agencies, TxDOT is also encouraging voters to save time and money by
exploring more cost-effective options to driving alone. Whether people are going down the
street or across the state, TxDOT wants everyone to arrive safely and on-time. Through
traditional and innovative solutions, TxDOT is dedicated to creating a positive
transportation experience every day for its customers in Round Rock.
66
3.2.2.4 Proposed KPIs
TxDOT should consider KPIs in the following five areas: 1) Fiscal Stewardship, 2)
Mobility, 3) Accessibility, 4) Economic Competitiveness, and 5) Safety. Table 9 shows
proposed customer-oriented KPIs for the Texas State Legislature.
Table 10: Proposed Customer-Oriented KPIs for the Texas State Legislature
Proposed KPIs
Fiscal Stewardship Projects Completed On-Time and On-Budget Return on Transportation Investment
Mobility
Annual Hours of Delay per Legislative District Annual Cost of Delay per Legislative District
Accessibility Residents within a half-mile of public transit Businesses within a half-mile of public transit
Economic Vitality Value and Quantity of Goods Shipped Over Texas Roadways
Safety Fatalities and Serious Injuries in Legislative District
67
3.2.3 Stakeholder Profile: Small Business
3.2.3.1 Ask
TxDOT should ask small businesses to trust the agency to provide reliable access
to product and labor markets. TxDOT should earn the trust of small businesses by
increasing travel time reliability and improving employee mobility.
3.2.3.2 Customer Experience
The State of Texas defines small business as an independent business with fewer
than 100 employees. Small businesses create seven out of every ten new jobs and employ
over half of the country’s private sector workforce (SBA Office of Advocacy, 2015). More
than 2.2 million small businesses constitute over 98.6% of all businesses in Texas, and
include over 725,000 women and minority owned small businesses. In 2014, Texas was
ranked number one in Economy and Infrastructure and second in Technology and
Innovation by CNBC, supporting the “Texas, Wide Open for Business” campaign. While
small business growth is continuing to rise, the environment is very competitive. The
Bureau of Labor Statistics shows that 49% of establishments survive five years or more;
34% survive 10 years or more, and 26% survive 15 years or more. Seven Texas counties
reported average weekly wages above the national average of $922 per week and Texas’s
six largest cities all have cost of living indexes under the national average of 100. Small
businesses seeking to develop or expand their operations must consider a large number of
factors, including reliable and efficient access to markets, access to an educated workforce,
costs of living, regulatory processes, and investment opportunities. It is important to
demonstrate how the transportation network and services can support the profitability and
success of the small business. Customer-oriented KPIs can be used to inform the decision-
making process and foster economic growth.
68
3.2.2.3 Value Proposition
With less than 50% of establishments surviving five years or more, small businesses
are competing for access to markets, low cost of living, and an educated workforce. Success
and survival depend heavily on the reliability of the transportation network in order to
guarantee on-time and on-budget operations. Furthermore, passionate employees, who are
the backbone of the organization, are demanding higher levels of mobility and access to
urban amenities. In order to meet client expectations, TxDOT understands that small
businesses cannot afford the wasted time and money and has developed a suite of initiatives
to facilitate small business supply chain management.
TxDOT is dedicated to enabling small businesses by increasing travel time
reliability and improving employee mobility. Using technology in order to generate reliable
travel time information, TxDOT enables small businesses to plan their operations to
achieve on-time and on-budget goals. Furthermore, TxDOT works closely with local
municipalities and other transportation agencies to offer multimodal interconnectivity that
supports first- and last-mile solutions. In order to maximize the value of goods moved
through the state, TxDOT is addressing congestion by building new lanes, enhancing
operations, and implementing traffic management systems. To provide access to a quality
workforce and improve worker productivity, TxDOT is encouraging small businesses to
save time and money by exploring flextime policies. Whether a small business is deciding
where to open or when to expand operations, TxDOT wants small businesses to succeed in
supply chain management and be proud to call Texas home. Through traditional and
innovative solutions, TxDOT is dedicated to creating a positive transportation experience
every day for its small business customers.
69
3.2.3.4 Proposed KPIs
TxDOT should consider KPIs in the following three areas: 1) Supply Chain
Management, 2) Local Economic Vitality, and 3) Access to Labor.
Table 11: Proposed Customer-Oriented KPIs for Texas Small Businesses
Proposed KPIs
Supply Chain Management Annual On-Time Delivery Percentage Buffer Index per Delivery Route per Mode Annual Cost of Delay per Delivery Route per Mode
Local Economic Vitality
Cost of Living Index Accessibility of business to multimodal transportation facilities
Access to Educated Workforce Annual Hours of Delay for Employees Annual Cost of Delay for Employees Time and Cost Savings from Implementing Flextime Policies
70
3.3 ANALYSIS
In an effort to validate the usefulness of the proposed stakeholder profiles, feedback
from each of the following stakeholder groups was gathered through a series of key person
interviews.
3.3.1 TxDOT Leadership
TxDOT leadership was asked to provide their reactions to the stakeholder profiles,
commenting on their usefulness in engaging the various stakeholder groups, offering
suggestions for improving their content, and comparing their format with the existing
reporting techniques. A Commissioner and Chief Strategy and Innovation Officer shared
their thoughts on the development of key performance indicators. The Commissioner
appreciated the focus of the KPIs over a multitude of metrics and saw the value in engaging
directly with the stakeholder. He also recognized the need to respond to the political context
and recommended to do “political reality checks” so the KPIs would resonate with the
legislature in the way that they were intended. The Chief Strategy and Innovation Officer
also appreciated the concise format that summarized the customer experience. He
recommended that additional customer segments should be studied: urban and rural
residents, technology-oriented and non-technology-oriented legislators, and major
corporations and small businesses.
3.3.2 Public
The Chief Communications and Marketing Officer and Director from the TxDOT
Communications Division offered their insights in communicating the agency’s message
to the public. From their experience, using narratives have been fundamental in painting a
picture that citizens can relate to and find meaningful. Distinguishing between urban and
71
rural needs was also identified as an important factor. Regarding technology, it is important
to relate it to the everyday context and eliminate all technical jargon.
3.3.3 State Legislature
The Director and Assistant Director from the State Legislative Affairs Section of
the TxDOT Government Affairs Office provided feedback on effective techniques for
communicating with the state legislature. The Director noted that there are a limited
number of legislators who are familiar with or receptive to technology, often making it
difficult to propose more innovative solutions. The Director also raised the point that the
traditional “highways and bridges” DOT model is still considered the paradigm for the
agency by many. In order to overcome these obstacles, it was suggested that technology
constitute part of a more comprehensive solution to complement and enhance conventional
infrastructure projects.
3.3.4 Businesses
The Director of Research and Technology Implementation who formerly worked
in supply chain management discussed the importance to focus on the business perspective
of profitability. By concentrating on how transportation impacts the margins, TxDOT can
become a trusted business partner. Furthermore, it was pointed out that many businesses
desire to locate within five miles of the interstate for access to markets. The impact of
traffic congestion on worker productivity was also raised and the opportunity to use KPIs
to support telecommuting policies was considered viable.
72
Chapter 4: Recommendations
Based upon the key person interviews described previously, the following
synthesizes their feedback and summarizes five best practices for State DOTs seeking to
develop a customer-oriented KPI framework.
4.1 LESS IS MORE
It is more effective to focus on the vital few. While State DOTs may measure an
array of things, it is important to recognize which metrics are driving strategic intent and
are relevant to the stakeholders of interest. If there are too many KPIs, State DOT
executives run the risk of overwhelming the stakeholder, diluting the message, or losing
stakeholder interest altogether. Confusion or apathy may arise instead of having the desired
effect of garnering stakeholder support.
4.2 DESIGN FROM THE OUTSIDE IN
Understanding the customer experience is key to developing a customer-oriented
KPI framework. Considering transportation from the stakeholder’s perspective ensures that
the agency’s performance message resonates with the audience. Furthermore, KPIs can be
used to manage or exceed customer expectations. Rather than relying upon the traditional
myopic KPIs, State DOTs can use customer-oriented KPIs to connect and engage with
stakeholders. By nurturing stakeholder relationships, State DOTs can progressively build
credibility, improve their public image, and secure critical investments.
73
4.3 POWER OF PERSONALIZATION
Stakeholder segmentation and mapping are critical tools for differentiating
stakeholder priorities and clarifying their individual relationships to the State DOT. Each
stakeholder group may be viewed as an audience. Developing customized communications
strategies tailored to specific interests ensures that each stakeholder’s contribution is
valued. Although this requires additional time, benefits of personalized messages include
constructive feedback, emotional connections, effective education, diverse perspectives,
proactive problem solving, improved decision making, and engaged stakeholder
development.
4.4 EMPOWER STAKEHOLDERS
Through the use of KPIs, State DOTs can translate data into information and
actionable items. Armed with the right information, a good KPI can enable a stakeholder
to make a well-informed decision in a timely manner. When coupled with incentives,
influential KPIs can be used to reinforce positive changes in behavior or perception. If
progress is demonstrated, trust is established and a stakeholder may become an advocate
for the agency’s activities. By leveraging brand advocates, State DOTs can accrue more
influence, reach, and stakeholder engagement.
4.5 EVOLVE OR RISK BEING LEFT BEHIND
In an environment of constant change, State DOTs need to be prepared to respond
to multiple forces of change. By institutionalizing the development process rather than the
product, State DOTs can establish a consistent methodology for updating their KPI
frameworks. By taking a proactive stance, State DOTs can anticipate the needs of
stakeholders and may even avoid additional oversight by including inherent transparency
and accountability mechanisms.
74
Chapter 5: Conclusion
With the erosion of public trust, increasing shortfalls in funding, and continuous
forces of disruption, State DOTs need to evolve their performance management
frameworks to better align the agency’s mission with stakeholder priorities. Using
stakeholder segmentation and mapping techniques, customized stakeholder profiles were
developed and KPIs were proposed to support the Texas Department of Transportation
leadership engage with the public, state legislators, and businesses. Feedback from the
stakeholder groups recognized the value of the proposal for its clarity, accessibility, and
relevance. Although a customer-oriented KPI framework requires more forethought than a
traditional DOT-centric approach, the rewards are advantageous. The customer-oriented
KPI development approach offers a successful process that is transferable to other State
DOTs. By understanding the context that surrounds performance management, State DOTs
can craft customized stakeholder profiles to build trust, garner support, and encourage
investment in their own states.
Areas of future research include refinement and expansion of the stakeholder
profiles to include more detailed customer segmentation, such as urban and rural
customers, Democrat and Republican legislators, small and large businesses. Additional
feedback could be gathered through stakeholder focus groups in order to improve
messaging. Other state DOTs could also be solicited for input regarding the transferability
of the customer-oriented approach. It is important to explore data integration and
interagency collaboration policies to identify areas in which TxDOT or the state could
serve as the central authority on information. Overall, this thesis serves as a good starting
point for state DOTs seeking to implement a customer-oriented KPI framework to improve
stakeholder engagement.
75
Appendix A: State DOT Scorecard Examples
Figure 14: FDOT 2014 Performance Highlights
(Source: http://www.dot.state.fl.us/planning/performance/2014/2014SISHighlights-print.pdf)
76
Figure 15A: CalTrans 2015 Q2 Mile Markers
(Source: http://www.dot.ca.gov/MileMarker/2015-2/index.html)
77
Figure 15B: CalTrans 2015 Q2 Mile Markers
(Source: http://www.dot.ca.gov/MileMarker/2015-2/index.html)
78
Figure 16A: MDOT 2015 MiScorecard Performance Summary
(Source: https://www.michigan.gov/documents/mdot/MDOT_Scorecard11-14-11_01-19-12_374118_7.pdf)
79
Figure 16B: MDOT 2015 MiScorecard Performance Summary
(Source: https://www.michigan.gov/documents/mdot/MDOT_Scorecard11-14-11_01-19-12_374118_7.pdf)
80
Figure 17A: MnDOT 2014 Transportation Results Scorecard
(Source: http://www.dot.state.mn.us/measures/pdf/2014PerformanceReport.pdf)
81
Figure 17B: MnDOT 2014 Transportation Results Scorecard
(Source: http://www.dot.state.mn.us/measures/pdf/2014PerformanceReport.pdf)
82
Figure 18A: ODOT 2015 Key Performance Measures
(Source: http://www.oregon.gov/ODOT/CS/PERFORMANCE/OnePagers/KPM%20Rollup.pdf)
Data as of Aug 25, 2014
Policy goal/Key Performance Measure
PreviousReporting period
CurrentReporting period
Goal
Goal met (w/in 10%)
Progress Made
Comments
Safety – Engineering, educating, and enforcing a safe transportation system
Number of traffic fatalities per 100 million vehicle miles traveled (VMT) in Oregon 1.02 .93 .90
Since 1999, Oregon's fatality rates have been consistently below the national average (Currently 1.14)
Number of traffic injuries per 100 million vehicle miles traveled (VMT) in Oregon
108.78 98.38 70 ‐‐‐
A system change in 2011 resulted inan increase of over 15% for injury and property damage data making it into the crash data file.
Percent of traffic fatalities that involved alcohol
37% 41% 35% ‐‐‐
According to 2013 NHTSA statistics, Oregon is #7 in the nation for lowest alcohol‐related fatalities.
Percentage of all vehicle occupants using safety belts 98% 98% 97%
In 214, Oregon’s observed safety belt use rate was reportedly 97.75%. The national average is 87%.
Number of large truck (commercial motor vehicle) at fault crashes per million vehicle miles traveled (VMT) in Oregon
.42 .44 .37 ‐‐‐
In 2013, Oregon ranked #1 in thenation, as inspectors placed 13.2 percent of drivers out of service for critical safety violations. The national rate is 5.5 percent. Most truck‐at‐fault crashes are caused by speeding, tailgating, or changing lanes unsafely.
Number of highway‐railroad at grade incidents 9 14 11 ‐‐‐
Oregon has been in or near the top 20 states for least number of motor vehicle incidents at public crossings.
Number of train derailments caused by human error, track, or equipment
20 23 25
Some increase may be attributed toincreased train volumes as the industry recovers from the recession.
Percent of public satisfied with transportation safety
83% 81% 74%
For the last three consecutive years,public opinion survey shows that over 80% of Oregon travelers feel safe on our roads.
Employee disabling (time loss ) claims rate per 100 ODOT employees
2.1 2.1 1.7 ‐‐‐
A comprehensive review of operations where workers are near moving equipment is underway. Changes in procedure and training will be implemented in 2015.
Mobility and Economic Vitality – Keeping people and the economy movingHours of travel delay per capita per year in urban areas 24 24 22
This statistic reflects Portland, Salem& Eugene metropolitan areas.
Average number of transit rides per each elderly and disabled Oregonian annually 19 20 24 ‐‐‐
Increases in the population of olderadults continue increase demands.
Number of rail service passengers 215,096 210,901 208,590
Since 2004, passenger rail ridership has increased by more than 92,000.
Percent of Oregon communities of 2.5K+ with intercity bus or rail passenger service 94% 95% 95%
Intercity bus connections remain steady.
Percent of Oregonians who do not commute alone to work during peak hours** 33% 30% 35% ‐‐‐
Education and awareness of alternatives to commuting alone can affect change.
Percent of lane blocking crashes cleared within 90 minutes
80% 81% 100% ‐‐‐
Clearing lanes is occasionally delayed due to accident investigations. Traffic incidents account for about 25% of the congestion on the highway system.
Oregon Department of Transportation Key Performance Measures Continuously updated as performance is reported
N
N
Y
Y
Y
N
N
N
Y
N
N
83
Figure 18B: ODOT 2015 Key Performance Measures
(Source: http://www.oregon.gov/ODOT/CS/PERFORMANCE/OnePagers/KPM%20Rollup.pdf)
Preservation – Preserving and maintaining the transportation infrastructure Percent of pavement mi les rated “fair or better” out of total miles on ODOT highway system
87% 87% 87%
ODOT’s pavement programs resurface less than one‐half the need and higher co st projects can’t be completed with available funds.
Percent of State highway bridges that are not distressed
78% 78% 78%
After 2017, bridge conditions willdecline exponentially. To maintain current bridge conditions through 2030, funding to state bridges would need to be tripled.
Sustainability – Sustaining the environment and communities Number of priority culverts that need work to improve fish passage 190 190 189
In the next 4 years, approximately1/2 its annual budget will fund storm water runoff retrofit projects.
Percent of urban state highways with bike lanes and sidewalks 43% 38% 48% ‐‐‐
ODOT is making strategic investments where communities have identified the greatest need.
Percent of ODOT sustainability performance measures maintaining steady or trending positive
93% 93% 90%
As with most new measures, additional data will be needed over time to better understand facility‐level practices and trends.
Stewardship – Maximizing value from transportation investments Number of jobs sustained as a result of annual construction expenditures
11,700 10,138 10,600
The 2013 model update calculatedthe 2013 fiscal year jobs impact factor at 10.5 jobs per $1M. The fiscal year 2015 jobs impact factor decreased to 10.1 jobs per $1M, due to inflation.
Percent of projects going to construction phase within 90 days of target date 96% 99% 90%
In 2014 ODOT continued to exceedthe 90% goal with 99% being on time.
Percent of projects with construction phase completed within 90 days of original date 76% 88% 80%
2014 results (88%) surpassed thegoal of 80% the first time since measurements started.
Percent of original construction authorization spent
106% 100% 99%
On average, overall project construction expenses are within 100% of their original authorization over the last 13 years.
Percent of ODOT contract dollars awarded to Disadvantaged Business Enterprise (DBE) businesses
9.0% 8.7% 16.95% ‐‐‐
The ODOT DBE Program is in the tophalf of the state reviews (45 to date).
Percent of ODOT customers who are satisfied with services
90% 89.5% 90%
Variations in results between 2006and 2012 are not statistically significant and have been near the target of 90%.
DMV Field office wait times (minutes) 15 mins 18 mins 11 mins ‐‐‐
ODOT is implementing a new measure to better reflecting the average customer experience.
DMV Phone queue wait times (seconds)
41 sec 45 sec 45 sec
ODOT continues to focus on providing consistent telephone answer time and cost‐effective service from three contact centers.
Vehicle Title transaction turnaround time (days)
24 days 24 days 21 days ‐‐‐
Agency is developing business processes to reduce the title wait time as transaction volumes increase.
Y
N
Y
Y
N
N
Y N Regression since last report Progress since last report Progress remains unchanged since last report
N
N
84
Figure 19A: VDOT 2015 Q4 Quarterly Report
(Source: http://www.virginiadot.org/VDOT/Info/asset_upload_file723_65707.pdf)
85
Figure 19B: VDOT 2015 Q4 Quarterly Report
(Source: http://www.virginiadot.org/VDOT/Info/asset_upload_file723_65707.pdf)
86
Appendix B: TxDOT Stakeholder Profile Examples
Figure 20: Proposed TxDOT Stakeholder Profile for General Public
90
Bibliography
Transportation for America. State Transportation Funding. 2015. Accessed: http://t4america.org/maps-tools/state-transportation-funding/#top
Turnbull. Katherine F. Aligning Data Systems to Communicate with Decision Makers: A Peer Exchange. Transportation Research Board of the National Academies. Washington, DC. 2014.
Cambridge Systematics. A Guidebook for Performance-Based Transportation Planning. NCHRP Report 446. Transportation Research Board of the National Academies. Washington, DC. 2000.
Molenaar, Keith R., Navarro, Desiderio. Key Performance Indicators in Highway Design and Construction. Transportation Research Record: Journal of the Transportation Research Board, No. 2228. Transportation Research Board of the National Academies. Washington, DC. 2011, pp. 51-58.
Federal Highway Administration (FHWA). Performance-Based Planning: Minnesota Department of Transportation. 2011. Accessed: http://www.fhwa.dot.gov/planning/performance_based_planning/case_studies/minnesota/
National Association of State Chief Information Officers (NASCIO). NASCIO Nomination Category: Data, Information, and Knowledge Management; Project Title: NC Department of Transportation Pre-Construction Project Status Dashboard. 2007. Accessed: http://www.nascio.org/portals/0/awards/nominations2007/2007/2007NC3-DOT.NASCIO%20application.pdf
Chisholm-Smith, Gwen. State DOT Public Transportation Performance Measures: State of the Practice and Future Needs. National Cooperative Highway Research Program. Transportation Research Board of the National Academies. Washington, DC. September 2011.
Virginia Department of Rail and Public Transportation (DRPT). Virginia Transit Performance Report (FY 2002 – FY 2006). 2007. Accessed: http://www.drpt.virginia.gov/media/1218/fy2002-2006-transit-performance-report-final.pdf
Minnesota Department of Transportation (MnDOT). 2014 Annual Minnesota Transportation Performance Report. 2014. Accessed: http://www.dot.state.mn.us/measures/
Oregon. Key Performance Measure Summaries. 2014. Accessed: http://www.oregon.gov/ODOT/CS/PERFORMANCE/Pages/PerformanceMeasureSummaries.aspx#Overview
91
Maryland Department of Budget and Management. Managing for Results in Maryland State Government. 1996. Accessed: http://www.dbm.maryland.gov/Pages/ManagingResultsMaryland.aspx
Maryland Department of Information Technology. Open Data Portal. 2015. Accessed: https://data.maryland.gov/
Google Now. What Is It. 2015. Accessed: https://www.google.com/landing/now/
Metropia. About. 2015. Accessed: http://www.metropia.com/about-metropia
Metropia. Metropia and the Central Texas Regional Mobility Authority Kick Off App-Based Traffic Solution. Austin, TX. May 6, 2015. Accessed: http://www.metropia.com/blog/metropia-and-central-texas-regional-mobility-authority-kick-app-based-traffic-solution
Ad Council. Buzzed Driving Case Study: Making a Difference, Drunk Driving Prevention. 2011. Accessed: http://www.adcouncil.org/Impact/Case-Studies/Buzzed-Driving-Prevention-2011
Padgette, Robert. Effective Organization of Performance Measurement. NCHRP 8-36, Task 47. Requested by American Association of State Highway and Transportation Officials (AASHTO). 2006. Accessed: http://statewideplanning.org/wp-content/uploads/19_PM-Organization.pdf
Missouri Department of Transportation (MoDOT). MoDOT’s Approach to Program Management. 2007. Accessed: http://www.modot.org/business/documents/Programmanagementsupportdocument.pdf
Rall, Jaime; Wheet, Alice; Farber, Nicholas J.; Reed, James B. Transportation Governance and Finance: A 50-State Review of State Legislatures and Departments of Transportation. National Conference of State Legislatures and AASHTO Center for Excellence in Project Finance. 2011. Accessed: http://www.mbta.com/uploadedfiles/About_the_T/Panel/TransportationGovernanceandFinance-50StateReview.pdf
Bremmer, Daniela; Cotton, Keith C.; Hamilton, Brooke. Emerging Performance Measurement Responses to Changing Political Pressures at State DOTs: A Practitioner’s Perspective. November 2004.
Federal Highway Administration (FHWA). MAP-21: Moving Ahead for Progress in the 21st Century. 2013. Accessed: https://www.fhwa.dot.gov/map21/factsheets/pm.cfm
Amekudzi, Adjo A.; Meyer, Michael. Best Practices in Selecting Performance Measures and Standards for Effective Asset Management. June 2011.
RideScout. About. 2015. Accessed: http://www.ridescoutapp.com/about-us/
92
Stein, Kathleen E. and Sloane, Robert K. Managing Change in State Departments of Transportation. Scan 4 of 8: Innovations in DOT Communications, Image, and Positioning. NCHRP Web Document 39, Transportation Research Board, National Research Council. May 2001. Accessed: http://onlinepubs.trb.org/onlinepubs/nchrp/nchrp_w39-4.pdf
Washington State Department of Transportation (WSDOT). Gray Notebook. Edition 58. June 30, 2015. Accessed: http://wsdot.wa.gov/publications/fulltext/graynotebook/Jun15.pdf
Van Wart, Montgomery. Public-Sector Leadership Theory: An Assessment. Public Administration Review. Vol. 63, No. 2. March/April 2003. Accessed: http://classes.maxwell.syr.edu/ppa753/PDF%20Files/Van%20Wart%202003.pdf
Kase, Philip “PK.” Interview. 2015.
Litman, Todd. Evaluating Transportation Economic Development Impacts: Understanding How Transport Policy and Planning Decisions Affect Employment, Incomes, Productivity, Competitiveness, Property Values, and Tax Revenues. 2010. Accessed: http://www.vtpi.org/econ_dev.pdf
Chisholm-Smith, Gwen. State Department of Transportation Role in the Implementation of Transportation Demand Management Programs. Research Results Digest 348. National Cooperative Highway Research Program. Transportation Research Board of the National Academies. Washington, DC. July 2010.
National Household Travel Survey. 2009. Accessed: http://nhts.ornl.gov/2009/pub/stt.pdf
FHWA. Virginia’s Dashboard: Driving VDOT Success. 2014. Accessed: https://www.fhwa.dot.gov/tpm/noteworthy/hif13014.pdf
TransTech Management. Strategic Performance Measures for State Departments of Transportation. 2003.
PricewaterhouseCoopers LLP. Guide to Key Performance Indicators: Communicating the Measures that Matter. 2007.
Rozner, Steve. Developing Key Performance Indicators: A Toolkit for Health Sector Managers. December 2013. Accessed: https://www.hfgproject.org/wp-content/uploads/2014/10/03-Developing-Key-Performance-Indicators.pdf
Michigan. Open Michigan: MI Dashboard. 2015. Accessed: https://midashboard.michigan.gov/
Office of the Auditor General of British Columbia. Guide for Developing Relevant Key Performance Indicators for Public Sector Reporting. 2010. Accessed: http://www.bcauditor.com/sites/default/files/publications/2010/report_10/report/OAGBC_KPI_2010_updated.pdf
93
TxDOT. Project Tracker. 2015. Accessed: http://www.txdot.gov/inside-txdot/projects/project-tracker.html
Cisco. Utah Department of Transportation Creates CommuterLink to Help Utah Residents “Know Before You Go.” 2004. Accessed: https://www.cisco.com/web/strategy/docs/trans/utah_dept_transportation.pdf
Govloop. The Govloop Guide: Big Data, Examining the Big Data Frontier, Innovations that Matter. Accessed: http://img.en25.com/Web/GovDeliveryInc/%7B2d226ca5-be34-4b05-9134-65f7925fd9fb%7D_GovLoop_Innovations_That_Matter_ Big_Data_Spring2014.pdf
American Association of State Highway and Transportation Officials (AASHTO). Communications Brief: State Departments of Transportation Lead the Way Using New Media. February 2010. Accessed: http://communications.transportation.org/ Documents/New_Media_Research_Brief%202010.pdf
Florida Transportation Commission. Performance and Production Review of the Florida Department of Transportation: FY 2013/2014. 2014. Accessed: http://www.ftc.state.fl.us/documents/reports/PPR/FY13-14FinalCombinedReport.pdf
Florida Department of Transportation (FDOT). 2014 Performance Highlights. 2014. Accessed: http://www.dot.state.fl.us/planning/performance/2014/2014PerformanceHighlights-print.pdf
Virginia Department of Transportation (VDOT). Quarterly Report: Fourth Quarter FY 2015. September 2015. Accessed: http://www.virginiadot.org/VDOT/Info/asset_upload_file723_65707.pdf
Utah Department of Transportation (UDOT). 2015 Strategic Direction. 2015. Accessed: http://www.udot.utah.gov/main/uconowner.gf?n=19974707633468335
Nevada Department of Transportation (NDOT). 2014 Performance Management Report. 2014. Accessed: https://www.nevadadot.com/uploadedFiles/NDOT/About_NDOT/NDOT_Divisions/Planning/Performance_Analysis/2014_Performance_Management_Report.pdf
Missouri Department of Transportation (MoDOT). Tracker. October 2015. Accessed: http://www.modot.org/about/documents/Oct2015Tracker.pdf
Caltrans. Strategic Management Plan: 2015-2020. 2015. Accessed: http://www.dot.ca.gov/perf/library/pdf/Caltrans_Strategic_Mgmt_Plan_033015.pdf
Sunset Advisory Commission. Summary of Sunset Legislation – 82nd Legislature. July 2011. Accessed:
94
https://www.sunset.texas.gov/public/uploads/files/reports/Department%20of%20Transportation%20SOL%202011%2082%20Leg.pdf
Texas Department of Transportation (TxDOT). 84th Legislature 2015: Mapping for the Future, Summary of Enacted Legislation. 2015. Accessed: https://ftp.dot.state.tx.us/pub/txdot-info/sla/84th-legislative-summary.pdf
Texas Government Insider. Barton Leaving TxDOT after Nearly Three-Decade Career with Agency: SB 20 Provision Could Lead to Exodus of Veterans throughout State Government. Volume 13, Issue 30. August 7, 2015. Accessed: http://www.spartnerships.com/newsletter/2015/tgi-8-7-15/texas-government-insider-8-7-15.html
Potter, Lloyd B.; Hoque, Nazrul. Office of the State Demographer. Texas Population Projections, 2010-2050. November 2014. Accessed: http://osd.texas.gov/Resources/Publications/2014/2014-11_ProjectionBrief.pdf
Batheja, Aman. Some Worry Traffic Issues Will Stem Texas’s Growth. The Texas Tribune. July 9, 2014. Accessed: http://www.texastribune.org/2014/07/09/traffic-forecasts-worsen-fears-miracle-slowing/
International Transportation Forum. What the Rise of the Sharing Economy Means for Transport. April 2015. Accessed: http://www.fiaregion1.com/en/fia_region_1/news/what-the-rise-of-the-sharing-economy-means-for-transport.htm
Batheja, Aman. TxDOT Refunding $1.7 Million for Botched Toll Bills. The Texas Tribune. March 17, 2015. Accessed: http://www.texastribune.org/2015/03/17/txdot-refunding-17-million-erroneous-toll-bills/
Strategy Analytics. 33 Billion Internet Devices by 2020: Four Connected Devices for Every Person in World. 2014. Accessed: https://www4.strategyanalytics.com/default.aspx?mod=pressreleaseviewer&a0=5609
Gonzales, Larry. On the Issues. 2015. Accessed: http://www.larrygonzales.com/meet-larry/
Collier, Kiah. Two Williamson County Republicans Will Help Negotiate State Budget. April 25, 2015. Accessed: http://www.larrygonzales.com/2015/04/two-williamson-county-republicans-will-help-negotiate-state-budget/
Stocker, Mike. The Definition of Omni-Channel Marketing – Plus 7 Tips. April 1, 2014. Accessed: http://blog.marketo.com/2014/04/the-definition-of-omni-channel-marketing-plus-7-tips.html