1

Data Visualization for Strategic Decision Making

Abstract

Organizations large and small continuously strive to improve strategic decision-making. Strategic decision-making (distinguished from operational decision making) involves making substantial investments of resources over long periods of time before results are evident. These decisions are made using quantitative and qualitative information - experience, intuition, and subjective assessment. These are people decisions, not decisions made by machines.

So, how can data interfaces be designed to support these most critical decisions of the organization?

This case study looks at approaches taken with one R&D (research and development) client to address their key strategic decisions: whether to move research efforts into the next stage of development or cancel the project. We will discuss how we’ve used web-based interface technologies to create visual metaphors for data including: visualizing time, collaborating, and modeling scenarios. We’ll also demonstrate approaches to embedding more abstract constructs like decision theory, statistical analysis, and competitive advantage into these interfaces.

Angela Shen-Hsieh Visual I/O 285 Washington St Somerville, MA 02143 angela@visual-io.com

Mark Schindler Visual I/O 285 Washington St Somerville, MA 02143 mschindler@visual-io.com

Permission to make digital or print copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage; the copyright notice, the full citation of the publication, and its date appear; and notice is given that copying is by permission of AIGA. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. This material is co-published with permission in the Association for Computing Machinery Digital Library.

©2002 American Institute of Graphic ArtsExperience Design Case Study Archive

2

Keywords

Data Visualization, Information Visualization, Information Architecture, Decision Science, Strategic Decision Making, Enterprise Information Systems (EIS), Enterprise Applications, Enterprise Resource Planning (ERP), Decision Support, Executive Dashboards.

Industry/category

Healthcare, Pharmaceuticals, Research and Development, Management Consulting, Finance.

Project statement

In 1999, Visual I/O (formerly Schindler+Associates) began this project.

This case study highlights two efforts in a three-year on-going engagement with a pharmaceutical research and development (R&D) organization. We have been working with this R&D company to improve their strategic decision-making through the design of interactive information interfaces. By combining visual �pictures� of data with a deep understanding of individual and organizational decision processes, we hope to generate new insight to users and decision makers.

This case study will focus primarily on the design and development of the data visualizations and their interactivity. Less attention will be given to the development of the applications and issues of data integration. This case study will discuss an early system designed in 1999 (our first work using visualization techniques to address strategic decision making and what we will call the �first generation tool� for the purposes of this case study) and a design from

a current undertaking. We hope to give a sense of the issues engaged when designing interfaces for enterprise (business) applications.

The first generation tool will explain the business problems our work addresses and frame our approach. We also hope it will provide some background about how we got involved with this type of work - where both the client organization and we were, when this project was started.

The second example (which we will call the �current design iteration�) incorporates more complexity in the issues, users, and functions it addresses. It also integrates current thinking on information management, web and database technologies, as well as the evolution of our approach to techniques and metaphors for visualizing large and complex data sets.

The first generation tool:

Sometime before this project was undertaken, it was rumored that tens of millions of dollars were misplaced in this pharmaceutical company�s budgeting cycle. The money was not taken, it was apparently �lost� by the financial tracking system. The organization had also had a number of drugs fail in late stage clinical trials (a huge financial loss as well as an embarrassment) and was being pressed by the parent company towards more operational accountability, faster and better decision making - identifying bad projects and cutting them earlier. Understandably, this organization was ripe for some new perspective on finance and portfolio management. A management consulting firm offered them a model which approached budgeting in a completely different way: where this client had always organized their budgets by �functional area�, (e.g., chemistry, toxicology, clinical trials) they now wanted

3

to see financial data across each �franchise� or therapeutic area (e.g., cholesterol, heart, or asthma medications) and across each project in development. This is akin to knowing how much of your grocery money you spend in the dairy section, in the produce aisle, in canned goods and at the meat counter, and then being asked how much a particular meal cost. The president of the company, a critical sponsor of this effort, felt he had no comprehensive �picture�. He had no way to �see� the whole operation - the risk and opportunity areas-- and current systems offered no way to evaluate the business� key strategic assets - its portfolio of R&D projects. Science had always been divorced from strategy; now the organization�s approach to and perspective on �stop or go� decisions needed to be broadened.

Figure 1. Example of a 'Before' view, a sample client report

We were brought in to design and develop this system by the consultancy. This firm, which consults to healthcare organizations, provided the domain knowledge essential to organizing the information for this application.

We were to deliver a working web-based prototype of the interface.

Figure 2.Example of an �after� view, Visual I/O approach to a similar data set

Current design iteration:

In early 2001, the CIO (Chief Information Officer) of the R&D organization was challenged to show the value of the Information Technology (IT) organization to his internal business partners. Like many IT organizations, this one had become focused around maintaining infrastructure: servers, email, applications, etc., and was not designed or motivated around adding value to the business. The CIO needed a way to demonstrate his thinking; he needed examples of the kind of applications that he felt could be built that would target the most critical needs of the organization and its most key decisions.

In the initial phase of this project (what this case study will describe), Visual I/O was asked to develop these examples which would be used as an internal sales tool by IT senior management. We were to use one specific

The Value of Better, Faster

Decision Making: Over a

period of 6-12 years, a

pharmaceutical company

spends between $350M-$600M

to bring a single drug to

market. Typically, 95% of the

total development costs will be

spent by the time the drug

completes clinical trials (testing

in humans.)

4

phase in the drug development process - its data, issues, and points of pain - as a testing ground for these prototype application interfaces. Our client was also very interested in injecting decision science concepts and methodologies into information systems and we were teamed with a statistical analyst from an enterprise application development firm. We were to analyze the data currently available to our target audience, evaluate concepts and methodologies from decision science theory, and integrated these with our own techniques for visualization data and collaborative decision making.

We would then take these prototype interfaces to key sponsors to generate buy-in for the project.

Project participants

Visual I/O:

Mark Abramson, Project Management

Chris Brewer, Data Architecture

Chris Edgett, Flash Programming

Kate Follen, Graphic Design

Tim Kelley, HTML Programming

Chris Jenks, Java Scripting

Scott Listfield, Graphic Design

Doug Marttila, Flash Programming

Denise McFadden, Research

Gibby Miller, HTML Programming

John Nakazawa, Project Management

Mark Schindler, Information Design and Project Lead

Angela Shen-Hsieh, Information Design and Project Lead

ISO Healthcare Group (ISOHCG):

Marvin Eng, Consulting

Scott Myers, Consulting Lead

Andy Robbins, Research

Software Associates International (SAI):

Michael Barnes, Analyst

Nicole Gleason, Research

SPS Infoquest:

Franky DeCooman, Data Warehouse Integration Lead

Christine Vander Vorst, Technical Project Manager

Gitek:

Corné DeKoning, Quality Assurance Lead

Project dates and duration

February 1999- December 2001

Design and development process

We first designed and built a proof-of-concept system which the finance department used for one budgeting cycle in 1999. This system used a single visual diagram which detailed the key events for each compound in development and the cost associated with each of these events.

5

Figure 3. Detail of Milestone View showing key events and

costs anticipated over life of the project

Given the success of and the interest generated by the proof-of-concept, senior management envisioned a system that would provide this view not just to Finance but one which could serve related views to multiple levels of the organization. The visual metaphor could �roll up� (aggregate) data to senior management and also �roll down� (detail) information for therapeutic area (product line) leads, project teams, and functional area leads. Because the same tool would be used across the company, accessibility, interactivity and exposure to the model would be increased. This would create a common metaphor visible to people at many levels - embedding organizational goals and metrics and building alignment across the organization.

We quickly realized that understanding not only the pharmaceuticals business but also this particular company was critical to producing accurate data visualizations (as opposed to the design of a more traditional reporting tool.) Fortunately, we had a great resource in our consulting partners. Some examples of company-specific questions about strategic decision making include:

> How does the company budget? (e.g., by department, product line, project)

> How does the company balance their portfolio with respect to:

! marketing data

! resource planning

! risk (and risk can breakdown into things like: patent protection, $, safety, etc.)

! distributions across target disease areas (e.g., cancer, diabetes, chronic pain)

! ratios of new to existing formulations

! scientific considerations (e.g., can the formulation be ingested rather than injected)

> What is the company�s approach to valuation? What valuation methods do they favor? What metrics do they prioritize? Are there better methods available?

> How are decision makers incentivized in their roles?

We were fortunate that both the client organization and their consulting partners gave us great latitude to explore a lot of ideas through drawing and sketching and did not put limits on the way in which we worked with them. Because neither we nor the consulting firm (acting as project managers) had experience with application development projects this extensive, we were not wedded to a more standardized software development process. We believe now that this was of great benefit to the end-product. Being new to this domain, we focused our requirements gathering around understanding the decisions people make and the processes and data around those decisions: we focused more on need and less on want because we needed to understand the context of the information for

How does Visual I/O define

a �Working Prototype�?: a

�working prototype� is an

interface using a static data

source. We build it in the

technology in which it is to be

deployed (MS Access, HTML,

Flash, Director, etc.). After

client approval and design

iterations, the prototype is

connected to dynamic data

sources.

6

ourselves. The client organization also did not require us to deliver a formal user requirements specification (URS) before granting approval for the project (as would normally be the case when developing a traditional reporting tool) and this enabled us to work in an unusually iterative way: conducting user interviews, designing, and developing the URS simultaneously over a period of months. We had monthly design reviews with the client and frequent brainstorms about the �concept� and the functionality required of each module of the system. Then intensive design sessions would follow to bring the visual metaphor in line with the expectations for the functionality, the use, and the overall organizational goals.

The URS was completed at approximately the three month mark and the working prototype was delivered about four weeks after that. This more flexible process also gave us the means to change users� preconceptions of what an information system could be. In this way, this system moved away from being a reporting tool built to provide reports and printouts and became an interactive way to navigate through information. To our thinking, a �visualization� approach to dynamic data should naturally create a very fluid way of working, where clicking points of data logically move the user to deeper or broader contexts for that piece of data, and working with information becomes a �screen-based� (rather than paper-based) activity.

The working prototype was then delivered to an application development firm who built the data warehouse and programmed to the web-based interface.

Current design iteration:

As stated, we were teamed with a statistical analyst and began to wade through mountains of research on decision theory and decision science. The areas of decision thinking we were most interested in were from the social sciences, statistics, and industry and business research. We felt one promising approach was to analyze historical information to look for data relationships, similarities, outliers, that might reveal early indicators for project failure or project success. At the same time we began to map all the data available to decision makers in our pilot area to processes, milestones, decision points, etc. We were particularly interested in the scientific data (laboratory test results, instrument readings, etc.) and in the qualitative data that the decision-making teams used. We specifically tagged data that was currently being captured electronically because we wanted to avoid introducing a data entry and maintenance burden.

This groundwork involved a tremendous amount of summarizing and detailing of the information available to us. Many matrices were created to cull relationships, trends, and patterns and to distill information into key, resonant categories. We knew their process often involved gathering expert opinions from around the world but we were also trying to get a handle on how exactly decisions came about. For example, we would try to extrapolate what kind of discussion a committee had had about a compound by looking at the documents they reviewed in the meeting.

One of the big challenges was getting a feel for how scientific the decision-making criteria really were and how much marketing, competition, cost, and other factors influenced their thinking. The other challenge was understanding the scientific data and whether it could be used in the ways we anticipated. Although we

Where does the data come

from? Large companies have a

tremendous amount of historical

data, often stored in some

electronic form. Frequently

though, it is not in the right form

for the purposes at hand (for

example, costs were captured by

project phase, not month) and so

a �business logic� layer is needed.

Data quality, data availability,

and how data will get into the

system are always tricky issues

for these types of applications.

7

did have access to content specialists within IT, because we were enlisted to develop a presentation for IT to take to the business partners working in other areas, we were not able to consult with the scientists or decision makers in this phase of the work. Still, we needed to get a very fast but fairly comprehensive understanding of their data.

We took our initial findings to our clients who suggested that we first develop non-pharmaceutical examples. The concern was that the principles behind the approaches to the data and to the interface might become overshadowed by discrepancies and inaccuracies in any sample data that we would use in our prototypes: a �lay� example might better convey the value of the approach to the scientists and other specialists.

The criterion for this lay example was that it address a �stop or go� decision and have data to support an historical analysis. We chose to use a sports metaphor: we compared the decision to cancel or continue an R&D project with the decision to pull a pitcher out of a baseball game.

Here is the data model we generated around this decision point:

Figure 4. Criteria used to evaluate the question: Should this pitcher be pulled? Values for each data set (pitch count, pitchers confidence level, etc.) are grouped and indexed on a scale of 1 (leave pitcher) to 5 (pull pitcher). The index is based on historical data about this pitcher in similar circumstances (this is a linear regression analysis). Criteria are then weighted according to importance to the overall decision.

Because we were trying to mirror the way the decisions we researched were made, we were particularly determined that the interface accommodate not just the quantitative data but the more qualitative - yet absolutely critical - human aspects of the decision making process (the experience, intuition, collaboration, negotiation, etc.) that is the real driver behind any complex decision.

We used a navigation schema which we had previously developed for collaboration and the design work went very quickly.

Here is an example demonstrating our approach to the data around �Pitch Count�:

8

Figure 5. One critical decision facing a baseball team is whether to pull a pitcher during a game. This diagram examines one particular aspect of this decision: how many pitches has the current pitcher thrown? Runs and hits given up by the pitcher are tracked along the y-axis while the pitch count increases along the x-axis. At 47 pitches, this pitcher is in what has been statistically his strongest stretch. The user may adapt the timescale at the bottom to measure the statistics deemed most relevant. This tool can then be paired with similar unique maps to gather a consensus around the fundamental decision point.

Images of the full screen interface follow in the Solution Details section of this document.

This baseball example was presented to senior IT management in Fall 2001.

Solution details

We will describe some static screenshots, focusing primarily on the ways we chose to depict data. The first generation tool modeled the essential metrics of dollars, people and time pertaining to a broad, high-level picture of the R&D organization and its portfolio of projects. It provided users information pertaining to their roles, responsibilities and the decisions they needed to make.

Figure 6 and the drilldown shown in figure 2 follow a line of thought a decision maker might take through the data interface. Figure 6, which we call the matrix map, gives a high-level view of the entire project portfolio across therapeutic area (product line) and project phase. Different metrics can be mapped onto the matrix (FTEs, or employee-hours, is shown in the figure.) The matrix map is a snapshot of the protfolio at one moment in time. The questions it responds to are: What is the current status of my projects? Where are my resources (people and dollars) allocated? In what phase or product area do I have the most or least sunk costs or value or competitive advantage? Is my distribution of resources aligned with value? Where is my pipeline of projects thin? At what point in the future might there be a gap in activity or revenues?

Figure 2 drills down into the system for more information about the developmental phase. By clicking the phase (A) in the matrix, the user moves to a more detailed distribution of projects, people (full-time

9

equivalents or FTEs), and schedule and priorities within phase A. A red ball indicates some sort of alert (e.g., over budget, over schedule) and clicking any one of the balls brings you to further detailed information about that project.

Figures 7 and 8 are additional views which portray this same data set (but were not included in the original system.) These more advanced portfolio views were developed in-house in order to explore the scope of information these visualizations, and their users, could process. Figure 7 was conceived to be similar to the matrix map, only with a different area of concentration. First, the element of time, and dynamic change over time, was introduced. Second, each project is visible as a small dot or ball, and further detail can be accessed directly by clicking on a project. In figure 8, which is called the pipeline flow, a much more complex system of time and developmental phase shift has been added to the screen in order to get a more complete view of the entire project portfolio.

In Figure 9, a circle diagram in the upper left corner shows an aggregate �finding� of whether the pitcher should be removed from the game in this circumstance. Each �slice� of the circle represents the data from one of the criteria detailed in Figure 4 where the size of the slice represents the weighting and the radius shows the value of that factor (on that 1-5 scale shown in Figure 4.) What this means at-a-glance is that the more filled the circle is, the more the data suggests that the pitcher should be pulled.

The open factor that in the interface in Figure 9 is the current batter history. This takes historic data about this pitcher�s experience with this particular batter and the batter�s experience with similar pitchers (e.g., left-handed fastball pitchers), compares those statistics

with league averages and feeds up an �opinion.� A user can also adjust the data surveyed by pulling the game slider (beneath pitcher/batter data) to the left or right, reviewing more or fewer games respectively. This allows the input of more qualitative aspects of this decision like the pitcher has been throwing very well the past few games, or the batter is slumping.

Figure 10 shows the �Opinion� toggle of this interface. Here the same �pie� visualization is used but here each slice represents one expert opinion. Each expert generates his/her own settings and weighting to the criteria and those aggregates are shown beside each person. In order to get an at-a-glance feel for what the verdicts might be, the criteria are grouped into shades of gray and there is a median represented by the light blue transparency (again, more filled indicates the pitcher should be pulled.)

This project has been an exciting challenge for our techniques of information visualization. Much more complexity has been introduced �multiple perspectives and opinions, multiple decision criteria seen side-by-side, the ability to model multiple scenarios, the �sense� of the algorithm generating data and how changes affect those calculations. And, in trying to represent to the decision making group what points of data each member might be focused on and how each member might be reaching his or her conclusion to the question at hand, we feel we have finally started to explore and inject those most qualitative, most human aspects of discussion, negotiation and persuasion that drive the decision making process and we are eager to test the usefulness of this visual metaphor in collaborative decision-making in the next phase of this work.

10

These examples represent some of Visual I/O�s thinking on the strategic R&D question of �stop or go�.

Post-launch result

The first generation tool has yet to be deployed. It went 6 months long in development during which time this pharmaceutical company was merged with another R&D group. In a major reevaluation of all the information technology and application development across the parent organization, this application was shelved (although some of the visual systems were used in a �home grown� project tracking application.) The interface is currently in redevelopment and we are updating the user requirements. Since this work is on going, we are not in a position to do a complete post-mortem. Still, it is worthwhile taking stock of the things we learned through the course of this project.

In the client organization there was disappointment that the original application was not deployed. Although most of the reasons for its demise were political and internal to the client organization, for us, this experience highlighted some problems inherent to the way in which we work. We continue to struggle to articulate our position on the scale between �custom� and �generic�. To make these tools truly useful, we believe they must be tightly integrated to the systems in place � the people, the processes, and the priorities. However, a tool built around an existing situation in this way is then at risk for obsolescence when there are changes to the infrastructure in that organization.

And, while it may seem surprising that so much time and effort (both internal and external) would, in effect, go to waste, we have come see that R&D organizations are very accustomed to making these types of decisions

and see throwing out large amounts of work a fully anticipated cost of doing business. So, while the deep knowledge of the client and the business was a great benefit to us in the development process, it did not, in the end, serve the overall effort.

We�ve come to look at decision support as a process, not an application. An evolving business is always going to be examining its operations and its goals and we�ve tried to address that by thinking about our solutions as being more lightweight and, if necessary, disposable. We have seen though, that ways of visualizing data and processes can be very powerful and we believe that when an organization buys into a visual metaphor, there can be a lasting resonance.

ACKNOWLEDGMENTS

Thanks to Scott Listfield and Doug Marttila for their help in pulling this paper together, and to Scott Myers for the inspiration for this line of work.

REFERENCES [1] Matheson, D., & Matheson, J. The Smart Organization.

Harvard Business School Press, 1998.

[2] Klein, G. Sources of Power. MIT Press, 1998.

[3] http://www.smartmoney.com. see the �Map of the Market.�

[4] Tufte, E. The Visual Display of Quantitative Information. Graphics Press, 1983.

[5] Tufte, E. Envisioning Information. Graphics Press, 1990.

[6] Tufte, E. Visual Explanations. Graphics Press, 1997.

[7] Proceedings of 5th International Conference on Information Visualization (London, 2001).

11

http://www.graphicslink.demon.co.uk/IV2001/KEYNOTE.htm.

[8] Multi Attribute Decision Making. Quantitative Applications in the Social Sciences #104. Sage University Paper, 1995.

[9] Hallett, P. Web-Based Analytics Improve Decision Making. DM Review, February, 2001, 1-5.

[10] http://decisionstrategies.com/apps1.html. Building a World Class Pharmaceuticals Portfolio. see the Integrated Decision Management® product.

12

A variety of different metrics can beviewed in the matrix. This allows different levels of management to access the statistics which are most vital to their decision making process.

Figure 6. This portfolio tool maps progressive levels of drug development (A - FN) against the various fields

of research. Areas of greater density or value are described by darker color hues. In this scenario, FTEs, the

equivalent of employee-hours, is represented by the shade, while number of total projects appears in the top

left corners of each square. This matrix also functions as a portal. Any development phase (A-FN), research

field (Immunosuppression, e.g.) or individual square can be accessed for further detail. If updated on a

regular basis, this view presents a comprehensive picture of the R&D portfolio.

13

Figure 7. This matrix map is a step further in development from Figure 6. The earlier view, while comprehensive, did not allow for dynamic change over time, nor did it include any reference to future predictions or events. The Portfolio Viewbox (as this tool is called) depicts individual projects (represented as small banded circles) in the pipeline. Developmental phases are depicted along the top and specific research fields along the side, and densities are represented by color saturation. Problems that are delayed, overdue, or problematically over budget are called out in red. The slider along the bottom allows the user to dynamically view their portfolio as it progresses through time. Trends, gaps, and slow moving projects can be easily detected. Any project in the pipeline can be selected to receive in-depth project related information.

The time slider shows portfolio shifts over time. Viewing future projections is especially helpful given the lengthy timeframe involved in the drug development process.

Cost and employee-hours are summed for each developmental phase. As the time slider is moved, the user can watch the shifting allocation of funds. Past examples of overspending and dead zones can be tracked. Projected over-runs or dry spells can be predicted.

14

Figure 8. In this alternate pipeline view, actual time flows horizontally and is shown at the bottom. The progression of phases is represented along the vertical axis, descending from Phase U to Phase Z. Each project, represented by a single line and labeled by its number, enters the pipeline at its start date. As the project progresses, it travels both forward in time and also vertically through the pipeline.

As projects travel through the pipeline, a great many of them will be canceled for various reasons. These project deaths are noted in the pipeline as raised blocks.

The height of the bars on each phase line indicates the number of projects within that phase.

The time slider, similar to figure 8, traces the course of the portfolio through time. The time slider highlights in yellow projects that are in the pipeline at that point in time, and then indicates their phaselocation in the blocks at right.

Budget and workflow numbers overthe whole portfolio, at the selected point in time, are tallied.

15

Figure 9. This system is designed around the fundamental concept of making an informed stop & go decision. In this case, the example used was a baseball decision regarding whether to pull a pitcher from the game at a crucial moment. This screen displays the statistical side of the equation. Information relevant to the decision is pooled into a simple pie chart, and further detail is attached by lines. In this particular screen, the Current Pitcher/Batter History section has been expanded for even further review and customization. Each detail on the screen is moveable and custom weighted so that the user can not only tailor the statistics to their personal preferences, but also rearrange the visual hierarchy on the screen.

This pie chart represents the entire decision. When completely full, it is strongly advised that progress is stopped - in this case, the pitcher would be pulled from the game. When relatively empty, the project should continue.

The section entitled Current Pitcher/Batter History has been expended to reveal more detail about previous meetings between the current hitter and the current pitcher, as well as similar scenariosinvolving other players. The user can change the timescale as well as determine the relative weight of this information.

The system is divided into two views, a statistical evaluation of the data, and also a more subjective opinion-gathering tool. This particular screen displays the statistical analysis side of the equation. These options are provided as a way for the user to get different perspectives on similar data sets.

16

Figure 10. Should the pitcher be pulled? Here the question is analyzed by polling the opinions of those involved in making major decisions. The statistical data is still available, however here the information has been filtered through the views of those who most intricately understand the problem. The Opinion aggregate can then be compared to the statistical model and vice-versa in order to form a more comprehensive image of the problem.

This pie chart signifies the aggregate opinion of the staff and managers. Each pie piece represents an individual, weighted according to his or her priority within the campaign.

Each individual opinion is broken out in a manner similar to the statistical analysis. Here you can view in detail the priority and rating given to each event by thosemaking the decisions. In this case, the General Manager feels uncertain about whether to pull thepitcher as his accumulated score of 3.005 is very nearly dead center. However the total opinion (represented in the pie chart) seems to favor pulling the pitcher from the game.

The blue shaded circle represents the midline. Pie charts larger than the circle endorse pulling the pitcher.

17