the value of vision in radical technological innovation …

THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INNOVATION

A DISSERTATION

SUBMITTED TO THE DEPARTMENT OF MECHANICAL ENGINEERING

AND THE COMMITTEE ON GRADUATE STUDIES

OF STANFORD UNIVERSITY

IN PARTIAL FULFILLMENT OF THE REQUIREMENTS

FOR THE DEGREE OF

DOCTOR OF PHILOSOPHY

Tammy L. Carleton

September 2010

This dissertation is online at: http://purl.stanford.edu/mk388mb2729

© 2011 by Tammy Lee Carleton. All Rights Reserved.

Re-distributed by Stanford University under license with the author.

ii

http://purl.stanford.edu/mk388mb2729

I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.

Larry Leifer, Primary Adviser


Riitta Katila


Chuck House

Approved for the Stanford University Committee on Graduate Studies.

Patricia J. Gumport, Vice Provost Graduate Education

This signature page was generated electronically upon submission of this dissertation in electronic format. An original signed hard copy of the signature page is on file inUniversity Archives.

iii

THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

iv

ABSTRACT

Is a technological vision needed to drive radical or disruptive innovations? Few studies

have discussed a possible relationship between the formation of a technological vision

and the sustained creation of radical innovation. Even fewer have analyzed a relationship

systematically, and finding a suitable data set has proven to be a challenge to scholars.

Since 1958, the U.S. Defense Advanced Research Projects Agency (DARPA) has

sponsored high risk, high reward research and development that bridge the gap between

fundamental discoveries and their military uses. The agency’s sole charter has been

radical innovation, providing 52 years of sustained practice. This study draws on 59

interviews with DARPA personnel and funding recipients, whose time at the agency

spans 45 of those years. Historical interviews and agency documents further validated the

data set. Using grounded theory methodology, this study identifies the importance of

vision in radical technological innovation, synthesizing prior studies from organizational

innovation, technology management, visionary leadership, and industrial research and

development. Four major findings are discussed, which address the timing and criteria for

innovation visions, specific mechanisms for the formation of visions, the role of

socialization on envisioning, and the corresponding governance model. These findings,

supported by empirical evidence, add to the current understanding of technological

visions and radical innovation research.


v

ACKNOWLEDGEMENTS

I would like to thank the following individuals, who have supported my research with

extraordinary contributions: Chuck House, Riitta Katila, and especially Larry Leifer for

being willing to think boldly in the spirit of DARPA Hard; John Cockayne for his

generosity of spirit and funds; William Cockayne for his steadfast faith and provocation;

and Jon and Suni Carleton for ensuring the final chapters were drafted.


vi

TABLE OF CONTENTS

Abstract......................................................................................................................... iv List of Tables ..............................................................................................................viii

List of Illustrations....................................................................................................... ix Chapter 1 Introduction ................................................................................................ 1

Research Opportunity ..............................................................................................................2 Research Plan ..........................................................................................................................5 Research Contribution .............................................................................................................7

Chapter 2 Literature Review ....................................................................................... 9 Radical Technological Innovation...........................................................................................9

A Time for Discovery & Invention...................................................................................12 Processes of Radical Innovation .......................................................................................13 Organizational Sustainability ............................................................................................17

Visions of Technology ..........................................................................................................17 Visionaries and Other Innovation Roles................................................................................21 Innovation and DARPA ........................................................................................................23

Organizational Structure ...................................................................................................24 Processes of Innovation.....................................................................................................29 Measures of Program Success...........................................................................................35

Conclusion.............................................................................................................................36 Chapter 3 Methodology.............................................................................................. 37

Grounded Theory Methodology ............................................................................................37 Iterative Process of Data Collection and Analysis ................................................................39 Theoretical Sampling.............................................................................................................43 Method Limitations ...............................................................................................................44 Conclusion.............................................................................................................................45

Chapter 4 Data Set...................................................................................................... 47 Data Sources ..........................................................................................................................49 Subject Profile .......................................................................................................................53

DARPA Personnel ............................................................................................................54 Funding Recipients............................................................................................................58 DARPA Office Affiliation ................................................................................................59

Conclusion.............................................................................................................................60 Chapter 5 Analysis...................................................................................................... 62

Process Model of Radical Innovation....................................................................................62 Stage 1: Recruitment .........................................................................................................63 Stage 2: Vision Formulation .............................................................................................63 Stage 3: Program Launch ..................................................................................................64 Stage 4: Portfolio Management.........................................................................................64 Stage 5: Technology Transfer ...........................................................................................65

Vision and Recruitment .........................................................................................................67 Recruiting Criteria and Evaluation....................................................................................70 Channels for Recruiting Technology Visionaries .............................................................74

Vision Formulation................................................................................................................76 Sources of Technology Visions ........................................................................................77


vii

Criteria of a Radical Vision...............................................................................................78 Advancing Partial Visions of Technology ............................................................................81

Expert Workshops.............................................................................................................83 Proof-of-Concepts .............................................................................................................84 Similarities Between the Mechanisms ..............................................................................85

Learning the Capacity for Envisioning..................................................................................86 A Community of Informal Learning .................................................................................87 An Entrepreneurial Spirit ..................................................................................................88

Converting Visions to Programs ...........................................................................................89 Conclusion.............................................................................................................................92

Chapter 6 Discussion .................................................................................................. 94 A Process Model of Radical Innovation................................................................................95 New Dimensions of Vision ...................................................................................................96

Visions at the Program Level............................................................................................97 Vision Quality ...................................................................................................................99 Visionaries of Technology ..............................................................................................104

The Development of Partial Visions ...................................................................................107 Learning Radical Innovation Through Socialization ..........................................................109 Internal Review of Radical Innovation Ideas ......................................................................111 Conclusion...........................................................................................................................113

Chapter 7 Conclusion ............................................................................................... 114 Contribution to the Field .....................................................................................................115 Contribution to the Practice of Innovation ..........................................................................115 Research Limitations ...........................................................................................................117 Directions for Future Research............................................................................................118

Appendix A: DARPA Technology Offices............................................................. 120

Appendix B: Federal Budget Categories ................................................................ 122 Appendix C: Research Interviews Conducted ....................................................... 124

DARPA Staff.......................................................................................................................124 Funding Recipients ..............................................................................................................125 Related Defense Roles.........................................................................................................125 Other Researchers................................................................................................................125 Oral History (Interview) Transcripts ...................................................................................126

List of References...................................................................................................... 127


viii

LIST OF TABLES

Page

Table 1: Uniqueness levels of a technological innovation ........................................... 11

Table 2: Twelve organizing elements of DARPA ........................................................ 33

Table 3: Common processes and interview quotes per DARPA innovation stage....... 67

Table 4: A selection of DARPA program visions ........................................................ 70

Table 5: Criteria for a radical vision at DARPA .......................................................... 78

Table 6: The concept of DARPA Hard......................................................................... 81

Table 7: U.S. federal defense budget categories......................................................... 123


ix

LIST OF ILLUSTRATIONS

Page

Figure 1: Open innovation and network partners of DARPA ...................................... 25

Figure 2: Basic staff structure at DARPA .................................................................... 26

Figure 3: Theory building and theory testing approaches to research .......................... 38

Figure 4: Iterative process of data collection and analysis ........................................... 44

Figure 5: Research interviews (n=59) by category....................................................... 54

Figure 6: Different roles held by DARPA subjects (n=29) .......................................... 55

Figure 7: Distribution of roles (n=42) held by DARPA subjects over time ................. 57

Figure 8: DARPA funding recipients (n=19) by organizational type........................... 58

Figure 9: Distribution of funding recipients (n=19) based on primary DARPA engagements over time ......................................................................................... 59

Figure 10: Subject affiliation (n=44) by DARPA office .............................................. 60

Figure 11: Process model of radical innovation at DARPA......................................... 63

Figure 12: Radical innovation starts with vision at DARPA........................................ 68

Figure 13: Rise of starting ages for new DARPA personnel ........................................ 73

Figure 14: Formulation of partial visions of technology for radical innovation .......... 83

Figure 15: Transition between stages of Vision Formulation & Program Definition .. 90

Figure 16: Comparison between DARPA’s process model and the stage-gate model. 95

Figure 17: Visions at DARPA operate at the program level ........................................ 98

Figure 18: A sample 7-point scale for quantification of human performance variables103

Figure 19: A radical technological vision relies on one big idea and one visionary .. 104

Figure 20: Efforts preceding a complete vision of technology................................... 107


CHAPTER 1

INTRODUCTION

• A computerized network of thinking centers operating remotely • An aircraft that cannot be detected by conventional radar • A navigation system that allows land, sea, and airborne users to determine their exact

location in all weather conditions • A doctor able to operate virtually on an injured soldier miles away • A robotic ground vehicle that can operate autonomously • A rough-terrain robot that walks, runs, climbs and carries heavy loads • An artificial arm controlled by the human mind • A personal transportation vehicle that can drive and fly

The above ideas are all examples of radical technology visions, funded by the

U.S. Department Advanced Research Projects Agency (DARPA). When these

engineering concepts were first imagined, they were at the forefront of technical

knowledge, pushing the limits of previous inventions and, in many cases, requiring the

creation of new technologies in order to be built. While scholars may refer to radical

innovation by a variety of synonyms, such as disruptive (Christensen, 1997) or

revolutionary (Dundon, 2002), all the terms share a core definition. Radical innovation is

not incremental development; instead, it is a new technology or solution that creates an

entirely new and often unexpected market, what industry might call “big R&D”.

Radical innovation is a popular topic in business and engineering literature,

largely due to a lack of understanding about what to do and how to do it (e.g.,

Christensen, 1997; Leifer et al., 2000; Christensen & Raynor, 2003; O’Connor, Leifer,

Paulson, & Peters, 2008). Two conflicting beliefs can be found in this literature. First,

radical innovation, like many business functions, is seen as a management process that

requires specific tools, rules, and discipline. Second, applying traditional evaluation

methods and criteria to radical innovation projects is inappropriate and


2

counterproductive. This study of successful radical innovation is intended to help

organizations be more deliberate in their pursuit of technology visions that will generate

greater business value and change society’s beliefs about what is possible.

Research Opportunity

One of the principal obstacles in studying radical innovation is finding

appropriate data sets (Van de Ven, Angles, & Poole, 2000; O’Connor et al., 2008). For

example, O’Connor and colleagues identified a small sample of companies, whose

management expressed a strategic intent to develop or evolve a sustained capability for

radical innovation. The majority of the companies had recently begun the process, and

after several years of study, many were still experimenting with different mechanisms to

facilitate their efforts in radical innovation. Although the researchers posit a set of shared

patterns of activity across these projects, their ability to describe a single process of

radical innovation is limited by the small number of projects completed and by the

distribution of successful projects across a variety of companies.

In the search for a larger pool of technology visions and a better ability to

compare successful outcomes, this study looks to the Defense Advanced Research

Projects Agency, or DARPA, as providing a repeated, and repeatable, model of radical

innovation. Founded in 1958 by the United States Department of Defense, DARPA

specializes in radical innovation. The mission of the federal agency is to “maintain the

technological superiority of the U.S. military and prevent technological surprise from

harming our national security by sponsoring revolutionary, high-payoff research bridging

the gap between fundamental discoveries and their military use” (DARPA, 2010b). The


3

agency’s name was changed from Advanced Research Projects Agency (ARPA) to

Defense Advanced Research Project Agency (DARPA), then back to ARPA, and in

1996, back to DARPA, purely for political expediency. For simplicity, DARPA is used

throughout this report.

DARPA has a long history of finding and building big engineering visions. At

DARPA, the conversion rate of new technologies to market has consistently been higher

than comparable private industry sectors over the agency’s long lifespan, and many

technologies have transitioned to commercial products, according to studies from

Richardson, Bosma, Roosild, and Larriva (1999) and Richardson, Larriva, and Tennyson

(2001). Easily recognized examples include the foundation of today’s Internet known as

ARPANET, the global positioning satellite (GPS) system, and the da Vinci Surgical

System used for laparoscopic surgery in hospitals (Belfiore, 2009).

DARPA’s organizational structure supports the pursuit of big ideas (Bonvillian,

2006). The organization is independent from other more conventional military research

organizations with approximately 100 technical personnel (the number changes as

personnel are intentionally rotated through), who directly manage a $3.25 billion R&D

budget (DARPA, 2009; M. Peterson, personal communication, June 10, 2010). DARPA’s

primary staff consists of program managers (PMs) hired on a short-term basis, typically

3-4 years, to find, fund, and foster new engineering ideas based on technology visions

that each one of them is expected to develop. While the PMs develop the overarching

visions, they do not bring the visions to life themselves. Instead, the PMs fund teams at

universities, industry, and government labs. With the combination of an achievable

technology vision and funding, these teams are induced to think and act like innovators


4

for an intense period in time, designing and building functioning prototypes that extend

today’s knowledge. This practice reflects a model of open innovation (Chesbrough,

2003), in which DARPA has relied on an external network, rather than internal staff, to

source and commercialize innovation ideas for over 50 years.

In short, DARPA is a model of a sustainable radical innovation engine.

Nonetheless, DARPA remains relatively unstudied by the academic community. Of the

few rigorous studies that exist, researchers have focused on DARPA-funded outcomes in

the early computing field (e.g., Hafner & Lyon, 1996; Waldrop, 2001; Roland & Shiman,

2002). DARPA’s model of innovation has been covered superficially, often through

personal reflections (Malakoff, 1999; Tennenhouse, 2004; Bonvillian, 2009). A former

agency director reinforced the need to study the innovation mechanisms within DARPA,

saying: “The literature on DARPA is large, though most is about projects and results, and

not about how the magic is performed. Hence a lot of the Agency’s operations are

shrouded in mystery” (S. Lukasik, personal communication, September 23, 2009).

A research question was crafted to delve into the mystery of DARPA. The

question is stated as such: how do creators communicate their long-range, technical

visions during early development? This question was further refined in order to

operationalize the various components. “Creators” refer to the individuals responsible for

defining and introducing the visions, and the literature notes a special role for the

technical visionary (Hebda, Vojak, Griffin, & Price, 2007; Deschamps, 2008). The act of

communicating focuses on the delivery of the visions from the creators to those

responsible for embracing and building the visions. In the case of DARPA, the receiving

group would be the various funding recipients, who hear about the new vision


5

opportunities through both private and public channels. The communication itself may

take several forms, including in-person discussions and public documents. In the

question, technical visions describe the future state of technology concepts that rely on

engineering knowledge to build. The qualifier long-range specifies visions that require

multiple years to address, and these visions are separate from goals in new product

development and immediate R&D initiatives. Lastly, early development refers to the start

of the innovation process, which includes conceptualization and prototyping, before

formal research and product development begins (Van de Ven, Polley, Garud, &

Venkataraman, 1999).

Research Plan

The research goal was to study the full lifecycle of a technology vision within

DARPA using grounded theory methodology. Grounded theory methodology is an

inductive research approach that draws insights directly from multiple cases in real-world

contexts for the purpose of constructing theory (Goulding, 2002; Bryant & Charmaz,

2007; Corbin & Strauss, 2008). This research method was chosen for several reasons.

First, as a qualitative method, grounded theory allows a researcher to gain insight into an

organizational culture, as well as people’s attitudes, behaviors, and motivations, which

cannot be understood through a statistical approach. Qualitative research seeks out the

‘why’, not the ‘how’ or ‘how many’ of its topic through the analysis of unstructured

information, such as interviews and observations. The objective is to study things in their

natural settings, attempting to interpret phenomena in terms of the meanings that people

bring to them. Grounded theory was a necessary starting methodology for a domain in


6

which the governing variables are not yet known. Once these variables are identified,

then specific hypotheses can be tested in future studies to validate the model.

As the second reason, grounded theory methodology is useful when theories are

not available. Limited research exists about DARPA’s innovation model, apart from

various anecdotes, and adopting a theoretical framework without first collecting data or

applying a framework out of context would be intellectually negligent. Grounded theory

enables the construction of new theory from data. As opposed to developing a theory that

attempts to correct or clarify previous theories, the aim of this study is to provide an

explanation derived directly from reality.

Third, grounded theory methodology considers a phenomenon within its socio-

historical context. A key assumption in this study is that organizational activity is heavily

dependent on time and place. Individual personalities and projects within DARPA change

constantly due to the rotating staff and network structure, so any in-depth research

requires placing multiple particular instances in a larger situation.

Fourth, grounded theory offers an emergent approach to data collection and

analysis. In particular, hypotheses are not formed in advance, and instead the literature

review is updated in parallel with the data collection, so that both efforts may

continuously inform each another. Many myths abound about DARPA, and a critical

researcher would avoid starting with one model too rigidly, particularly if there may be a

danger of missing nuances that could inform deeper understanding. Like the

anthropologist entering an alien culture for the first time, it seemed wise to start without

undue preconceived notions in order to view the actions at DARPA with clear eyes

(Howe, 1952).


7

In this study, data is comprised of personal stories and archival documents.

Interviews provided the most effective way to hear about the development of visions

within DARPA. The most direct source was the program managers, followed by related

staff roles (e.g., office director) and then the funding recipients. In total, 47 in-depth

interviews were conducted with agency personnel and funding recipients, and the

subjects’ time at DARPA span the entire lifespan of the agency and range across a variety

of programs.

An additional set of 12 interview transcripts with DARPA personnel were

obtained from the Charles Babbage Institute (CBI) archive, which were recorded from

1988 to 1993. Several of the interviews touch on innovation practices at DARPA, helping

to validate the responses from current subjects, as well as providing some perspective

from early computing pioneers who have since passed away.

Finally, agency documents were reviewed as a secondary source of data. These

documents were written at different points in DARPA’s history (1958 – present) and

offer another view of agency procedure. The documents also provide a formal and public

record of DARPA’s activities related to vision setting. In summary, by triangulating

multiple sources, the aim is to reduce potential bias in the interpretation of the data.

Research Contribution

This study extends the current understanding of radical innovation by describing

the central role that vision plays, building on studies of organizational innovation,

technology management, visionary leadership, and industrial research and development.

This study further examines the specific mechanisms involved in developing visions of


8

radical innovation. Part of the study’s learning reinforces the deep-seated belief within

the innovation community that processes of radical innovation should be treated

separately from processes of incremental innovation. However, new data reveals that the

actual processes for the selection and evaluation of ideas within radical innovation runs

counter to the existing literature of radical innovation. The details behind these insights

are found in the next few chapters.


9

CHAPTER 2

LITERATURE REVIEW

This chapter summarizes the state of knowledge about processes of radical

innovation, concentrating on the early stages when a daring technology idea is first

formed and evaluated. In addition, the scholarly literature about technology visions is

discussed in order to provide broader context for the big ideas that drive radical

technological innovation. Lastly, the Defense Advanced Research Projects Agency

(DARPA) is presented as an organization entirely inspired, structured, and measured by

radical technological innovation. Existing studies and reports about DARPA are reviewed

in order to examine what has been documented about the organization’s internal

innovation processes. Overall, research is drawn from multiple disciplines, primarily

studies of organizational innovation, technology management, leadership, and industrial

research and development.

Radical Technological Innovation

Innovation is a broad term, often used to describe both the process and its output

interchangeably. Innovation is a relatively new area of inquiry. In 1942, Schumpeter first

defined the modern innovation process, introducing the concept of creative destruction in

which new technologies and new skill sets either create or redefine firms and existing

markets. Since then, scholars have classified different types of innovation output. One

category of innovation is technological innovation, which is defined as “the act of

introducing a new device, method, or material for application to commercial or practical

objectives” (Schilling, 2005, p. 1). Other example innovation categories are related to


10

services, processes, and organizations. Technological innovations can be further

classified as either incremental or radical. The general consensus among scholars is that

incremental innovations are solutions that improve an existing product or service, such as

adding new features that do not create significant changes in the marketplace (Harvard

Business Essentials, 2003; Schilling, 2005). In contrast, radical innovation entails

solutions that are very new and different from prior solutions, often generating new

markets or industries.

Scholars differ on how radical should be perceived. A recent definition, revised

by O’Connor and her management colleagues (2008), defines radical innovation relative

to a company’s R&D portfolio, and both risk and impact are based on the management’s

perception. Other scholars define radical innovation in terms of a technological

breakthrough, which brings new knowledge to an industry overall. Lynn and Akgün

(2001) characterize radical innovations along two dimensions of very high market

uncertainty and very high technology uncertainty, yet they do not describe nor attempt to

quantify either dimension in detail.

To provide clearer guidelines, Abetti (2000) introduced a five-level scale to

identify innovation uniqueness for technologies. In Abetti’s scale, which is reproduced in

Table 1, levels 1 and 2 correlate with “highly radical” and “radical” technological

innovations, respectively. While straightforward, his taxonomy does not seem to be

widely adopted by other innovation scholars. In addition, while his categories describe

the level of “radicalism” for a new technology, they do not fully address the scale of

commercial impact or societal change. Considering economic impact, Betz (2003)

distinguishes between three scales of technological innovation, which span radical


11

innovation, incremental innovation, and next-generation technology innovation. Betz

defines radical innovation as a “basic technological innovation that establishes a new

functionality”, such as a steamboat engine (p. 73). However, impact is noted in terms of

the solution’s functionality, versus other benefits to humanity. In short, a good, all-around

workable definition of radical innovation is lacking in the literature.

Level Type of Innovation Description 1 Highly Radical Unique original product or system, which will obsolete

existing ones, based on proprietary technology beyond the state-of-the-art, highly specialized and customized, major R&D

2 Radical New product or system, with original state-of-the-art proprietary technology, that will significantly expand the capabilities of existing ones, specialized product with many adaptations, significant R&D

3 Intermediate New product with proprietary technology, but may be duplicated by others, mix of standard and special features, average R&D

4 Significant Incremental

Significant extension of product characteristics with original adaptations of available technology, product with standard variations, limited patent protection, minor R&D

5 Minor Incremental Incremental improvement over existing products, application of current technology, standardized product, no patent protection, no R&D

Table 1: Uniqueness levels of a technological innovation

Abetti’s five-level scale (2000, p. 209) identified innovation uniqueness for technologies, in which levels 1 and 2 correlate with “highly radical” and “radical” technological innovations, respectively.

Radical is only one term to describe high-risk innovation. Scholars have used

other various terms to describe radical innovation, such as disruptive (Christensen, 1997;

Kostoff, Boylan, & Simons, 2004), breakthrough (Fleming, 2002; O’Connor, 2008),

game changing (Edelheit, 2004), blue oceans (Kim & Mauborgne, 2005), discontinuous

(Watts, 2001; Norling & Statz, 1998; Rice et al., 1998), transformative / transformational


12

(House & Price, 2009; Johnson, 2010), technology push (Cotterman et al., 2009; Tzeng,

2009), and revolutionary (Dundon, 2002). Regardless of the exact term, radical

innovation is critically needed in society as a way to drive progress. As economist Perez

(2002) points out, “technological change occurs by clusters of radical innovations

forming successive and distinct revolutions” (p. 6). Additional studies of radical

innovation will contribute to the growing scholarship and understanding of a process that

has significant consequence on society and individual wellbeing.

A Time for Discovery & Invention

Generally, the start of the innovation process is considered a time of discovery

(O’Connor et al., 2008). In the literature of new product development, scholars refer to

this stage as the “fuzzy front end”, which entails activities for the scouting, generation,

and screening of ideas (Smith & Reinertsen, 1998; Rhea, 2003). There is less clarity in

studies of radical innovation. Many of the companies that O’Connor and her colleagues

studied pursued a practice of “open innovation”, looking outside the organization to

customers, partners, and other groups for new innovation ideas (Chesbrough, 2003).

Ultimately, O’Connor and her team believe that the discovery process should be

centralized internally into what they call an “innovation hub” for greater impact and

effectiveness within an organization. Van de Ven and his colleagues (1999) conclude

differently. Based on extensive longitudinal studies over 17 years, looking at all types of

innovation, they believe that organizations with more divergent and chaotic activities

were more likely to develop a tradition of innovation. In particular, processes for


13

technically complex innovations proved to be more disorderly than processes for

technically simple innovations.

Specific to radical innovation, Betz (2003) posits that the process begins with new

scientific knowledge, and he draws a linear connection from scientific discovery to basic

invention to ultimately economic exploitation. Science looks to uncover and understand

new knowledge. Next, invention puts new knowledge to practice, and invention is a

critical step along to path to innovation. Prototypes are then used to test the technical

feasibility of an invention before any product design begins. These early steps define the

start of the innovation process, which then leads to technology development and

commercialization.

Overall, Betz discusses the difficulty that management faces when integrating

these different steps functionally within an organization and also in handling intellectual

property rights appropriately with multiple partners. He argues that management should

support the national development of technologies because next-generation technologies

essentially occur at the industrial level, not at the organizational level. In particular, Betz

makes a case that the overlap between proprietary and nonproprietary research happens at

the step of functional prototyping, which determines the critical decisions in direction and

pace for innovation.

Processes of Radical Innovation

Within organizations, the formula to operationalize radical innovation remains

murky. O’Connor and her colleagues (2008) outline several historical approaches that

established companies have used, which comprise intrapreneuring, skunks works, and


14

corporate venture units. Intrapreneuring depends on enterprising individual employees,

who act as entrepreneurs in residence by taking direct responsibility for turning an idea

into a profitable business. A “skunks works” approach is another model, in which

companies allow a small group of experts, typically isolated and protected from

headquarters, to work on a risky project through unconventional means. Well-known

examples include Lockheed Martin (Johnson & Smith, 1989), Apple’s original Macintosh

team (Levy, 2000), and arguably Xerox PARC (Hiltzik, 1999). Corporate venture units

have undergone cycles of popularity, based on the success of venture capitalism, and

companies use their venture units to invest strategically in startups in order to later adopt

or acquire promising new technologies (O’Connor et al., 2008).

The main issue in all these models is that corporate management generally lacks

the experience and expertise in understanding how to start and manage radical innovation

separately from other R&D initiatives (O’Connor et al., 2008). Dekkers (2005) states:

“Where incremental innovation reinforces the capabilities of established organizations,

radical innovation does force them to ask a new set of questions, to draw on new

technical and commercial skills, and to employ new problem-solving approaches” (p.

189-190). Few empirical studies have documented which capabilities are needed for

radical innovation versus traditional R&D work. An earlier study of food processors

found that decision centralization and informal structures tend to support radical process

adoption, regardless of organizational size (Ettlie, Bridges, & O’Keefe, 1984).

Other scholars have found a similar issue in competence within companies

attempting to create platforms of disruptive growth (Stringer, 2000; Christensen &

Raynor, 2003). In order to develop the necessary capabilities for radical innovation,


15

Christensen and Raynor (2003) argue that companies should focus on processes as a way

to create and transform raw resources into value. Processes may be formal, informal, or

embedded in the organizational culture. The problem occurs when innovating managers

follow processes intended for new product development.

Similar to O’Connor and colleagues (2008), Christensen and Raynor argue that

radical innovation requires an entirely different set of processes to be conducted

effectively; however, large established companies tend to default to familiar practices.

The problem is compounded for these companies, who tend to judge radically new ideas

with the same criteria and short-term biases as other R&D investments. Stringer (2000)

posits four reasons why large companies struggle with radical innovation: the cost to

change status quo, inflexible bureaucracy in place to maintain organizational stability, an

over-reliance on internal R&D projects, and an inability to attract and reward radical

innovators meaningfully.

Several management practices have been consistently identified that facilitate

radical innovation. Many case studies of organizations show that significant consensus or

committee measures – such as oversight boards, governance teams, and commercial

councils – are used in order to continuously scan, surface, and select potential new

opportunities (O’Connor et al., 2008). These consensus measures are similar with

recommendations from other researchers, who discuss the use of innovation boards for

reviewing and selecting promising new ideas during the early stages of innovation

(Bacon & Butler, 1973; Hamel, 2002; Snyder & Duarte, 2003; Skarzynski & Gibson,

2008). Innovation boards are typically represented by senior executives who manage

successful existing business units, along with members from outside the company and


16

occasionally employees from different levels of the company. Innovation boards rotate

members and are seldom dissolved as needed. One limit of these case studies is that they

focus on large corporations, which tend to default to familiar patterns of committee

decision-making. Corporations concentrate on building broad consensus to ensure

organizational alignment and leadership support.

The equivalent in academic settings is peer review. Similar to innovation boards,

a peer review model relies heavily on group consensus to ensure representative authority

(Daniel, 1993). In terms of supporting the pursuit of cutting-edge innovation, there has

been a growing criticism that organizations and funding agencies, which rely on peer

review to discover, evaluate and ultimately fund new research ideas, are less willing to

sponsor scientists working outside traditional fields or across multiple fields (Chubin &

Hackett, 1990).

Another process often cited in the innovation literature is the stage-gate model

(Cooper, 2001), which organizes efforts in new product development in terms of stages

and gates. Before the first stage formally occurs, a period of discovery is used to screen

and identify new ideas, and at the end of each stage, a “gate” provides a deliberate

decision point for project continuation. The stage-gate model is considered the dominant

structure for decision-making in corporate R&D. However, some scholars posit that the

stage-gate model is more effective for market-pull innovation within existing markets

(Christensen, 1997; Gassmann & von Zedtwitz, 2003), while other scholars argue that the

underlying issue is with the criteria used in screening potential new ideas (Harvard

Business Essentials, 2003).


17

Organizational Sustainability

The literature shows few examples of lasting practice. Despite management’s

desire to pursue truly radical innovation, finding a repeatable organizational model is

difficult. O’Connor and her colleagues (2008) conducted a longitudinal study from 1995

to 2007 of multiple companies, all of which were pursuing or planned to pursue efforts in

radical innovation. They did not find a sustained model in the literature, and none of the

companies in their data set had developed an enduring set of practices. In fact, their

research found that the average life span of a radical innovation program is four years.

They state, “History shows that new business incubators, new ventures divisions, or other

groups that have been founded to accomplish major innovation in companies often don’t

last” (p. 22). This finding is consistent with research from Deschamps (2008), who states

that most companies are unable to sustain innovation performance, largely due to poor

leadership commitment.

Visions of Technology

Generally, scholars mark the start of the innovation process with the generation of

new ideas (Harvard Business Essentials, 2003); however, recent work from Cockayne

(2004) demonstrates that significant groundwork is spent in the formative period

preceding a complete idea. This is consistent with work from Van de Ven and colleagues

(1999), which found an extended gestation period, often lasting several years, occurs

before the development of an innovation. Sources of new ideas are individuals, firms,

universities, private nonprofits, or government-funded research, as well as the linkages

between them (Schilling, 2005).


18

There is limited research that demonstrates the connection between radical

technological innovation and vision. Generally, studies about the processes of

technological innovation do not mention vision (e.g., Roberts, 1987; Tornatzky,

Fleischer, & Chakrabarti, 1990). Lynn and Akgün (2001) studied how vision impacts

new product success, finding that “research on vision at the project level is curiously

lacking” (p. 374). They defined a vision as the desired future state and impact of an idea,

what one might consider as a big idea. Examples of technological visions from the

broader literature are largely anecdotal, such as the story of 3M listing vision as one of its

six principles for the company’s successful tradition of innovation (Van de Ven et al.,

1999) or field advice that companies are missing vision as the key ingredient in their

innovation strategy (Lundquist, 2004). Cases of radical product innovation, such as about

Hewlett-Packard’s thermal ink-jet (Fleming, 2002), are markedly absent mentioning

vision.

Drawing from studies of organizational vision, Lynn and Akgün (2001)

introduced three components to product vision that they note as clarity, support, and

stability, which when combined, provides a clear image of what to do. According to their

work, vision clarity refers to having a goal that is “a well-articulated, easy to understand

target” (p. 375). Vision support is about securing the necessary commitment from the

company to execute the vision, and vision stability ensures a consistent vision over time.

Lynn and Akgün found that only vision clarity was significantly associated with new

product success for radical innovations, at least as defined by the subjects’ perception of

market and technology uncertainty. One weakness of the study is that their sample of

successful radical products was drawn entirely from the computing industry. In addition,


19

while some products may have introduced a new product category, none established a

new market category, a core aspect that defines radical innovation. In comparison, Betz

(2003) describes two types of visions for the leaders who manage technology. The first is

a vision of the enterprise system, and the other is a vision of the technologies within the

enterprise system. A criticism is that his definition lacks a sense of future possibility or

technical challenge; instead, he characterizes vision solely in terms of the engineering

solution, either as a total integrated system or as a set of individual components.

Although literature about vision at the project level is scarce, in contrast, there is a

much richer stream about vision at the organizational level. Several studies, such as by

Lynn and Akgün (2001) and Deschamps (2008), base their models of technological

vision on these studies of organizational vision as a way to address this gap. Most

frequently cited is the study about visionary companies by Collins and Porras (1991,

1996, 1997), who analyzed 20 Fortune 500 companies, selected from a popularity poll

with CEOs. They posit that a vision consists of two major components: core ideology,

which is captured in core values and core purpose, and envisioned future, which consists

of a 10-to-30-year daring goal, plus vivid descriptions of what it will be like to achieve

the goal. Collins and Porras found that vision was a tricky term, noting that, “vision has

become one of the most overused and least understood words in the language” (1996, p.

65). As good management consultants, they coined the term “Big, Hairy, Audacious

Goal”, or BHAG for short, to describe an envisioned future (J. Porras, personal

communication, May 5, 2008). The test for a successful BHAG includes several criteria,

notably that it is clear, concise, compelling, “serves as a unifying focal point of effort”,

“acts as a catalyst for team spirit”, engaging, tangible, energizing, and highly focused


20

(2006, p. 75). The authors do not present a process for developing visions, although they

describe the components of a great vision.

Other scholars (e.g., Nanus, 1992; Quigley, 1993) have emphasized similar vision

components within the organizational literature and describe different processes for

formulating corporate visions. Quigley (1994) points out that that “little has been written

on how to communicate vision, how to renew it, and how to sustain it over long periods”

(p. 37). He describes a corporate vision as clear, stimulating, accessible, as well as open-

ended, and he argues that vision must precede strategy within a company. Quigley

outlines a consensus-driven process for the formation of visions, which relies on a series

of internal leadership conferences to build alignment and support from the start.

Deschamps (2008) takes a similar view. He recognizes that a vision for

technology requires special formulation and argues that the process follows analogous

mechanisms as a business vision. Deschamps states, “That process, which can be called

‘technology visioning’, is similar to what process management teams use to sharpen,

reorient or renew their business vision” (p. 147). In his view, a technology vision is a

means of implementing the business vision, and he notes that a typical output is a short

policy statement or charter that guides R&D initiatives, not necessarily a driving view of

the future. Overall, he mixes a technology vision with operational aspects within

corporate R&D, such as group organization and technology acquisition strategy. He shifts

the focus from building radical innovations to the procedures for technology deployment

and management. In a related vein, Christiansen (2000) describes the process of vision

formulation based on near-term innovation horizons:

Vision-building, in any particular market, involves identifying market problems or opportunities, and technical means to solve them. So, if you are looking for a


21

vision, ask yourself if there are customer problems or opportunities you could address. (p. 178)

Christiansen takes a more reactive stance, in which the process of building visions entails

finding an existing customer need for new products, versus creating an entirely new

market. Overall, the scholarship about technology visions, particularly how these visions

are crafted and communicated, would benefit from more in-depth study.

Visionaries and Other Innovation Roles

When discussing visions, it is almost inevitable to extend the discussion to

visionaries. The business literature has a deep history of leadership studies that address

the unique role of visionaries, typically chief executive officers (CEOs) who inspire the

staff through top-down measures (e.g., Quigley, 1993; Sigismund, 2000). Based on a

historical study of HP, a technology company that has pioneered many new product

markets, House and Price (2009) distinguish between two approaches of visionary

leadership. Either top management directs change visibly or instead fosters an

organization culture that allows other visionaries and their visions to emerge.

There is limited scholarship about technology visionaries, who are technically

trained and personally responsible for spearheading the creation of a new market-

disrupting technology. Two related studies by Hebda, Vojak, Griffin, and Price (2007)

and Vojak, Griffin, Price and Perlov (2006) differentiated the technical visionary as being

separate from the CEO role and leading innovation efforts as the big idea driver. They

note: “Technical visionaries are technologists who were involved intimately involved in a

project, from its inception, which represented a significant technical breakthrough, came

to market, and significantly changed life” (2007, p. 433). Rhetorical perspectives from


22

the field, such as commentary by Maccoby (2007), reinforce that another class of

visionary leader exists within technology separate from the CEO. Deschamps (2008)

argues a similar line of thinking, and he considers chief technology officers (CTOs) to be

the primary innovation leaders in companies. Deschamps posits six attributes that define

an innovation leader, which entail disciplined creativity, ability to handle uncertainty and

risk, passion, willingness to explore and experiment with external ideas, the courage to

stop projects, and a talent for recruiting and building innovation teams.

Beyond visionaries, other roles are described in the innovation literature, namely

gatekeepers, project champions, business innovators, technical innovators, and user

innovators (Howell & Higgins, 1990). Gatekeepers interface with the world outside an

organization by helping to acquire, translate, and disseminate company information and

resources. Project champions serve as evangelists, promoting a particular idea. Business

innovators shepherd emerging ideas by providing political coverage and access to

resources, and technical innovators are responsible for the idea’s design and

development. Lastly, user innovators ensure user adoption through training and other

efforts. Of these various roles, Howell and Higgins associate a vision most with the

project champion, stating:

The literature on champions and innovation highlights the capacity of champions to inspire and enthuse others with their vision of the potential of an innovation, to persist in promoting confidence in themselves and their mission, and to gain the commitment of others to support the innovation. (p. 320)

Their data suggests that champions are effective because they appeal to the larger

principles and values, including “the organization’s dream of what it can be”, in others

(p. 336). In their view of the literature, the technical innovator, the one most likely


23

technically trained and familiar with technology, does not take on the visionary role and

instead focuses on an idea’s implementation.

Innovation and DARPA

The Defense Advanced Research Projects Agency (DARPA) is the central

organization for research and development (R&D) for the United States Department of

Defense (DoD). DARPA is an organization entirely defined and driven by radical

technological innovation. Founded in 1958 as a response to the Soviet Union’s launch of

Sputnik, DARPA’s mission has always been to prevent technological surprise. Neil

McElroy, then U.S. Secretary of Defense, stated, “I want an agency that makes sure no

important thing remains undone because it does not fit somebody’s mission”

(Richardson, Larriva, & Tennyson, 2001, p. v).

The current agency director Regina Dugan has continued to uphold DARPA’s

focus, stating: “From its founding more than 50 years ago to current day, this mission

implies one imperative for the Agency: radical innovation for national security”

(DARPA, 2010b). DARPA focuses on revolutionary initiatives with high technical risk,

which are differentiated from the “typically evolutionary efforts” appropriate to the R&D

programs undertaken in the individual military services (Defense Science Board, 1999, p.

vi). The Defense Science Board (1999) considers DARPA to be an “idea” agency

because DARPA looks beyond today’s known needs and requirements. Other scholars

echo this point; DARPA encourages and supports technology advances with non-defense

organizations, both academic and commercial, rather than defense-only organizations, as

its best means of gestating new concepts into implementation (Bonvillian, 2009).


24

Most DARPA projects range in time horizons, in which short-term is defined as

within two years and long-term is beyond 10-15 years (Defense Science Board, 1999).

DARPA also funds disruptive capabilities, which are defined as “more than just new

technologies; they are transformations in operations and strategy enabled by synergistic

combinations of technologies” (Van Atta et al., 2003a). Over the years, DARPA’s

programs have concentrated on major national issues, such as missile defense, nuclear

test detection, information processing, and counterinsurgency.

Organizational Structure

DARPA operates with limited overhead and does not maintain any research

facilities or labs. The rationale is to “minimize any institutional interests that might

distract the Agency from its imperative for innovation” (DARPA, 2007). Much of the

work occurs offsite at the organizations funded by DARPA, presented in Figure 1. How

DARPA works is often attributed to the people within the organization, and these people

have then reinforced and institutionalized certain procedures over time, which then has

established a unique culture at DARPA. As Bonvillian (2006) writes, “An agency is its

culture, not just its enabling statute or organizational chart” (p. 47).


25

Figure 1: Open innovation and network partners of DARPA

DARPA relies on a network of multiple external organizations, which perform the work of innovation offsite, in a business model known as “open innovation”.

Three main staff roles work at DARPA regarding its technology programs: the

agency director, the office director, and the program manager, illustrated in Figure 2. The

DARPA agency director steers the organization, serves as the primary spokesperson, and

determines the overall investment strategy and office structure. There have been 18

agency directors in DARPA’s history, averaging 2.8 years in office (DARPA, 2008). A

deputy director assists the agency director, typically overseeing internal operations and

management.

!"#$%&'()*

+,-.,*

/&'0(1)'2()*3%2"&%4*

4%5"1%6"1'()*

-()(%178*&"&$1"96)*

:"0(1&#(&6-;+*7(&6(1)*

<'4'6%1=*)(10'7()*


26

Figure 2: Basic staff structure at DARPA

A small number of roles are responsible for the technology programs at DARPA. The basic technical roles include the agency director, office director, and program manager, and all personnel in these roles serve short-term contracts.

Reporting to the agency director, an office director oversees a DARPA

technology office, and typically, there are 6-8 active office directors at a given time.

Offices are organized around key development areas in technology and science, and they

have been created and disbanded, as needed, throughout DARPA’s history (DARPA,

2010, April 7). The themes of each office are derived from DARPA’s strategy and set by

the agency director as a result of assessments of technical advances worldwide and

interactions with DoD stakeholders (DARPA, 2007). As examples, the Microsystems

Technology Office (MTO) spearheads pioneering research in integrated microsystems in

order to enable revolutionary performance and functionality for future DoD systems, and

the Information Processing Techniques Office (IPTO) supports research, development,

and prototyping that spans the information lifecycle. Disbanded offices include the Joint

!"#$%&'()*#%+,*'

-.%#'()*#%+,*'

-.%#'()*#%+,*'

/*,"*01'20$0"#*'

(#34+&'()*#%+,*'

/*,"*01'20$0"#*'

/*,"*01'20$0"#*'

5433,*+'5+06'


27

Unmanned Combat Air Systems (J-UCAS) Office and Information Awareness Office.

See Appendix A for details on current DARPA technology offices.

A set of program managers is affiliated with a particular DARPA Office. A

program manager, often abbreviated as PM, is responsible for a particular program or

technology initiative, and programs often serve as an organizational structure for multiple

related projects. Program managers are recruited from multiple areas, including

companies, universities, national laboratories, nonprofit research organizations, federally

funded R&D centers, and the military services. DARPA maintains its entrepreneurial

edge by hiring expert program managers from industry at competitive salaries and

through faster procedures than normal civil service rules due to special authority from

Congress (DARPA, 2007). Program managers bring deep technical knowledge and

connections:

So what makes a DARPA PM unique? First and foremost, they’re subject matter experts, either already a leader in their technical community, or prepared to become a leader as they take on the challenge of creating and managing DARPA programs. They’re driven with the zeal of a venture capitalist, driven by the conviction that their program is creating a major technological advance. They’re ready and willing to take risks. Not a venture capitalist’s financial risk, but more precisely, a professional risk, the risk of trying to transform a far-side scientific or engineering insight into demonstrable warfighter capability, and with no guarantees of success. (Leheny, 2007) Approximately 140 technical program managers serve at DARPA, and they often

work with light oversight from DoD management (DARPA, 2007). While program

managers were always hired into DARPA on temporary contracts, a four-year rotational

cycle was enforced in the early 1990s as a way to improve quality and technical expertise

(Defense Science Board, 1999). By shortening contracts, at least three challenges arose

(ibid.). First, internal collaborations, particularly between DARPA Offices, were driven


28

more by staff personality than programmatic needs. Second, program managers were

motivated to plan in shorter shifts based on their contracts, rather than longer-term

horizons. Third, institutional memory and continuity dropped significantly, resulting in

an annual personnel turnover rate of approximately 25 percent (DARPA 2008b).

Due to the staff rotation policy, DARPA program managers keep the

organizational culture dynamic by regularly infusing the agency with new ideas and

visions. The program managers are perceived as being visionaries because they are

directly responsible for imagining and realizing their visions of technology. One report

states the association clearly: “These individuals must be highly motivated visionaries

who are successful at both management and technology” (Defense Science Board, 1999,

p. 7, emphasis added). Moreover, DARPA program managers are expected to seek

results, rather than just explore their visions. Having both a vision and the ability to

facilitate building the vision is a critical hiring attribute because program managers bring

with them “an inherent desire to accomplish a vision or goal within their tenure” (p. 8).

Program managers have developed an almost mystical power in scientific and

popular press for their ability to imagine and build new ideas in a compressed timeframe.

For example, Malakoff (1999) notes in Science that “some [program managers] become

influential figures in their subfields, capable of nudging established research communities

in a particular direction or creating collaborations where none existed before.” Esquire

magazine (Junod, 2003) wrote similarly:

All of them are on contracts that are finite, by nature and definition. That’s the signal element of the DARPA culture: It’s a government job, but it’s the opposite of the civil service, and even of academia. You don’t get tenure. You don’t get security. What you get, generally, is about four years – four years to do something that’s never been done before. Four years to do the impossible… It’s what accounts for the fact that whole fields of inquiry rise up around the enthusiasms of


29

program managers – fields of inquiry that weren’t there before and that keep going after the program manager is gone. Call them program managers if you want, but really, if the whole DARPA ethos works, they become fathers of their fields. Mothers of invention. Inventors. Several additional groups work at DARPA, and supporting personnel include the

DARPA Small Business Programs Office (SBPO), contract management office, and

facilities office (DARPA, 2010d). In addition, Systems Engineering and Technical

Assistance (SETA) contractors are civilian employees or government contractors who are

hired to assist with DoD programs, and a DARPA program managers may assign several

“SETAs” to a program who provide project management and administrative assistance.

In addition, various contracting officer representatives, who are trained and certified by

the DoD, help DARPA program managers to negotiate and maintain project contracts

with the teams that they fund. Operational liaisons from each U.S. military service and

the National Geospatial Intelligence Agency further facilitate the technology transfer

process for DARPA programs (DARPA, 2010e).

Processes of Innovation

DARPA’s focus is radical innovation, and its processes are designed to enable

this goal. The agency does not have a systematic process for deliberately planning for the

future; instead, DARPA encourages a decentralized process, trusting office directors and

program managers to set their own research portfolios (Defense Science Board, 1999).

One recent report summarized the internal process as a series of “fast, flexible, and

informal cycles of ‘think, propose, discuss, decide, and revise’ (DARPA, 2007). DARPA

has been remarkably consistent in its approach to innovation. Work from Fuchs (2010)

examined the way program managers planned new technologies within one DARPA


30

Office, finding that the specific processes have not changed over time, but rather the

situations in which program managers have applied these processes. The National

Research Council (2005) describes DARPA’s decision-making process as unique among

other government agencies, stating: “It is informal and relatively flexible, taking a top-

down approach to problem definition and a bottom-up approach to generating ideas and

solutions, with the key emphasis on technical merit” (p. 14).

The Institute for Defense Analyses identified six management practices that have

facilitated DARPA’s development and exploitation of various disruptive capabilities for

military advantage (Van Atta et al. 2003a, Van Atta et al. 2003b). Across different

programs and applications, these six practices are identified as follows (p. S-1):

• Investing in basic technologies that can lead to fundamental technical advantages;

• Building communities of change-state advocates; • Defining strategic challenges in detail across multiple scenarios; • Supporting the conceptual development of integrated, disruptive capabilities; • Testing promising disruptive capabilities in large-scale, proof-of-concept

demonstrations; and • Working with leadership from the Office of the Secretary of Defense (OSD)

to broker Service commitment to the implementation of particular disruptive capabilities.

Research from Richardson and his colleagues (1999) mention similar

management practices. For DARPA’s Technology Reinvestment Project (TRP), one of

the largest commercial investment programs ever undertaken by the DoD, the agency

pioneered a new mode of development called “dual use”, which refers to technology

development that accommodates commercial interests while maintaining military utility.

Several new management processes and tools were introduced to make this program

work, which included the use of cost share, agreements tailored to projects, consortia, and


31

flexibility on intellectual property rights (IPR) and foreign participation, many of which

helped influence other DoD organizations.

Bonvillian (2009) introduces the notion of challenges, which he attributes to the

influence of Vannevar Bush during WWII. In short, project teams were brought together

around and inspired by major technology challenges, such as radar or atomic weapons.

More importantly, the people who worked on these challenges were able to connect their

work on technology breakthroughs to the follow-on stages of development, prototyping

and production. Bonvillian feels that DARPA effectively upholds this connected model

by tying invention to innovation.

In addition, Bonvillian compares DARPA’s way of working with great groups

from history, namely Edison’s “Invention Factory” in the 1900s, MIT’s Rad Lab in the

1940s, and Bell Labs in the late 1940s, which all produced amazing new developments in

technology. Bonvillian notes similarities with other high-risk R&D efforts: “Many of the

organizational features of these three ‘great groups’ are common to others, including the

development of atomic weapons at Los Alamos, the integrated circuit and microchip at

Fairchild Semiconductor and Intel, the aeronautics and stealth advances at Lockheed’s

Skunk Works, the personal computer at Xerox Parc and Apple, biotech at Genentech and

Craig Venter’s genomics projects” (p. 218). According to Bonvillian, these organizations

share common characteristics, which include a highly collaborative team of experts

pulled from multiple disciplines, a flat and meritocratic organizational structure, and

direct access to top decision makers. DARPA develops this model further, and Table 2

summarizes the 12 organizing elements that define DARPA’s model for successful


32

innovation, as presented by the agency across its various materials (Bonvillian, 2006, p.

46-47).

Element Description Small and flexible DARPA consists of only 100–150 professionals; some have referred to

DARPA as “100 geniuses connected by a travel agent.” Flat organization DARPA avoids military hierarchy, essentially operating at only two

levels to ensure participation. Autonomy and freedom from bureaucratic impediments

DARPA operates outside the civil-service hiring process and standard government contracting rules, which gives it unusual access to talent, plus speed and flexibility in organizing R&D efforts.

Eclectic, world-class technical staff

DARPA seeks great talent, drawn from industry, universities, and government laboratories and R&D centers, mixing disciplines and theoretical and experimental strengths. This talent is hybridized through joint corporate-academic collaborations.

Teams and networks

At its very best, DARPA creates and sustains great teams of researchers that are networked to collaborate and share in the teamʼs advances, so that DARPA operates at the personal, face-to- face level of innovation. It isnʼt simply about funding research; its program managers are dynamic playwrights and directors.

Hiring continuity and change

DARPAʼs technical staff are hired or assigned for three to five years. Like any strong organization, DARPA mixes experience and change. It retains a base of experienced experts who know their way around DoD, but rotates most of its staff from the outside to ensure fresh thinking and perspectives.

Project-based assignments organized around a challenge model

DARPA organizes a significant part of its portfolio around specific technology challenges. It works “right-to-left” in the R&D pipeline, foreseeing new innovation-based capabilities and then working back to the fundamental breakthroughs that take them there. Although its projects typically last three to five years, major technological challenges may be addressed over longer time periods, ensuring patient investment on a series of focused steps and keeping teams together for ongoing collaboration.

Outsourced support personnel

DARPA uses technical, contracting and administrative services from other agencies on a temporary basis. This provides DARPA the flexibility to get into and out of a technology field area without the burden of sustaining staff, while building cooperative alliances with the line agencies it works with.

Outstanding program managers

In DARPAʼs words, “The best DARPA Program managers have always been freewheeling zealots in pursuit of their goals.” The DARPA directorʼs most important job historically has been to recruit highly talented program managers and then empower their creativity to put together great teams around great advances.

Acceptance of failure

At its best, DARPA pursues a high-risk model for breakthrough opportunities and is very tolerant of failure if the payoff from potential success is great enough.


33

Element Description Orientation to revolutionary breakthroughs in a connected approach

DARPA historically has focused not on incremental but radical innovation. It emphasizes high-risk investment, moves from fundamental technological advances to prototyping, and then hands off the production stage to the armed services or the commercial sector. From an institutional innovation perspective, DARPA is a connected model, crossing the barriers between innovation stages.

Mix of connected collaborators

DARPA typically builds strong teams and networks of collaborators, bringing in a range of technical expertise and applicable disciplines and involving university researchers and technology firms that are usually not significant defense contractors or beltway consultants (neither of which focus on radical innovation). The aim of DARPAʼs “hybrid” approach, unique among American R&D agencies, is to ensure strong collaborative “mindshare” on the challenge and the capability to connect fundamentals with applications.

Table 2: Twelve organizing elements of DARPA

Bonvillian (2006, p. 46–47) outlines 12 organizing elements that define DARPA’s model for successful innovation, many of which are shared with other great groups in history, such as MIT’s Rad Lab and Bell Labs in the 1940s.

At DARPA, project managers are empowered to make decisions quickly, and the

agency minimizes bureaucracy by distributing administrative tasks and contract

negotiations through a large support staff. For example, in one large initiative, program

managers were not required to follow normal acquisition procedure, instead skipping

initial decisions concerning “make or buy” and “co-develop or procure” (Richardson et

al., 1999, p. vii). In fact, DARPA personnel are exempt from standard DoD training for

acquisitions (L. Schuette, personal communication, October 16, 2009). In addition,

DARPA received additional authority in 1993 to conduct prototype projects outside the

traditional contracting statutes (Defense Science Board, 1999).

Among its network, DARPA encourages diverse collaboration on projects. For

example, projects in the Technology Reinvestment Program regularly combined defense

teams with teams from academia, the commercial sectors, and other nonprofit

organizations, and with the right combination, teams were often able to address a broader


34

set of problems than an organization working alone (Richardson et al., 1999). A former

program manager describes similar mixing while at DARPA, noting cycles of

competition and hybridization between university and industry researchers: “These cycles

of competition and hybridization lead researchers to quickly adopt the best of each

other’s ideas and are highly effective in accelerating the transfer of people and

technology across university and industry boundaries” (Tennenhouse, 2004, p. 22). Fuchs

(2010) has found that collaboration across groups within DARPA has increased since

2001. Before 2001, DARPA program managers relied more heavily on successfully

published professors, whom Fuchs calls “star scientists”, to work on DARPA-funded

projects, and after 2001, program managers engaged a broader mix of subjects, namely

established vendors within universities, industries, and start-ups. One common

mechanism for exchanging knowledge was through invite-only workshops (Fuchs, 2010).

In addition, DARPA takes an unusual approach to intellectual property rights

(IPR). Unlike most businesses and government labs, which rely heavily on IPR to

maintain a competitive advantage, DARPA does not file patents or claim legal ownership

to the various technologies that it funds. Instead, the agency allows ownership and

control to reside within the teams. DARPA desires visibility to ideas of radical

innovation, as well as the influence to ensure that these ideas are provoked and built

(Defense Science Board, 1999). Richardson and his colleagues (1999) explain, “By

allowing industry to retain rights to the intellectual property generated during their

project, government gained early access to some of industry’s best ideas” (p. xi).


35

Measures of Program Success

Arguably, the first measure of success is the actual physical instantiation of a

daring idea. In other words, can you build it? For its 50-year anniversary, DARPA

chronicled its major contributions to the world, which include interactive computers,

internet computing, virtual reality, intelligent systems, stealth technology, and more (Van

Atta, 2008). Once new technologies are built, DARPA considers the effective transfer of

technology to the U.S. military services as its primary measure of success. One internal

report (DARPA, 1998) concludes that: “The success of DARPA has been measured

historically by the transition of its technologies and concepts into military capabilities in

the hands of U.S. forces. By that measure, the Agency has been phenomenally successful,

considering its size” (p. 9). The same report notes that nearly every current military

system has been affected to a substantial degree by DARPA work in information

technology, microelectronics, and materials.

Collaboration with industry partners plays a significant role in the success of

DARPA’s programs in the marketplace. According to Richardson, Larriva, and Tennyson

(2001), part of the agency’s success comes from its ability to seek a broad range of

customers, unlike similar R&D counterparts in the military services that typically address

one or two groups of users. After tabulating transfers of technology to the military

services for three periods in DARPA’s history, they found that that highest impact came

from working with industry partners, which was pursued roughly 30 percent of the time.

In DARPA’s Technology Reinvestment Program, which funded dual use technologies, 33

percent of the projects were already selling their technologies or products in the

commercial market (Richardson et al., 1999).


36

Conclusion

Two general conclusions can be drawn from the past research on radical

technological innovation, which identify a critical gap in the literature. First, although

organizations deeply desire radical innovation, their efforts tend to be short-lived, often

lasting less than four years. Little is understood about the specific mechanisms that

organizations should pursue on a repeatable basis and which factors are influential or

decisive to the start of the overall process. Second, the role of vision in radical innovation

has not been well understood or documented. Scholars have been heavily influenced by

numerous organizational studies, which address corporate vision, and a research

opportunity exists to define and clarify different dimensions of vision as related to

technological innovation.

DARPA provides an example of an organization that has consistently sustained

radical technological innovation over 50 years. As a networked organization with high

staff turnover, the agency presents several challenges in data collection. The next chapter

presents the methodology used to document and uncover DARPA’s model of innovation.


37

CHAPTER 3

METHODOLOGY

A qualitative approach was selected for this research study in order to most

effectively address the proposed research question. When variables are unknown, as in

this case, a qualitative approach enables the discovery and identification of the governing

variables. The goal of qualitative research is to discover patterns forming a complex and

holistic picture, which emerge after close observation, careful documentation, and

thoughtful analysis of the research topic. In addition, using qualitative methods in

empirical studies within engineering allows researchers to address the human aspects that

are often overlooked (Seaman, 1999). This section describes grounded theory

methodology as the research method.

Grounded Theory Methodology

In this study, data was collected and analyzed using grounded theory

methodology. This methodology has a long history, and since the late 1960s, it has been

used extensively to address questions of interest to researchers in various fields, including

medicine, social science, and management (Glaser & Strauss, 1967; Goulding, 2002;

Bryant & Charmaz, 2007; Corbin & Strauss, 2008). Instead of testing theory developed

by other scholars, grounded theory methodology builds new theory directly from data, as

illustrated in Figure 3 (adapted from de Vaus, 2001, p. 6).


38

Figure 3: Theory building and theory testing approaches to research

As a theory-building approach, grounded theory methodology uses inductive reasoning to craft a new theory from observations in the field (adapted from de Vaus, 2001, p. 6). Theory-building approaches are recommended when studying new or uncharted areas of research.

Grounded theory begins with observations and uses inductive reasoning to derive

a theory from these observations. Theories are general explanations based on a large

amount of data. The aim is to generate hypotheses that fit the data, not to build up the

weight of evidence necessary to confirm existing hypotheses. This method was adopted

because it allows for discovery when studying new or uncharted areas of research in

which little theory has been developed, as evidenced by the literature review about

DARPA’s innovation process. The method also supports the study of things in real-world

!"#$%&'()*+,+

!"#$%&'()*+-+

!"#$%&'()*+.+

!"#$%&'()*+/+

0$1++23$)%4+

!"#$%&'()*+,+

!"#$%&'()*+-+

!"#$%&'()*+.+

!"#$%&'()*+/+

567#(*8+23$)%4+

!"#$%&'()*(+,-"."/)

0(#$%&'()*(+,-"."/)

23$)%4+"97:;7*8+'<<%)'=3+

23$)%4+>$#(*8+'<<%)'=3+


39

contexts, attempting to interpret phenomena in terms of the meanings that people bring to

them.

Grounded theory seeks the discovery of unforeseen activities in natural activities

and settings. By applying grounded theory to DARPA, the intention is to uncover the

inner workings of the agency in terms of radical innovation. Hypotheses are not

developed in advance, and working explanations are instead allowed to emerge

throughout an iterative process of data collection and analysis. Data saturation occurs

when data categories and themes become repetitive, and nothing substantially new is

learned.

The final output is a new theory, which may be explanatory (what is the nature of

the phenomenon), descriptive (why the phenomenon occurs), or predictive (how the

phenomenon could reoccur under similar future conditions). In applied fields, good

theory is required to be of practical value, ultimately to “explain the meaning, nature, and

challenges of a phenomenon, often experienced but unexplained in the world in which we

live” (Lynham, 2002, p. 222). Good theory results in two kinds of knowledge: outcome

knowledge, which is usually used to help explain and predict a phenomenon, and process

knowledge, which is used to increase understanding about how something works and

what it means (ibid.).

Iterative Process of Data Collection and Analysis

Grounded theory methodology relies on an iterative and systematic process of

data collection and analysis, and each cycle of data analysis determines the next cycle of

data collection. Data collection is based on empirical evidence, and interviews are


40

frequently the main source of information in grounded theory. Based on the study

requirements, interviews were chosen in this study. Direct observation was not possible

due to the historical perspective desired, security considerations of multiple projects, and

distributed nature of the innovation work at DARPA. An excerpt from a raw interview

transcript follows as an example of data collected in this study:

Data Example: Excerpt From an Interview Transcript “One of the things about program managers – I donʼt know if the others found this as well – but as a program manager, my perspective was that once you funded somebody, you remained involved. At those times, you were remaining involved and you would go out every 3-6 months or sometimes more frequently, but you would go out a visit with them with the purpose of seeing how things were going, and if they are not going as well as youʼd like, see what you can do to help them, even if it required more money, but whatever it would take, just to help them get either back on track or hire somebody else to get something done that they werenʼt able to get done. It was truly, in my mind, a partnership between the program manager and the contractors, the ones that were actually doing the work, and I almost felt like a venture capitalist – if you need more money Iʼll get it for you, type of thing.”

In grounded theory, data analysis begins by coding the interview transcripts and

field notes. The researcher defines a set of different codes, or categories, based on a

particular theme or idea that is of interest in the study. Coding is a dynamic, intuitive and

creative process of inductive reasoning, thinking and theorizing. These codes are then

assigned to specific portions within the text or interview transcript. In coding concepts, a

researcher must pay close attention to the words and meanings of the subjects, as

contained in the field notes, to make sure that they are being interpreted accurately.

Codes can be either preformed or post-formed based on the objectives of the study.

Various passages of text are then grouped into patterns according to the codes and sub-


41

codes they have been assigned. Seaman (1999) notes that “intuition may help guide the

process of analyzing the data, but it does not constitute conclusions unless it is clearly

supported by the data” (p. 567).

For this study, coding started with open coding, in which codes were identified

without any restrictions. Example codes in this study included “processes for discovery”,

“definitions”, “DARPA Hard”, and “community”. The purpose was to compare emerging

properties and dimensions between codes, as well as to discover promising nuggets that

could lead to meaningful findings. Open coding produces a preliminary structure of

categories and sub-categories, which are then refined through subsequent cycles of data

collection and analysis.

As part of each cycle of coding, a researcher writes a field memo that articulates a

proposition (working hypothesis) or an observation synthesized from the coded data. The

field memo is meant to be an informal way to record a researcher’s discoveries quickly

before they are lost. Throughout analysis, a researcher attempts to gain a deeper

understanding of what he or she has studied and to continually refine interpretations of

the data. More detailed memos also show how strong or weak the support for a particular

proposition is thus far in the data collection process. An excerpt from a field memo in this

study follows as an example:


42

Data Example: Excerpt From a Field Memo August 23, 2009 One observation emerging from my data is that the type of PM has significantly changed over the years. At least within IPTO, in the early days, several PMs were graduate students – some fresh starting their doctorates. Now they are largely senior and accomplished (narrow?) experts. While youth brings enthusiasm, age has experience… and more connections? As Iʼve seen in the foresight work at Stanford, age matters. Graying PMs would also age the agency in many ways, bringing / creating different views of innovation. I should explore this idea further in the next interviews.

A specific type of memo is an integrative memo, which aims to synthesis previous

concepts and coding threads, including making connections with previous research. An

example of an integrative memo for this study follows:

Data Example: Excerpt From an Integrative Memo November 7, 2009 I feel that OʼConnor (2008) and team are dismissive of projects, which they felt were disconnected and one-off efforts. Instead, they recommend a platform approach because platforms “can be the foundation for a variety of business models, products, and applications” (p. 62) either around a technology or a business problem. It is interesting to compare this viewpoint to DARPAʼs structure. Perhaps the agencyʼs offices are equivalent to meta-platforms, in which the programs serve as platforms, and the projects are units of innovation that can be easily funded, tracked, and transferred.

In this study, the process of coding and drafting field notes was manually handled.

Done mainly for convenience, manual coding in a field log (notebook) supported faster

reflection in a greater variety of locations, particularly when a computer was not readily

accessible. The decision to use manual coding is dependent on the size of the project, the

funds and time available, and the inclination and expertise of the researcher (Basit, 2003).


43

Theoretical Sampling

Ideally, data collection and analysis occur concurrently, and the feasibility of the

new proposition is then checked in the next round of data collection. This procedure is

called “theoretical sampling”, which lets the research guide the data collection (Corbin &

Strauss, 2008). Instead of sampling people, the focus is on the core concepts within the

data. The act of theoretical sampling is used to maximize opportunities to develop

concepts in terms of their properties and relationships with other concepts, and the

sampling decisions evolve during the research study. This means that relevant concepts

are elaborated upon and refined through purposeful gathering of data pertaining to these

concepts. Corbin and Strauss explain that, “The researcher is purposefully looking for

indicators of those concepts so that he or she might examine the data to discover how

concepts vary under different conditions” (p. 144).

New concepts might generate questions, and these questions lead to more data

collection. As new insights are gained or new leads open up, the researcher can shift

perspective quickly and explore new areas of inquiry. This is not possible with most

quantitative instruments, such as surveys – because once survey questions are set, all are

asked in the same way for all respondents. In contrast, grounded theory allows the

researcher to have greater flexibility in gathering relevant data. Theoretical sampling

continues until the point of saturation, when all concepts are well defined and explained.

In this way, the research process feeds on itself as continual redesign, and Corbin and

Strauss liken the researcher to a detective, who follows the leads of promising concepts


44

as clues. Figure 4 illustrates the iterative process of data collection and analysis in

grounded theory methodology.

Figure 4: Iterative process of data collection and analysis

Grounded theory methodology relies on an iterative and systematic process of data collection and analysis, and each cycle of data analysis determines the next cycle of data collection.

Method Limitations

As with all methods of research, grounded theory methodology has limitations

that can be easily minimized during a research study (Corbin & Straus, 2008). First,

careful training and strong attention to detail can help prepare the researcher to conduct

interviews and record observations completely and accurately. Considerable amount of

training and skills is required in collecting qualitative data appropriately. In these studies,

# ___

INTERVIEW FIELD NOTESPrimary Role? Director / Program Manager / ~~tor / Other:DARPA technical area (office)? f/’S-~, /f’1~, ~5S Primary location? (U.S. state) 13<

Start date with DARPA: ~ ~7Contract length:

Reports to: ~/~w~5# of Projects: ______ Team size: _________

Gender? MAge bracket? ~2J2 / 21-30

_____ / 41-50 / 51-60 / 61-70 /≥2.Q

4/~

~-~~ ~

~~ ~

- •j~I~~/~f~

~

___ ~1~~C~7 ~~

~- /k~- mA~t~1~,( ,~ ~t~Jci2~ ~ ~r1~ t~5~j~ /e~’~

~

~~t~-~cy

~Ic ~7~ ~A~PPr~~‘k ~

~J.Pm ~i~e /~ ~ L~~4kr, ~:~(

4 ~~efr /i~s h~. ‘~‘~

~ /~/~ 7~— ~~ ~

~&~

7~~~jJQ~~y

T~7J~)~~fr/fl~)~7-~~5o~,~I

pp~~1~~

~t~J~

/1’~r~7~/ia’1I~w~/~~iP~‘~

~~~-

~)~YI)~2~~~~d~

~)2~47~~/‘I~

~~~i))r~~1~S~rT~2V~~I~

~1~~-rJ~1~r~~I~-~1Y~fC~71~7I~?°~~~

j7’v2f~~

IrI)

Tr~2~ ‘I

~5/J

Primary Role?_____

__________

DARPA technicalarea (office)?

Start date with DARPA: / 9’~’ 5~

Reports to:I~

Genderi~/ F

~W

INTERVIEW FIELD NOTES

Pro~r~rn Manager / Contractor/ (Wh~r~

# ____

Primary location?(U.S. state) (~4

Contract length:Ic 741,4C_{

#ofProjects: _____

Teamsize: ________

Age bracket? ~2J2 / 21-30 / 31-40 /41-50 / 51-60 / 61-70

/~)

(k~’e ~1 4~€L~~~

~ (7~’ ~

Lv1~ttI,

y’k~~~ 5~fr (mi~) - 4~s~~~

~

~75 ~

C’ ~~

F’ ~ /~ ~

~ /~ ~

~x ~~o,v€~,, J,~JJ(P

frt~C-(

~~

~ ~

li 11~ fl4~Qf/ ~4~-d~ ~hi~t/9~P4-, A~

1-~11~~ ,1 ~

j~tf d~- c,~4 (‘~I-€~ ~ ‘I ~/~

— ~iW4 ~ -

~z~LU~ Ci~—&o( 4~- 1,~ ~‘/~‘~“~ ~

~ ~rf ~;.-~.~

__

~~~ II __

~ /t~

~) /~~ (~ )~ ~j

~

~‘- ~- ~

‘~‘ ~

- I, ~~

!"##$%&'()&)'!  "#$%&'()*$(+,-*+./)0$%)'01(&,+)2+3%)$#(+/))

!  4,*(+)2+3%)5+5#/)(#)0,6'&30(+)$+.)1,#1#/*6#$/)

!  7+-+3#1)%*08,05/)0$%)5#%+3/)

!"*$'()&)'!  "#%+)%0(09)(,0$/',*1(/)0$%)%#'&5+$(/)

!  :,#'+//)#;)<(=+#,+6'03)/0513*$8>)(#)*%+$6;?)@+?)*$%*'0(#,/)&1)(#)(=+)1#*$()#;)%0(0)/0(&,06#$)


45

the investigator is the research instrument. He or she conducts the interviews, records the

observations, and then analyzes and interprets them.

Second, ingrained beliefs and cultural predispositions are often hard to escape,

and the researcher may bring a potential bias that influences the direction of data

analysis. Therefore, a researcher can help minimize bias by keeping a personal journal

along with the field documentation and being relentlessly reflective throughout the

process of data collection and analysis.

Third, there is a risk that deficient coding categories support the final theoretical

argument. Researchers trained in grounded theory methodology become so familiar with

the data that they rigorously examine it at each cycle of data collection, and the constant

refinement process removes the likelihood of incomplete or subjective analysis. Corbin

and Strauss (2008) suggest relying on additional data sources to verify the coding

categories to further strengthen the analysis. In this study, historical interviews and

archival documents help to crosscheck interpretations in the primary interview data.

Conclusion

As a theory-building approach, grounded research methodology provides the

framework for data collection and analysis in this study. Grounded theory methodology is

valuable when studying new or uncharted areas of research in which little theory has been

developed. Data collection begins with observations and interviews in the field, which are

then coded in categories. As part of coding, field memos are written to reflect on and

synthesize emerging insights. This process of analysis determines the next cycle of data


46

collection, and through theoretical sampling, the analysis is ultimately refined until the

point of data saturation. The next section describes the data set in detail.


47

CHAPTER 4

DATA SET

This study was designed as exploratory research, intended to generate a plausible

and compelling new theory, as well as thick description about a particular phenomenon.

Primary data was gathered through interviews and further supplemented and validated

through historical interviews and documents. Research protocols are described in detail

below.

DARPA is a good data set for at least three reasons: predefined scope, long

timeframe, and a clear unit of study. The first reason relates to organizational clarity. As

described in the previous chapter, the agency’s entire charter is predicated on radical

innovation, which DARPA defines as “high risk, high reward”. All efforts from planning

to application undertaken by DARPA personnel fall under radical innovation, based on

the agency’s own interpretation, which removes the need to filter conflicting or

unnecessary activity. In a study commissioned by DARPA, Van Atta and his colleagues

(2003a) state: “Vision, the primary role of DARPA management, involves conceiving,

developing, and demonstrating disruptive capabilities… Perhaps the most important

effect of DARPA’s work is to change people’s minds as to what is possible” (p. S-12).

Subsequently, studying DARPA provides a clear and predefined scope for radical

innovation.

The second reason is an extended timeframe, and DARPA provides a repeated

process of radical innovation. The agency demonstrates over 51 years of sustained and

successful practice, evidenced by multiple new and different technologies built over time


48

– including the Internet, the global positioning system (GPS), virtual reality, and stealth

airplanes. In terms of national impact, DARPA (1998) reports that nearly every current

military system has been affected to a substantial degree by their work. In terms of

business impact, 33% of dual-use projects were already selling their technologies or

products in the commercial market (Richardson et al., 1999). In all cases, an assumption

can be made that organizational processes are both successful and consistent across the

different eras within DARPA’s history.

A third reason is that it is relatively easy to identify the unit of study, which is the

DARPA program manager and his/her program vision. Each DARPA program manager

manages a small number of programs, and each program has one vision that results in a

formal document called a Broad Agency Announcement (BAA).

Two questions may arise when selecting DARPA as an organization to study: Is

DARPA actually innovative? And does DARPA’s work in technology generalize outside

the realm of national defense? It is true that DARPA program managers do not build the

innovations with their own hands; instead, based on their personal visions of technology,

they define, fund, foster, and facilitate innovation with the help of others. As an

organization, DARPA is critical to the overall lifecycle of innovation because many new

engineering ideas and their related implementations may not have been developed as

rapidly or invented without the agency’s support. Van Atta and his colleagues (2003a)

explain DARPA’s role as an accelerator: “DARPA acted as a catalyst for innovation by

defining research programs linked to DoD strategic needs, seeding and coordinating

external research communities, and funding large-scale demonstrations of disruptive

concepts” (p. S-10). Some critics might constrain DARPA’s influence to invention only,


49

arguing that DARPA funding recipients deliver singular prototypes. This is an incorrect

view. In actuality, there is both intent and action to commercialize. DARPA program

managers are expected to guide all their projects toward military and commercial

applications and, when ready, transition the technologies to the appropriate users. Thus,

DARPA technical staff demonstrates an unmistakable understanding of their role in the

broader innovation lifecycle.

Lessons from DARPA also extend past the realm of defense. Although DARPA-

funded projects are largely designed for military use, scholars such as Bonvillian (2009)

point out the shared relationship between innovation pursuits in the commercial and

defense sectors. He writes, “Defense technology cannot be discussed as though it is

separate and apart from the technology that drives the expansion of the economy – they

are both part of the same technology paradigms” (p. 207). Both types of technology

ultimately contribute to societal wealth and economic growth in the same economic

system. One national newspaper made a similar point to its readers: “While many

military technologies go on to find civilian applications, one might say that this is

especially true of DARPA’s successes – the agency is perhaps already quite interested in

new technologies as much for their general coolness and applicability as for their military

potential” (Page, 2009). Returning to two of DARPA’s many innovations, today’s world

of GPS in your hand and Internet everywhere refute this claim.

Data Sources

In this study, primary research was gathered through in-depth interviews.

Interviews offered a way to hear directly from subjects and gain further insight into their


50

motivations and beliefs about their experiences at or with DARPA. The assumption was

that personal stories would provide meaning and context for behavior. Generally,

interviews allow greater depth for research questions where knowledge is often taken for

granted, not readily articulated by most members, and/or subjected to multiple

interpretations and perspectives – all of which hold true at DARPA – than other research

instruments.

Interviews were conducted over a three-month period from September to

November in 2009. Subjects were recruited through purposeful sampling, in which

subjects are selected based on a particular characteristic. Subjects were first screened to

confirm that they were related to DARPA in one of three groups: DARPA personnel,

DARPA funding recipients, or staff who worked with DARPA in related R&D roles

(e.g., director of innovation at the Office of Naval Research). Subjects who were DARPA

personnel worked either as an agency director, office director, deputy director, or

program manager at the organization.

The study’s sample was then expanded through a snowballing process, a research

technique that relies on existing study subjects to suggest future subjects from among

their personal network. (As the term indicates, the sample group appears to grow like a

rolling snowball.) This sampling technique is useful when hidden populations are

difficult for researchers to access. In this case, previous studies and knowledge about

DARPA demonstrated that program managers and funding recipients are tightly

connected through closed networks. As more subjects were recruited, they were

deliberately selected to reflect different offices and points in the agency’s history in order

to maximize population diversity. To avoid a sampling bias, a broad net was cast to


51

recruit subjects who came from different industries, served in a range of DARPA

program offices, and lived in different U.S. regions.

Conducted by phone or in person, interviews were semi-structured in format,

following standard protocols (Seidman, 2006). Interview questions were drawn from

themes within the literature review and included a mixture of open-ended and specific

questions, designed to elicit both foreseen and unexpected information. For example,

subjects were asked to explain the concept of “DARPA Hard”, which is described in

detail in the next chapter, and their experience with any efforts pre-program. Following

grounded research methodology, interview questions were further refined in each data

cycle to gather additional evidence to clarify and/or refute potential propositions.

Interviews averaged 60 minutes in length, totaling 2632 minutes or approximately 44

hours of interview time. All responses were treated as confidential and anonymous. Field

notes were also taken during the interviews and provided 157 pages of raw

documentation. When possible, interviews were audio recorded with each subject’s

permission; however, not all interviews could be recorded.

An additional 12 interviews were drawn from an oral history database maintained

by the Charles Babbage Institute (CBI), a research center dedicated to promoting the

study of the history of computing. The CBI interviews were conducted from 1983 to

1993 with key figures in DARPA’s history, including several individuals who have since

passed away, such as J. C. R. Licklider (a well-known DARPA office director, who

provided the initial vision for the Internet) and Charles Herzfeld (a DARPA agency

director). Running 475 pages, the transcripts from the CBI interviews helped supplement

and validate the emerging themes in the primary interviews. A coding process similar to


52

the primary interviews was applied to the analysis of these oral history transcripts. An

example excerpt from a CBI historical transcript follows:

Data Example: Excerpt From an Historical Interview Transcript “One thing that stands out was there was a large number of helper organizations that had been hired to help design, run and evaluate programs. A lot of money went into that, and we were under constant pressure to explain what were we doing with all these helper organizations. So we finally put together a sort of master plan about who does what, and who is good for what, at what, and so on. We had all that worked out. Then there was a kind of built-in natural, but I think very appropriate, tension between the people who were thinking in terms of systems, people who were thinking in terms of the technology and science, and people who were thinking about the military utility.”

Documents from DARPA’s public archive and personnel provide a third data

source. These documents include program technology reports, reflections from the

agency’s 50-year anniversary commemoration in 2008, and various agency symposium

proceedings. Similar to the CBI oral histories, the documents helped check the language

and details provided by DARPA personnel in the primary research interviews. An excerpt

from DARPA archival documents follows as an example:


53

Data Example: Excerpt From a DARPA Document “DARPAʼs mission implies one imperative for the Agency: radical innovation for national security. DARPAʼs business processes reflect this in a straightforward way: bring in expert, entrepreneurial program managers; empower them; protect them from red tape; and quickly make decisions about starting, continuing, or stopping research projects. To maintain an entrepreneurial atmosphere and the flow of new ideas, DARPA hires program managers for only 4 to 6 years because the best way to foster new ideas is to bring in new people with fresh outlooks. New people also ensure that DARPA has very few institutional interests be- sides innovation. New program managers are willing to redirect the work of their predecessors – and even undo it, if necessary. Since program managers are not at DARPA for a career, they are willing to pursue high-risk technical ideas even if there is a reasonable chance the idea will fail.” (DARPA, 2009, p. 1)

Subject Profile

A total of 47 primary interviews were conducted, which comprise 24 interviews

with DARPA personnel, 18 with DARPA funding recipients, and 5 with related R&D

roles. Related R&D roles included, for example, the director of innovation at the Office

of Naval Research and the deputy director of the National Science Foundation, both of

whom have worked closely with DARPA in their careers.

The supplemental CBI interviews comprise 10 interviews with DARPA

personnel, one with a DARPA funding recipient, and one with a related role. As a side

note, five of the CBI transcripts are with subjects who were interviewed in this study. For

example, Robert Taylor was interviewed in 1989 by CBI staff and then again by this

researcher in 2009. These historical interviews provide an earlier capture of similar

concepts, further validating each individual’s comments and perceptions about DARPA

captured in this study.

Figure 5 presents the combination of primary and historical interviews, which

total 59 interviews in this study.


54

Figure 5: Research interviews (n=59) by category

This study draws on a total of 59 research interviews, which includes 47 primary interviews and 12 historical interviews categorized in three groups: DARPA personnel, funding recipients, and related R&D roles. Five of the historical transcripts are with the same subjects in the primary interviews.

DARPA Personnel

A total of 34 interviews were with DARPA personnel. The roles for DARPA

personnel were typically fluid for subjects. It was common for a subject to begin as a

funding recipient and then be recruited later as a DARPA program manager.

Alternatively, a program manager could transition into a new internal role, such as an

office director or deputy director, or return to the agency as a program manager for

another term at a different point in the future. When removing the five historical

interviews conducted with the same subjects later, approximately 66 percent of the

overall group had been program managers.

Subjects held a total of 38 different roles at DARPA. Six subjects served in

multiple roles, such as first as an office director and later as a deputy director. Subjects

!"#$%#

&#

$'#

$#

$#

'#

&#

$'#

$&#

!'#

!&#

('#

(&#

"'#

)*+,*#,-./011-2# 3415617#+-8696-1:/# +-2;:-5#+<)#+02-/#

!"#

$%&'()'*+&,-./+

012'

3%2%+&-4'*+&,-./+012'506789'':;'<+1%=(&;'

!"#$%&"'()*"+$,&-",.*

/&"0(&1*"+$,&-",.*

!"#"$%&


55

who may have returned for another term as a program manager were not counted because

they held the same role again. Figure 6 shows the specific breakdown of agency roles,

which accounts for multiple different roles per individual. It was valuable to hear from

multiple roles within the organization’s hierarchy for two reasons. First, each role

influenced pre-program efforts differently and could provide greater context. Second, the

senior roles of office director, deputy director, and agency director could help verify the

various accounts from the program managers.

Figure 6: Different roles held by DARPA subjects (n=29)

Accounting for the overlap of five interviews in the primary and historical interviews with the same subjects, a total of 29 DARPA subjects worked in a range of functional roles. Some subjects served multiple roles at the agency; for example, a program manager might continue as an office director.

!"#

"# "#

$#

%#

&#

'#

(#

%#

)#

!&#

!'#

!(#

!%#

!)#

'&#

*+,-+./#0.1.-2+#

3452#67+258,+#

629:8;#67+258,+#

<-215;#67+258,+#

=2>.82?#=,>2#

!"#

$%&'()'*+,-

+'-%&.(//

%0'

*+,-+',(0%'

*+,-+'-%&.(//%0'1/2345''67',(0%'


56

Subjects spent an average of 6.7 years with DARPA, which ranged from one to 13

years. This average tenure is longer than a typical contract, noted as 3-4 years, because

several subjects transferred or returned to DARPA in other roles. For example, a program

manager recruited as an office director would serve longer. Figure 7 presents the

distribution of roles held by DARPA subjects over time in this study, which total 42

roles. In some cases, a single subject may have held multiple roles; for example, a

program manager may have returned as an office director. This distribution shows that

accounts were provided for all decades in DARPA’s history. In addition, accounts were

continuous; instead of hearing about a one-time event, subjects could describe multiple

activities over a given timeframe.


57

Figure 7: Distribution of roles (n=42) held by DARPA subjects over time

Interviews with DARPA subjects span across the organizational life of DARPA, providing detailed accounts for all decades in DARPA’s history. DARPA subjects interviewed for this study held a total of 42 roles. In some cases, a single subject may have held multiple roles; for example, a program manager may have returned as an office director. Roles held by DARPA subjects from the primary interviews are coded in red, and DARPA roles from the historical transcripts are coded in blue.

!"#$% !"#&% !"'$% !"'&% !"($% !"(&% !""$% !""&% )$$$% )$$&% )$!$%

*+,-./%012+3.42%563+%012+3.42%

*+,-./%012+3.42%*+,-./%012+3.42%563+%012+3.42%563+%012+3.42%

724829:%:9;98+2%*+,-./%012+3.42%

724829:%:9;98+2%724829:%:9;98+2%724829:%:9;98+2%724829:%:9;98+2%724829:%:9;98+2%724829:%:9;98+2%724829:%:9;98+2%724829:%:9;98+2%

563+%012+3.42%563+%012+3.42%

724829:%:9;98+2%563+%012+3.42%

724829:%:9;98+2%724829:%:9;98+2%724829:%:9;98+2%

563+%012+3.42%724829:%:9;98+2%724829:%:9;98+2%<8+;3/%012+3.42%563+%012+3.42%

724829:%:9;98+2%*+,-./%012+3.42%

724829:%:9;98+2%724829:%:9;98+2%724829:%:9;98+2%<8+;3/%012+3.42%

724829:%:9;98+2%724829:%:9;98+2%<8+;3/%012+3.42%*+,-./%012+3.42%563+%012+3.42%563+%012+3.42%

*+,-./%012+3.42%563+%012+3.42%

!"#$"%$&'()**&+%,*-./0%12(&3%)*%"4&*56%#)+&%

!"#$%&'$()*"+,-.)

/01%2$3)

4'5#0%'267)*"+8.)


58

Funding Recipients

DARPA funding recipients spanned three main groups: universities, research labs,

and corporations, all of which are depicted in Figure 8. Not one group dominates the data

sample. Interviews from funding recipients helped validate the various accounts from

DARPA personnel, particularly program managers, who potentially could have

emphasized certain details in recollection.

Figure 8: DARPA funding recipients (n=19) by organizational type

DARPA funding recipients span three main groups: universities, research labs, and corporations. This figure shows the organizational affiliation of the subjects in this study.

Based on the length of their primary engagements with DARPA, the interviews

with funding recipients covered the majority of DARPA’s history from the early 1960s to

recent years, which is depicted in Figure 9. This group averaged 17 years of financial

support from DARPA.

!"

#" #"

$"

#"

!"

%"

&'()*+,(-*," .*,*/+01"2/3," 45+65+/-5',"

!"#

$%&'()'*+&,-./+

012'

345*4'6"07.08'5%-./.%012'90:;<=''>?'@&8+0.A+,(0+B'C?/%'''


59

Figure 9: Distribution of funding recipients (n=19) based on primary DARPA engagements over time

Interviews with DARPA funding recipients cover the majority of DARPA’s history from the early 1960s to recent years. This group averaged 17 years of financial support from DARPA.

DARPA Office Affiliation

A total of 44 subjects were affiliated with a specific DARPA office, in contrast to

general management positions. This number includes subjects who served as program

managers, but excludes agency directors and deputy directors who oversaw multiple

agency offices. Figure 10 shows the distribution of DARPA personnel and funding

recipients across various DARPA offices. Approximately 57 percent of these subjects are

affiliated with DARPA’s Information Processing Technology Office (IPTO), which was

expected due to the snowball sampling, as well as the computing focus of the Charles

Babbage Institute. The other 43 percent were distributed across multiple other offices,

!"#$% !"#&% !"'$% !"'&% !"($% !"(&% !""$% !""&% )$$$% )$$&% )$!$%

*+,+-./0%1-2%*+,+-./0%1-2%*+,+-./0%1-2%*+,+-./0%1-2%

3456+.,578%9:.;:.-<:4%3456+.,578%

*+,+-./0%1-2%9:.;:.-<:4%3456+.,578%

*+,+-./0%1-2%3456+.,578%

9:.;:.-<:4%9:.;:.-<:4%3456+.,578%

9:.;:.-<:4%9:.;:.-<:4%3456+.,578%

!"#$%"&'()*+),+!-./-+0)*$%12$)%+3*$4%5"46#+71#48+)*+-94*2:+;4*'%4+

!"#$%&'$()*"+,-.)/01%2$3)

4'5#0%'267)*"+,.)


60

which provided an additional check of internal validity that processes reported within

IPTO were consistent with processes reported from subjects in other DARPA offices.

Figure 10: Subject affiliation (n=44) by DARPA office

A total of 44 subjects were affiliated with a specific DARPA office, either as an employee or funding recipient. Approximately 57 percent of subjects were affiliated with one office, and the remaining 43 percent were distributed across multiple other DARPA offices.

Conclusion

The study draws on 47 primary interviews with DARPA personnel, funding

recipients, and related R&D roles. The subjects’ time with DARPA span the entire life of

the agency, allowing commonalities to be found across agency eras and offices. An

additional set of 12 interviews from historical transcripts was used to validate the primary

interviews and temper any potential interviewer bias. Agency documents were also

reviewed as a secondary source of data. The results of this data set are discussed in the

next section.

!"

#$"

$" $" #"!"

$"

#%"

$" #" #"&"

'"

#&"

#'"

$&"

$'"

%&"

()*" +,-*" .-*" )-*" --*" */012"

!"#

$%&'()'*+&,-./+

012'

345*4'*&(6&+#'78-%'

*+&,-./+01'489.+,(0':0;<<=''>?'345*4'78-%'

!"#$%#&'()*%+%)#,-'

./(0/'0)1-2##)3'

!"#"$%&


61


62

CHAPTER 5

ANALYSIS

Four major research findings are derived directly from the data. First, the process

of radical technological innovation starts with vision. All subjects in the study who were

DARPA personnel reported starting their work with a big technology idea that then led

into an executable program. In all cases, the visions were characterized as “DARPA

Hard”. Second, subjects refined any incomplete technology visions through interaction

and iteration. Consistently throughout DARPA’s history, subjects described two main

mechanisms, specifically expert workshops and proof-of-concepts, to advance partial

visions into clear visions. Third, the skills for vision development are learned largely

through socialization. Subjects reported that DARPA does not provide training for new

program managers, expecting them instead to learn through experience and informal

mentoring. Fourth, the approval of technology visions excludes majority opinion. The

practice of consensus is actively discouraged in the DARPA vision approval process. The

subjects described direct reviews that allowed them to take their technology visions

forward rapidly. Additional findings are discussed in light of the four major findings in

this section.

Process Model of Radical Innovation

After reviewing the data, a high-level process model of innovation activities at

DARPA emerged, which is depicted in Figure 11. The process presents five stages, which

generally follow a linear progression starting with Recruitment and then leading to Vision

Formulation, Program Launch, Portfolio Management, and finally Technology Transfer.


63

The five stages were identified from subjects’ recollections and descriptions across all

eras and program offices. Each of these stages is described below.

Figure 11: Process model of radical innovation at DARPA

A high-level process model of innovation activities at DARPA emerged from subject accounts, spanning five stages from Recruitment to Technology Transfer.

Stage 1: Recruitment

Recruitment refers to efforts in identifying and hiring talented candidates, mainly

as program managers, to work at DARPA. Technical personnel are hired explicitly with

new and daring ideas of innovation, and this stage tightly links people and ideas. Subjects

mentioned campus visits and networking as common activities during this phase. The end

result of this stage is a new hire for a program manager.

Stage 2: Vision Formulation

Vision Formulation is focused on the initial work that new programs managers do

when they first join DARPA. Some program managers arrive with a clear and long view

of their programs, while others start with partial ideas that they then refine quickly

through different activities, such as workshops and proof-of-concepts, the latter of which

were called “seedlings” under the directorship of Tony Tether. Some may consider this

step as a process of envisioning. Towards the end of this stage is a Broad Agency

Announcement (BAA), in which each vision is internally formalized, documented, and

!"#$%&'(")'*+&,&-)*

.-$(%/01-)*2$-3$0(*40%)#5*

2-$6-/&-*70)03"(")'*

8"#5)-/-39*8$0),:"$*


64

put forward for approval as a new program. The final step in this stage is the approval of

the BAA for external distribution.

Stage 3: Program Launch

Once a program is approved, DARPA program managers proceed next to a stage

of Program Launch, in which they publicly announce and start a new program through an

official solicitation procedure. In the early days of DARPA, program managers would

solicit bids or request participation on a more ad hoc basis. After 1984, Congress required

the selection and evaluation of DARPA performers, those responsible for exploring and

implementing a particular vision, to be handled formally through the BAAs. A BAA is

different from a Competitive Request for Proposal (RFP) procedure, a more common

document within the U.S. federal government that is related to the development of a

specific system or hardware procurement (DARPA, 2010a). The end result of this stage is

the final selection, funding, and orientation of team(s) external to DARPA to explore and

build the vision. One DARPA program can have multiple projects being executed by

different teams.

Stage 4: Portfolio Management

In the stage of Portfolio Management, DARAP program managers oversee the

development of all active team projects within a new program, as well as end any

unproductive projects that do not meet the original vision or offer meaningful learning to

the organization. This stage emphasizes the practical management skills of a DARPA


65

program manager, who handles project evaluations and budget reviews for all his/her

technology investments, typically on an annual basis.

Stage 5: Technology Transfer

The final step is Technology Transfer, in which program managers transition

promising technologies, often as a functional prototype, to early adopters in the military

services or relevant industries. Memorandums of Understanding (MOUs) and user testing

procedures are frequently part of this stage. Although not all DARPA projects end with

technology transfer, this step is considered the critical closure in “bridging the gap” –

borrowing the slogan of DARPA’s 50-year anniversary campaign – between far-reaching

ideas and near-term solutions.

Table 3 summarizes several common processes that occur at each stage in

DARPA’s process model, as well as presents multiple supporting quotes from the

interviews that are representative of the broader data set.


66

Stage Example Process Example Quotes From Research Interviews Recruitment • Campus visits

• Mentoring

• “If youʼre going to foster innovation, you should get smart people.”

• “The hiring process becomes an important part of the process.”

• “Go find good people; go do good things.” • “DARPA is shaped by people. Its content is a

function of the people.” • “There is a kind of networking. You learn to trust

certain people, and they expand your acquaintance.”

• “It was literally all word of mouth.” • “I teamed up with members of my senior

community; that was the recipe then.” • “My bias was toward very bright people. I did not

care very much what they knew or what they were interested in.”

• “Well, certainly when I arrived it was, as I mentioned before, an environment in which you gave money to good people and expected good results.”

Vision Formulation

• Workshops • Proof-of-

concepts

• “I came with a technology vision.” • “In some sense, everything I knew I got either

from thinking myself or from visiting labs and talking with people.”

• “You have to have a new unique idea.” • “I sure as hell knew what I wanted to do.” • “When I came in, I had a couple of ideas about

things I was interested in… but after I got there, I actually changed course entirely.”

• “Most of the programs I eventually got started ended up because I saw this idea over here, and I saw that idea over there, and if I put them together, we could really do something incredible.”

• “We were constantly inundated by ideas, some good, some not – lots of good ones. So this was a very lively time. People would come in and out all the time with new ideas. And there were indeed many places where they could sell these, but in the end they would come to us. So there was a great traffic of ideas, flow of ideas.”


67

Stage Example Process Example Quotes From Research Interviews Program Launch

• Industry Day • Contract

negotiations

• “I am sure you have heard the fact that we could get an idea in the morning and have the guy working under a Letter of Intent by 4:00 the next day.”

• “One thing that I learned early on, and I was doing back in the 90ʼs, is that when you do a project you had to have a contracting officerʼs representative.”

• “As written it looks like I had it all scheduled out, but there was a lot more of improvising and ʻoh my gosh, I have to do this now if I want anything to happenʼ in it.”

Portfolio Management

• Project evaluations

• Budget reviews

• “I believe the most important word in our [DARPA] name is projects.”

• “Part of the DARPA recipe … is that projects are the output. They are the units of success and failure.”

• “We donʼt want to own it; we actuate it.” • ”DARPA program managers really act like

managers and manage the research." Technology Transfer

• Memorandums of Under-standing (MOUs)

• User testing

• “I picked the government agent based on the application area, which helps with the technology transfer.”

• “I consider myself a technology harvester.” • “Most of the people, 90% of the DARPA program

managers, in my mind, are 100% clueless about technology transfer – they have no idea. I was very naïve about it, even when I thought I knew.”

• “The research that we understand as revolutionary research is only 10% of the problem, and 90% of the work in order to get a new idea successful is after youʼve got the prototype, not before.”

Table 3: Common processes and interview quotes per DARPA innovation stage

DARPA’s process of innovation follows five general stages: Recruitment, Vision Formulation, Program Launch, Portfolio Management, and Technology Transfer. Each stage is associated with a set of common processes, which were identified based on dominant themes from the research interviews.

Vision and Recruitment

Vision plays a central role in the first two stages, Recruitment and Vision

Formulation, starting the process of radical innovation at DARPA, depicted in Figure 12.

During these two stages, a vision for a radical technological innovation is identified,


68

defined, evaluated, and approved as a DARPA program. These first two stages in the

process are tightly entwined because DARPA program managers are hired in part for

their initial technology visions. Moreover, each DARPA program requires a technology

vision, and this vision precedes any efforts for a bid solicitation or project team selection.

Simply put, a new DARPA program is not approved for funding until it has established a

vision. One internal report indicates this inseparable combination: “DARPA insists that

all programs start with good ideas and good people to pursue them; without both of these

things, DARPA will not start a program” (DARPA, 2007).

Figure 12: Radical innovation starts with vision at DARPA

During the first two stages, a vision for a radical technological innovation is identified, defined, evaluated, and approved as a DARPA program. These first two stages are tightly entwined because DARPA program managers are hired in part for their initial technology visions.

Each vision is associated with one person: a DARPA program manager, who

serves as the vision’s champion. A program manager from the 1990s stated, “A

technology vision is the way that I came in,” and similar words were echoed by a

program manager serving a decade later, who said, “I came with a technology vision.”

All program managers have the sole responsibility for realizing their particular visions

during their time at DARPA by pursuing and funding relevant projects. When their tenure

ends, if any substantial work remains on a project, they may be able to hand it off to

another program manager; however, the vision itself rarely transfers between program

!"#$%&'(")'*+&,&-)*

.-$(%/01-)*2$-3$0(*4"5)&1-)*

2-$6-/&-*70)03"(")'*

8"#9)-/-3:*8$0),;"$*


69

managers. One exception was the computing vision behind the ARPANET project in the

early 1970s, which engaged multiple program managers over an extended period. While

the data does not describe a specific reason for this, various accounts from subjects

support the interpretation that this vision was bigger than any one person.

At this point, it may be helpful to see some examples of technical visions that

DARPA program managers ultimately document in their Broad Area Announcements

(BAAs). The BAA serves as a formal and public description of the initial program vision.

The Augmented Cognition program described a vision for a new type of soldier helmet or

cockpit, which then lays the groundwork for the field of empathetic avionics, among

other areas (BAA, 2001). Another program, the Legged Squad Support System, described

a vision for a new type of military pack mule (BAA, 2008). While DARPA program

managers present their general requirements for a range of radical ideas, they look to the

funding recipients to decide exactly how to build and execute on the respective vision.

Table 4 presents two visions as documented in BAAs.

DARPA Program

Excerpt From Broad Area Announcement Example Prototype Built

Augmented Cognition

“The goal of the DARPA Augmented Cognition effort is to extend, by an order of magnitude or more, the information management capacity of the human-computer relationship for the 21st century warfighter. This will be achieved by developing and demonstrating quantifiable enhancements to human cognitive ability in diverse, stressful, operational environments. Specifically, this effort will empower one humanʼs ability to accomplish the functions currently carried out by three or more individuals and will improve and enhance the quality of military decision-making.” (BAA, 2001)

Boeing Phantom Works, AugCog


70

DARPA Program

Excerpt From Broad Area Announcement Example Prototype Built

Legged Squad Support System (LS3)

“The Legged Squad Support System (LS3) Program is an effort to develop a walking platform, preferably a quadruped, which can accompany dismounted soldiers and increase their combat capability. LS3 is envisioned to augment squads by maneuvering with them in complex terrain where wheeled tactical vehicles cannot go, carry traditional squad equipment (in an effort to improve squad performance), carry new squad equipment (in an effort to give new combat and sustainment capabilities to the squad), and do so in a self-controlled fashion (requiring minimal human interaction and control).” (BAA, 2008)

Boston Dynamics, BigDog

Table 4: A selection of DARPA program visions

A DARPA program manager broadcasts his program vision in a document called a Broad Area Announcement (BAA), which officially launches the program and begins a public competitive solicitation procedure. The BAAs provide a public record of DARPA program visions, and a program vision may generate multiple and different prototypes from the funding recipients.

Recruiting Criteria and Evaluation

Program visions are vital to the recruiting stage, and an important finding in this

study was that vision is considered during the hiring of program managers. A DARPA

program manager is recruited primarily based on having an initial vision of technology,

plus the ability to communicate that vision to different groups. An office director

explained the focus simply, “We look for people with good ideas,” and another director

felt an ideal candidate was someone who could “articulate what they want to do and

why”. A program manager compared his role to a technology evangelist, saying,

“Sometimes I feel a little like Billy Graham”, because he was endlessly communicating

and exciting others about his ideas.


71

The contents of a vision and its presentation differed by personality; however, the

common theme across subjects was that each vision be radical and actionable. In other

words, each person described a technology concept that could be achieved within several

years, potentially creating an important new application and user base (e.g., market) that

had never existed before. One program manager explained this focus as a combination of

vision and action:

I mean it’s a unique combination of strategic vision and tactical implementation that is required to be a very effective program manager. You can dream all you want, but if you don’t have the ability to tactically implement your dreams, then all you are is a dreamer. If all you do is tactically implement, then you’re just putting one foot in front of the other without picking your head up and looking to the horizon and understanding where you are going. In nearly all interviews, DARPA personnel were technically trained with

advanced degrees in engineering or science. DARPA does adapt a general boilerplate

hiring description for its technical offices (M. Peterson, personal communication, June

10, 2010). However, no subjects described an official job profile, and instead they listed

several gut criteria for finding the right program manager, including: technical expertise,

visionary thinking, leadership, communication skills, and fiscal responsibility. A deputy

director described his view of the hiring practices at DARPA, highlighting the

importance of creativity coupled with technical knowledge:

We need people to be thinking big thoughts, really pushing the edge, but doing it in a rigorous way. It’s not just to say: I want to build a teleporting machine, exactly how are you going to do that. And why should we believe it is going to happen. So that’s what I mean by technical creativity, rigor and honesty. It’s creativity, but it’s tied to things that make sense.

A recent public job listing from DARPA (Defense AT&L, 2007) called for similar traits,

highlighting the need at the personal level for a radical idea that generates lasting change:


72

Job Listing for a DARPA Program Manager Are you a scientist or engineer with a radical idea (or ideas) that you believe could provide meaningful change of lasting benefit for the U.S. military? Would you like to lead the countryʼs most capable academic and industrial experts to make that idea become reality in a period of just a few years? If so, you should consider joining the Defense Advanced Research Projects Agency (DARPA) as a program manager. What is a DARPA PM? A DARPA program manager is ... An idea generator A technical expert An entrepreneur A visionary A patriot dedicated to national service

Generally, DARPA’s hiring process does not rely on specific evaluation tests,

case interview formats, or personality profiling. With the exception of J.C.R Licklider

who was known to assess graduate students by their test scores on the Miller Analogies

Test and Graduate Record Examination (GRE) in the 1970s (Aspray & Norberg, 1988),

candidate evaluation has historically been more informal at DARPA. An office director

explained, “It’s not a formal process. We don’t give them tests or anything.” Instead, a

candidate is largely evaluated based on the relevancy of his/her technology vision, as well

as cultural fit with the agency. Overall, subjects indicated that hiring visionaries has

benefited DARPA in at least two ways. First, these visionaries open new areas of inquiry,

which often do not exist before. Second, office directors can influence the broader

defense research agenda by strategically hiring program managers with big visions.

Based on the research sample, a secondary finding emerges related to recruiting.

In the instances when subjects offered their ages when starting at the agency, a clear trend

emerges that the average hiring age of DARPA technical staff has increased, particularly

among its program managers, throughout the agency’s history. Presented in Figure 13,


73

this trend was focused more in the computing areas, particularly within DARPA’s

Information Processing Technology Office (IPTO).

Figure 13: Rise of starting ages for new DARPA personnel

The data shows a preliminary trend that the average hiring age of DARPA technical staff, particularly among program managers, has increased throughout the agency’s history.

While subjects did not explain or address the issue, several subjects shared anecdotes that

supported the general trend. Before the 1980s, they described hiring university graduate

students and newly minted doctorates, who brought their energy and enthusiasm into the

agency. One program manager said, “You took smart, young, energetic people and let

them try.” Similarly, an office director said, “I even made an agreement with all these

kids: I told them that if they wanted to keep on working on their Ph.D. thesis project, they

!"#

!$#

%"#

%$#

&"#

&$#

$"#

$$#

'"#

()'"# ()*"# ()+"# ())"# !"""# !"("#

!"#$%

&#'$()*#+

$,-'$.#/*01$

.#/*$23/*3#+$/3$4!56!$


74

could do that up to 50% of their time.” More broadly, an agency director explained his

focus on younger talent for the agency:

[We should] produce graduates of those organizations who were expert in the technologies that we were promoting… what I wanted were more Ph.D.s who could go out into industry, and particularly in the defense industry… Now that guy is a product of ARPA. That company is a product of ARPA’s activities, and it’s going to have an influence on the commercial enterprises that this country has. In more recent years, much older and established professionals have been

recruited to the agency.

Channels for Recruiting Technology Visionaries

How does DARPA find new personnel? The use of internal referrals has been the

dominant vehicle for hiring new program managers. Office directors are often groomed

from talented program managers or enlisted by former DARPA personnel. Subjects

indicated the importance of highly specialized talent, which required other experts in the

same area or field to qualify. An open job call would be unsuitable, and even the current

agency website does not offer the standard career information or link to a hiring page.

Instead, DARPA expects potential recruits to be already embedded in the relevant R&D

community. Repeatedly, stories were told in which the individual was part of a team

funded by DARPA or working in a related defense area; subsequently, the current

director or program manager could observe that recruit in action and then reach out

directly. The following interview quotes all underscore the importance of leveraging

existing relationships to find talent: “There’s a fair amount of relying on past

connections,” “Every PM has some connection to a DARPA PM before they got there [to


75

DARPA],” and “DARPA is always looking for people who’ve had previous

interactions.”

In addition, former program managers tend to serve as volunteer talent scouts. A

deputy director described the role of DARPA alumni:

They are people who can feed us – because they know what the job is like, they know what are the characteristics of a good PM and a good Office Director, or whatever. They can be the people who help us recruit the talent to the agency. That’s the way it has always been, and I think that’s the way we get the best people.

The alumni typically remain active in the extended DARPA community. After their

contracts ended, many program managers and directors spoke about referring potential

new candidates, informally mentoring new program managers, competing for DARPA

funding, serving on technical advisory boards for DARPA projects, and consulting on

different DARPA initiatives. One program manager explained, “You don’t really leave,”

and another said, “It’s a fraternity… not an honor society.” (As a side note, historically,

program managers at DARPA have been mainly men.) These close alumni relationships

offer another value to DARPA’s model of open innovation because these relationships

reinforce the network strength and emphasis on people. However, several subjects noted

the limits of a network model of recruiting. For instance, one funding recipient observed

that, “DARPA has always operated with a set of insiders.”

An average recruiting period could not be determined from the data. The

recruiting period varied widely from weeks to years, heavily dependent on the recruit’s

career schedule, as well as openness to working at DARPA. For example, a program

manager said, “I was targeting and hassling people for three years.” For personnel

coming from academia, DARPA often accommodates the constraints of tenure, and some


76

subjects noted that they joined the agency only after their faculty positions were secured

at their universities. Ultimately, subjects noted that recruiting is about finding the right

people at the right time, and in addition to being persistent, several of them described the

importance of timing. One deputy director said:

Many times, most people think that science is just some linear progression, but a lot of times it’s like a startup. If you get the right group of people together at just the right time, great things happen. If you wait six months, the person who would have been the great integrator has moved on to another job and is no longer available, and you’ve lost that.

Vision Formulation

The second stage in DARPA’s model of radical innovation focuses on the first

few months that a program manager spends at DARPA. Program managers indicated that

they begin their roles with either complete or partial visions of technology, which then

they must shape and pitch as a new program. In all cases, as one program manager put it,

“You have to have a new unique idea.” When the vision is incomplete, program

managers have relied on several mechanisms to formulate and advance partial visions,

and these mechanisms have been used consistently through the agency’s history.

Collectively, the visions from all program managers determine the focus of the agency. A

program manager explained this shared direction: “The scope of what DARPA works on

is a function of what its people works on, not a function of its physical facilities.”

Operating without labs or research space, DARPA directly relies on its people and their

radical ideas to spur innovation efforts in the teams that they fund and foster.


77

Sources of Technology Visions

A related finding has to do with where to look for possible visions. Although

program managers are responsible for bringing a vision to DARPA, many credit the

broader network of partners for inspiration. All of the program managers advocated being

in the field and interacting with other researchers. They felt that this immersion allowed

them to directly hear the latest thinking of the R&D community, as well as develop

personal knowledge of the problem context. For example, a program manager said, “In

some sense, everything I knew I got either from thinking myself or from visiting labs and

talking with people.” Another program manager explained, “They are my ideas, but they

are inspired by other people, people that I’ve seen. The more that you go out and the

more different things that you see, you integrate them into what you think would be a

great idea to do.” An office director was more blunt when he said, “The ideas don’t really

come from program managers. They come from the researchers [funded by DARPA].” In

short, program managers find that their ideas advance through community participation

and interactions. The common theme was that new ideas occur more readily through two

simple actions, when program managers leave the office and then talk with a diversity of

people. A deputy director underscored these points with DARPA’s R&D community

during a speech:

So how does the process work? Often it begins with you, a potential contractor, articulating your idea, your vision, in a conversation with a DARPA Program Manager, who, caught up with your enthusiasm, might help you flesh this idea out by finding funds to support you in a seedling effort, just to see where it might lead. More typically, he may try mapping what you want to accomplish against either his existing program portfolio, or incorporate your thinking with his own nascent ideas to create a broader plan, one that might capture the imagination of a whole community, one that might result in a new DARPA program. (Leheny, 2007, p. 1)


78

Criteria of a Radical Vision

Another related finding identifies the criteria for a radical program vision at

DARPA. Although individual visions ranged widely in content, four general attributes for

a program vision can be drawn from subject accounts. Radical visions must be

technically challenging, actionable, multidisciplinary, and far-reaching. These attributes

are summarized in Table 5.

Vision Attribute Description Technically Challenging

Visions must extend beyond the current limits of technology and state of knowledge

Actionable Visions are expected to be built and produced Multidisciplinary Visions draw on multiple experts and areas of

knowledge Far-reaching Visions are ambitious about advancing society on a

grand scale

Table 5: Criteria for a radical vision at DARPA

Four attributes for a good DARPA program vision were drawn from subject accounts, and together these attributes characterize a notion of “DARPA Hard”.

First, at the core, all visions at DARPA constitute a challenge and must be almost

technically impossible to achieve. The challenge is not based on the limits of a particular

organization. Instead, visions must extend beyond the current limits of technology, and

the current state of knowledge in the world for a given problem determines the difficulty

of challenge. These problems are focused on technological breakthroughs. DARPA

program managers described relying on multiple experts in the field, often across

different domains of knowledge, to help them qualify true technical challenges.

Second, the visions must be actionable. The visions instill an expectation for

action, and the right people can see a path to the impossible and can make progress


79

beginning today. Ultimately, program managers are responsible for delivering at least

part of their idea during their time at DARPA, and all subjects underscored the pressure

of this expectation. Successful program managers must be both dreamers and producers,

even if they leave the implementation to the project teams. An agency director explained

the importance of application: “We required what I called a pin thread of logic that got

you from what you were trying to accomplish in each one of the programs that you were

supporting to a capability that would make a real difference in the future.” This belief

was echoed by other subjects, including a deputy director who noted, “Again it is not

enough to say, ‘hey, I think it’s really cool if we were to look at this phenomenon.’ Why

is it interesting from a capability perspective? That’s the most fundamentally important

question [DARPA program managers] need to answer.”

Third, visions are multidisciplinary. The inherent problem in the vision is

complex because it sits at the intersection of multiple disciplines. Several subjects noted

this multidisciplinary aspect, in which the problems became so complex that they

required a team approach. As subjects explained, achieving a good vision that would lead

to radical innovation requires more than one person. If one person can build the vision,

then the vision is not big enough. One program manager said, “I don’t remember a single

case where we ever funded a single individual’s work.”

Fourth, good visions were far-reaching as a mix of long-range and broadly

complex, which ultimately produced a broad impact on society and overall progress for

humanity. In terms of long range, some subjects described efforts that simply required

more time to pursue, often because certain advances in science and engineering research

could not be skipped. In other cases, subjects described efforts that entailed integrating


80

multiple parts into a complete and complex solution, and while many of these parts did

not have a long development lead-time in themselves, the sheer complexity of integration

made the vision incredibly challenging to achieve. One office director spoke about

making “an order of magnitude difference”. Another program manager described how he

directly shortened the timeline for an ambitious program by increasing the available

funding:

[DARPA management] asked me why [the problem] was so hard, and I told them why it was so hard… I just went ahead and had what I thought was a really cool idea, which would be virtual reality for simulation in for robot and we put them all together and put them on the switchable robot. That was something I thought was a really hard program, and I wasn’t sure that [the funding recipient] was going to be able to do it in the time or the amount of money that he said, so that’s why I doubled the amount that I wanted to make sure he would be getting. We went forward, and we dumped the money on him, and they performed admirably. In sum, these four attributes of a good vision – technically challenging,

actionable, multidisciplinary, and far-reaching – have become encapsulated into a term

known as “DARPA Hard” within the agency. DARPA Hard is a recent term, coined

during the directorship of Tony Tether (2001–2009). Subjects who worked at DARPA

before the 1990s described the same concept and, in many cases, adopted the recent

jargon of DARPA Hard because they felt it was apt terminology and evocative. Table 6

highlights several select quotes about DARPA Hard from the research interviews, as well

as from the literature review.

In short, DARPA Hard is a daring technology challenge that cannot be solved

easily, typically requiring new knowledge or invention to be truly addressed through bold

measures. Subjects shared a sense of pride that the challenge was especially worthy of

DARPA’s attention and support. A deputy director stated, “Sometimes it’s a great

opportunity, sometimes [DARPA program managers] can answer the why now, but they


81

can’t really answer why DARPA.” When program managers were able to address why

tackling a given vision was genuinely DARPA Hard, they were able to move onto the

next stage.

Source Example Quotes Research Interviews

• “Some ideas in technology areas you really canʼt initiate unless youʼre here at DARPA because theyʼre seemingly impossible.”

• “So they provided a leading edge where they addressed some hard problems early on. A lot of the knowledge that they gained from that work fed through the rest of the computer research community. A synonym for being the leading edge might be the lunatic fringe.”

• “If it is successful, it will change by an order of magnitude.” • “I really didnʼt have a concept of DARPA Hard until the second or

third year or so.” • “We were explicitly looking for things which, if they could

succeed, would really be a large step beyond what technology could then permit.”

Literature Review

• “DARPAʼs approach has often been to take on projects that seem impossible, and the term DARPA Hard – now a part of the agencyʼs legacy – refers to technology challenges considered too difficult for other parts of the militaryʼs research and development laboratories.” (Weinberger, 2006, p. 4, italics included)

• “DARPA focuses its investments, therefore, in an area referred to as ʻDARPA Hard,ʼ a set of challenges that, if solved – even with a solution that initially poses a high risk of technical failure – will be of enormous benefit to U.S. military capabilities and to national security.” (National Research Council, 2005, p. 13)

• “DARPA prides itself on addressing only the most difficult ʻDARPA-hardʼ problems, the ʻgrand challengesʼ in technology, by nurturing the science needed to meet them.” (Weiss & Bonvillian, 2009, p. 163)

Table 6: The concept of DARPA Hard

Multiple quotes drawn from the research interviews and the literature review about innovation show a coherent understanding about the concept of DARPA Hard.

Advancing Partial Visions of Technology

Visions may start as either clear or partially formulated at DARPA. Many

program managers arrive with an unambiguous vision of they want to do and spend the


82

initial months at DARPA finessing program mechanics and budgets, so they do not feel

the need to spend much time clarifying or validating their ideas. What does a clear vision

look like? Several subjects described their visions very simply, and examples include

swimming robots, portable ultrasound devices, nonstick paint, direct robotic surgery,

microsystem fabrication at the nano scale, artificial blood, and a reusable, rapid-strike

hypersonic weapons system. In the 1970s, J. C. R. Licklider introduced one of the more

well-known DARPA visions that took on a life of its own, ultimately inspiring the

invention of the Internet:

The thing called “Man Computer Symbiosis”… No, that didn’t come out of any particular research. That happened frequently: you do a lot of work, you get in a position to see some relationship or make some decision. And then it was obvious. In fact, even before you could quite get finished, you knew how it was going to come out.

Years later, a program manager described a similar process, in which the vision was

evidently clear, as well as defined by aspects of DARPA Hard:

So about two, three years into the MEMS program, there were some people who said you know you can make Radio Frequency components from MEMS. You can make oscillators, you can make detectors, you can make mixers, modulators. Now we can make whole types of RF systems using MEMS. Well, I wasn’t convinced you could do that, and so I said, well what’s the hardest thing you can do in RF MEMS was build some high quality resonators.

When program managers felt that their visions were clear, they quickly moved to pitch

these visions internally as new programs and secure funding. Little time was spent during

the Vision Formulation stage.

Although other program managers felt that they had a good understanding of the

research limits and market demands to their roles, these managers had not defined their

visions of technology as clearly as their counterparts. Instead they worked more from a

gut feeling or half-formed concept, trusting their intuition that they were pursuing the


83

right path. Program managers used descriptors such as “informal”, “concept”, and

“demos” to describe their early ideas. Their language indicates partial visions, or visions

that required some additional effort to finalize to a more advanced state of readiness.

Program managers described two mechanisms to develop their partial visions into

a complete and clear vision, which is illustrated in Figure 14. These mechanisms are

expert workshops and proof-of-concepts, which have consistently been applied by

program managers throughout DARPA’s history. Program managers and other DARPA

personnel described no other methods for refining their visions.

Figure 14: Formulation of partial visions of technology for radical innovation

Expert workshops and proof-of-concepts have consistently been applied by program managers throughout DARPA’s history to refine any partial visions into clear and complete visions.

Expert Workshops

Expert workshops are invite-only, small group discussions between the program

manager and key members of the R&D community, planned over 2-3 days offsite, that

emphasizes the exchange of ideas and the potential demonstration and application of a

new technology. The objective is to gather feedback about a particular topic, its potential

technical issues, and overall direction. Participants are not paid for attending the event,

!"#$%&&'()(*+&

,$%-$"&'()(*+&

./0#%1&2*%3)4*0&

,%**56*56&!*+7#01&8#9*&

:##;<$73&=&>#$%+(+?&


84

and primarily, only experts who specialize in a particular problem domain are invited.

Program managers were adamant about the value of a workshop, saying: “The first thing

I did was a workshop,” and “I did two to three workshops over different subjects to

understand the state of the art.” Several funding recipients echoed this point, and one

subject said, “The program manager is trying to come up with a germ of an idea.”

Proof-of-Concepts

At DARPA, proof-of-concepts are small design studies that explore and pre-test a

particular technology concept. Proof-of-concepts may result in written documentation,

physical prototypes, or simulations that confirm initial assumptions. Generally, a DARPA

program manager provides a set of early requirements to a specific R&D contractor, who

then undertakes the project as a paid assignment. These proof-of-concepts were dubbed

“seedlings” during the agency directorship of Tony Tether (2001–2009) based on the

germination of an idea.

Program managers described efforts for proof-of-concepts that took several

months to conduct. The outcome may not be clear, and a program manager explained,

“It’s a fishing expedition in advance of a program.” Comments from funding recipients

matched descriptions from DARPA program managers, and one funding recipient stated,

“The aim is always the same: help the PM to do a successful pitch or write a good BAA

or both.”

The process is intentionally emergent. Another program manager explained his

approach to proof-of-concepts, while visiting teams in the field:

As I was walking down the corridors visiting the lab, I saw a 3-D image that looked like an X-ray. I said that is absolutely incredible, but you know what, I


85

would like to get an ultrasound that is that good – something small and portable that I could get a full 3-D image without having to have X-rays. They said that’s the ultrasound laboratory. So we stop what we’re doing, we go into the laboratory, and I immediately funded these people with a seedling to get up and running on 3-D ultrasound.

This quote shows the DARPA program manager out in the field, interacting with his

teams, scouting for possible new ideas, and deliberately comparing his observations with

the big vision in his head. When he saw a promising part of the vision, the program

manager took instant action to support and accelerate its growth as a small focused study.

He had the discretion and control to make these decisions while in field. All of these

elements in this example match other descriptions from subjects about an effective proof-

of-concept.

In addition, subjects who served as program managers at DARPA before the

1980s spoke about a greater flexibility to pursue proof-of-concepts without additional

approval. One program manager said, “When I was program manager, I had a

discretionary budget of $250,000. If I wanted to go ahead and spend it, I didn’t need

approval. I didn’t need approval from my office director. I had pre-approval from the

director.” The ability to act quickly and with minimal oversight was crucial to subjects,

who felt empowered by their organization to ensure that the technology vision was

complete and truly DARPA Hard from the start.

Similarities Between the Mechanisms

The expert workshops and proof-of-concepts are similar in three ways. First, the

objective is to gain continual learning and feedback from the relevant R&D community,

which then is used to sharpen the original technology vision. A program manager


86

explained his process, saying, “You muddle along, and whatever appears in front of you

is the course of action that you judge to be the one that is going to lead you in the right

direction, and you make a lot of false steps.” Goals are loosely set in order to support an

unexpected result. Another program manager described the importance of staying

flexible: “Sometimes you do workshops, sometimes you do seedlings, sometimes it just

falls in your lap.” Second, these activities provide rapid feedback because they can be

conducted within a matter of days or months. One program manager explained the time

range for two of his seedling efforts: “One of them took six months; one of them took

almost 18 months.” Third, both mechanisms are planned during the embryonic phases of

a project, and once the vision is clear, program managers do not rely on these

mechanisms further.

The frequency and success rate of these activities was discussed in a few of the

interviews. One program manager shared his personal experience, saying, “I’ve probably

held a dozen workshops, and I would tell you a quarter of them have led to programs.

Half of them turned out to be useless, and a quarter of them led to a program for

somebody else.” Another program manager explained, “Of the programs I’m working

with, I think I started something like nine programs in my time here, maybe 10. That’s a

relatively high number and very few of those started as a seedling.”

Learning the Capacity for Envisioning

As the third major finding, DARPA does not directly help its program managers

learn how to develop their visions of technology. Although some subjects described

previous work experience with technology funding or startup development, the


87

entrepreneurial skills and business mindset required to be a DAPRA program manager

are typically learned “on the job”. There is no direct teaching. Aside from standard

review of DoD policies, DARPA does not require new hire training or provide an

orientation specific to program creation for new program managers.

Instead, the prevailing beliefs are to start quickly and learn by doing. One

program manager mentioned, “There really weren’t any guidelines. The best preparation

was to be agile… and honest.” Another program manager shared a similar viewpoint,

noting, “Most of these people are experienced researchers, so there isn’t a lot of training

required… There’s not a whole lot of instruction here.” The socialization occurs during

the first two stages at DARPA, in which new program managers are shaped, both

deliberately and informally, by the organizational culture and ultimately assimilate the

agency’s practices, rituals, and values regarding radical innovation.

A Community of Informal Learning

Based on subject accounts, DARPA’s culture of innovation is based on experts

learning from other experts through informal channels. As one subject said, “It’s about

the history, the continuity, the sociology.” Although many subjects struggled to explain

the specifics of DARPA’s tacit culture, a clear thread emerged across their descriptions

that former and current personnel served as mentors and masters of art to new personnel.

Program managers said they would observe others around them and emulate their

behavior. One program manager explained the importance of the informal interactions

between the groups, saying:

There is not really training on how to manage a program. There’s training on the mechanics, training on contracting, training on security, training on those kinds of


88

things, but management of programs is really done by trial and error and an informal network that forms the day you walk in with the other program managers in your office or in your field, which may actually be outside of your office, but related field, that you run into and bump elbows with and talk to. Those people who’ve been here before, they learn lessons, and you really learn by talking and going down and sitting in their office for an hour and just talking through issues and problems. It’s like having an encyclopedia of how to do things right and wrong sitting in front of you and having direct access to it all the time.

An office director shared a similar story, in which relaxed dialogue helped him to

advance his ideas:

We would get our gang together, and there would be lots of discussion, and we would stay up late at night, and maybe drink a little alcohol and such. So I thought I had a plan at that level. I could talk about it to people in ARPA. It was easy to have plenty of topics in the outline carried down three or four levels if I found a guy who wanted it that way. An implicit theme emerged from subjects that effective program managers

learned the necessary skills to start a program at DARPA faster than their less effective

colleagues. Being more effective generally entailed having greater program funding,

longer employment contracts, and fewer review sessions with directors. This learning

occurred informally, and a certain amount of ego was involved in absorbing the

organizational culture and belonging to the “in” community who understood how to start

new programs.

An Entrepreneurial Spirit

Based on subject accounts, the culture at DARPA relies heavily on self-initiative.

In lieu of formal training, one aspect raised repeatedly by subjects was learning how to

think as a technology investor and also as an entrepreneur. Program managers fancied

themselves as venture capitalists, seeing a similarity in funding early ideas. Beyond the

role of venture capitalists, DARPA program managers first and foremost seek to identify


89

new technologies without regard to market potential, provide seed funding for ideas that

may never come to fruition, and help to drive these ideas toward reality. Further, DARPA

program managers have no vested interest in the actual outcome of the teams and ideas

that they fund.

In addition, several subjects compared DARPA program managers to

entrepreneurs. They felt that both roles exhibited similar characteristics to find execution

teams, pitch and evangelize new ideas, and seek high-risk, high-payoff opportunities. For

instance, one program manager stated, “DARPA is the Venture Capitalists of the DoD

driven by a herd of entrepreneurial Program Managers from industry, government &

academia” (Pappert, 2009, pg. 5). Similarly, a deputy director perceived the program

managers as acting as entrepreneurs when they sought funding internally for a new

program. He said, “The way we help [the program managers] is really through a process

by which they sort of pitch their idea for a new start… we act like a venture capital

corporation – where someone is coming in to pitch a program, you’re really pitching your

company.” Unlike real-world entrepreneurs, who ultimately have to choose one team and

one goal, DARPA program managers can foster multiple teams, engage in the creation of

multiple ideas, and seek to build a bigger vision than a classic entrepreneur.

Converting Visions to Programs

By “pitching”, program managers meant that they presented their visions for new

programs internally for funding approval. The step between the vision formulation and

program definition is a critical point because not all visions are automatically allowed to


90

move forward. Figure 15 illustrates the transition between the two stages, which was seen

an important part of the overall process.

Figure 15: Transition between stages of Vision Formulation & Program Definition

The step between the stages of Vision Formulation and Program Definition is a critical point because not all visions are automatically allowed to move forward as new programs at DARPA.

DARPA program managers rarely present an idea until they feel it is subjectively

“ready and complete”. Unlike corporations, there are no annual reviews or comparable

stage gate procedures. Program managers schedule a review session as needed. A deputy

director explained the importance of putting control in the hands of the program manager,

who decides when and how to pitch to his superiors:

[Program managers] are empowered to make funding decisions. I think that’s really important. They obviously take a lot of advice; they get input from various scientists and advisors to a program… But ultimately they’re the ones that make the decision. So, it’s not a peer review model, which is perhaps the traditional one that’s employed in other federal agencies. Subjects reported that the respective office director and, ultimately, the agency

director provided the final decision for a new program. This proved true across agency

offices and eras. While other individuals might be present for input and perspective, such

as other DARPA office directors and military liaisons, the primary decision-maker was

always the agency director. One program manager said:

!"#$%&'(")'*+&,&-)*

.-$(%/01-)*2$-3$0(*4"5)&1-)*

2-$6-/&-*70)03"(")'*

8"#9)-/-3:*8$0),;"$*


91

Now, going back to the joy of [D]ARPA: you know, if a program manager has a good idea, he has got two people to convince that that is a good idea before the guy goes to work. He has got the director of his office and the director of [D]ARPA, and that is it. It is such a short chain of command.

Another program manager spoke of “three signatures, one of which was yours and two

were other people within DARPA… [who] would agree that this was a good project and

then you would be able to go forward”. Similarly, a program manager from the mid-

1970s described his experience presenting a technology vision to only his office director

and the agency director:

Well, before then, you would go up to the director’s office with [the office director] and sit with your feet on the desk and [the office director] would tell him, "Well, gee, I have got this great idea," and you could talk and knock it back and forth. I do not think that [the office director] had ever, ever had anything turned down by [the agency director], because [the agency director] understood. Decisions for new programs were described as occurring quickly, generally under

an hour. The decision was made and communicated by the end of the meeting, and no

additional waiting time was required for evaluation. Once the vision was approved, it was

considered as a new start program and on its way to documentation as a Broad Area

Announcement (BAA).

All of the subjects described DARPA’s evaluation process as distinct from peer

review and other forms of committee or consensus decision-making. Multiple statements

from the subjects demonstrate the strong sentiment and deliberate intention behind this

aspect of DARPA’s process. A program manager noted, “We were the only agency that

didn’t require external peer review. I think that was probably one of the most important

things about DARPA. It was the only reason DARPA succeeds.” Another program

manager expressed it as:


92

Outside reviews get rid of any innovative ideas… They mediocritize any revolutionary idea. You can guarantee that the project will fail, or you won’t get the revolutionary project you want if it’s subjected to external peer review.

A further comment, “Outside review, clearly and unequivocally, gets rid of any

innovative ideas.” An office director said, “That process [peer review] is guaranteed to

never fund an idiot. It’s also guaranteed to never fund a visionary.” One subject felt

strongly that DARPA’s model of evaluation was misperceived as elitist. He said:

Now, with regard to whether or not that’s elitism, or on the other hand whether or not having a committee decision structure as the NSF [National Science Foundation] does is really more democratic, I think is subject to argument. I can just as easily argue that having a committee of folks make a decision about supporting a particular proposal that goes to the National Science Foundation easily is an example of elitism. That is to say, there is this club of people who all work, let’s say, in natural languages. There are a set of people who are accepted in that club, and there are a set of people who aren’t. You float that proposal around to a group of people that NSF has chosen to be its peer review group. And if you’re in the club you might get accepted, and if you’re not you might not get accepted. So, I think I reject the notion of elitism versus democracy, and just say there was a different set of objectives, a different modus operandi. ARPA initially was to be a quick reaction operation – let’s see where we make a big difference in a few places.

In short, the 52 years of success at DARPA are built on a model that eschews peer review

and consensus decision-making. DARPA’s process of evaluation is viewed as positive

and highly efficient in advancing technology visions in the overall mandate of radical

innovation.

Conclusion

The research data revealed five stages that define the process of radical innovation

at DARPA: Recruitment, Vision Formulation, Program Definition, Portfolio

Management, and Technology Transfer. This study investigated the start of the process,

specifically describing activities that occurred in the first two stages of Recruitment and


93

Vision Formulation. Recruitment begins with both the people and their ideas. In terms of

people, new program managers are hired into DARPA for their radical visions of

technology in addition to their technical expertise. In terms of ideas, the attributes of a

good vision are to be technically challenging, actionable, multidisciplinary, and far-

reaching – DARPA Hard. If program managers arrive to DARPA with partial visions,

they develop them through a combination of expert workshops and proof-of-concepts.

However, DARPA does not teach program managers how to build visions. Instead, new

program managers learn the necessary skills for finding and formulating visions through

socialization at DARPA, and all learning is gained through immersive experience and

interaction. Finally, when program managers are ready to take their visions forward for

new program status, they submit to a highly efficient and, by extension of 52 years of

operation, successful process of review by the organization’s leaders.


94

CHAPTER 6

DISCUSSION

This study provides empirical evidence of the role of vision in fostering

technological invention, adding to the existing literature about radical innovation (e.g.,

Roberts, 1987; Tornatzky, Fleischer, & Chakrabarti, 1990; O’Connor et al., 2008).

DARPA provides a long history of examples of technical program visions and how these

visions are formed and communicated time after time. In this section, the four main

findings of the study are discussed in detail and in context of the literature.

First, this study shows a relationship between the formation of a technological

vision and the sustained creation of radical innovation, providing new knowledge about

the role of vision in radical innovation. Since its inception in 1958, new programs at

DARPA have required a vision to be started, which then guides subsequent work and

development. Several dimensions arise regarding the role of vision, which entail

functioning primarily at the program level, characterized as DARPA Hard, and relying on

the program manager as a vision champion. Second, this study describes the use of expert

workshops and proof-of-concepts, used steadily by DARPA to shape partial visions into

complete visions, which demonstrates critical efforts occurring pre-vision. Third, this

study describes the importance of socialization in order to prepare and instruct program

managers in their envisioning skills. Immersed in the culture at DARPA, new program

managers learn from each other and their network connections. Fourth, this study

provides new evidence about radical innovation governance models. DARPA relies on

small group decisions by organizational leadership to approve promising new visions,


95

running counter to the dominant literature about stage-gate reviews, peer reviews, and

extended consensus-seeking processes.

A Process Model of Radical Innovation

As described in the previous chapter, DARPA follows certain high-level steps in

its quest for radical innovation, and this process is reproduced in Figure 16. By

documenting the process at DARPA, this study helps other researchers and practitioners

to understand one organization’s formula for sustained radical innovation. Documented

processes are the basis for repetition and become the springboard for continuous and

measurable performance.

Figure 16: Comparison between DARPA’s process model and the stage-gate model

Although DARPA’s process model of innovation looks similar to the typical stage-gate model for new product development (Cooper, 2001), the two models differ in terms of objectives, activity timing, and evaluation mechanisms.

!"#$%&'()*%+,((

)*-%&.--(/0-.(1.2.+#$3.&4(

5.-6&'(7(80+%,06#&(

90*&":(

;."<*%43.&4(8%-%#&(

=#<3*+06#&(><#'<03(90*&":(

>#<?#+%#(@0&0'.3.&4(

5.":&#+#'A(5<0&-B.<(

1C;>CD-($<#".--(3#,.+(#B(<0,%"0+(%&&#206#&(

5A$%"0+(-40'.E'04.(3#,.+(B#<(&.F($<#,*"4(,.2.+#$3.&4(


96

In addition, some scholars may see a similarity between the depiction of

DARPA’s process model and the typical stage-gate model for new product development

(Cooper, 2001), depicted in Figure 16. Both models are comprised of five stages that

sequence categories of cross-functional activities, which help to invite a comparison.

However, there are at least three key differences between the two models. First, the two

models differ in objectives. DARPA’s goal is radical innovation, which is intended to

produce new technologies that ultimately may lead to new products. In contrast, the

stage-gate process is designed to build and launch new products.

Second, the two models differ in their activity timing. DARPA’s model is focused

on the early stages that precede project scope. The stage-gate model is missing the

preliminary or ideation phase, often called Discovery, which occurs before the start of the

first stage of scoping.

Third, the two models differ in evaluation mechanisms. DARPA’s process is

fluid, and although transition arrows are noted between stages, formal decision points are

not necessarily required before proceeding onto the next set of activities. In comparison,

the stage-gate model is predicated on predefined deliverables and checkpoints with go/no

go criteria at the end of each stage (what are called gates).

New Dimensions of Vision

Vision plays a central role in DARPA’s process of innovation; indeed, DARPA

starts its process with vision. It matters where and how a vision is started, as does who

starts and maintains the vision. DARPA program managers are hired deliberately for their

visions of technology, even if partially formed. Then, program managers codify their


97

visions at the start of each new program in a specialized document called a Broad Area

Announcement (BAA), which are used to generate interest in the broader R&D

community. Thus, the vision is formulated before groups are funded because DARPA’s

funding recipients rely on these BAAs to determine potential solutions.

Visions at the Program Level

By studying the role of vision within DARPA, this study reveals several new

dimensions of vision as related to innovation. One dimension is the level at which a

vision operates. The dominant business literature has largely studied vision at the

organizational level (e.g., Collins & Porras, 1991); at the other end of the literature,

several studies have investigated technological visions at the product or project level

(e.g., Lynn & Akgün, 2001). Within DARPA, work is broken down at three levels:

organizational, program, and project, and the data shows that vision is introduced and

functions primarily at the program level. Figure 17 illustrates the multiple levels of

visions that could exist within an organization, and visions at DARPA address the gap

between the organizational and project/product levels.


98

Figure 17: Visions at DARPA operate at the program level

The literature on innovation predominantly discusses technological visions at the organizational level, and several studies have investigated technological visions at the project level. The literature fails to discuss vision at the program level, which is equivalent to the business unit or market level. At DARPA, technological visions function at the program level.

In fact, DARPA lacks a traditional corporate vision, which identifies a set of

organizational values and direction for the enterprise. Since its inception in 1958, the

agency has not defined (or even reinvented) its long-term goals, aspirations, and values at

the organizational level. Instead, DARPA emphasizes visions at the program level, which

correlates with a traditional business unit or market focus. Multiple visions – often

totaling over a hundred, depending on the number of program managers actively serving

at DARPA – exist in parallel at a given time. Programs serve as new, broad-scale

technical initiatives that typically encompass multiple projects, and projects are

equivalent to product teams in industry. Again, a DARPA program could be considered

equivalent to a business unit or new market category. At DARPA, a program is more of

an open-ended question or challenge posed to the R&D community, which might have

multiple solutions and product possibilities, and scholars have documented the benefit of

a challenge model within an R&D setting (Bonvillian, 2006, 2009).

!"#$%&'$()%$*+,&-&)%+

.")#"$/+,&-&)%+

.")0123+4+.")5623+,&-&)%+

!"#$"%&'()*+*%

,+-++

./0*123%-+*+4-(,%

546%02%7,+%807+-47)-+%

./0*123%-+*+4-(,+


99

In addition, visions at the program level allow DARPA program managers to

direct multiple projects, multiple teams, and even multiple products over multiple years.

Through visions at the program level, DARPA can excite and rally interest across several

different technical areas, helping to distribute resources more effectively. Program

visions provide a way to organize multiple projects and smaller-scale efforts across a

range of funding recipients, who each may interpret the vision differently in application.

This approach, in turn, increases the likelihood of a greater diversity of solutions. A

program structure also allows for greater flexibility in engendering commitment.

Vision Quality

A second dimension is the quality of the vision. In the literature, few studies focus

on technological visions, and most scholars draw on studies of corporate vision. For

example, Lynn and Akgün (2001) describe product visions as a combination of clarity,

support, and stability, which are determined relative to the larger organization. While

these attributes offer a sense of an ideal vision, they do not provide meaningful guidelines

on how to develop a vision, including the type of vision to create in the technology space.

This study shows that technological visions at DARPA have several attributes that are

essential to the creation of its visions. Since its inception, DARPA has socialized a catch

phrase known as DARPA Hard. Drawn from the data, a DARPA Hard program vision is

characterized as technically challenging, actionable, multidisciplinary, and far-reaching.

Taken apart, these attributes can be found discussed in prior studies.

The first attribute of technically challenging is understood within the operations

research and engineering design community as a “wicked problem” (Buchanan, 2009). A


100

wicked problem is a technically difficult problem that is nearly impossible to solve due to

complex interdependencies, a high level of ambiguity, and conflicting interests from

stakeholders. Wicked problems cannot be solved through classic experimentation and

logic, instead requiring a different and more creative strategy of reasoning.

By focusing on these types of problems at DARPA, program managers have

ensured that they push the limits of innovation sought, what might be interpreted as

“highly radical” innovation according to Abetti’s scale (2000). When most definitions of

radical innovation argue for market changes, DARPA is pushing for a radical technology

shift, which then may lead to a radical market shift. Each attempt at creating a new

technical solution changes the understanding of the problem in two fundamental ways.

First, more information helps to reformulate the initial requirements, and second, every

prototype and implementation built advances the state of knowledge overall in the world.

In other words, there is no turning back or reverting to the former understanding of the

problem. The vision for a DARPA program provides the high-level guidelines to inspire

potential funding recipients, and by engaging both more and different groups to respond,

DARPA is able to cast a wider net for solutions and likewise accelerate the

experimentation process.

This approach helps to drive toward action, and actionable is the second attribute

of DARPA Hard. Program visions are intentionally grounded in reality because they are

expected to improve and extend the limits of existing technologies. Visions cannot exist

as science-fiction fantasy, political rhetoric, or policy scenarios. This attribute is partly

captured in earlier research about the reflective practitioner, in which Schön (1983)

describes how professionals, such as engineers, address problematic situations that are


101

fraught with uncertainty, disorder, and indeterminacy by taking action through real-time

cycles of feedback and learning. In DARPA’s case, program managers rely on their

visions as a way to simulate broader learning in their research networks.

A growing body of research about learning in inter-organizational networks

shows that networks facilitate rapid responses. Powell (1990) states that, “Whether it is

the case that one firm’s technological competence has outdistanced the others, or that

innovations would be hard to replicate internally, as suggested by the growing reliance on

external sources of research and development, network forms of organization represent a

fast means of gaining access to know-how that cannot be produced internally” (p. 316).

The third attribute of multidisciplinary is equally critical to forming the right

program visions at DARPA. As many subjects noted, they needed to redefine problems

outside of usual boundaries, and complex situations required drawing from more than one

discipline. Multidisciplinary efforts are not new to government-sponsored R&D and can

be evidenced in the rise of systems engineering in the 1950s that supported large-scale

efforts, such as the Atlas missile program and ARPANET (Hughes, 1998). This type of

approach encourages less hierarchical control and more network-based management

techniques.

The fourth attribute of far-reaching is important when creating program visions at

DARPA. One part of far-reaching is about having a broad impact in society. Subjects

spoke about making a difference in magnitude. DARPA program managers stated that

they need to think big in order to have big results. Another aspect of far-reaching is the

ability to plan long-term. The importance of planning long-term has its roots in World


102

War II, notably the founding of RAND (Campbell, 2004), which transition into today’s

management science through thinkers such as Drucker (1959, 1973).

The real test of a good vision in R&D is whether others will commit resources to

action, which will bring results in the future. DARPA deliberately couples action with

future intent. However, the conundrum is that traditional R&D results may not be

produced or easy to measure because the extent of far-reaching effects take time and are

broadly distributed across society. The attribute of far-reaching is consistent with recent

work in foresight engineering, which focuses on long-range technology cycles as part of

an organization’s ongoing search for innovation opportunities (Carleton & Cockayne,

2009).

Together, these four attributes – technically challenging, actionable,

multidisciplinary, and far-reaching – that make up a DARPA Hard program provide a

metric that can be instrumented and tested. Based on pioneering work in taxonomies

(Bloom, 1956; Fleishman & Quaintance, 1984), Figure 18 presents a sample

classification using a 7-point scale that was used for the quantification of human

performance variables, specifically describing human ability for side-to-side equilibrium

(Cockayne & Darken, 2004). This type of scale could be adapted in order to classify each

of the four attributes characterizing DARPA Hard. Follow-on studies can further define

and test the scale values as related to radical innovation. Ultimately, if other

organizations seek to recreate a variant of DARPA Hard, they will benefit from defining

and using a clear classification of technological visions.


103

Figure 18: A sample 7-point scale for quantification of human performance variables

A sample 7-point scale, drawn from another study, could be adapted to classify and evaluate each of the four attributes characterizing DARPA Hard. Follow-on studies can define and test the scale values as related to radical innovation.

1

2

3

4

5

6

7


104

Visionaries of Technology

A third dimension of vision is the person responsible for fostering it. Visions

cannot exist without creators, who must imagine and invent them. Within DARPA, the

work on innovation is driven as much by ideas as by individuals. Program managers are

hired as technical visionaries, and they are solely responsible for shaping, spearheading,

and promoting their respective visions of technology. The project champion is a critically

recognized role in innovation, and findings from this study are consistent with literature

on this topic (Howell & Higgins, 1990). At DARPA, a new program is not confounded

with multiple organizational champions; instead, there is a clear relationship in that each

program manager builds one vision per program. Figure 19 depicts this relationship.

However, the DARPA program manager does not operate in isolation. He (or she) is part

of a broader ecosystem and network, in which multiple players – both internally and

externally to the agency – are engaged to support the formation and execution of a

program vision.

Figure 19: A radical technological vision relies on one big idea and one visionary

At DARPA, a program vision relies on a program manager, who serves as the vision’s primary champion internally and externally. Moreover, there is a clear relationship in that each program manager builds one vision per program.

!"#$$%&'$()#*$

!"#$+&,&-"*./$

0*)&1*2$3#14"-2-'&1*2$

+&,&-"$


105

DARPA program managers serve in other innovation roles that have been

documented separately in literature. For example, they share some characteristics with

business innovators because DARPA program managers provide substantial funding, as

well as some organizational credibility and access to other resources (Howell & Higgins,

1990). Although DARPA program managers do not build and develop their own visions,

instead relying on the various funding recipients, they do act as technical innovators in

other ways (ibid.). More informed than the usual project champion, DARPA program

managers are nearly all technically educated and bring deep expertise from various fields

of engineering and science. This background allows them to more effectively understand

the given technical problem, as well as advise and guide the technical teams that they

sponsor. A growing number of studies discuss the special role of a technical visionary,

who combines technical knowledge with project oversight (Hebda et al., 2007;

Deschamps, 2008).

DARPA program managers also play the role of technology licenser or

technology transfer manager. They are directly responsible for finding potential user

groups, typically in the U.S. military services, who might test and ultimately adopt a

functional prototype. The final success of DARPA program visions hinges on user

adoption.

At DARPA, potential program managers – the champions of new technological

visions – are found and recruited through the extended research network. Studies show

that as networks mature, they tend to petrify (Powell, 1990). People prefer to work with

familiar connections, which limits network access to new connections. Current program

managers will find new program managers based on similar qualities and will continue


106

funding the same relationships. When this happens, an innovation network does not

diversify, and the development of new ideas can be potentially severely limited. DARPA

has addressed this limitation by deliberately hiring program managers new to the

network, who, in turn, bring new visions of technology. Subsequently, the new-to-the-

network program manager finds and funds research groups that bring additional new

ideas to the network, which helps to refresh institutional thinking and challenge

engrained assumptions.

Lastly, DARPA is now over 50 years old as an organization, and historically, the

agency has relied on its network for internal job referrals. As the people in DARPA’s

network have aged, they may not be cultivating as many new relationships with other

research groups or also with junior engineers and scientists. Age plays a substantial role

in creating new fields, and research shows that younger scientists are more likely to be

drawn to a new field than older scientists (Rappa & Debackere, 1993).

Some scholars have studied how large mature organizations must continually

reconfigure their systems of power in order to sustain innovation (Dougherty & Hardy,

1996). Recently, DARPA leadership has recognized the need to recruit younger program

managers into its mix. For example, the press observed former agency director Tony

Tether “has managed to draw younger researchers into an agency whose stalwart backers

are growing greyer every year” (Nature, 2008). However, more research is needed to

understand the affects of age on DARPA’s ability to foster radical innovation.


107

The Development of Partial Visions

In the key texts that mention vision, few descriptions are provided about how to

generate a vision or develop a partial vision into a complete technological vision (e.g.,

Roberts, 1987). Scholars underscore the importance of having a vision, yet they assume a

complete vision. Findings from this study demonstrate that multiple steps are consistently

taken by DARPA program managers in order to advance their early ideas and thinking

before the complete vision is formed. Figure 20 illustrates the actions that must occur

before a complete vision is achieved. In addition, while the technological idea drives

action, the path to the vision itself is emergent.

Figure 20: Efforts preceding a complete vision of technology

Earlier actions occur before a complete vision is achieved at DARPA.

This study describes the formation of partial visions via two primary mechanisms,

specifically expert workshops and proof-of-concepts, which are used consistently

throughout DARPA’s history to develop partial visions into clear visions. While details

may differ, the objective is the same between the two mechanisms: to gain more insight

into a promising yet incomplete vision. Expert workshops and proof-of-concepts address

the people and the idea, respectively. Through expert workshops, each program manager

!"#$%&&'()(*+&

,$%-$"&'()(*+&

./0#%1&2*%3)4*0&

,%**56*56&!*+7#01&8#9*&

:##;<$73&=&>#$%+(+?&


108

engages his (or her) network, and the network serves as a way to gain perspective through

dialogue among trusted colleagues. In studies about knowledge networks and

communities of practice, network members regularly share information through both

formal and informal channels (Hildreth & Kimble, 2004; Powell & Grodal, 2005), and

the DARPA workshops positively exploit the broader knowledge network for the agency.

The DARPA workshops are effective because they draw on the collective wisdom for a

field, helping DAPRA program managers to gain access to the latest knowledge about a

particular topic.

If the workshops rely on people, the proof-of-concepts depend on the idea. The

objective of the proof-of-concepts is to explore and test the feasibility of an emerging

idea. Each proof-of-concept serves as a directed demonstration. Proof-of-concepts are

regularly discussed in engineering design research and business studies as a form of

prototyping (Schrage, 1999; Betz, 2003; Moss & Atre, 2003), and specifically, Carleton

and Cockayne (2009) discuss the growing role that physical prototypes serve in long-

range planning. This study provides new information about the use of proof-of-concepts

in vision development as a way to demonstrate feasibility and test early hunches before

undertaking a new technical initiative. There is an opportunity to expand on the

relationship between prototyping and vision formation.

It is important to note that this combination of expert workshops and proof-of-

concepts has provided the primary mechanisms for converting partial visions into full

visions at DARPA; no other mechanisms were pursued as long or as reliably, as reported

by DARPA program managers and funding recipients. This approach has implications for

organizations pursuing radical or disruptive innovation. O’Connor and her colleagues


109

(2008) discuss the different experiments that big companies have attempted in order to

scout for and generate radical ideas. Some of these experiments resemble the expert

workshops at DARPA. IBM has held a large annual R&D event to order to stimulate new

ideas internally and identify potential emerging business opportunities. This event has

been called by multiple names – including idea jams, idea cafes, and deep dives – and

while the organizers continually tinker with the process, the event itself remains constant

every year. The annual event has led to a high number of opportunities, which in turn

have become profitable business lines at IBM.

Learning Radical Innovation Through Socialization

The third finding relates to the culture of innovation at DARPA. Program

managers come from a variety of backgrounds. While they have impeccable academic

and professional credentials, many lack direct experience with certain innovation skill

sets, such as documenting a vision, recruiting and leading others, and technology transfer.

Regardless of their background, expectations are high for DARPA program managers to

develop and deliver on their program visions quickly.

In addition, DARPA does not provide formal training in innovation “know how”,

particularly the skills needed to develop program visions. Is staff training necessary for

radical innovation? According to subjects, DARPA has not codified much of its internal

procedures historically; so new program managers cannot rely on manuals or similar

process guides. Instead, knowing is a matter of participating. At DARPA, subjects

reported learning primarily from immersion. From the start, a candidate for a new

program manager has to be already embedded in the research community to be


110

considered for recruiting. Once at DARPA, program managers described learning by

doing, particularly by proactively reaching out to colleagues, alumni and other members

in the network for advice and resources, as well as by gaining new knowledge from

regular field visits.

In many ways, DARPA is a culture of show, not tell. Through a process of

socialization, program managers acquire the habits, beliefs, and accumulated knowledge

of the organization. In sociology, this period is known as metamorphosis, when a

newcomer becomes an established organizational member (Kramer, 2010). How people

behave and interact with one another over time shapes an organizational culture, and the

data from DARPA is consistent with prior studies about tacit knowledge and informal

learning occurring within innovation organizations and communities of practice (Von

Krogh et al., 2000; Wenger, 2007).

If an organization is to survive, then research shows that stability over time is

required, so that one generation of employees transmits the dominant social and cultural

patterns to the next generation (Alvesson, 1995). In other words, practice is transferred

from those who have done it to those who need to do it. At DARPA, this transfer of

knowledge occurs through informal conversations, and given the short contracts of

DARPA technical staff, the cycle of generations is rapid. It is remarkable that a

knowledge-generating organization over 50 years old, which has resisted lasting

knowledge capture, has maintained such a stable set of practices as DARPA has. Based

on subject reports, two factors have likely contributed most to the unusual stability of

DARPA’s culture. First, the broader infrastructure supporting DARPA program

managers, namely the support staff, provide continuity across leadership turnovers. This


111

support staff functions as an underlying layer of institutional permanence, handling the

same routines and project coordination tasks. Second, the agency’s network structure

supports ongoing learning. For example, even when program managers leave their

agency roles officially, they typically stay connected to DARPA in other ways. This

connection creates additional channels of knowledge sharing between staff and also

ensures that some institutional memory is maintained across staff rotations. New staff

relies on the stories and experiences shared within the network in order to prepare

themselves at DARPA.

Internal Review of Radical Innovation Ideas

Even with the right person and the right idea, a promising technological vision

may not become a new program at DARPA. There is one final test before a Broad Area

Announcement (BAA) is released to the public. A program manager must pitch his vision

internally with a small audience for funding approval, and decision making authority

resides namely with the agency director and respective office director. Subject reports

demonstrate that DARPA has consistently followed this governance model over the

years, actively discouraging larger evaluations in the agency’s innovation process.

Subjects especially note the benefits of speed, convenience, and flexibility from these

small group reviews.

DARPA’s model runs counter to the literature and practice of innovation, which

discusses consensus-based governance models – such as innovation boards, technology

councils, R&D committees, task forces, and stage gates – as a dominant best practice

(Bacon & Butler, 1973; Hamel, 2002; Snyder & Duarte, 2003; O’Connor et al., 2008;


112

Skarzynski & Gibson, 2008). These models provide a decision-making framework that

help to define evaluation criteria, grant decision-making power, and verify feasibility of a

new research idea. A growing body of literature has noted that certain models have

limitations for radical innovation. Gassmann and von Zedtwitz (2003) note:

In industries or projects where the science or technology push is the dominant driver of innovation, stage-gate processes are too rigid and slow. Innovations that are triggered by a technological invention with unknown market potential need different processes and techniques to succeed. (p. 704) Overall, innovation studies endorse a strong philosophy that the processes for

radical or disruptive innovation must differ from traditional R&D processes in order to be

effective within an organization. By deliberately adopting a model of limited, leadership-

driven review and following it for over 50 years, DARPA provides empirical support for

this belief. Instead of creating large task forces, DARPA relies on its leadership to

approve and support the visions. Instead of formally scheduled sessions, DARPA

program managers arrange meetings when they feel that their new program visions are

ready for funding. Most of corporate R&D, the work of funding agencies, and academic

research are actually structured in direct opposition to this approach. Members of the

science community, who believe that DARPA provides an enduring and effective model

for advancing radical innovation, understand this difference. Penman and Bates (1999)

write, “Those wishing to emulate the success of DARPA and Bell Labs might consider

another important aspect: freedom from the so called ‘peer review’ that weighs down

most National Institutes of Health (NIH) and National Science Foundation efforts.”


113

Conclusion

Four main findings were discussed in relation to the literature review. By

describing how visions serve an integral role in DARPA’s innovation process, the first

finding brings new perspective to innovation studies about the role of visions in radical

innovation. In particular, new program visions must meet the criteria of being DARPA

Hard, and this term of art introduces a working metric for technical breakthroughs that

are nearly impossible to achieve based on the current state of knowledge and tools.

Second, the discovery that expert workshops and proof-of-concepts have been used

repeatedly to convert partial visions into complete visions at DARPA shows that

activities exist pre-vision and directly influence the formation of technological visions.

Third, the discovery that new program managers receive no formal documentation or

training for their roles and instead rely on acculturation is consistent with prior research

on innovation networks and communities of practice. Finally, by showing that DARPA

has a leadership-driven, decision-making model, in which leadership approves a new

program vision, the fourth finding introduces contradictory evidence to the dominant

literature. These four findings, supported by empirical evidence, add to the current

understanding of technological visions and radical innovation research.


114

CHAPTER 7

CONCLUSION

In conclusion, this research investigated the role of vision in the process of radical

technological innovation. DARPA provides a relevant data set because the federal

agency’s mandate has explicitly focused on sponsoring technology ideas that are

revolutionary and high-payoff. Moreover, DARPA has been actively pursuing radical

innovation since 1958, which demonstrates over 50 years of sustained practice. By

drawing on 47 interviews with DARPA personnel and funding recipients, plus archival

documents and an additional set of 12 historical interviews, a dominant set of practices

were discovered that span multiple DARPA eras and offices.

This study relied on grounded theory methodology to analyze the data. The

purpose of an inductive study, such as the one reported here, is to guide and inspire new

research, not to validate existing ideas. Four major findings demonstrate that vision

fosters radical innovation at DARPA. Specifically, vision starts DARPA’s process of

radical innovation through the recruitment of innovation champions who develop

DARPA Hard visions at the program level; partial visions can be advanced to complete

visions through expert workshops and proof-of-concepts; job skills are learned largely

through socialization within DARPA’s culture; and promising new visions are ultimately

selected through leadership-driven approval. The extent that these findings generalize to

other innovation-seeking organizations can be determined through additional research;

however, data from existing literature suggest that any resemblance to efforts in other

organizations is piecemeal at best. Follow-on studies can further quantify these findings.


115

Contribution to the Field

Outsiders have long viewed DARPA as a black box, in which the agency’s input

and output are clear, yet knowledge of its internal workings remains murky. By studying

DARPA’s internal operations, this study provides new empirical evidence about the role

of vision and its relationship to the early stages of radical innovation in an organizational

setting. In addition, by taking a multidisciplinary stance, this study synthesizes research

conducted about radical technological innovation across different fields, including

organizational innovation, technology management, visionary leadership, and industrial

research and development. Therefore, a framework is introduced to unify the literature

around the concept of technological visions. This study also provides a novel data set.

While most studies about DARPA have been fragmented or largely anecdotal to date, this

study provides a coherent and complete data set on the innovation processes at DARPA.

Contribution to the Practice of Innovation

This study provides lessons to industry, and the four major findings offer insights

for both researchers and practitioners of radical innovation. Multiple organizations –

including corporations, research labs, and funding agencies – can gain insight in

facilitating efforts during the early stages of innovation, notably in recruitment and vision

formation. Specifically, if vision operates effectively and powerfully at the program level,

then R&D management may want to consider how their current organization is structured

and at which levels of execution. In addition, the four attributes that characterize DARPA

Hard – technically challenging, actionable, multidisciplinary, and far-reaching – provide

a set of evaluation criteria for a good vision that can be instrumented and tested.


116

Moreover, DARPA recruits new program managers from a broad and diverse

network that includes universities, corporations, national laboratories, research

nonprofits, and military services. If R&D organizations want to enhance their own

research talent, then they should consider adopting DARPA’s procedures. In particular,

DARPA program managers are hired for their technological visions; thus, other

innovation-seeking organizations should make vision a critical qualification for their

intrapreneurs and radical innovation leaders. Envisioning ability may be the most

important skill for organizations seeking radical innovation.

In addition, DARPA has relied on expert workshops and proof-of-concepts as the

sole means to refine partial visions. Other innovation-seeking organizations should

evaluate and potentially eliminate all other mechanisms in idea scouting and formation,

which function as extracurricular activities.

This study raises implications for building an organizational culture that fosters

radical technological innovation. Innovation-seeking organizations can look to DARPA’s

model of acculturation as an effective way to socialize innovation leaders, who must act

as champions of new technological visions.

Lastly, DARPA’s model of leadership-driven decision-making challenges

dominant practices and beliefs within R&D. Fully embracing DARPA’s model may be

the most disconcerting change that an organization will face in its quest for radical

technological innovation.


117

Research Limitations

First, this study relies on interview data. Although hearing from subjects provided

direct and first-hand accounts, subjects may potentially inflate their own roles in context

or forget certain details over time. Reviewing agency documents as a secondary data

source, as well as identifying stable patterns of behavior across various agency eras and

directorships, addressed this limitation.

A second limitation is a hindsight bias, in which subjects emphasize only

successes from their past. Several subjects remarked about the strong ego of DARPA

program managers, which might influence how they perceive their self-worth and impact.

By interviewing DARPA funding recipients and related R&D roles, the various accounts

from program managers and other DARPA personnel were further verified and cross-

checked.

A third limitation is a researcher bias. A researcher may project, consciously or

unconsciously, personal beliefs and values into the choice of methodology and the

interpretation of findings. Comparing primary interview data to historical interviews, the

latter of which were conducted by other scholars for different objectives, helped reduce a

researcher bias.

Lastly, the findings reported here could be idiosyncratic to DARPA. While the

agency was deliberately selected as a data source due to exceptional status, in order to

more effectively study radical innovation in its purest form, the findings may not be as

broadly generalized to other organizations or settings. By gathering data from different

DARPA offices, as well as from multiple programs across the agency’s entire history, the

applicability of the findings has been increased.


118

Directions for Future Research

As primary research, this study conducted extensive foundational work to

describe innovation practices at DARPA and outline a working model of vision

formation. Building on the data and insights in this study, a logical next step is to

formulate and test hypotheses for comprehensive investigation and experimentation.

Multiple areas remain unexamined and are suggested below as potential directions for

future research.

One research thread could probe the development of DARPA program visions

more deeply. For example, how has the quality of DARPA visions changed over time?

Could these visions be categorized in some fashion?

Another thread is to study the relationship between DARPA projects and their

overarching programs. Specifically, how do programs serve as strategic or technical

platforms for projects? How have project characteristics, such as the time required to

deliver solutions or the frequency and quality of interaction between DARPA program

managers and funding recipients, changed over time in relation to the program visions?

From an organizational view, a research opportunity exists to understand DARPA

in terms of other organizational structures and roles. For instance, how does DARPA’s

model of radical innovation compare to corporate R&D or national funding agencies?

How could DARPA’s innovation practices be transferred to other types of organizations?

And at a personal level, DARPA program managers could be analyzed against

complementary roles in a national innovation ecosystem, such as venture capitalists or

entrepreneurs.


119

A related thread is to compare DARPA’s mode of operations and historical output

to other R&D institutions, such as Stanford Research Institute / SRI, Bell Labs, HP Labs,

Apple Research, and Interval Research (C. House, personal communication, June 11,

2010). In addition, given the high turnover rate, the pool of DARPA alumni continues to

grow. How do former program managers influence other R&D organizations in their later

careers, and which innovation practices from DARPA do they continue, adapt, or even

reject in their subsequent work?

Lastly, DARPA’s pursuit of radical innovation does not function in isolation. An

additional research thread could investigate how DARPA operates as a pure model of

open innovation and how DARPA fits into the broader ecosystem of innovation, both

nationally and globally. In short, this study presents an early step in understanding how

one exceptional organization has organized and sustained radical innovation for so long

and as consistently as DARPA has.


120

APPENDIX A:

DARPA TECHNOLOGY OFFICES

As of the time of writing, DARPA programs are organized in the following technology offices (DARPA, 2010d): Adaptive Execution Office (AEO) – Conceive and execute novel technology and system developments that are adaptive both in end function and in the process by which they are developed. Improve the transition worthiness of DARPA programs across the enterprise. Defense Sciences Office (DSO) – Bridge the gap from fundamental science to applications by identifying and pursuing the most promising ideas within the science and engineering research communities, and is committed to transform these ideas into new DoD capabilities. Information Processing Techniques Office (IPTO) – Supports research, development, and prototyping that span the information lifecycle: sense, process, understand, and apply. Microsystems Technology Office (MTO) – Leading pioneering research in integrated Microsystems as “platforms-on-a-chip” to enable revolutionary performance and functionality for future DoD systems. Strategic Technology Office (STO) – Focus on technologies that have a global theater-wide impact and that involve multiple Services. Tactical Technology Office (TTO) – Engage in high-risk, high-payoff advanced technology development of military systems, emphasizing the “system” and “subsystem” approach to the development of aerospace systems. Transformational Convergence Technology Office (TCTO) – Advance new crosscutting capabilities derived from a broad range of emerging technological and social trends, particularly in areas related to computing and computing-reliant subareas of the life sciences, social sciences, manufacturing, and commerce.


121


122

APPENDIX B:

FEDERAL BUDGET CATEGORIES

The U.S. government classifies federal budgetary activities into functional and sub-functional categories that represent the major purposes of the federal government (U. S. Department of Defense, 2008). DARPA follows this model to characterize its investment portfolio. Generally, R&D activities are divided into either Science & Technology (S&T) or Research & Development (R&D) categories. Science & Technology encompasses budget categories 6.1, 6.2, and 6.3, which correlate with basic research, applied research, and development, respectively. Budget categories 6.4 through 6.7 correlate with testing, evaluation, and design activities. Categories are summarized in Table 7. DARPA personnel use these budgeting terms to differentiate their areas of investment, funding projects in categories 6.1 through 6.3. For example, a recent strategic plan stated, “DARPA uses its 6.1 funding to ‘get a seat at the table’ with basic research performers so that DARPA can understand what is going on at the forefront of science and can mine that research for new Defense capabilities” (DARPA, 2007). In addition, the taxonomy helps to show the relationships between the various funding areas, which are addressed by the other Department of Defense (DoD) agencies and military labs.

Category Category Name Description Science & Technology 6.1 Basic Research “Basic research is systematic study directed toward

greater knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific applications towards processes or products in mind…” (p. 5-2)

6.2 Applied Research “Applied research is systematic study to understand the means to meet a recognized and specific need. It is a systematic expansion and application of knowledge to develop useful materials, devices, and systems or methods…” (p. 5-2)

6.3 Advanced Technology Development

“This budget activity includes development of subsystems and components and efforts to integrate subsystems and components into system prototypes for field experiments and/or tests in a simulated environment…” (p. 5-2)

Research & Development 6.4 Advanced

Component Development & Prototypes

“Efforts necessary to evaluate integrated technologies, representative modes or prototype systems in a high fidelity and realistic operating environment are funded in this budget activity…” (p. 5-3)


123

Category Category Name Description 6.5 System

Development & Demonstration

“SDD programs have passed Milestone B approval and are conducting engineering and manufacturing development tasks aimed at meeting validated requirements prior to full-rate production…” (p. 5-3)

6.6 RDT&E Management Support

“This budget activity includes research, development, test and evaluation efforts and funds to sustain and/or modernize the installations or operations required for general research, development, test and evaluation…” (p. 5-4)

6.7 Operational Systems Development

“This budget activity includes development efforts to upgrade systems that have been fielded or have received approval for full rate production and anticipate production funding in the current or subsequent fiscal year…” (p. 5-4)

Table 7: U.S. federal defense budget categories

DARPA’s estimated FY2010 budget is $3.25 billion (DARPA, 2009; personal communication, Mark Peterson, June 10, 2010). DARPA typically accounts for about 25 percent of DoD’s S&T budget (DARPA, 2007). DARPA finds this emphasis in line with common industry practice, which devotes about 75 percent of R&D funding to product improvement and the remaining 25 percent to new ideas, products, and markets (ibid.).


124

APPENDIX C:

RESEARCH INTERVIEWS CONDUCTED

A total list of the research interviews is provided below as reference. The list of DARPA staff includes both current and former personnel, and interviews with current DARPA staff were conducted with DARPA’s full support. Several of the interviewees are also subjects of the oral histories done by the Charles Babbage Institute. DARPA Staff

Agency Directors & Deputy Directors

1. Ken Gabriel 2. Bob Leheny 3. Steve Lukasik Office Directors

4. Robert Kahn, Information Processing Technology Office 5. Greg Kovacs, Microsystems Technology Office 6. Ivan Sutherland, Information Processing Technology Office 7. Robert Taylor, Information Processing Technology Office 8. David Tennenhouse, Information Processing Technology Office Program Managers

9. Ralph Chatham, Cybernetics Technology Office 10. Vinton Cerf, Information Processing Technology Office 11. Joseph Cohn, Defense Sciences Office 12. Steve Crocker, Information Processing Technology Office 13. John Dexter Fletcher, Cybernetics Technology Office 14. Tom Kenny, Microsystems Technology Office 15. Doug Kirkpatrick, Defense Sciences Office 16. John Main, Defense Sciences Office 17. Dylan Schmorrow, Information Processing Technology Office 18. Barry Wessler, Information Processing Technology Office Related Personnel

19. Henry Girolamo, U.S. Army, Advanced Technology Manager – Integration & Transition

20. John Murphy, U.S. Navy, DARPA Special Assistant


125

Funding Recipients

Universities

21. Paul Cohen, University of Arizona 22. Les Earnest, Stanford University 23. Edward Feigenbaum, Stanford University 24. Richard Fikes, Stanford University 25. Sebastian Thrun, Stanford University 26. Bill Wulf, University of Virginia 27. Mike Zyda, University of Southern California Research Labs

28. Jim Allen, Sandia National Labs 29. Nate Blaylock, Institute for Human and Machine Cognition 30. Greg Hebner, Sandia National Labs 31. David Israel, SRI International 32. Cynthia Matuszek, Cycorp 33. Jonathan Salton, Sandia National Laboratories Corporations

34. Dalibor Hodko, Nanogen 35. Harvey Lehtman, retired (formerly SRI) 36. John Lekashman, Change Research Inc. (formerly NASA) 37. Jeff Rulifson, Sun Microsystems Laboratories 38. Bob Sproull Jr., Sun Microsystems Laboratories Related Defense Roles

39. Joe Bordogna, National Science Foundation, former Deputy Director and COO 40. Harry Rowen, Hoover Institution, former DoD Assistant Secretary of Defense 41. Larry Schuette, Office of Naval Research, Director of Innovation 42. Col. Jocelyn Seng, U.S. Department of Defense, Acquisition Integration 43. Rudy Darken, Naval Postgraduate School, Professor Other Researchers

44. Michael Belfiore, freelance technology writer 45. Andreu Veà i Baró, Internet Society (ISOC)


126

Oral History (Interview) Transcripts

The Charles Babbage Institute recorded multiple oral histories from 1988 to 1993, and the transcripts have been made available to the public for free. In addition, separate interviews were also conducted with Vinton Cerf, Stephen Crocker, Robert Kahn, and Steve Lukasik as part of this research study, which are noted with asterisks below.

46. Allan Blue 47. Bruce G. Buchanan 48. Vinton Cerf * 49. Robert S. Cooper 50. Stephen Crocker * 51. Charles M. Herzfeld 52. Robert E. Kahn * 53. J. C. R. Licklider 54. Steve Lukasik * 55. Robert L. Simpson, Jr. 56. Charles A. Zkaret


127

LIST OF REFERENCES

Abetti, P. A. (2000). Critical success factors for radical technological innovations: A five

case study. Creativity and Innovation Management Journal, 9(4): 208–221.

Alvesson, M. (1995). Management of knowledge–intensive companies. New York, NY:

Walter de Gruyter.

Aspray, W., & Norberg, A. (1998, October 28). An interview with J. C. R. Licklider

(transcript). University of Minnesota, MN: Charles Babbage Institute, Center for the

History of Information Processing.

Bacon, F. R., Jr., & Butler, T.W., Jr. (1973). Achieving planned innovation: A proven

system for creating successful new products and services. New York, NY: Simon &

Schuster.

Basit, T. N. (2003). Manual or electronic? The role of coding in qualitative data analysis.

Educational Research, 45(2): 143–154.

Belfiore, M. (2009). The department of mad scientists: How DARPA is remaking our

world, from the Internet to artificial limbs. New York, NY: Smithsonian Books /

HarperCollins.

Betz, F. (2003). Managing technological innovation: Competitive advantage from change

(2nd edition). New York, NY: John Wiley.

Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of

educational goals: Handbook I, cognitive domain. New York, NY: Green.

Bonvillian, W. B. (2006). Power play: The DARPA model and U.S. energy policy.

The American Interest, 39–48.


128

Bonvillian, W. B. (2009). The connected science model for innovation – The DARPA

Role. 21st century innovation systems for Japan and the United States: Lessons from

a decade of change: Report of a symposium. Washington, D.C.: National Academies

Press.

Broad Area Announcement: Augmented Cognition. (2001, July 27). Solicitation Number:

DARPA–BAA 01–38. Commerce Business Daily, PSA #2902.

Broad Area Announcement: Legged Squad Support System. (2008, Oct. 24). Solicitation

Number: DARPA–BAA–08–71. Retrieved from

https://www.fbo.gov/index?id=6c67f326ad4b836ce3217e3b7c56f45c.

Bryant, A., & Charmaz, K. (Eds.) (2007). The SAGE handbook of grounded theory. Los

Angeles: Sage Publications.

Buchanan, R. (2009). Thinking about design: An historical perspective. In A. Meijers,

(Ed.). Philosophy of technology and engineering sciences. Amsterdam, The

Netherlands: Elsevier B.V.

Campbell, V. (2004). How RAND invented the postwar world. Invention & Technology,

20(1): 50-59.

Carleton, T. and Cockayne, W. (2009). The power of prototypes in foresight engineering.

In M. Norell Bergendahl, M. Grimheden, L. Leifer, P. Skogstad, & U. Lindemann

(Eds.), Proceedings of the 17th International Conference on Engineering Design

(ICED’09), 6: 267–276. Stanford, Calif.: The Design Society.

Chesbrough, H. W. (2003). Open innovation: The new imperative for creating and

profiting from technology. Boston, MA: Harvard Business School Press.


129

Christensen, C. M. (1997). The innovator’s dilemma: When new technologies cause great

firms to fail. Boston, MA: Harvard Business School Press.

Christensen, C. M., & Raynor, M. E. (2003). The innovator’s solution: Creating and

sustaining successful growth. Boston, MA: Harvard Business School Press.

Christiansen, J. A. (2000). Building the innovative organization: Management systems

that encourage innovation. New York, NY: St. Martin’s Press.

Chubin, D. E., & Hackett, E. J. (1990). Peerless science: peer review and U.S. science

policy. Albany, NY: State University of New York Press.

Cockayne, W. R. (2004). A study of the formation of innovation ideas in informal

networks (Doctoral dissertation). Available from Dissertations and Theses database.

(UMI No. 3128637)

Cockayne W., & Darken, R. (2004). The application of human ability requirements to

virtual environment interface design and evaluation. In D. Diaper, & N. Stanton

(Eds.), The Handbook of Task Analysis for Human-Computer Interaction. Mahwah,

NJ: Lawrence Erlbaum Associates.

Collins, J. C., & Porras, J. I. (1991). Organizational vision and visionary organizations.

California Management Review, 34(1): 30–52.

Collins, J. C., & Porras, J. I. (1996). Building your company’s vision. Harvard Business

Review, 74(4), 65–77.

Collins, J. C., & Porras, J. I. (1997). Built to last: Successful habits of visionary

companies. New York, NY: HarperBusiness.

Cooper, R. G. (2001). Winning at new products: accelerating the process from idea to

launch (3rd edition). Cambridge, MA: Basic Books.


130

Corbin, J. M., & Strauss, A. (2008). Basics of qualitative research: Techniques and

procedures for developing grounded theory (third edition). Los Angeles, Calif.: Sage

Publications.

Cotterman, R., Fusfeld, A., Henderson, P., Leder, J., Loweth, C., & Metoyer, A. (2009).

Aligning marketing and technology to drive innovation. Research Technology

Management, 52(5): 14–20.

Daniel, H.–D. (1993). Guardians of science: Fairness and reliability of peer review.

Weinheim, Germany: VCH. Retrieved from

http://onlinelibrary.wiley.com/doi/10.1002/3527602208.fmatter/summary

Defense Advanced Research Projects Agency. (1998). DARPA technology transition.

Retrieved from http://www.darpa.mil/Docs/transition.pdf

Defense Advanced Research Projects Agency. (2003, February). Strategic plan.

Defense Advanced Research Projects Agency. (2007, February). Strategic plan.

Retrieved from

http://www.darpa.mil/Docs/DARPA2007StrategicPlanfinalMarch14.pdf

Defense Advanced Research Projects Agency. (2008a). DARPA: 50 Years of bridging the

gap. Retrieved from http://www.darpa.mil/50thanniversary.html

Defense Advanced Research Projects Agency. (2008, April 7). Human Resources (HR)

recruitment – sourcing and placement of DARPA Program Managers (PMs). Request

for Information, reference number SN08–27.

Defense Advanced Research Projects Agency. (2009). Department of Defense Fiscal

Year (FY) 2010 Budget Estimates, Volume 1 – Defense Advanced Research Projects

Agency. Retrieved from http://www.darpa.mil/budget.html


131

Defense Advanced Research Projects Agency. (2010a). Broad Area Announcements.

Retrieved from http://www.darpa.mil/cmo/baa.html

Defense Advanced Research Projects Agency. (2010b). DARPA mission. Retrieved from

http://www.darpa.mil/mission.html

Defense Advanced Research Projects Agency. (2010b). Director’s biography. Retrieved

from http://www.darpa.mil/directorbio.html

Defense Advanced Research Projects Agency. (2010d). Offices. Retrieved from

http://www.darpa.mil/offices.html

Defense Advanced Research Projects Agency. (2010e). Technology transition team.

Retrieved from http://www.darpa.mil/techtransteam.html

Defense AT&L. (2007, November 1). Joining DARPA as a program manager starts with

your idea [advertisement copy].

Defense Science Board. (1999, July). Investment strategy for DARPA (Storming Media

Report No. A014763). Washington, DC: Pentagon reports.

Dekkers, R. (2005). (R)evolution: Organizations and the dynamics of the environment.

New York, NY: Springer Science+Business Media.

Deschamps, J. (2008). Innovation leaders: How senior executives stimulate, steer and

sustain innovation. Hoboken, NJ: Jossey-Bass.

de Vaus, D. (2001). Research design in social research. Thousand Oaks, CA: Sage

Publications.

Dougherty, D., & Hardy, C. (1996). Sustained product innovation in large, mature

organizations: Overcoming innovation–to–organization problems. Academy of

Management Journal, 39(5): 1120–1153.


132

Drucker, P. F. (1959). Long–range planning: Challenge to management science.

Management Science, 5(3): 238–249.

Drucker, P. F. (1973). Management: Tasks, responsibilities, practice. New York, NY:

Harper Colophon.

Dundon, E. (2002). The seeds of innovation: Cultivating the synergy that fosters new

ideas. New York, NY: AMACOM.

Edelheit, L. S. (2004). Perspective on GE research & development. Research Technology


Ettlie, J. E., Bridges, W. P., & O’Keefe, R. D. (1984). Organization strategy and

structural differences for radical versus incremental innovation. Management Science,

30(6): 682–695.

Fleishman, E., & Quaintance, M. (1984). Taxonomies of human performance: the

description of human tasks. Orlando, FL: Academic Press.

Fleming, L. (2002). Finding the organizational sources of technological breakthroughs:

the story of Hewlett–Packard’s thermal ink–jet. Industrial and Corporate Change,

11(5): 1059–1084.

Fuchs, E. R. H. (2010, forthcoming). Rethinking the role of the state in technology

development: DARPA and the case for embedded network governance. Research

Policy. Retrieved from http://ssrn.com/abstract=1545155

Gassmann, O., & von Zedtwitz, M. (2003). Innovation processes in transnational

corporations. In L. V. Shavinina (Ed.), The international handbook on innovation.

Oxford, UK: Elsevier Science.


133

Glaser, B. G. & Strauss, A. L. (1967). The discovery of grounded theory. New York, NY:

Aldine.

Goulding, C. (2002). Grounded theory: A practical guide for management, business and

market researchers. Thousand Oaks, CA: Sage Publications.

Hafner, K., & Lyon, M. (1996). Where wizards stay up late: The origins of the Internet.

New York, NY: Touchstone.

Hamel, G. (2002). Leading the revolution: how to thrive in turbulent times by making

innovation a way of life. Boston, MA: Harvard Business School Press.

Harvard Business Essentials. (2003). Managing creativity and innovation. Boston, MA:

Harvard Business School Press.

Hebda, J. M., Vojak, B. A., Griffin, A., & Price, R. L. (2007). Motivating technical

visionaries in large American companies. IEEE Transactions on Engineering

Management, 54(3): 433–444.

Hildreth, P. M., & Kimble, C. (Eds.) (2004). Knowledge networks: innovation through

communities of practice. Hershey, PA: Idea Group Publishing.

Hiltzik, M. (1999). Dealers of lightning: Xerox PARC and the dawn of the computer age.

New York, NY: HarperBusiness.

Howe, H. E. (1952). ‘Space men’ make college men think. Popular Science, 161(4):

124–127.

House, C. H., & Price, R. L. (2009). The HP phenomenon: Innovation and business

transformation. Stanford, CA: Stanford Business Books.

Howell, J. M., & Higgins, C. A. (1990). Champions of technological innovation.

Administratively Science Quarterly, 35: 317–341.


134

Hughes, T. P. (1998). Rescuing Prometheus: Four monumental projects that changed the

modern world. New York, NY: Pantheon Books.

Johnson, C. L., & Smith, M. (1989). Kelly: More than my share of it all. Washington,

DC: Smithsonian.

Johnson, M. W. (2010). Seizing the white space: Business model innovation for growth

and renewal. Boston, MA: Harvard Business School Press.

Junod, T. (2003, December 1). Science & industry: DARPA. Esquire. Retrieved at

http://www.esquire.com/features/best–n–brightest–2003/ESQ1203–DEC_DARPA

Kim, W. C., & Mauborgne, R. (2005). Blue ocean strategy: How to create uncontested

market space and make competition irrelevant. Boston, MA: Harvard Business

School Press.

Kostoff, R. N., Boylan, R., & Simon, G. R. (2004). Disruptive technology roadmaps.

Technological Forecasting and Social Change, 71: 141–159.

Kramer, M. W. (2010). Organizational socialization: Joining and leaving organizations.

Cambridge, UK: Polity.

Leheny, R. (2007, August 7). What is a DARPA Program Manager? Speech presented at

DARPATech, DARPA’s 25th Systems and Technology Symposium. Anaheim, CA.

Leifer, R., McDermott, C. M., O’Connor, G. C., Peters, L. S., Rice, M. P., & Veryzer, R.

W. (2000). Radical innovation: How mature companies can outsmart upstarts.

Boston, MA: Harvard Business Press.

Levy, S. (2000). Insanely great: The life and times of Macintosh, the computer that

changed everything. New York, NY: Penguin Books.


135

Lundquist, G. (2004). The missing ingredients in corporate innovation. Research–

Technology, 47(5): 11–12.

Lynham, S. A. (2002). The general method of theory-building research in applied

disciplines. Advances in Developing Human Resources, 4: 221–241.

Lynn, G. S., & Akgün, A. E. (2001). Project visioning: Its components and impact on

new product success. Journal of Product Innovation Management, 18: 374–387.

Maccoby, M. (2007). Developing research/technology leaders. Research–Technology


Malakoff, D. (1999). Pentagon agency thrives on in–your–face science. Science,

285(5433): 1476–1480.

Moss, L. T. & Atre, S. (2003). Business intelligence roadmap: the complete project

lifecycle for decision–support applications. Boston, MA: Addison-Wesley.

National Research Council. (2005). Government/industry/academic relationships for

technology development: A workshop report. Steering Committee for Workshops on

Issues of Technology Development for Human and Robotic Exploration and

Development of Space, Aeronautics and Space Engineering Board, Division on

Engineering and Physical Sciences. Washington, D.C.: The National Academies

Press.

Nanus, B. (1992). Visionary leadership: Creating a compelling sense of direction for

your organization. San Francisco, CA: Jossey-Bass.

Norling, P. M., & Statz, R. J. How discontinuous innovation really happens. Research

Technology Management, 41(3): 41–44.


136

O’Connor, G. C., Leifer, R., Paulson, A. S., & Peters, L. S. (2008). Grabbing lightning:

Building a capability for breakthrough innovation. San Francisco, CA: Jossey–Bass.

O’Connor, G. C., & Rice, M. P. (2001). Opportunity recognition and breakthrough

innovation in large established firms. California Management Review, 43(20: 95–116.

Page, L. (2009, January 21). Spy chief to Obama: Let DARPA fix economy. The

Register. Retrieved from

http://www.theregister.co.uk/2009/01/21/darpa_economy_fix_plan/

Pappert, S. (2009). Succeeding with DARPA/MTO: Tips from an ex–Program Manager.

Proceedings from the 2009 Microsystems Technology Office Symposium, March 2–

5.

Penman, S., & Bates, C. C. (1999). DARPA in the spotlight. Science, 286(5438): 239.

Perez, C. (2002). Technological revolutions and financial capital: The dynamics of

bubbles and golden ages. Northampton, MA: Edward Elgar Publishing.

Powell, W. W. (1990). Neither market nor hierarchy: Network forms of organization.

Organizational Behavior, 12: 295–336.

Powell, W. W., & Grodal, S. (2005). Networks of innovators. In J. Fagerberg, D.

Mowery, and R. R. Nelson (Eds.), The Oxford handbook of innovation (pp. 56–85).

New York, NY: Oxford University Press.

Quigley, J. V. (1993). Vision: How leaders develop it, share it, and sustain it. New York,

NY: McGraw–Hill.

Quigley, J. V. (1994). Vision: how leaders develop it, share it, and sustain it. Business

Horizons, 37(5): 37–41.


137

Rappa, M., & Debackere, K. (1993). Youth and scientific innovation: The role of young

scientists in the development of a new field. Minerva, 31(1): 1–20.

Rhea, D. (2003). Bringing clarity to the “fuzzy front end”: A predictable process for

innovation. In B. Laurel (Ed.), Design research: methods and perspectives. Boston,

MA: MIT Press.

Richardson, J. J., Bosma, J., Roosild, S., & Larriva, D. (1999). A review of the

technology reinvestment project. Arlington, VA: Potomac Institute for Policy Studies.

Richardson, J. J., Larriva, D. L., & Tennyson, S. L. (2001). Transitioning DARPA

technology. Arlington, VA: Potomac Institute for Policy Studies.

Rice, M. P., O’Connor, G. C., Peters, L. S., & Morone, J. G. (1998). Managing

discontinuous innovation. Research Technology Management, 41(3): 52–58.

Roberts, E. B. (Ed.). (1987). Generating technological innovation. New York, NY:

Oxford University Press.

Roland, A., & Shiman, P. (2002). Strategic computing: DARPA and the quest for

machine intelligence, 1983–1993. Cambridge, MA: MIT Press.

Schilling, M. A. (2005). Strategic management of technological innovation. New York,

NY: McGraw–Hill/Irwin.

Schön, D. A. (1983). The reflective practitioner: how professionals think in action. New

York, NY: Basic Books.

Schumpeter, J. (1942). Capitalism, socialism, and democracy. New York, NY: Harper &

Row.

Schrage, M. (1999). Serious play: How the world’s best companies simulate to innovate.

Boston, MA: Harvard Business School Press.


138

Siedman, I. (2006). Interviewing as qualitative research: A guide for researchers in

education and the social sciences. New York, NY: Teachers College Press.

Sigismund, C. G. (2000). Champions of Silicon Valley: Visionary thinking from today’s

technology pioneers. New York, NY: John Wiley.

Smith, P. G., & Reinertsen, D. G. (1998). Developing products in half the time: New

rules, new tools (2nd edition). New York, NY: John Wiley.

Snyder, N. T., & Duarte, D. L. (2003). Strategic innovation: Embedding innovation as a

core competency in your organization. San Francisco, CA: Jossey-Bass.

Stringer, R. (2000). How to manage radical innovation. California Management Review,

42(4): 70–88.

Skarzynski, P., & Gibson, R. (2008). Innovation to the core: A blueprint for transforming

the way your company innovates. Boston, MA: Harvard Business School Press.

Tennenhouse, D. (2004). Intel’s open collaborative model of industry–university

research. Research Technology Management, 47(4): 19–26.

Tether, T. (2005, May 12). Multidisciplinary research. Presented to Committee on

Science, United States House of Representatives. Retrieved from

http://www.darpa.mil/testimony/HseSciCom51205url.pdf

Tornatzky, L. G., Fleischer, M., & Chakrabarti, A. K. (1990). The processes of

technological innovation. Lexington, MA: Lexington Books.

Tzeng, C. (2009). A review of contemporary innovation literature: A Schumpeterian

perspective. Innovation: Management, Policy & Practice, 11: 373–394.

United States Department of Defense. (2008, July). DoD’s Financial Management

Regulation (Publication No. DOD 7000.14–R), volume 2B, chapter 5 (Research,


139

Development and Evaluation Appropriations). Washington, DC: U.S. Government

Printing Office.

Urban Dictionary. (2010). DARPA hard. Retrieved from

http://www.urbandictionary.com/define.php?term=DARPA%20hard

Van Atta, R. H., Lippitz M. J., Lupo, J. C., Mahoney, J., & Nunn, J. H. (2003a).

Transformation and transition: DARPA’s role in fostering an emerging revolution in

military affairs. Volume 1 – Overall assessment. Washington, DC: U.S. Government

Printing Office.

Van Atta, R. H., Cook, A., Gutmanis, I., Lippitz, M. J., Lupo, J., Mahoney, R., & Nunn,

J. H. (2003b). Transformation and transition: DARPA’s role in fostering an emerging

revolution in military affairs. Volume 2 – Detailed assessments. Washington, DC:

U.S. Government Printing Office.

Van Atta, R. (2008). Fifty years of innovation and discovery. In DARPA: 50 years of

bridging the gap. Washington, DC: U.S. Government Printing Office.

Van de Ven, A. H., Angles, H. L., & Poole, M. S. (2000). Research on the management

of innovation: The Minnesota studies. New York, NY: Oxford University Press.

Van de Ven, A. H., Polley, D. E., Garud, R., & Venkataraman, S. (1999). The innovation

journey. New York, NY: Oxford University Press.

Vojak, B. A., Griffin, A., Price, R. L., & Perlov, K. (2006). Characteristics of technical

visionaries as perceived by American and British industrial physicists. R&D


Waldrop, M. M. (2001). The dream machine: J. C. Licklider and the revolution that made

computing personal. New York, NY: Viking Press.


140

Watts, R. M. (2001). Commercializing discontinuous innovations. Research–Technology


Weinberger, S. (2006). Imaginary weapons: A journey through the Pentagon’s scientific

underworld. New York, NY: Nation Books.