a conceptual understanding of requirements for theory-building research: guidelines for scientific...

A CONCEPTUAL UNDERSTANDING OF REQUIREMENTSFOR THEORY-BUILDING RESEARCH: GUIDELINES FOR

SCIENTIFIC THEORY BUILDING�

JOHN G. WACKERArizona State University

Business academics have focused their attention on empirical investigation ofprograms’ effect on organizational competitive performance. These studies pri-marily emphasize theory building. With the many definitions of theory, aca-demics are not certain whether their research papers meet the specificrequirements for theory development required by the academic field of thephilosophy of science. Certainly, supply chain academics generally believe thattheir academic articles fulfill the requirements of theory building. Althoughmany of these articles do have elements of theory, more focus is needed on thespecific requirements of theory to assure that academic research is ‘‘good’’ theorybuilding. The primary purpose of this research paper is to logically develop a setof guidelines to assist empirical researchers to assure that their studies fulfill therequirements of good theory based upon traditional scientific theory building.By fulfilling the requirements of good theory, researchers will develop studiesthat will have a lasting impact on their academic field. To achieve a lasting im-pact on an academic field, it is necessary to follow a logical plan. This articleprovides a plan for logical guidelines for developing an understanding of howand why ‘‘good’’ theory building is achieved. This article logically develops aformal conceptual definition of theory along with its related properties to un-derstand these guidelines. Next, it analyzes the requirements of theory, ‘‘good’’theory, and their properties. These guidelines are included in the existing phi-losophy of science publications. However, this article consolidates these sourcesand logically explains why these guidelines are needed. In the conclusion, theguidelines are summarized to serve as a summary checklist for supply chainresearchers to use for ensuring their articles will be recognized as a contributionto the academic field. So in that sense, this article does not develop a revolu-tionary new insight into theory-building empirical articles, but rather integratesdiverse traditional philosophy of science requirements into a much simpler setof guidelines. Through logical development of these guidelines, researchers willunderstand the structure of theory and how to ensure their studies can bemodified to have a lasting impact on the field of supply chain management.

Keywords: supply chain management; empirical theory testing

INTRODUCTIONOver the last several decades, there has been a dramatic

increase in empirical studies testing theory (Swamidass1991). The use of statistical techniques has been an im-portant contribution to understanding the wide variety ofpractices’/programs’ effects on international competi-

tiveness. Most of these articles focus on the testing anddevelopment of business ‘‘theory.’’ In general, academicshave a basic understanding of what theory is. However,this understanding is not precise enough to explain

exactly how and why theory building is driven by specific

�Like all invited papers and invited notes, the original version of this

manuscript underwent a double-blind review process.

Acknowledgment: The author wishes to thank many of his close

friends who have commented and discussed these materials over

many years. The author especially wishes to thank Lawrence Fred-

endall (Clemson University) for his careful reading of this study andmany corrections that were important for the development of the

study. Additionally, the author wishes to thank his many close ad-

visors. Most notable of these advisors are R. Kenneth Teas (IowaState University), Shelby Hunt (Texas Tech University), George Mar-

icoulides (California State University-Fullerton) and Daniel Samson

(University of Melbourne). Naturally, any errors, omissions, logic

faults, and so forth are entirely the fault of the author.

July 2008 5

requirements. Without this understanding of theory re-quirements and how it relates to empirical research, it isdifficult to assess if a statistical set of relationships are or

are not a formal theory. This issue was addressed inmarketing in books by Shelby Hunt (1991), in manage-ment in the Academy of Management Review’s special issue(1989) and several subsequent articles, and in operations

management in the Journal of Operations Management’s(1998) special issue on theory. Although these studiesprovided insightful understanding of what constitutes atheory-building article, they did not provide a concise,

logical plan for authors to ensure their articles followclassical guidelines for developing theory-building re-search. This article presents the logic behind theory-building articles and why specific properties are necessary

for the article to be considered an important contributionto the academic field. So in one sense, the article is notpresenting new astounding conclusions but rather ispresenting an integrated framework for understandinghow to develop theory-building articles.

Although outside of academia there is not a wide-spread appreciation of theory (see Shubik 1987), toacademics the need for theory is paramount to under-standing how theories can be applied (Poole and Van de

Ven 1989; Van De Ven 1989; Hunt 1991; Klein, Tosi, andCanella 1999, and numerous others). Consequently,there is a need to increase the precision of theory-building procedures for developing theories that are

important contributions to the academic field. Yet, thereis a need to clearly understand what a theory precisely isand what properties are essential for making a theoryuseful to the pragmatic managerial world.

The literature on the philosophy of science is well over2,500 years old. Many of the critical thoughts haveevolved over those millennia. Upon reading the philos-ophy of science literature, it is difficult to discern to

whom the original ideas and thoughts should be attrib-uted. Many philosophers of science did not cite theoriginal authors of these ideas and thoughts. Conse-quently, attributing thoughts to the original authors is

not possible. This article apologizes for any omission ofcitations that may have addressed these theory-buildingissues.

To begin, there is a difference between a lay under-standing and an academic understanding of precisely

what comprises a theory (most recently seen in Popper1957; Kaplan 1964; Bunge 1967; Hempel 1970; Hunt1991; and numerous others). The purpose of this study isto address key issues for developing theory. There are two

important issues: (a) theory’s conceptualization and(b) theory’s effect on conducting empirical tests. Becausethe conceptualization of theory drives statistical estima-tion, it should precede the statistical tests.

To understand theory conceptualization, it is necessaryto understand precisely what theory is and to understandprecisely what ‘‘good’’ theory is. The literature on these

topics is based in the formal study of the theory of the-ories: the philosophy of science. Somewhat unfortu-nately, the philosophy of science literature is not easy to

understand and even more difficult to apply. Yet under-standing the basic properties of theory and ‘‘good’’ theoryis essential for the conceptualization of theory and thestatistical tests of theory. Consequently, one purpose of

this article is to translate the philosophy of science lan-guage into specific, concise guidelines for testing theory.

In order to achieve this goal, this study clarifies thedefinitions of what is meant by theory conceptualization

and demonstrates how theory can be evaluated. Thereare three different levels of detail for examining theory.These are Theory, Good Theory, and Guidelines forGood Theory. ‘‘Theory’’ is used to determine if any set of

relationships (hypotheses) are really a theory or justlay conjectures: Do these relationships have definitions,domains, and predictions? ‘‘Good’’ theory determinesif a theory can be adequately tested. And ‘‘guidelines’’are used to apply specific requirements to elements of

theory.Although it is possible to give statistical examples of

how each of the guidelines affect empirical estimates, thelength of the discussion of those examples require: (A) a

complete literature justification, (B) a development ofthe definitions, (C) an internally consistent conceptualmodel, (D) a specification of the sample, (E) a completestatistical explanation, and (F) a complete substantive

explanation. In short, each one of the guidelines wouldrequire a separate full-length paper to empirically dem-onstrate how statistics support or question any theory.Consequently, this study presents logical insights to these

problems so interested readers may apply them to theirown research. This paper will leave it to future research todemonstrate how the guidelines for good theory devel-opment lead to specific statistical tests.

The remainder of this paper is organized as follows:First, it develops definitions of theory and ‘‘good’’ theory.Then it gives ‘‘good’’ theory formation guidelines basedupon the properties of ‘‘good’’ theory.

WHAT IS THEORY?The foundation of understanding of all theory is based

on the philosophy of science where theory is developedfollowing strict rules of logic. In the philosophy of sci-ence, the same rules apply for all theory in all scientific

fields. The classical philosophical properties of theoryapply to determine exactly what a theory is. In order tounderstand these guidelines, a formal conceptual defi-nition of theory is necessary. By definition, a theory is an

explained set of conceptual relationships. To determine ifa theory is actually a theory, there have to be measures forwhat is meant by ‘‘an explained set of conceptual rela-tionships’’. These measures are the called properties of

theories and they are used for determining if any set of

Journal of Supply Chain Management

Volume 44, Number 36

conceptual relationships is actually a theory (Bunge1967).

Figure 1 illustrates the relationship between the formalconceptual definition of theory and its properties. The

key terms in the formal conceptual definition of theoryare ‘‘an explained set of conceptual relationships.’’ To beexplained, a theory must answer all the common ques-tions: Who? What? When? Where? How? Why? and

precisely what Would, Should, or Could Happen? (if theprevious questions are answered.) These questionscomprise four basic properties of the formal conceptualdefinition of theory: Definitions (Who? and What?),

Domain (When? and Where?), Relationships (How? andWhy?), and Predictions (Would? Should? and Could?).Further, definitions explain the terms used in the theory,the domain is when and where the theory applies, rela-

tionships explain how and why the relationships exist,and what should, could, and would is predicted by thetheory (Bacharach 1989; Whetten 1989). Without anyone of these properties, any conjecture, inference, sup-

position, hypothesis, or set of hypotheses, is just not atheory. Calling any set of relationships a theory does notmake it theory.

There is an important pragmatic point concerningtheory: should any academic article be published that

does not have any theory in it? The answer to thatquestion can be phrased another way. Should an aca-demic article be published that does not answer the tra-ditional questions? Who? What? When? Where? How?

Why? and what Would? Should? Could happen? If so,which questions should not be answered? It is difficult toimagine any practical application that does not requireanswering all those questions. It is even more difficult to

imagine any pragmatic manager who would not wantthose questions answered before they apply any tech-nique (practice, program, etc.) to their organization. Yet,

these properties of theory can be met and the theory canstill not be what philosophers of science call a ‘‘good’’theory. ‘‘Good’’ theory has additional restrictions thatindicate faults in any theory that may indicate a poor

theory. Precisely what constitutes a ‘‘good’’ theory is thenext section’s topic.

ANY THEORY VS. A ‘‘GOOD’’ THEORYPhilosophers of science differentiate any theory from a

‘‘good’’ theory. ‘‘Good’’ theory has more restrictive prop-erties that further limit any theory from being a ‘‘good’’theory. To understand these properties, the formal con-ceptual definition of ‘‘good’’ theory must be understood.

A ‘‘good’’ theory is a fully explained set of conceptualrelationships used for empirical testing. The first keyterms are ‘‘fully explained.’’ ‘‘Fully explained’’ means thatnot just any set of conceptual relationships will serve as

‘‘good’’ theory because many theories are not fully ex-plained. The properties of good theory are more fullyexplained in Bunge (1967), Hunt (1991), Wacker (1998)and the aforementioned special issues of Academy of

Management Review (1989) and Journal of OperationsManagement (1998).

Figure 2 illustrates the relationships of all theory’sproperties and the restrictions on those properties forthe theory to be ‘‘good’’ theory. These are definitions

(conservatism, uniqueness, and parsimony), domain(generalizability, abstraction), relationships (fecundity,internal consistency, statistical parsimony), and predic-tions (refutability). To review these properties of ‘‘good’’

theory, this study will review them one by one. However,although the properties of ‘‘good’’ theory are associatedwith specific properties of theory, in many instances these‘‘good’’ theory properties may be associated with other

theory properties as well. For simplicity’s sake, each

Definition of theory:It is an explained set of conceptual relationships.

When and WhereDomain

Who and WhatDefinitions

How and Why:Relationships

Should, Could, WouldPredictions

Properties of All Theory

FIGURE 1The Relationship Between Theory and Its Properties

A Conceptual Understanding of Requirements for Theory-Building Research

July 2008 7

‘‘good’’ theory property will only be addressed here aslimitations on a theory property.

An important caveat! A ‘‘good’’ theory may not be a‘‘true’’ theory. It may be that all the above ‘‘good’’ theorycriteria are met and the ‘‘good’’ theory is just plain wrong.Yet, a wrong ‘‘good’’ theory will be identified early. On

the other hand ‘‘bad’’ theory can never be disprovedbecause there are so many conceptual ‘‘loopholes’’ thatwould never allow the theory to be disproved. The pointis that science advances through ‘‘good’’ theory and ‘‘bad’’

theory is a major challenge in all scientific fields.In summary to this section, all four properties of theory

are needed for a set of relationships to be a theory andadditional properties are needed to assure that the theory

is a ‘‘good’’ theory. ‘‘Good’’ theory is needed for the ad-vancement of science. The next four sections elaborate oneach property of ‘‘good’’ theory.

‘‘GOOD’’ THEORY’S PROPERTIES PROVIDEGUIDELINES FOR SCIENTIFIC THEORY

BUILDINGThe above discussion provides the basis for a set of

guiding principles for authors to use in their research.These principles will be called guidelines for ‘‘good’’ the-ory building. Each property of theory (definition, domain,relationships, and predictions) has specific challenges that

need more explanation to provide more clarity for theseguidelines. The next four sections are explanations of theseguidelines so they can be implemented.

Definitions: Science Can Only Advance as Rapidlyas the Language that Expresses Its Concepts

It has long been recognized that all theory must de-

velop an artificial language. That is, lay language is not

precise enough for scientific purposes. This artificiallanguage requires all theory to precisely define its terms

(see Bollen 1989; Bunge 1967; Hunt 1991; Teas andPalan 1997; and many others). Unfortunately in manyacademic articles, many researchers use ill-defined and/orassume previously ill-defined concepts are ‘‘good’’ defi-

nitions. The ‘‘previous definitions are precise enough’’assumption leads to ill-defined concepts being assumedto be adequate. Using ill-defined concepts prevents the-ory from being ‘‘good’’ theory. All concepts need to be

precisely defined inside of each specific theory. Theproperties of definitions that need to be considered areconservatism, definition parsimony and uniqueness.‘‘Good’’ theory has good definitions that are conservative,

parsimonious (short), and unique. A ‘‘good’’ definition isformally defined as a concise, clear verbal expression of aunique concept used for empirical testing.

The first question is: do business academics define their

concepts? Unfortunately, academics usually assume thateveryone implicitly understands conceptual definitionsbecause they are widely used. However, one of the fewacademic articles that ever studied the use of definitions

found that the majority of articles do not formally definetheir concepts. The study by Burgess, Singh and Koroglu(2005) found that out of 100 articles only 42 definedtheir terms. In short, only 42 of 100 articles preciselyknew what they were measuring, testing or discussing.

How can a researcher precisely measure something that isimprecisely defined? A concept’s measure is only as goodas its formal conceptual definition (Bunge 1967 andnumerous others). Yet there is an additional problem

when researchers utilize previously published definitionsthat have not been analyzed for conceptual problems. Toavoid these problems, ‘‘good’’ definitions have restrictiveproperties that limit how to define terms (see Bunge

1967; Teas and Palan 1997; Wacker 2004).

Definition of ‘’good’’theory:A fully explained set of conceptual relationships used for empirical investigations.

Domain

Definitions

Relationships

Predictions

‘ Good ’ Theory ’ sProperties

• Conservatism • Theory Parsimony• Uniqueness

• Generalizability • Abstractness

• Fecundity • Internal Consistency • Statistical Parsimony • Substantive vs. Statistical

Significance

• Falsifiability

Properties of All Theory

FIGURE 2The Relationship of ‘‘Good’’ Theory to Its Properties



A Definition of Conservatism. One frustration foracademics occurs in articles that use new, poorlydefined terms that are seemingly very similar (oridentical) to earlier concepts. It is a violation of the‘‘good’’ definitions property to use new terms withoutcarefully distinguishing them from similar, existingterms. This inclusion violates the conservation propertyand in many cases, may violate the uniqueness propertyof ‘‘good’’ definitions. Although this property may soundas if it restricts concepts and theory to the past, thisproperty actually serves the very important purpose ofassuring that concepts are not just redefined and calleddifferent names (Bunge 1967; Teas and Palan 1997).

Definition of Uniqueness. One very common difficultywith definitions is that many definitions are not unique.That is, they include terms that are used in otherconceptual definitions. When this happens, there is atautological relationship between the two concepts thatcause the two concepts to be statistically significantlyrelated. This problem has been noted as the ‘‘overlappingdefinition problem’’ or ‘‘concept stretching’’ (Osigwehand Chimizie 1989). In short, academic authors shouldbe very careful not to use terms found in other con-ceptual definitions.

Definition Parsimony. Long definitions carry manyproblems. Most of these definitions seem to includeeverything to avoid being non-unique. In some fields,this is very common in such definitions as total qualitymanagement, lean manufacturing, totally integratedsupply chain and even ‘‘big’’ just-in-time. Many of thesedefinitions confuse the relationships property of theorywith the definition property of theory (for a morecomplete explanation see Bunge 1967; Teas and Palan1997; Wacker 2004). Definitions should precede thetheory’s relationship property because researchers mustunderstand precisely what the concept is before theycan state how it is related to other concepts. A relateddifficulty with many definitions is that some publi-cations include other broad concepts in their defi-nitions thereby making the definition even broader(concept stretching). In sum, shorter definitions arepreferred to longer definitions to prevent concepts frombeing confused.

Good Measures of Concepts: You Cannot PreciselyMeasure What You Cannot Precisely Define. There is acommon misunderstanding among academics thatseeking specific empirical properties (characteristics,metrics, traits, measures, operational definitions, etc.) isthe best way to clarify concepts before they have devel-oped formal conceptual definitions. This empiricalsearch is illogical. How can any concept be measured,if the concept is not clear? Bunge (1967) and others havenoted that the measure of a concept is defined as aninformal definition (in the philosophy of science,measures are also known as accidental definitionsand in the business literature they are known asoperational definitions). Ambiguous and vague terms

lead to innumerable measures that authors may use toempirically test theory, causing innumerable contradictoryresults depending upon the selection of measures. Thequest for better measures detracts from the real problem:‘‘good’’ formal conceptual definitions (Bunge 1967).Worse, it leads to very confused measurements. Con-sequently, researchers should search for better formalconceptual definitions rather than measures or metrics.The fallacy of seeking properties to clarify bad definitionshas long been known in the philosophy of science (Bunge1967; Hunt 1991; Teas and Palan 1997 and going back toPlato in the parable of the cave).

Many concepts in business seemingly have animportant meaning but cannot be adequately defined.In general, for these ill-defined terms there is a heavyemphasis on developing measures for the ill-definedconcept. In many cases, the term is not clarified and isincluded in some form of measurement instrument. Thisinclusion causes the ill-defined ‘‘concept’’ to bestatistically significant but practically unimportant.More about this problem will be discussed below whendiscussing statistical versus substantive significance.

Measuring what is not clearly defined is an exercisein academic frustration. ‘‘Good’’ formal conceptualdefinitions have very few measures. Even with measuresdeveloped from ‘‘good’’ formal conceptual definitions, allmeasures are not created equal. Each measure has twocritical dimensions: the reason for using it and the degreeof necessity for the formal conceptual definition. Thecomplete explanation of these requirements is beyondthe intention of this paper. However, for the purposeshere, the property (measure) of a concept should bedirectly derived from the formal conceptual definition(called the interpretive property). For a more completeexplanation please see Wacker (2004).

The underlying principle of this discussion is basedupon an important academic tenet: science can onlyadvance as fast as there is a language to express it(Bunge 1967 and other earlier authors). Too frequently,as science advances, academics readily accept ambiguous(several distinct properties) or vague (innumerableproperties) definitions without trying to refine them.Without this refinement, the science cannot progress.All conceptual theory is derived from the existingliterature. It is important that the theory derived fromthe literature drives the conceptual model. Frequently,there are competing conceptual models. For business’theories, it is important to review the literature beginningwith the formal conceptual definitions to determinewhich of the competing theories has the better ‘‘good’’formal conceptual definitions. If one theory does notdefine its terms or does not evaluate existing definitionswhile a second does and offers better formal conceptualdefinitions, then the second theory should be consideredsuperior to the first. Without ‘‘good’’ formal conceptualdefinitions, it would be difficult for any theory to be‘‘good’’ theory.


July 2008 9

One major problem in business is the use of the term‘‘constructs.’’ Although it is not the focus of this article, acomment on the term ‘‘construct’’ is necessary. Onecommon definition of a construct is ‘‘a concept devel-oped for describing relationships among phenomena, ora theoretical definition in which concepts are defined’’(Vogt 1999). Vogt’s definition is ambiguous, because itpoints to at least two conceptual definitions in thephrases: (1) ‘‘describing relationships among phenom-ena,’’ and (2) ‘‘or a theoretical definition.’’ Vogt’sdefinition of construct is neither clear, nor precise, norunique. Consequently, it is not a ‘‘good’’ definition,because when an academic uses the term ‘‘construct’’they can mean either a relationship or a definition. Recallthat a formal conceptual definition must lead to a uniqueconcept. This property of ‘‘good’’ definitions is contraryto some of the current literature that use constructs formultidimensional concepts (note that the definitionof the term ‘‘construct’’ can be made clearer by making‘‘construct’’ into two definitions: abstract conceptconstruct and a theory relationship construct).With that stated, many times it is necessary to usemultidimensional terms to explain broad concepts forease of explanation.

In sum, great care should be taken when definingconcepts used in theory building. For theory building,all concepts should have good formal conceptualdefinitions. All formal conceptual definitions should beunique, conservative, and short. Unfortunately, there is anatural predisposition to accept traditional termswithout evaluating them for precision. Without theserefinements, theory and science cannot advance.

Domain of Theory: When and Where Did the EventHappen?

A theory’s domain is when and where a theory is to beapplied. It is a critical property of ‘‘good’’ theory becauseit limits when and where the theory applies. However,

the more important theories can be applied at moretimes and in more places than less important theories,other things being equal. There are two related propertiesof ‘‘good’’ theory: generalizability and abstractness. For

this study, a theory’s generalizability is defined as thedegree to which a theory can be applied to existingpopulations. The wider the existing populations wherethe theory applies, the more generalizable the theory is.On the other hand, abstraction is defined as a theory’s

application being void of time and space requirements.Put another way, an abstract theory can be applied overall times and all places. Abstraction at its highest level iscalled a grand theory and is considered the ideal goal of

theory because the theory applies to all times and allplaces (Osigweh and Chimizie 1989).

The choice of sample is dictated by the theory’s do-main. Reviewers of empirical articles uniformly challenge

the researcher’s choice of sample. The sample is always

heterogeneous because every sample point is gatheredat a different time and place. Consequently, authorsshould justify the sample they are planning to use to

test a specific theory. It should be apparent that thesample should be based upon the theory’s domain. Afrequent criticism of an empirical study is that the sampleused to test a theory is biased (a biased sample is defined

as the a priori belief that a sample is not representative ofthe theory’s population). It is always possible for anycritic to state that the sample was not representative ofthe population. So, authors should carefully account for

sample demographics to assure that when and where thesample is collected is well specified (Mitchell and James2001).

Which demographics (characteristics) should the re-

searcher expect to account for variations in the expectedrelationships? Here are some typical questions thatshould be included in the discussion of the chosensample: These characteristics can be classified as: (1)Countries, (2) Industries, (3) Respondent’s Position, and

(4) Respondents’ Demographics. Compounding all thesefactors, there are innumerable differences between thesedatum characteristics. Because the number of character-istics is larger than sample size, it would be not be pos-

sible for any statistical technique to control for all thedifferences. It would be relatively easy for any reader/re-viewer to summarily reject any empirical study on thegrounds that the data were heterogeneous, leading to all

empirical articles being rejected. One resolution for thisconundrum is the existence of empirical evidence thatsuggests that specific characteristics are related to im-portant conceptual variables. If there is no empirical ev-

idence in the academic literature, then the sample cannotbe biased and should not be questioned. The require-ment of empirical literature to support a reader’s claimthat the data were too heterogeneous to draw conclu-

sions is known as the previous literature convention. If thereis no previous literature to suggest some characteristic isrelated to the theory’s important variables, then thesample is considered unbiased.

‘‘Any Statistical Result Can Be Explained’’The theory’s relationship property has four key re-

quirements for ‘‘good’’ theory: Fecundity, Internal Con-sistency, Substantive versus Statistical Significance, and

Statistical Parsimony.Relationship’s Fecundity Property. The fecundity of a

‘‘good’’ theory is interpreted to mean a ‘‘new’’ theory mayexplain the current phenomena but also may offer newareas to research. A ‘‘new’’ theory that offers new areas toexplore is considered superior to existing theory thatexplains and explores fewer issues. Other things beingequal, if the new theory offers more new areas to explore,then it is preferred to the existing theory. However, as acaution it is important that the new theory explains thephenomenon as clearly as the existing theory. In short,



the fecundity property of relationships serves the purposeof gaining new, wider research areas to explore.

‘‘New’’ theories integrate existing theories. Thisintegration is important for theory development. Froma strict philosophy of science perspective, theory buildsupon previous theory (Bunge 1967; Hunt 1991 goingback to Plato). Theory building comes from integratingexisting theories into a larger framework and higher leveltheory. By building upon previous theory, ‘‘good’’ theoryintegrates more and more concepts into a larger theorythat can explain many events. This integration is animportant property for the advancement of science.

Relationship’s Internal Consistency Property. Theinternal consistency of ‘‘good’’ theory is interpreted tomean that the theory is logically consistent. The internalconsistency property is very difficult to demonstrate. At avery simplistic level, it is a graphical model showing therelationships between all variables. At the more advancedlevel, the researcher will illustrate that the theory givesvery specific mathematical relationships between allrelevant variables from extant business specific liter-ature. In some academic fields, symbolic logic gives thedeductive reasoning for the relationships but more typi-cally mathematics is the preferred method for internalconsistency (Moorthy 1993).

Although the internal consistency property seems likeit would be relatively easy to demonstrate, in practice it isquite difficult. Certainly, statistics cannot give theconceptually correct theory and its related model.Consider a simple model of just 10 variables. There are3,628,800 possible models. At the a55% level, there are(181,440) statistically significant models by mere chance.So only the abstract conceptual world can sort out whichones of these models really make logical sense. One ofthe oldest maxims in statistics and econometrics is ‘‘Anystatistical result can be explained.’’ Unfortunate as it mayseem, any one of those models could be explained withsome reasoning (convoluted or not). Without logicalintegration into the larger body of business knowledge,results become statistical artifacts that could have hap-pened by mere chance. Only through conceptual devel-opment can statistical estimates point to an integratedunderstanding among many related concepts. Regardlessof what is found during statistical modeling it can beexplained . . . many times with the tragic result of beingsubject to random chance.

For the last 20 or so years statistics have been a centraldriving force in many highly ranked academic journals.To some academics, statistical estimation is the only wayan article can be called a study, although this beliefignores the definition of a study (study is usuallydefined as the pursuit of knowledge, by reading, obser-vation, or research). The basic purpose of research is thecomplete understanding of phenomena. Sometimes thiscomplete understanding can be enhanced by under-standing conceptualization of the statistical results. Tonon-academics, statistics are the ‘‘real world’’ while the

abstract conceptual world is just an abstruse understand-ing that is open to any interpretation. This interpretationmay lead non-academics to believe that their effortsshould be highly focused on statistical methods andnot on conceptual understanding. Yet when thestatistical results are explained, the explanation providesan insight into the application of the results to specificorganizations and thus provides important implicationsfor pragmatic managers.

Relationship’s Parsimony: Explain More Theory andReport Less Statistics. It may seem obvious that thepurpose of any empirical article is to test and toexplain a theory. Yet when reading articles, it isapparent that a hefty portion of the page count is spenton explaining statistics rather than completely explainingthe model and how the empirical results of the theoryrelate to the overall business field. One result of this over-emphasis on statistics is the lack of truly integratedtheories. Empirical results tend to stand alone withoutintegration into a larger body of knowledge that is basedupon existing theory. From a scientific philosophy ofscience perspective, the empirical investigation shouldfocus on specific theory development and on the use ofspecific statistical techniques to test that theory.

Recently there have been many newly developed sta-tistical techniques to test theory. Sometimes researchersfrequently focus on these abstruse statistical techniques.The reasons may be: (1) It is easier to report statisticsthan it is to develop a full conceptual model before thedata are gathered; (2) sophisticated statistical analysescan easily disguise faulty conceptual analyses by report-ing specific advanced statistical techniques; and (3) manyacademics are better trained in using statistics than inunderstanding the underlying traditional statisticalmathematical difficulties and the relationship of thesedifficulties to explain the theory.

Many conceptually developed empirical articles inbusiness need to have sophisticated statistical tech-niques. Yet some articles overemphasize the importanceof the statistical findings. Because academic studies arerestricted in their length, the presentation and discussionof sophisticated statistical techniques are not withouttrade-offs. Unfortunately, sometimes what is lost is thecomplete examination and explanation of the theory thatis being tested. It would be difficult to imagine anyacademic wishing to read an article that did not carry acomplete explanation of how and why a theory isimportant to the pragmatic managerial world.

Academics should heed the words of the leadingstatistical scholars of the APA task force (Wilkinsonet al., 1999, p. 10) on reporting of statistical results:

The enormous variety of modern quantitativemethods leaves researchers with the nontrivial task

of matching analysis and design to the researchquestion. Although complex designs and state-of-the-art methods are sometimes necessary to address


July 2008 11

research questions effectively, simpler classicalapproaches often can provide elegant and sufficientanswers to important questions. Do not choose an

analytic method to impress your readers or todeflect criticism. If the assumptions and strength ofa simpler method are reasonable for your data andresearch problem, use it. Occam’s razor applies to

methods as well as to theories.

The APA task force indicates that scholars must explainthe results in the conceptual relationship of the a priorimodel rather than trying to impress the reader/reviewerwith non-meaningful statistics. Just because the statisticssoftware program output provides impressive statistics, itdoes not mean that authors should report them. Thisview was also supported historically by others (see Fisher1935 and others).

Statistical vs. Substantive Significance: Statistics Are Nota Substitute for Conceptualization. There is a disquietingtrend in empirical research that has not been adequatelyaddressed in the business literature and not practiced inmany empirical articles. That trend is the trade-off be-tween statistical significance and substantive significance.The statistics and the abstract theory should go hand-in-hand to understand the underlying causes of empiricalresults. Researchers should remember that statistics aremere facts and do not relate to theory without concep-tualization. It does not seem possible to make predic-tions without conceptualization.

Unfortunately, some articles seem to confuse theconceptual model with the statistical estimating model.This confusion is a major conceptual flaw in explanationbecause the statistical model does not integrate theory, itmerely tests it. This problem was not as severe 30–40years ago when econometric books such as Johnston(1972) clearly differentiated between the conceptualmodel and the statistical model. Currently, there is a re-emphasis on this problem of confusing the conceptualmodel with the statistical model. McCloskey and Ziliak(1996) have examined this difficulty in the economicsliterature. McCloskey and Ziliak examined the differencebetween the statistical model and the substantive model.They then differentiated statistical significance (confi-dence level) and substantive significance (conceptualsignificance). Their key point is that there is a largedifference between the conceptual model (developedfrom the theory) and the statistical model (developedto test the theory’s conceptual model). They warned thatthis confusion causes an over-emphasis on the statisticalresults and attributes less importance to the theory beingtested. The important conclusion is that there are manystatistical results that are not substantive results but aremerely statistics that could have occurred by merechance. During their survey of the economics literature,McCloskey and Ziliak found that 70% of the papers didnot differentiate between statistical significance and sub-stantive significance. Additionally, 53% used statistics for

including or excluding a variable that is a clear violationfor ‘‘good’’ theory building. In short, there is a criticaldifference between statistical significance and substantivesignificance. Statistics are not the determining factor if aconcept is theoretically important. The conclusion is thatonly the conceptually important variables should beincluded in any estimating procedure.

In short, there are many wrong variables in a statisticalestimate that are statistically significant but aremeaningless because they have occurred due to chance.Also more unfortunate is that there are many importantconceptual variables that are excluded due to statisticalinsignificance from the sample. Only a substantivelysignificant variable is important while statisticallysignificant variables may not be substantively significant.

The old econometrics maxim still applies today tomost empirical estimates: ‘‘if a variable is conceptuallyimportant enough to be included in the estimate, it isimportant enough to leave in the estimate.’’ In short, donot drop conceptually important variables just becausethey are statistically insignificant.

PredictionsThe prediction property of ‘‘good’’ theory is refutability

(falsifiability). Sir Karl Popper has pointed to this prop-

erty as being a critical aspect of ‘‘good’’ theory. Thisproperty states that the more unlikely a prediction is, thebetter the theory is. A theory that predicts a very likelyevent is not considered a ‘‘good’’ theory. Sometimes

when reading a conclusion of an empirical article, areader may be left with an empty feeling that there isabsolutely nothing in the article that is not obvious. If thereader asks the question: How likely are the article’s

conclusions to occur? Frequently, the answer would be:‘‘It would be revolutionary if the conclusion did notoccur.’’ The concern with obvious conclusions violatesthe falsifiability criterion for good theory. On the other

hand, if the conclusion is logical and offers new insightsinto conceptual relationships, then the reader is left withthe feeling that these findings are very important becausethe relationships were not obvious.

Frequently, academics during empirical investigationsfind a new statistical relationship between variables(concepts). These relationships frequently surprise theresearcher and are important for future studies. However,these relationships were not predicted and should not be

portrayed as being predicted. The explanation of any re-sult that was not predicted has a very special name in thephilosophy of science: it is ‘‘calling in the conventionalstratagem.’’ It is considered a major logic research flaw in

the philosophy of science. With that said, researchersshould attempt to explain unexpected results from theliterature. Explaining unexpected results is only a prob-lem when the researcher states that the results were pre-

dicted. Predictions must be predictions and not what was



Definition of ’’good’’ theory:A fully explained set of conceptual relationships used for empirical investigations.

Domain

Definitions

Relationships

Predictions

‘‘Good’’ Theory’sAdditionalProperties

Conservatism Parsimony Uniqueness

Generalizability Abstractness

Internal Consistency Fecundity Parsimony

Simplest Explanations

Falsifiability

Guidelines for ‘‘Good’’Theory Building

Generalizability: Abstraction: Specific times and places to test theory

Fecundity: Internal consistency: Relationship Parsimony: Statistical Parsimony: Substantive vs. Statistical Significance

Falsifiability:

All Theories’Properties

Conservatism: Definition Parsimony: Uniqueness:

FIGURE 3The Derivation of the Guidelines for ‘‘Good’’ Theory

TABLE I

A Summary of Guidelines for Evaluating All Empirical Theory Building

1. Is the proposed theory really a theory? Does the theory explicitly state all four properties of theory(definitions, domain, relationships, and predictions)?

2. If the proposed theory is actually a theory, how ‘‘good’’ of a theory is it? Where are its weaknesses:precision in definitions, specified domains, logically consistent relationships, falsifiable predictions?

3. How ‘‘good’’ are the definitions?a. Does the author(s) define all concepts or present definitions to clearly understand what they are

proposing?b. Does the author(s) eschew from renaming an existing concept without clearly differentiating why it

is necessary?c. Are the definitions concise (short as possible)?d. Do the definitions lead to a unique concept or are they ambiguous or vague?e. Do the definitions pass the guidelines for ‘‘good’’ definitions?

4. How carefully is the theory’s domain specified?a. Does the study use the literature to indicate where the previous studies’ domains were gathered:

(1) Countries, (2) Industries, (3) Respondent’s Position, and (4) Relevant Respondents’ Demographics?b. Does the study specify ‘‘when’’ and ‘‘where’’ the theory can be applied and tested?

5. For theory’s relationships’ empirical tests:a. Does the theory offer new areas to explore (fecundity)?b. Are the relationships graphically/mathematically examined and explained to be internally consistent?c. Are all the relationships from the extant literature included in the explanation?d. Does the author(s) use the simplest statistics possible to develop their tests and eschew from presenting

statistics that are not germane to the discussion suggestions (Wilkinson and et. al. 1999 task force)?e. Does the author(s) differentiate between substantive significance and statistical significance McCloskey

and Ziliak (1996) ?i. Do the authors drop variables due to statistical insignificance?ii. Do the authors include variables due to statistical significance rather than substantive significance?ii. Are the relationships conceptually explained versus statistically explained?

6. For predictions:a. Do the conclusions offer new insights and explanations for the theory?b. Are the conclusions predicted from the extant literature avoiding ‘‘calling in the conventional stratagem’’?


July 2008 13

found during empirical research. It is easy to forget thatwithout the a priori models all statistical results aremerely artifacts because they are subject to random as-

sociation.

Guidelines to Evaluate TheoryThe purpose of this section is to summarize guidelines

for evaluating articles as to their importance to theory(for this article a theory-building guideline is defined as

an authoritative, prescribed direction for conductingempirical research). Whether the reader is a beginningacademic student or the most recognized scholar in thefield, it is important to be extremely critical of all ac-ademic literature using logical reasoning (this article is

not exempt from this evaluation). The above discus-sions give an outline of a logical method for evaluatingthe contribution to the academic literature and toimprove the significance of the reader’s own studies.

Figure 3 gives the relationships between all theory andthe guidelines for ‘‘good’’ theory building for academicarticles.

It should be noted that regardless of the research

method, all empirical research should follow these guide-lines (for different research methods and data gatheringmethods see Meredith, Raturi, Amoako-Gyampah andKaplan 1989; Wacker 1998, pp. 378–380). The guide-

lines for good theory building are summarized inTable I. These guidelines follow the outline of this paper:Theory and Good Theory, Formal Conceptual Defini-tions, Theory Domain, Theory Relationships, and

Predictions. As such, it provides a framework for theorydevelopment.

CONCLUSIONThe general conclusion is that most academic articles

might relatively easily be refocused to develop ‘‘good’’

theory. The purpose of this article is not to criticize ex-isting business literature, but to offer suggestions onhow to improve the theory. The article’s purpose was toprovide a framework for improving theory-building

to raise theory building’s importance to the academicfield.

There are numerous areas of future study to illustratemany of the points addressed in this study—specifically,how theory’s conceptualization affects specific advanced

statistical methods used for theory testing. Most impor-tant would be the statistical demonstration of how defi-nitions affect concept measurement—or how and whysome samples are biased and why others are not. Also

helpful would be relationship estimation procedures tosuggest specific estimation procedures for addressingspecific academic issues. In short, this article waswritten not as a criticism of current or past research

but with the goal of providing helpful suggestions to

improve future theory-building research in supply chainmanagement.

BIBLIOGRAPHYBacharach, S.B. ‘‘Organizational Theories: Some Criteria

for Evaluation,’’ Academy of Management Review,(14:4), 1989, pp. 496-515.

Bollen, K.A. Structural Equations with Latent Variables, JohnWiley & Sons, New York, 1989.

Bunge, M. Scientific Research I: The Search for System,Springer-Verlag, New York, 1967.

Burgess, K., P.J. Singh and R. Koroglu. ‘‘Supply chainmanagement: A structured literature review andimplications for future research,’’ InternationalJournal of Operations and Production Management,(26:7), 2006, pp. 703-729.

Fisher, R.A. The Design of Experiments, Oliver & Boyd,Edinburgh, Scotland, 1935.

Hempel, C.G. ‘‘Methods of Concept Formationin Science.’’ In Otto Neurath, Rudolf Carnapand Charles Morris (Eds.), Formations of the Unityof Science, University of Chicago Press, Chicago,1970.

Hunt, S.D. Modern Marketing Theory: Critical Issues in thePhilosophy of Marketing Science, SouthwesternPublishing Co, Cincinnati, OH, 1991.

Johnston, J. Econometrics Methods, 2nd ed., New York,McGraw-Hill, 1972.

Kaplan, A. The Conduct of Inquiry: Methodology forBehavioral Sciences, Chandler Publishing Company,San Francisco,CA, 1964.

Klein, K.J., H. Tosi and A.A. Cannella Jr. ‘‘MultilevelTheory Building: Benefits, Barriers, and New Devel-opments,’’ The Academy of Management Review, (Apr24), 1999, p. 2.

McCloskey, D. and S. Ziliak. ‘‘The Standard Error ofRegression,’’ Journal of Economic Literature, (34:1),1996, pp. 97-114.

Meredith, J.R., A. Raturi, K. Amoako-Gyampah and B.Kaplan. ‘‘Alternative Research Paradigms in Opera-tions,’’ Journal of Operations Management, (8:4), 1989,pp. 297-326.

Mitchell, T.R and L.R. James. ‘‘Building Better Theory:Time and the Specification of when Things Happen.Academy of Management,’’ The Academy of Man-agement Review, (26:4), 2001, pp. 530-547.

Moorthy, K.S. ‘‘Theoretical Modeling in Marketing,’’Journal of Marketing, (57:April), 1993, pp. 92-106.

Osigweh, C.A.B. and A.B. Chimizie. ‘‘Concept Fallibilityin Organizational Science,’’ Academy of ManagementReview, (14:4), 1989, pp. 579-594.

Poole, M.S. and A.H. Van De Ven. ‘‘Using Paradox toBuild Management and Organizational Theories,’’Academy of Management Review, (14:4), 1989, pp.562-578.

Popper, Sir K.‘‘Philosophy of Science: A Personal Report.’’In British Philosophy In Mid-Century, C.A. Mace (Ed.),1957.



Shubik, M. ‘‘What Is an Application and When Is Theorya Waste of Time?,’’ Management Science,(33:December), 1987, pp. 1511-1522.

Swamidass, P.M. ‘‘Empirical Science: New Frontier inOperations Management,’’ Academy of ManagementReview, (18:4), 1991, pp. 793-814.

Teas, R.K. and K.M. Palan. ‘‘The Realms of ScientificMeaning Framework for Constructing TheoreticallyMeaningful Nominal Definitions of MarketingConcepts,’’ Journal of Marketing, (2:61), 1997, pp.52-68.

Van De Ven, A.H. ‘‘Nothing Is Quite So Practical as aGood Theory,’’ Academy of Management Review,(14:4), 1989, pp. 486-489.

Vogt, W.P. Dictionary of Statistics & Methodology, Sage,Thousand Oaks, CA, 1999.

Wacker, J.G. ‘‘A Definition of Theory: ResearchGuidelines for Different Theory-Building Re-search Methods in Operations Management,’’Journal of Operations Management, (16), 1998, pp.361-385.

Wacker, J.G. ‘‘A Theory of Formal Conceptual Definitions:Developing Theory-Building Measurement Instru-ments,’’ Journal of Operations Management, (22),2004, pp. 629-650.

Whetten, D.A. ‘‘What Constitutes a Theoretical Contri-bution?,’’ Academy of Management Review, (14:4),1989, pp. 490-495.

Wilkinson, L. et al. ‘‘Task Force on Statistical Inference,Statistical Methods in Psychology Journals,’’ AmericanPsychologist, (54:3), 1999, pp. 594-604.

John G. Wacker (Ph.D., Wayne State University,Detroit, MI) is a visiting professor of supply chainmanagement at Arizona State University in Tempe, Ari-zona. He has published 45 journal articles in Journal of

Operations Management, Decision Sciences, InternationalJournal of Production Research, Journal of Marketing Re-search, International Journal of Production Economics andnumerous other journals. His research has covered a

wide variety of topics such as the use of theory for sta-tistical methods, manufacturing implementation andforecasting. He has been on the Editorial Review Boardfor Journal of Operations Management for the last 20 years.He acted as President of the Global Manufacturing Re-

search Group (www.gmrg.org). He has taught in over 16international MBA and PhD programs for operations andsupply chain management including Chinese Universityof Hong Kong (1 year) and Troy State Europe program (1

year), University of Melbourne, MacQuarie University,Tatung Institute of Technology, and Hangzhou Instituteof Electrical Engineering.


July 2008 15

a conceptual understanding of requirements for theory-building research: guidelines for scientific...

Documents