crowdsourcing new product ideas over time: an analysis of the...

MANAGEMENT SCIENCEVol. 59, No. 1, January 2013, pp. 226–244ISSN 0025-1909 (print) � ISSN 1526-5501 (online) http://dx.doi.org/10.1287/mnsc.1120.1599

© 2013 INFORMS

Crowdsourcing New Product Ideas over Time:An Analysis of the Dell IdeaStorm Community

Barry L. BayusKenan-Flagler Business School, University of North Carolina, Chapel Hill, North Carolina 27599,

[email protected]

Several organizations have developed ongoing crowdsourcing communities that repeatedly collect ideas fornew products and services from a large, dispersed “crowd” of nonexperts (consumers) over time. Despite

its promises, little is known about the nature of an individual’s ideation efforts in such an online community.Studying Dell’s IdeaStorm community, serial ideators are found to be more likely than consumers with only oneidea to generate an idea the organization finds valuable enough to implement, but they are unlikely to repeattheir early success once their ideas are implemented. As ideators with past success attempt to again come upwith ideas that will excite the organization, they instead end up proposing ideas similar to their ideas that werealready implemented (i.e., they generate less diverse ideas). The negative effects of past success are somewhatmitigated for ideators with diverse commenting activity on others’ ideas. These findings highlight some of thechallenges in maintaining an ongoing supply of quality ideas from the crowd over time.

Key words : innovation; marketing; ideation; creativity; fixationHistory : Received June 24, 2011; accepted May 4, 2012, by Kamalini Ramdas, entrepreneurship and innovation.

Published online in Articles in Advance November 5, 2012.

1. IntroductionThe need for innovation is consistently a top businesspriority among CEOs (Andrew et al. 2010, Jaruzelskiand Dehoff 2010) and a key issue in academic research(Krishnan and Ulrich 2001, Hauser et al. 2006). Giventhe need for a continual stream of new products andservices, firms have traditionally relied on an inter-nal staff of professional inventors to generate ideas(Ernst et al. 2000, Schulze and Hoegl 2008). Despitethese investments in traditional innovation activities,however, firms continue to be disappointed with theirinnovation outcomes (Andrew et al. 2010, Jaruzelskiand Dehoff 2010).

Many organizations are now outsourcing theirideation efforts in an attempt to get fresh ideas intotheir innovation process. One approach that is receiv-ing substantial attention is “crowdsourcing,” a neol-ogism created by Wired magazine contributor JeffHowe (Howe 2008). As he defines it, crowdsourc-ing is the act of taking a task once performed byan employee and outsourcing it to a large, undefinedgroup of people external to the company in the formof an open call. Several organizations have imple-mented online crowdsourcing systems that gatherideas for new products and services from a large,dispersed “crowd” of nonexperts (e.g., consumers).Community members can typically propose newproduct ideas as well as comment on the ideas ofothers. These Web-enabled systems for ideas have

been called the new and improved Suggestion Box 2.0(Weiss 2006).

Some crowdsourcing applications take the form of aone-time contest or multistage tournament (Terwieschand Xu 2008, Terwiesch and Ulrich 2009). For exam-ple, consider the recent Betacup Challenge, whichreceived a lot of offline and online press attention.This contest was created to reduce the number of non-recyclable cups that are thrown away every year bycreating a more convenient alternative to the reusablecoffee cup (Bostwick 2010). More than 400 ideas weresubmitted by several hundred individuals from allover the world between April and June 2010 withthe winner receiving $10,000 (Elliott 2010). The LEDEmotionalize Your Light competition, on the otherhand, was a three month contest in 2009 with twostages (Bullinger et al. 2010). Sponsored by Osram (asubsidiary of Siemens), participants were invited topropose ideas for LED solutions with a wellness orwell-being focus. Almost 600 ideas were submittedduring the first phase (with three winners split-ting E5,000), of which 10 ideas moved into the sec-ond improvement phase (where three winners splitE2,000). The limited empirical research on innova-tion contests include descriptive case studies (e.g.,Bullinger et al. 2010) as well as studies of the relation-ship between individual and contest characteristicsand the number of ideators/solvers in a contest (Yanget al. 2009) or problem solving effectiveness (Jeppesenand Lakhani 2010, Boudreau et al. 2011).

226

Bayus: Crowdsourcing New Product Ideas over TimeManagement Science 59(1), pp. 226–244, © 2013 INFORMS 227

Other crowdsourcing applications, and the typeconsidered in this paper, involve individuals generat-ing ideas repeatedly over time. For example, Dell (com-puter hardware) and Starbucks (coffee) were recentlyin the headlines for their ongoing efforts in hav-ing a large consumer community suggest, discuss,and vote on thousands of new product and ser-vice ideas (Sullivan 2010). Unlike one-time challengeswhere ideators typically only submit one idea dur-ing a limited timeframe and a winner is selectedbased on the “best” submitted idea, participants inthese ongoing crowdsourcing communities are usu-ally asked to keep on proposing any big or smallideas that might improve the organization’s productsand services (Dell’s IdeaStorm has been collectingconsumer ideas since February 2007 and Starbucks’MyStarbucksIdea since March 2008). With the excep-tion of a few case studies (Howe 2008, Di Gangi andWasko 2009, Di Gangi et al. 2010), there is a dearthof published empirical studies involving this type ofcrowdsourcing community.

Companies are very interested in ongoing crowd-sourcing communities because consumers presum-ably have specialized knowledge about their ownproblems with existing products, and they are intrin-sically motivated to freely contribute their ideas (vonHippel 2005, Fuller 2010). Moreover, under the rightconditions, individuals can generate ideas that anorganization finds valuable enough to implement(Kavadias and Sommer 2009, Magnusson 2009, Poetzand Schreier 2012, Girotra et al. 2010). In additionto an almost limitless source of ideas, possible ben-efits from these ongoing communities include directcontact with customers as well as consumer inputinto the innovation process that is better, faster, andcheaper than traditional market research (Boutin 2006,Howe 2008). Both Dell and Starbucks report that theyhave already implemented a few hundred consumerideas submitted through their crowdsourcing com-munities. Despite its intriguing promises, however,very little is known about the nature of an individ-ual’s ideation efforts in a crowdsourcing communityover time. Understanding the key factors that drivethe repeated generation of ideas that an organizationwants to implement is necessary to fully appreciatethe potential of these crowdsourcing communitiesand thus, their effectiveness. Here, an ideator’s pastparticipation in the community is of interest, and akey question is whether ideators with past successin proposing ideas that are implemented continue togenerate the types of ideas an organization desires toimplement.

In this paper, two years of publicly available datafrom Dell’s IdeaStorm community are used to studythe nature of a crowdsourced idea generation pro-cess over time. Although the majority of ideators only

propose a single idea, very few of their ideas areimplemented (Lotka 1926). Instead, most of the imple-mented ideas are proposed by serial ideators (i.e.,individuals submitting ideas on at least two separateoccasions). Building on the established theory aroundcognitive fixation (Jansson and Smith 1991, Smith2003, Burroughs et al. 2008), an individual’s past suc-cess in generating implemented ideas is shown tohave negative effects on their subsequent likelihoodof proposing another idea the organization wants toimplement. As ideators with past success attempt torepeatedly come up with ideas that will excite theorganization, they instead end up proposing ideassimilar to their ideas that were already implemented(i.e., they generate less diverse ideas). Although thereare no sure-fire ways to overcome fixation effects, thebrainstorming literature suggests that context shiftingby interacting with diverse others can increase thequality of an individual’s output (Dugosh et al. 2000,Nijstad et al. 2002, Smith 2003). Following this line ofreasoning, the diversity of an individual’s past com-menting activity is found to have positive effects onan individual’s subsequent likelihood of generatinganother idea the organization finds valuable enoughto implement. Thus, the negative effects of past suc-cess are somewhat mitigated for ideators that com-ment on a diverse set of others’ ideas.

2. New Product Ideas from the CrowdBefore developing the theoretical framework forthis study, let us consider some of the ideas fromDell’s IdeaStorm crowdsourcing community, listedin Table 1. These ideas span a diverse range oftopics (categories) and typically include informationabout customer needs (problem information) as wellas ways of satisfying these needs (solution infor-mation). Because they are voluntarily offered, ideasfrom the crowd often show a low degree of elabo-ration and thus can sometimes be vague and imma-ture (Magnusson 2009, Di Gangi and Wasko 2009,Di Gangi et al. 2010). In addition, it should not besurprising if some of the proposed ideas are alreadyknown to the organization. For example, several con-sumer ideas in Table 1 are currently offered by Dell(e.g., Dell has offered gift cards for many years andhas had its own magazine Power Solutions since wellbefore 2005). Clearly, these ideas are not novel (as willbe discussed later, all ideas tagged as being alreadyoffered are dropped from the analysis in this study).

Several ideas from the crowd seem to be verycreative. Creative ideas are both novel (relativelynew compared to other available ideas) and poten-tially useful to an organization in the short orlong run (Amabile 1996, Shalley et al. 2004, George2007, Burroughs et al. 2008). From Table 1, “Have

Bayus: Crowdsourcing New Product Ideas over Time228 Management Science 59(1), pp. 226–244, © 2013 INFORMS

Table 1 Selected Ideas from IdeaStorm (Ideator; Category; Date Submitted/Date Implemented)

Already offeredDell Technology and Applications Magazine (tonyman262; advertising and marketing; February 20, 2007)Offer a student discount (dormlord52; education; July 9, 2007)Contribute to Ubuntu (darkproteus66; Linux; April 24, 2008)Wireless headphones for mp3 (cargwella; accessories; July 1, 2008)Gift vouchers (sh; advertising and marketing; December 9, 2008)Laptop cover (matchew; accessories; January 5, 2009)

Not implementedHave Michael Dell in the Dell commercials (carap; advertising and marketing; February 16, 2007)Have Firefox preinstalled as default browser (robinjfisher; software; February 18, 2007)Dell EV: Design and sell an electric car (dhart; new product ideas; February 21, 2007)Dell should sponsor American Idol (guardianxps; advertising and marketing; March 24, 2007)Biodegradable computers (reg; desktops and laptops; March 30, 2007)Dell—Offer the blank keyboard (jorge; accessories; June 2, 2007)Start offering Dell products to general public in Poland (lukasz_wisniewski; Dell; November 12, 2007)Add an automatic spell check to IdeaStorm (jervis961; IdeaStorm; February 10, 2008)Make it easier to clean the fans on laptops (pwl2706; laptops; July 10, 2008)Discount coupons for top IdeaStorm users (bbr; advertising and marketing; July 16, 2008)Can we get Studio hybrid with Ubuntu? (arhere; Linux; August 1, 2008)IdeaStorm Live!! (aikiwolfe; advertising and marketing; November 6, 2008)Advertise on www.Hulu.com (jervis961; advertising and marketing; November 30, 2008)Buy Lenovo (jervis961; Dell; January 9, 2009)

Partially implementedNo extra software option (ootleman; software; February 16, 2007/July 20, 2007)Preinstalled Linux; Ubuntu; Fedora; openSUSE; Multiboot (dhart; desktops and laptops; February 16, 2007/July 20, 2007)Multitouch screen (wkornewald; monitors and displays; March 8, 2007/July 17, 2008)Offer more configurations with 64-bit Windows Vista (hbruun; desktops and laptops; February 21, 2008/November 18, 2008)

ImplementedRugged laptop for Marine use (hawk473vt; laptops; February 17, 2007/March 9, 2009)Implemented: Ubuntu Dell is Le$$ than Windows Dell (thebittersea; Linux; May 5, 2007/May 24, 2007)Invest in miniprojectors (badblood; new product ideas; July 23, 2007/September 25, 2008)Vostro 1,500 with 7,200 RPM hard drive option (liraco; servers and storage; August 21, 2007/April 15, 2008)Dell community member awards (jervis961; Dell community; August 27, 2007/October 1, 2007)Post a video of your global mobility event (jervis961; advertising and marketing; August 8, 2008/August 21, 2008)You ask us questions (brokencrystal; IdeaStorm; August 19, 2008/January 5, 2010)Children’s PC (jotje; education; October 7, 2008/August 11, 2009)

Note. Idea categories include accessories (keyboards, etc.), Adamo, advertising and marketing, broadband and mobility, Dell, Dellcommunity, Dell website, desktops, desktops and laptops, digital nomads, dimension, education, enterprise, environment, gam-ing, IdeaStorm, Inspiron, laptop power, laptops, Latitude, Linux, monitors and displays, netbooks, new product ideas, operatingsystems, Optiplex, PartnerStorm, Precision workstations, printers and ink, retail, sales strategies, servers and storage, service andsupport, simplify and save, small business, software, Studio, Vostro, and XPS.

Michael Dell in the Dell commercials,” “Advertiseon www.Hulu.com,” and “Buy Lenovo” all havetheir own underlying logic—and all were not imple-mented by Dell! Thus, creativity alone is not enough.Consider, for example, that the vast majority ofreally creative ideas have no commercial value (Levitt1963, Silverberg and Verspagen 2007) and that manypatented ideas end up on one of the numerousweird and wacky patent websites (Czarnitzki et al.2011). Achieving the organization’s innovation goalsrequires that some ideas are actually valuable enoughto be implemented (Mumford and Gustafson 1988,West 2002, Franke et al. 2006). In the words of Levitt(1963, p. 79), “Ideas are useless unless used. Theproof of their value is in their implementation.” Con-sequently, this study focuses on quality ideas, i.e.,ideas that an organization does not already offer (new

ideas) and considers valuable enough to implement.It is worth noting that unlike the prior literature thatrelies on subjective rater assessments of idea quality(e.g., Magnusson 2009, Girotra et al. 2010, Kornishand Ulrich 2011, Poetz and Schreier 2012), the presentstudy considers ideas that were actually implementedby an organization (see also Strobe et al. 2010).

Although many people only link innovation withradical breakthroughs that may change a company’sexisting way of doing business, it is important torecognize that innovation also encompasses ideas forincremental improvements to existing products andservices (Mumford and Gustafson 1988, Shalley et al.2004, Robinson and Schroeder 2005, Vandenboschet al. 2006). In most cases, the implemented incremen-tal improvements are considered to be quick wins bythe organization (Dahl et al. 2011, Silverman 2011).


For example, the suggestion by jervis961 in Table 1to post a video from one of Dell’s global events canprobably be considered to be an incremental idea—implementing it made sense to Dell, it was not toocostly, and it probably increased goodwill amongsome customers. On the other hand, some of thecrowd generated ideas are highly valuable (Bjellandand Wood 2008, Jouret 2009, Killian 2009). For exam-ple, the idea by dhart listed in Table 1 (“PreinstalledLinux; Ubuntu; Fedora; openSUSE; Multiboot”) gen-erated thousands of votes and hundreds of commentswithin Dell’s community. Because of the tremendoussupport for offering computer systems with Ubuntu(an open source Linux desktop operating system),Dell quickly surveyed their customers about preferreddistribution options (more than 100,000 people com-pleted the survey). Working through the associatedproduction issues, Dell began selling three computersystems with Ubuntu 7.04 preinstalled a few monthslater (Menchaca 2007). This idea was tagged as par-tially implemented because the original idea called forthe preinstallation of a wide variety of open sourcesoftware that is not as yet offered.

A unique feature of IdeaStorm is that implementedideas are publicly identified (e.g., see Table 1). Almosthalf of all implemented ideas involve changes inthe styling, design, and hardware of Dell products.Another third of the implemented ideas deal withopen source software, the IdeaStorm community, andwebsite. The remainder of the implemented ideasconcern other suggestions involving the environ-ment, service and support, and retail operations. Notsurprisingly, implementation costs vary considerablyacross ideas. Unfortunately, there is no available infor-mation on the costs or organizational impact associ-ated with each implemented idea. But what is knownis that very few ideas, including ideas that mayseem like only small improvements, pass Dell’s inter-nal quality screening process (Dell reports on theirIdeaStorm website that less than 4% of all submittedideas have been fully or partially implemented). Thus,whether or not an idea is actually implemented is thekey success outcome considered in this study.

3. The Theoretical FrameworkIn this section, the theoretical framework that guidesthe empirical study is discussed. A common themein the new product literature is the importance ofnovel combinations and rearrangements of ideas,components, products, technologies, strategies, etc.(Simonton 2003, Fleming and Szigety 2006). Addition-ally, research in cognitive psychology supports thenotion that quality ideas result from new and orig-inal arrangements of elements from existing knowl-edge bases (Ward 1994, Dahl and Moreau 2002).

The larger and more diverse an individual’s domain-relevant knowledge base, the more alternative ideascan be obtained by combining, recycling, recombin-ing, and further developing these pieces of informa-tion (Amabile 1988, 1996; Hargadon and Sutton 1997;Fleming and Szigety 2006). To generate an idea theorganization finds valuable enough to implement, anindividual must access relevant information, oftenfrom diverse knowledge bases (Amabile 1988, 1996).Indeed, high-quality ideas are rare because of the dif-ficulty individuals have in accessing this information(Fleming and Szigety 2006). Research indicates thatthere are both positive and negative factors that caninfluence the retrieval of pertinent information duringthe idea generation process.

3.1. Negative Effects of Past SuccessThere is a large and growing literature in cogni-tive psychology and creativity taking the positionthat past experience is detrimental to future ideationefforts. In particular, experimental research finds thata pervasive impediment to accessing relevant anddiverse knowledge bases is cognitive fixation (Janssonand Smith 1991, Smith et al. 1993, Ward 1994, Smith2003, Cardoso and Badke-Schaub 2011)—people tendto fixate on the principles and features of prior exam-ples, leading to ideas that are less original (Smith et al.1993, Marsh et al. 1996). Although Dahl and Moreau(2002) find that individuals generate less novel ideaswhen examples are provided, they also find thatthese ideas are less valuable (consumers have rel-atively low willingness to pay for the suggesteddesigns). As defined by Smith (2003, p. 16), fixationis “something that blocks or impedes the success-ful completion of various types of cognitive opera-tions, such as those involved in remembering, solvingproblems, and generating creative ideas.” Also calledunconscious plagiarism (or cryptomnesia), individu-als are often unaware that they are fixating on thecharacteristics of past examples (Marsh and Landau1995, Marsh et al. 1999). Research into cognitive fixa-tion goes back to early experiments by Maier (1931),Duncker (1945), and Birch and Rabinowitz (1951) thatdemonstrate that individuals have great difficulty indeviating from previously successful problem solv-ing strategies even when a problem requires a newsolution approach. In other words, past experiencecan limit the knowledge and heuristics used in theideation process, leading to lower quality ideas.

Jansson and Smith (1991) were the first researchersto demonstrate fixation effects in the product designprocess. Individuals in their experiments were askedto come up with as many ideas as they could tosolve various design problems (e.g., measuring cupfor the blind, spill-proof coffee cup, medical moni-toring device). Subjects in the control group received


only a one-page problem statement, whereas subjectsin the fixation group also received a second pagewith an example diagram of a possible design for theproblem. They found that the design solutions gener-ated by the fixation group were much more likely toinclude features of the example designs than the con-trol group. Moreover, the fixation group also tendedto include flawed aspects of the example designsthat violated the problem statement. These generalfindings have been confirmed in many other situa-tions, including those involving novices and experts(Jansson and Smith 1991), externally provided exam-ples and examples generated by the individual them-selves (Ward 1994), exemplars in the form of picturesas well as detailed verbal descriptions (Purcell andGero 1992), and non-Western, nonindustrialized cul-tures with limited technology (German and Barrett2005). Fixation means that the entire solution space isnot completely explored (i.e., providing design exam-ples may restrict an individual from seeing otheralternatives or better solutions). In this case, individu-als do not sufficiently access their diverse knowledgebases, reducing their chances of coming up with anidea that an organization wants to implement. Thesestudies demonstrate that product designs tend to con-form to the provided examples. Although the pro-posed ideas are less original, they can also be lessuseful (especially because the proposed designs mayinclude undesirable features carried over from theexamples). Because these ideas are not novel (andsometimes not useful), they are less likely to be imple-mented by an organization.

Purcell and Gero (1992, 1996) extend the Janssonand Smith (1991) experiments by showing that fix-ation effects only occur with example designs thatinclude principles that are already familiar. Further-more, Perttula and Sipila (2007) find that fixationeffects are highest when the exemplars contain fea-tures that are commonly known. These studies sug-gest that cognitive fixation is related to an individual’sestablished knowledge base—people are more likelyto become fixated when prior examples are familiar.This is particularly relevant in a crowdsourcing set-ting where an organization publicly identifies onlya few ideas that are implemented. Here, an individ-ual’s own1 implemented ideas are clearly very famil-iar and thus are highly salient exemplars of the typesof ideas that the organization desires (Weiner 1985,

1 A similar line of reasoning can be used to hypothesize the effectsof others’ implemented ideas. However, unlike the case of ownimplemented ideas (in which Dell informs the original ideator),there are no good measures of whether an individual is even awareof others’ ideas after their implementation in the Dell IdeaStormdata used in this study (e.g., there are very few comments onan idea after it is implemented). This presents an opportunity forfuture research.

Lindsley et al. 1995). Similarly, the examples in thepublished experiments are highly salient to subjectsbecause very few are used.

Taken together, this discussion suggests that anindividual’s past success in proposing implementedideas is detrimental to their subsequent ideationefforts. This literature and research findings are sum-marized in the following hypothesis.

Hypothesis 1 (H1). An individual’s likelihood ofproposing an implemented idea is negatively related to theirpast success in generating implemented ideas.

Various explanations that have been suggested toaccount for the negative effects of cognitive fixationare reviewed by Marsh et al. (1996) and Perttula andLiikkanen (2006). One rationale that is generally con-sistent with reported experimental findings is a vari-ant of Ward’s (1994) structured imagination theory.According to this theory, people unconsciously sum-marize the general features of known exemplars bycreating a new mental category such as “ideas desiredby Dell.” Further examples (i.e., implemented ideas)help to define (or redefine) this new category. Thisseems to be consistent with the crowdsourcing experi-ences of the top contributor to Dell’s IdeaStorm web-site (Jervis 2010):

Users sometimes have ideas that would force Dell togo outside their comfort zone and go in a directionwhere Dell would take the lead 0 0 0 0 This mindset hascaused Dell to often only partially implement a user’sidea or just not get it right (Dell doesn’t really discussor clarify ideas with users in most cases). Sometimesthe result is that the user will narrow their focus in futureideas based on the part that Dell did adopt. In essence theybecome less innovative and fall more into line with the safeapproach Dell usually follows. [emphasis added]

Thus, previous implemented ideas may influencewhat individuals believe to be “acceptable” ideas.In agreement with this explanation, Dahl and Moreau(2002) find that prior examples structure the form ofsubsequent ideas (i.e., ideas tend to conform to thehigh-level aspects of prior examples). If this mecha-nism is at work, then individuals with past success ingenerating implemented ideas will propose ideas thatwill be related to their previous implemented ideas,i.e., their subsequent ideas will be less diverse. Thisdiscussion is summarized in the following hypothesis.

Hypothesis 2 (H2). An individual’s likelihood ofproposing diverse ideas (i.e., ideas that differ from their pre-vious implemented ideas) is negatively related to their pastsuccess in generating implemented ideas.

3.2. Positive Effects of Past CommentingActivity

Research generally recognizes that interaction andidea exchange among individuals can facilitate the


retrieval of relevant and diverse knowledge duringthe idea generation process (Hinsz et al. 1997, Kohnand Smith 2011). Here, interactions typically involveone-on-one or group discussions and commentingactivity. A fundamental belief in brainstorming is thatinteracting with diverse others can stimulate associ-ations in memory that lead to higher quality ideas(Osborn 1953). In traditional brainstorming studies,these interactions include face-to-face as well as com-puter mediated discussions. Several researchers con-firm that interactions with diverse others involvingthe sharing of information and ideas has a positiveeffect on ideation efforts (Amabile 1996, Perry-Smithand Shalley 2003). These general results have beenfound in offline and online settings (Nijstad andStroebe 2006). Interactions and communication withdiverse others helps individuals to generate variousalternatives, revise their own knowledge, and refinetheir ideas, making it more likely that an organizationwants to implement the proposed ideas (Perry-Smithand Shalley 2003, Perry-Smith 2006).

Although there are no sure-fire methods to over-come fixation effects, the literature suggests thatcontext shifting by making use of others’ perspec-tives and ideas can increase the quality of an indi-vidual’s output (Dugosh et al. 2000, Nijstad et al.2002, Smith 2003). Prior research shows that shar-ing diverse ideas can stimulate other ideas or cat-egories of ideas when individuals are motivated todirectly attend to the stimulus ideas (Dugosh et al.2000, Dugosh and Paulus 2005, Rietzschel et al. 2007,Kohn et al. 2011). In crowdsourcing communities,ideas are shared online among members via reading,voting, and commenting. Because detailed informa-tion is available,2 this study focuses on an individual’scommenting activity. Although their motivations toparticipate in crowdsourcing communities vary, indi-viduals that actively interact by commenting on oth-ers’ ideas generally perceive more benefits, tend tofeel a greater sense of community membership, andtake their contributions seriously (Preece et al. 2004).Thus, it seems reasonable to assume that commentersfirst attend to the focal ideas of their comments byreading them. Interacting with others via online com-ments also has been shown to promote active andcritical thinking (Garrison et al. 2001, Schellens andValcke 2005). Ideators that comment on a diverseset of other members’ ideas should develop a betterunderstanding of consumer needs, leading to ideasthat are more likely to be valuable to the organizationand thus have greater chances of being implemented.

At the same time, arguments from the previous sec-tion might suggest that cognitive fixation on others’

2 Unfortunately, within the IdeaStorm community studied in thispaper there are no complete data on the ideators that only readideas, nor on voter IDs or voting dates.

ideas can impede ideation efforts. However, unlike atraditional brainstorming situation in which a smallgroup generates relatively few ideas in a fixed timeperiod, in an ongoing crowdsourcing communitythousands of ideas are proposed (few of which aredeemed by the organization to be valuable enoughto be implemented) over time (typically there areseveral days between idea generation and comment-ing activity). As compared to an ideator’s own ideasthat are actually implemented, the large number ofothers’ ideas in a crowdsourcing community is notexpected to be as salient. Moreover, fixation is gener-ally reduced when there is time between idea genera-tion sessions (i.e., because of incubation effects, Smith2003, Linsey et al. 2010). Finally, Dugosh et al. (2000)find that the ideation efforts of individuals separatelyexposed to others’ ideas (via a tape recording) washigher than interacting groups (which discussed eachother’s ideas)—suggesting that fixation is less impor-tant when individuals are asynchronously exposed toothers’ ideas. Thus, idea exchange via commenting ina crowdsourcing community is not likely to be influ-enced by fixation effects.

Taken together, this discussion suggests that therewill be a positive relationship between the diversityof an individual’s commenting activity and their sub-sequent chances of proposing an idea that the orga-nization finds valuable enough to implement. Thefollowing hypothesis summarizes these arguments.

Hypothesis 3 (H3). An individual’s likelihood ofproposing an implemented idea is positively related to thediversity of their past comments on others’ ideas.

Flexibility (or breadth) is traditionally consideredto be an important dimension of creativity (Torrance1966, Guilford 1967); e.g., breadth of attention hasbeen found to be beneficial for innovation outcomes(Friedman et al. 2003). Moreover, Nijstad et al. (2002)find that individuals generate more diverse ideas afterattending to stimulus ideas from a diverse set of cat-egories. Together, these studies suggest that individ-uals commenting on a wide set of others’ ideas willtend to generate more diverse ideas that will differfrom their previously implemented ideas. Thus, thefollowing hypothesis is offered.

Hypothesis 4 (H4). An individual’s likelihood ofproposing diverse ideas (i.e., ideas that differ from their pre-vious implemented ideas) is positively related to the diver-sity of their past comments on others’ ideas.

4. DataData for this study come from the publicly availableinformation on Dell’s IdeaStorm website. Accordingto their website, “The goal is for you, the customer, totell Dell what new products and services you’d like


to see Dell develop” (http://www.ideastorm.com/,accessed April 1, 2009). IdeaStorm has won a num-ber of awards, including the 2008 PR Innovation ofthe Year and the 2008 Award for Collaboration andCo-Creation by the Society for New CommunicationsResearch. IdeaStorm is lauded as being an excellent,best-in-class crowdsourcing application (Howe 2008,Sullivan 2010).

To participate, individuals must join the IdeaStormcommunity (at no cost) by selecting an anonymoususername (you do not have to be a Dell customerto join). Like most crowdsourcing applications, infor-mation on demographics and personal characteris-tics are not collected (the IdeaStorm community is a“large, undefined” crowd). Ideastorm members canpropose ideas as well as comment and vote on theideas of others. Anyone submitting an idea agrees togive Dell a royalty-free license to use the idea with norestrictions.

In addition to titling their idea and giving a descrip-tion, consumers can tag their idea based on 39 differ-ent categories (e.g., new product ideas, laptops, salesstrategies). Each idea can be classified in up to threecategories; typically, however, ideas are assigned asingle category. Similar to prior research (Ward 1994,Audia and Goncalo 2007), these idea categories arean objective measure of the extent to which an ideadiffers from an individual’s previous ideas. A list ofthese categories is in Table 1.

The IdeaStorm review team reads each idea within48 hours of posting to ensure that it meets Dell’s termsof use. This team has educated Dell executives andemployees how to best utilize IdeaStorm to find andact on ideas, and how to communicate the status ofany ideas they use so this can be reported on the Idea-Storm website. Answering a query by a user abouthow decisions on ideas are made, Dell’s IdeaStormmanager states that “the formal governance processconsists of a senior-level idea review team that reviewsthe top ideas on IdeaStorm and works them throughthe right departments for further study or implemen-tation” (Dell_Admin1 2007). This manager goes on tonote that there are two additional informal governanceprocesses: “One is a top-down process: a weekly sum-mary of the top ideas and major discussion threadsis sent to our executive leadership team, who discussand monitor the status of those ideas. Another is abottom-up process: employees in a number of depart-ments review ideas that relate to their area of exper-tise.” Most of Dell’s departments have identified acategory expert who serves as a business champion forIdeaStorm. This champion helps vet ideas and com-municates with the IdeaStorm review team regardingthe status of ideas in their department. Additionally,community members are vigilant in monitoring theideas that Dell eventually adopts, informing the Idea-

Storm team when one of their ideas (or even the ideasof other members) is implemented. To cut down onduplicate ideas (Kornish and Ulrich 2011), contribu-tors are encouraged to first search the IdeaStorm sitefor similar ideas. In addition, the IdeaStorm team andcommunity members patrol the website for duplicateideas (which are then merged together). Thus, thereis every reason to believe that Dell considers IdeaSt-orm to be an integral part of the way they do business(Killian 2009).

IdeaStorm was officially launched in February 2007.Information on all of the ideas proposed between itsinception and June 2009 was collected. To allow a timebuffer for community activity around an idea to sta-bilize, ideas generated during the last four monthswere dropped from the analysis. Over the two-yearperiod from February 2007 to February 2009, theavailable data include 4,285 individuals who gener-ated a total of 8,801 ideas.3 During the same period,348 ideas (or about 4% of all ideas proposed) wereimplemented. Individuals that have an idea acceptedby Dell (about 5% of all ideators) receive a pen in anengraved box (Sullivan 2010); no monetary compen-sation is awarded.4 Implemented ideas are also pub-licly discussed by Dell administrators in their “Ideasin Action” blog, and are appropriately tagged on theIdeaStorm website. Overall, Dell seems to be happywith the general quality of ideas (Killian 2009).

Descriptive information on the IdeaStorm popula-tion is in Figure 1. Almost 85% (Figure 1: 3,589/4,285)of all ideators only offered an idea on a single occa-sion (note that an individual can submit more thanone idea during the same day). Consistent with H1,Dell implemented 11.9% (Figure 1: 110/923) of theideas from serial ideators with one or two imple-mented ideas versus only 7.0% (Figure 1: 95/1,356)of the ideas from serial ideators with more than twoimplemented ideas (z = 4002; p < 0000). Furthermore,in line with H2, about 23% of the ideas proposedby serial ideators with more than two implementedideas were in new categories (as compared to 67% ofthe ideas from serial ideators with one or two imple-mented ideas and 83% of ideas from serial ideatorswith no implemented ideas).

In agreement with the literature (Simonton 2003,2004), the chances an individual generates an imple-mented idea is directly related to their ideation efforts.As indicated in Figure 2, most of the implemented

3 Ninety-four ideas (around 1% of all submitted ideas) alreadyoffered by Dell (including 42 people) were dropped from the anal-ysis because they are not novel.4 According to their terms of service, Dell can acquire any intel-lectual property rights associated with the materials submitted toIdeaStorm for $1,000. However, there is no evidence that Dell hasever exercised this option.


Figure 1 IdeaStorm Population (February 2007–February 2009)

Ideators4,285 individuals

8,801 ideas348 implemented ideas

One-time ideators3,589 individuals


Serial ideators696 individuals


0 ideas implemented3,477 individuals

3,908 ideas

≥ 1 ideas implemented142 individuals

179 ideas143 implemented ideas

0 ideas implemented591 individuals

2,435 ideas

1 or 2 ideas implemented93 individuals

923 ideas110 implemented ideas

>2 ideas implemented12 individuals1,356 ideas

95 implemented ideas

ideas are proposed by serial ideators—only 4% of theone-time ideators in IdeaStorm had their idea imple-mented, and over 15% of serial ideators (who hadan average of over six ideas per individual) had atleast one idea eventually implemented (z = 11046; p <

Figure 2 Ideator Productivity

Ideators4,285 individuals

5.8% had animplemented idea26.9% commented

on ≥ 1 ideas

One-time ideators3,589 individuals

4.0% had animplemented idea22.0% commented

on ≥ 1 ideas

Serial ideators696 individuals15.1% had an

implemented idea47.4% commented

on ≥ 1 ideas

0 multiple submission days3,253 individuals

3.6% had an implemented idea20.7% commented on ≥ 1 ideas

≥1 multiple submission days336 individuals


0 multiple submission days410 individuals


≥ 1 multiple submission days286 individuals


0000). Importantly, serial ideators do not seem to bemotivated by an early success (e.g., only about 6% ofserial ideators had their first or second idea imple-mented, and less than half of these individuals pro-posed a subsequent idea after one of their ideas was


Figure 3 Number of Ideas over Time in IdeaStorm

0

20

40

60

80

100

120

140

Feb-07 May-07 Aug-07 Nov-07 Feb-08 May-08 Aug-08 Nov-08 Feb-09

Num

ber

Ideas (x10)

Implemented ideas

implemented). Ideators that submit more than oneidea in the same day also are significantly more likelyto generate an idea that Dell finds valuable enoughto implement (Figure 2). As suggested by Figure 2,ideation efforts and community participation seem togo hand in hand—serial ideators and ideators withmultiple submission days are also more likely to com-ment on others’ ideas. Furthermore, active communityparticipation via commenting seems to be associatedwith serial ideators rather than ideators with an imple-mented idea (54% of serial ideators with an imple-mented idea, and 47% of serial ideators with noimplemented ideas, commented on at least one idea,whereas only 27% of one-time ideators with an imple-mented idea, and 22% of one-time ideators with noimplemented ideas, commented on at least one idea).

Monthly counts of proposed ideas are shown inFigure 3. During the first few weeks after the websitewas unveiled, ideas poured into IdeaStorm. From a

Figure 4 Number of Ideators over Time in IdeaStorm

0

200

400

600

800

1,000

1,200


Num

ber

Ideators

New ideators

peak of almost 1,200 ideas during the first month, the

number of suggested ideas rapidly declined over thenext six months, eventually steadying at a constantlevel. As shown in Figure 4, the number of individ-uals proposing ideas exhibits a similar exponentiallydeclining time pattern. Figure 4 also suggests that alarge fraction of the ideas in any month are generatedby new members. The reason for this is that most peo-ple offer just one idea (although one person suggestedmore than 250 ideas).

Monthly counts of proposed ideas that were imple-mented are also shown in Figure 3. Not surprisingly,the number of implemented ideas is closely related tothe total number of submitted ideas. As a result, themonthly proportion of implemented ideas is relativelyconstant over time. Although few consumers proposemore than one idea that is implemented (althoughone person did suggest more than 25 ideas that wereeventually implemented), the majority of individu-als (almost 95%) never propose an idea that Dell


Figure 5 Cumulative Number of Ideas over Time in IdeaStorm

20

40

60

80

100

120

140

160

180

200


Cum

ulat

ive

num

ber Serial ideators

Ideators with only one idea

found valuable enough to implement during this two-year period. Over one-third of the implemented ideasfrom one-time ideators were submitted in the firstmonth after IdeaStorm was initiated (less than 10%of the implemented ideas from serial ideators wereproposed in the initial month).

Finally, Figure 5 confirms that the majority ofimplemented ideas come from serial ideators. Dur-ing the first few months after IdeaStorm began oper-ations, many ideators offered a single idea that wasimplemented by Dell. But after six months, the num-ber of implemented ideas suggested by serial ideatorswas double those by one-time ideators.

5. The Empirical StudyIn this section, the hypotheses developed earlier areformally examined. To do this, an unbalanced paneldata set was constructed based on two years of daily5

IdeaStorm data at the individual level (February2007–February 2009). Here, observations are onlyincluded in the estimation sample if an individual iproposed an idea not already offered by Dell in day t.Definitions for all the variables are in Table 2.

5.1. Measures

5.1.1. Dependent Variables. Because H1 and H3concern an individual’s likelihood of proposing animplemented idea, an appropriate dependent mea-sure to test these hypotheses is yit , a binary6 variable

5 Prior studies also have aggregated Internet activity data to thedaily level (Park and Fader 2004).6 In less than 0.2% of all observations, an individual proposed morethan one implemented idea in the same day (eight observationshad two and two observations had three implemented ideas). Forthe IdeaStorm data, count model estimation results parallel thoseof the logit model to be discussed.

where a value of one indicates that individual iproposed an idea in day t that was eventually imple-mented (otherwise zero), i.e., yit = 1 if individual iproposed an implemented idea in day t. H2 andH4 involve an individual’s likelihood of proposingdiverse ideas (ideas that differ from their previousimplemented ideas). In this case, the dependent vari-able analyzed is a count (that takes on positive inte-ger values) yit , defined to be the number of ideasproposed by individual i in day t that are not inthe same categories as their prior implemented ideas.Importantly, both of these dependent measures are(1) publicly available from the IdeaStorm website,(2) objective (they do not rely on subjective assess-ments), and (3) consistent with the related literature(Ward 1994, Dugosh et al. 2000, Nijstad et al. 2002,Smith 2003, Audia and Goncalo 2007).

5.1.2. Past Success. The measure of past successin this research is based on the cumulative number ofimplemented ideas proposed by individual i beforeday t. In practice, however, it can take some timeafter an idea was initially submitted to decide if itmerits implementation. Updates posted as comments(which are dated) are used to determine the specificdate when an idea was implemented. For example,on May 29, 2008, jervis961 proposed an idea titled“Don’t put the Dell logo upside down on the MiniInspiron” (it seems that the earlier logo on the coverlooked upside down when viewed by someone usingthe laptop). However, it was not until September 4,2008, that vida_k (the new Dell IdeaStorm managerat the time) posted a comment “Changed status to∗∗IMPLEMENTED∗∗” under this idea. In this case, pastsuccess in generating implemented ideas (defined as thecumulative number of implemented ideas proposedby individual i that were known to be implementedbefore day t) is constructed based on implementa-tion dates, not submission dates. As the previously


Table 2 Variable Definitions and Summary Statistics for Serial Ideators

Variable Definition � � Min Max

Implemented ideas =1 if individual i proposed an idea in day t that was eventually implemented 0006 0023 0 1(0 otherwise)

Number of proposed ideas not in Number of ideas proposed by individual i in day t that are not in the same 1073 1010 0 10already implemented categories categories as their prior implemented ideas

Past success in generating Cumulative number of implemented ideas proposed by individual i that were 0074 2043 0 27implemented ideas known to be implemented before day t

Diversity of past commenting activity −∑

j pj ln4pj 5, where pj is the proportion of others’ ideas in category j 1067 1011 0 3004individual i commented on before day t

Age Time in the IdeaStorm community of individual i at day t (days) 123043 167098 0 739Diversity of past ideas −

∑

j pj ln4pj 5, where pj is the proportion of (own) ideas in category j 1064 0084 0 2095individual i proposed before day t

Past experience in generating ideas Cumulative number of ideas generated by individual i before day t 28058 51037 1 252Ideator with ≤ 2 implemented ideas =1 if individual i had a total of ≤ 2 implemented ideas; 0 otherwise 0075 0044 0 1Ideator with > 2 implemented ideas =1 if individual i had a total of > 2 implemented ideas; 0 otherwise 0025 0044 0 1

Notes. N = 31498. Includes 696 serial ideators; see Figures 1 and 2.

proposed ideas are filtered, there is a slight upwardtrend over time in the number of reported ideas thatwere implemented—the mean time between submis-sion and reported implementation is just over sixmonths.

Because the measure of success in this study isdetermined by the company (i.e., ideas they imple-ment), an alternative explanation for H1 is that Dell“spreads the love around” by systematically ensur-ing that any individual does not have multiple imple-mented ideas, no matter the quality of the proposedideas. Although this possibility cannot be ruled outwith the available data, there are two main reasonsagainst this argument. First, it is clearly not in Dell’sbest interest to do this because of the implementationand opportunity costs involved. Second, there is noevidence that Dell does this, especially because Idea-Storm is an important input into their new productdevelopment process.7

5.1.3. Past Commenting Activity. Idea exchangewithin the IdeaStorm community is measured basedon an individual’s previous activity in commentingon the ideas of other community members. Like ideas,comments are tracked by commenter and date. Acrossall the categories in which they commented on oth-ers’ ideas, almost 75% of the time ideators made acomment in a category before submitting their firstidea in that category. This is consistent with the notionthat commenting activity generally precedes ideationefforts across categories. To construct diversity of pastcommenting activity (defined using an entropy measureover the idea categories: −

∑

j pj ln4pj5, where pj is theproportion of others’ ideas in category j individual i

7 For example, there is no evidence that the ideas of IdeaStorm topcontributors (i.e., individuals listed on the IdeaStorm leaderboard)are systematically discounted (e.g., over 6% of the top 20 contribu-tors’ ideas were implemented versus only about 3% of the ideas ofeveryone else; z= 5053, p < 0000).

commented on before day t; Harrison and Klein 2007),comment dates were compared with dates of submit-ted ideas.

5.1.4. Control Variables. Several variables to con-trol for possible effects due to other individual or sit-uational factors are also included in the analysis (seeTable 2). To control for the possibility that successis simply because of greater cumulative productivity(Simonton 2003, 2004), prior experience in generatingideas (defined as the cumulative number of ideas gen-erated by individual i before day t) is included inthe analysis. An individual’s age (defined as an indi-vidual’s time in the IdeaStorm community) is usedto control for any effects of aging on output. To con-trol for the possible effects of the diversity of anideator’s own ideas, diversity of past ideas (definedusing an entropy measure over an individual’s ownideas) is included in the analysis. Category dummiesare included to control for any inherent differences inthe propensity for implemented ideas across topics,and monthly time dummies are used to account forany other unobserved time-varying effects.

Seminal work in creativity argues that an individ-ual’s productivity in generating quality ideas is highlyrelated to their total output (Simonton 2003, 2004).Thus, the absolute number of ideas generated by anindividual in a day should be positively related totheir likelihood of proposing an implemented ideain that day. The standard approach of including thenumber of ideas generated by individual i in day tas an exposure constraint8 is followed in this study.Across all consumers in the IdeaStorm data set, 495(over 10%) had two or more ideas in a single day (oneperson proposed 20 ideas in the same day).9

8 Including this constraint (the coefficient of ln(Number of ideas)= 1)controls for possible differences across ideators in sheer quantity ofideas proposed in a single day (Cameron and Trivedi 2009).9 The submission date is assumed to be the same as the date whenthe idea was conceived. Although it cannot be completely ruled


5.2. Estimation ApproachPanel logit models are used to estimate the effects ofthe explanatory variables when the dependent vari-able is binary and panel Poisson10 models are usedwhen the dependent variable is a count (Cameronand Trivedi 2009). The individual-effects logit modelis Pr4yit = 15 = å4�i + �xit + �zi5, where yit is thedependent variable (that takes on the values of 0 or 1)and å4w5 = ew/41 + ew5; the individual-effects Pois-son model assumes that yit (that takes on values ofpositive integers) is Poisson distributed with meanå4�5= e� . Here, xit are all the variables that vary overindividuals and time, zi are the variables that describethe individuals but do not vary over time, and �i cap-tures all the unobserved differences between ideatorsthat are stable over time and not otherwise accountedfor by zi.

Based on a Hausman type test (see Allison 2005),fixed-effects models are preferred over random effectsmodels for the IdeaStorm data. It is worth notingthat the conclusions to be discussed are generallyrobust to random effects models. Because of the largenumber of panels (individuals), estimation of thefixed-effects models is accomplished using a condi-tional maximum likelihood estimator where all time-invariant individual effects �i are conditioned out ofthe model using an individual’s total count (Cameronand Trivedi 2009). Importantly, a fixed-effects modelremoves any unobserved, time-invariant heterogene-ity across ideators. Although this approach allows theindividual specific effects to be correlated with theindependent variables (and thus is less likely to bebiased), a conditional fixed-effects model has someimportant caveats. In particular, ideators that haveyit = 0 (or 1 for a logit model) for all t are elimi-nated from the estimation sample because of the con-ditioning, and time-invariant variables zi (includingthe constant) cannot be estimated. In other words,one-time ideators are not included in the estimationsample; instead, serial ideators are used to estimatethe Poisson models and serial ideators with one ormore implemented ideas are used to estimate the logitmodels (see Figure 1). It is worth noting that the rel-atively large number of zeroes (i.e., 85% of the ideasfrom serial ideators are not implemented) in the datadoes not lead to a sampling bias in favor of the pro-posed hypotheses. Instead, the large number of zeroobservations will make it more difficult to find sta-tistical support for the hypotheses. More important,

out, there is no evidence that individuals actually generate manyideas over the course of several days and then submit them inbatches during one day. Active IdeaStorm members log in to thecommunity every day and use automatic activity updates throughtheir IdeaStorm dashboard, RSS, Twitter, and Facebook feeds.10 Panel Poisson models generally provide a more stable fit to theIdeaStorm data than panel negative binomial models.

Cramer et al. (1999) report that parameter estimatesare quite robust to relatively large imbalances in sam-ples sizes involving zero observations.

5.3. ResultsDescriptive statistics for all the variables are inTable 2. Because all of the explanatory variables (withthe exception of the diversity measures and dummyvariables) are highly skewed, their log transforms areused in the estimations.

Estimation results are in Tables 3 and 4. These mod-els provide very good fits to the IdeaStorm data asindicated by the significant Wald chi-square statis-tics. In general, the estimated coefficients for the timedummies do not reveal any significant pattern.11 Theresults for the category dummies across models sug-gest that ideas related to the Dell community, web-site, IdeaStorm, retail operations, printers, Precisionworkstations and Vostro computers have significantlyhigher chances of being implemented. Although thedetails are not shown here, there is no evidence thatideators systematically propose more ideas in thesecategories over time.

As shown in Table 3 (Model 1), the negativeand significant estimated coefficient for past successin generating implemented ideas strongly supports H1whereas the positive and significant estimated coeffi-cient for diversity of past commenting activity is in linewith H3. Model 2 demonstrates that these results arerobust to an alternative, stricter definition of imple-mented ideas. Here, ideas that were implementedvery quickly (possibly because the proposed ideawas already being considered by Dell or was anerror) or very slowly (possibly because the proposedidea involved technology that Dell was already wait-ing for) are not considered. For example, the bit-ter sea proposed an idea about pricing differencesbetween systems that was implemented within onemonth because, as it turns out, there was an erroron the website (see Table 1). An idea proposed byhawk473vt in February 2007 that Dell should developa rugged laptop for marine use was tagged as beingimplemented more than two years later (see Table 1).In this case, however, Dell had already signaled theirintentions of developing a rugged product line byintroducing a semi-rugged laptop (ATG D620) inJanuary 2007, which was eventually followed by theirXFR product line (that exceeds military standards) in2008 and 2009. Trimmed implemented ideas, there-fore, only include implemented ideas with implemen-tation times of between 30 days and one year.

11 Only information after February 2007 (see Figure 3) was also con-sidered in a separate analysis not reported here. Not surprisingly,very similar conclusions to those that will be discussed in this sec-tion are obtained.


Table 3 Panel Logistic Fixed-Effects Regression Results for Implemented Ideas

Model 1 Model 2 Model 3Variables (Trimmed)

Explanatory variablesln(Past success in generating implemented ideas) −1008∗∗ (0.36) −1036∗∗ (0.51)

× Ideator with ≤2 implemented ideas −4081∗∗ (1.62)× Ideator with >2 implemented ideas −1000∗∗ (0.37)

Diversity of past commenting activity 0069∗ (0.30) 0085∗ (0.40)× Ideator with≤2 implemented ideas 0066∗ (0.31)× Ideator with >2 implemented ideas −0009 (0.45)

Controlsln(Age) −0049∗∗ (0.15) −0034 (0.19) −0047∗∗ (0.15)Diversity of past ideas −0053 (0.44) −0072 (0.58) −0028 (0.45)ln(Past experience in generating ideas) 0042 (0.35) 0058 (0.45) 0047 (0.35)Time dummies Included Included IncludedCategory dummies Included Included Included

Log-likelihood −289040 −177006 −284036�2 (df) 152068∗∗ (61) 103059∗∗ (61) 162076∗∗ (63)N 1,539a 1,354b 1,539a

Note. Standard errors are in parentheses.aIncludes 102 serial ideators with >0 implemented ideas and >0 ideas not implemented; see Figures 1 and 2.bIncludes 65 serial ideators with >0 implemented ideas and >0 ideas not implemented; see Figures 1 and 2.∗Significant at the 0.05 level (two-tailed); ∗∗significant at the 0.01 level (two-tailed).

Additional robustness analyses are also shownin Table 3 (Model 3). Here, estimation results forsubsamples of ideators with few and many imple-mented ideas are reported. To do this, past successin generating implemented ideas and diversity of pastcommenting activity are multiplied by dummy vari-ables for whether the individual had ≤ 2 or > 2implemented ideas (see Table 2). Strong support forH1 is again found, indicating that the detrimentaleffects of past success are not only observed for serialideators with many implemented ideas but also forthose with relatively few implemented ideas. Theseresults suggest that the negative effects of past successare not simply due to the fact that many ideas arenot implemented. Interestingly, strong support forH3 only comes from serial ideators with few imple-mented ideas. The reason for this seems to be thatIdeaStorm members with a lot of past success alreadyhave a relatively high diversity of commenting activ-ity and thus there is little variance on this measurefor them (the diversity measure of ideators with > 2implemented ideas is almost twice as large as thatof ideators with < 2 implemented ideas; t = 26027,p < 0000). This is consistent with the significantlylower estimated coefficient for the effects of past suc-cess for serial ideators with many implemented ideas(Wald �2 = 5069; p < 0002), i.e., the negative effectsof past success are somewhat mitigated for serialideators who already are engaged in diverse com-menting activity.

As shown in Table 4 (Model 1), the negative andsignificant estimated coefficient for past success in gen-erating implemented ideas strongly supports H2. Thisresult in consistent with Ward’s (1994) structured

imagination theory that ideators in an ongoing crowd-sourcing community tend to fixate on their past suc-cess by generating less diverse ideas that are similarto their previous implemented ideas. Model 2 demon-strates that these results are robust to an alternativedependent measure involving ideas in any new cate-gory for the individual (defined to be the number ofideas proposed by individual i in day t that are innew categories for them, i.e., this is their first idea inthat category).

There is no support for H4 because of the insignifi-cant coefficient estimates for diversity of past comment-ing activity in Table 4 (Models 1 and 2). This resultsuggests that serial ideators with diverse comment-ing activity do not in turn propose more diverseidea themselves. A possible reason for this findingcomes from a study of “deep exploration” duringbrainstorming by Rietzschel et al. (2007). Rather thanincreasing the breadth (i.e., diversity) of idea genera-tion, past commenting activity may instead be relatedto increased depth of idea generation in relativelyfew categories and thus should lead to productiv-ity gains in the number of ideas generated (see alsoDugosh et al. 2000, Nijstad et al. 2002). This possibil-ity is explored by estimating the relationship betweenthe explanatory variables and a dependent variableinvolving the number of additional ideas proposedby individual i in day t. As shown in Table 4(Model 3), the coefficient estimate for diversity of com-menting activity is positive and significant. Thus, thereis strong evidence that serial ideators with diversecommenting activity propose more ideas, some ofwhich are eventually implemented by Dell.


Table 4 Panel Poisson Fixed-Effects Regression Results for Diverse Ideas

Model 1 Model 2 Model 3Variables (Implemented ideas) (Ideas) (Additional ideas)

Explanatory variablesln(Past success in generating implemented ideas) −0034∗∗ (0.06) −0060∗∗ (0.14) 0009 (0.13)Diversity of past commenting activity −0003 (0.03) −0001 (0.04) 0029∗∗ (0.10)

Controlsln(Age) −0001 (0.02) 0004 (0.02) −0009 (0.06)Diversity of past ideas −0015∗ (0.06) −0038∗∗ (0.08) −0012 (0.16)ln(Past experience in generating ideas) 0024∗∗ (0.05) −0015 (0.08) −0009 (0.12)Time dummies Included Included IncludedCategory dummies Included Included Included

Log-Likelihood −31345041 −21178064 −11140028�2 (df) 361015∗∗ (61) 678021∗∗ (61) 11015007∗∗ (61)N 3,498a 3,498a 2,349b

Note. Standard errors are in parentheses.aIncludes 696 serial ideators; see Figures 1 and 2.bIncludes 286 serial ideators with >0 additional proposed ideas; see Figures 1 and 2.∗Significant at the 0.05 level (two-tailed); ∗∗significant at the 0.01 level (two-tailed).

6. DiscussionIn this research, the nature of a crowdsourced ideageneration process over time is empirically inves-tigated. Two years of panel data involving severalthousand ideas and individuals are studied in thecontext of Dell’s IdeaStorm ongoing crowdsourcingcommunity. Most consumers only propose a singleidea and few of these are ideas the organizationwants to implement (Figure 1). Instead, serial ideatorsaccount for the largest share of implemented ideas(Figures 2 and 5). Among serial ideators, past suc-cess in generating implemented ideas is found to havedetrimental effects on their subsequent likelihood ofproposing another idea the organization eventuallyimplements (Table 3). Past success is also shownto be negatively related to the number of diverseideas proposed (Table 4). Thus, as serial ideators withpast success attempt to come up with ideas thatwill excite Dell, they instead end up proposing lessdiverse ideas that are similar to their ideas that werealready implemented. In addition, the diversity ofcommenting activity on others’ ideas is found to havepositive effects on an individual’s subsequent likeli-hood of generating another implemented idea, but isnot related to the number of diverse ideas proposed(Table 4). Instead, commenting activity seems to berelated to an increase in the quantity of ideas pro-posed (Table 4).

These findings are in stark contrast to a plau-sible alternate perspective involving individual dif-ferences. Specifically, some ideators may simply bebetter at generating quality ideas (e.g., Terwieschand Ulrich 2009, Giotra et al. 2010), suggesting thatpast success should be positively related to an indi-vidual’s chances of proposing another implementedidea. This perspective would also be consistent withthe empirical literature showing that past cumulative

knowledge is positively related to creativity (Weisberg1999, Simonton 2004). For IdeaStorm, a Web-basedcrowdsourcing community where individuals gener-ate ideas repeatedly over time, this is not the case.Instead, there is strong empirical evidence that pastsuccess is negatively related to the likelihood that anideator proposes another quality idea. This findingis consistent with experimental research on cognitivefixation (Jansson and Smith 1991, Burroughs et al.2008)—people tend to fixate on the principles and fea-tures of their prior ideas that have been implemented,thus generating less diverse ideas (i.e., they proposeideas that are similar to those that have already beenimplemented).

The panel analyses in the present study extend theexisting literature by considering the key individual-level factors associated with idea generation in acrowdsourcing community. In particular, an indi-vidual’s past success and past commenting activ-ity are shown to directly relate to their chancesof proposing another successfully implemented idea.In contrast to the research on the patenting activ-ity of inventors (Audia and Goncalo 2007, Contiet al. 2010), the present study empirically demon-strates that individuals in the crowd are unlikelyto generate additional implemented ideas oncesome of their ideas are implemented. Methodolog-ical and/or contextual differences between studiesmay account for these distinct results. Method-ologically, research involving patents only consid-ers granted patents—i.e., these studies ignore patentapplications that were never granted,12 meaning thatthey do not explain the differences between success-ful and unsuccessful patenting activity. Contextually,

12 Audia and Goncalo (2007) note that up to half of all patent appli-cations are rejected.


Figure 6 Proportion of New IdeaStorm Members That Become Serial Ideators

0.00

0.05

0.10

0.15

0.20

0.25

0.30


Rat

io

professionals (employees) are expected (and oftenrewarded) to continually generate patents as part oftheir job whereas ideas are voluntarily contributedin crowdsourcing communities. In the case of patent-ing therefore, organizational learning theories suggestthat past success will be positively related to futureproductivity (March 1991).

6.1. ImplicationsThis research involving an analysis of individual-levelideation efforts is a first step toward addressing thekey question of whether the supply of quality ideascan be sustained by an ongoing crowdsouring com-munity over time. In the case of Dell’s IdeaStorm,the majority of implemented ideas come from serialideators. As suggested by the declining proportion ofnew IdeaStorm members that become serial ideatorsshown in Figure 6, Dell’s future supply of qual-ity ideas may be drying up. Currently, managementclosely tracks the site’s activity (number of ideas, com-ments, votes), but not member statistics. As suggestedby Figure 4, Dell should be monitoring the total num-ber of individuals proposing an idea as well as thetotal number of individuals proposing an idea for thefirst time. Similar values for these metrics indicatethat new members are not becoming serial ideators,which will ultimately lead to a reduction in the num-ber of ideas offered by the crowd that the organizationfinds valuable enough to implement.

Furthermore, as their ideas are implemented andexternally recognized, serial ideators become lesslikely to subsequently propose further ideas theorganization wants to implement. Ideators with pastsuccess tend to propose less diverse ideas because theyfocus on the general aspects of their previously imple-mented ideas. This suggests that an organization likeDell should attempt to (1) entice new members into

the community who are unencumbered by past suc-cess, and (2) convert these ideators into serial ideators.Even then, serial ideators tend to only generate imple-mented ideas over a relatively short period of time.

At the same time, however, the diversity of a serialideator’s commenting activity is found to mitigate thenegative effects of past success (i.e., diversity of pastcommenting activity is positively related to the likeli-hood of proposing an implemented idea). Thus, com-menting activity by serial ideators (particularly acrosscategories) should be encouraged and made as easy aspossible because it is associated with the generationof ideas the organization finds valuable enough toimplement. More generally, it is known that success-ful online communities have active members that con-tinually make contributions (Tedjamulia et al. 2005).Ongoing research suggests that the engagement ofcommunity members depends on individual person-ality characteristics (e.g., intrinsic motivation, per-sonal goals, need to achieve, group identity, personalresponsibility) as well as feedback and reinforcerssuch as financial and social rewards (Tedjamulia et al.2005). For example, individuals that are intrinsicallymotivated and passionate about the organization andcommunity are most likely to propose ideas and com-ment on others’ ideas. In IdeaStorm, this seems to beserial ideators with many implemented ideas as theyare also actively engaged in commenting on others’ideas. Research also finds that individuals who go onto become long-lived, highly engaged members expe-rience significantly better treatment (e.g., they receivethoughtful comments on their initial ideas) from themoment they join the community (Backstrom et al.2008). Thus, to ensure that there is active and diverseparticipation, organizations should pay close atten-tion to the how the community is structured andmanaged.


Although there are no sure-fire ways to overcomefixation effects (Smith 2003), published research sug-gests that the use of analogies can help (Dahl andMoreau 2002, Linsey et al. 2010, Cardoso and Badke-Schaub 2011). Various engineering related design-by-analogy methods are reviewed by Linsey et al.(2010). Interestingly, there is also some evidence thatfixation can be reduced by simply instructing indi-viduals to not focus on prior examples (Chrysikouand Weisberg 2005). Thus, ongoing crowdsourcingcommunities might introduce explicit instructions forideators to use analogies and to not focus on theirprior ideas that were implemented.

Finally, it is interesting to note that Dell hasrecently introduced Storm Sessions to its IdeaStormcommunity—a place where community members canparticipate in “hyper-focused idea-generation ses-sions.” These brainstorming sessions center on a spe-cific topic (question) and last for a relatively shortamount of time (e.g., a couple of weeks). Althoughonly a dozen or so sessions have been completed todate and participation has been relatively low, thesesessions have the potential to reduce fixation effectsby getting ideators to shift their attention away fromtheir own previously implemented ideas.

6.2. Limitations and Future ResearchAlthough Dell’s IdeaStorm represents the gold stan-dard for new product idea crowdsourcing applica-tions, the generalizability of the specific results fromthis study may be limited. Future research might,therefore, attempt to confirm the roles of past suc-cess and commenting activity found in this study inother settings besides computer hardware and soft-ware. The present study is also limited to the pub-licly available data on the IdeaStorm website. Morerefined measures of community activity (e.g., com-ment and vote valence), idea creativity (e.g., percep-tions of novelty, usefulness, and feasibility), and ideadiversity (e.g., based on the idea’s description andcontent) might lead to other hypotheses and, con-sequently, a deeper understanding of the nature ofcrowdsourced ideation over time.

The IdeaStorm data only allow the effects of anindividual’s own implemented ideas to be studied.It would be very interesting to also consider the possi-ble effects of others’ implemented ideas (which wouldnecessitate a measure of whether ideators attend toothers’ implemented ideas). Do ideators also fixateon the general characteristics of others’ implementedideas, leading to less valuable ideas? The theory ofcognitive fixation and structured imagination has notso far considered the possible effects of differentexemplar sources.

In all likelihood, commenting activity by itself isa conservative measure of community member inter-actions. Although the potential differences between

simply reading someone else’s idea versus alsocommenting on that idea could not be examined here,the brainstorming literature indicates that innovationoutcomes are improved when individuals directlyattend to these other stimulus ideas (Dugosh et al.2000, Dugosh and Paulus 2005, Kohn et al. 2011).Future research might, therefore, examine whetherother community activities like voting can also mit-igate the negative effects of fixation on prior suc-cess. Recent experimental research also confirms thatbuilding off the ideas of others can be beneficial inthe idea generation process (Kohn et al. 2011). Thus,it would be interesting to see if collaboration duringthe ideation process is helpful in repeatedly generat-ing quality ideas over time.

The marketing literature has a long-standing inter-est in methods for improving idea generation (e.g.,Goldenberg et al. 2001, Toubia 2006) but is gener-ally silent on “best” approaches for idea selection.From the present study, it is clear that the crowd cangenerate ideas an organization finds valuable enoughto implement. However, much less is known aboutwhether consumers are effective in selecting ideasthat can be implemented (West 2002, Rietzschel et al.2010). And, next to nothing is known about idea selec-tion in crowdsourcing applications where ideas arepublicly submitted and rated by consumers over time.

Finally, improving our understanding of rewardand feedback mechanisms in ongoing crowdsourc-ing communities presents an excellent opportunity forfuture research. For example, several crowdsourcingsystems, such as Threadless and Innocentive, offermonetary rewards for innovative ideas, yet report thatvery few individuals win more than once (Jeppesenand Lakhani 2010). At the same time, thousands ofsoftware programmers willingly contribute their timeto various open source software projects for no tangi-ble rewards (Shah 2006). Future research might ana-lyze secondary data involving actual crowdsourcingapplications and/or construct experimental scenar-ios to determine the proper rewards and feedbackto maintain (or increase) the crowd’s output quantityand quality over time (e.g., Toubia 2006).

6.3. ConclusionsOrganizations are very interested in the crowdsourc-ing model because consumers presumably have spe-cialized knowledge about their own problems withexisting products, and they are intrinsically motivatedto freely contribute their ideas for new products andservices. Many companies and entrepreneurs haverushed to develop and implement crowdsourcingcommunities, even though very little is known abouttheir effectiveness. Most crowdsourcing communi-ties have not been in existence very long, so thereis no established history of successes and failures.


This empirical study of Dell’s IdeaStorm communityreveals that serial ideators are more likely than con-sumers with only one idea to generate an idea theorganization finds valuable enough to implement butare unlikely to repeat their early success once someof their ideas are implemented. The negative effectsof past success, however, are somewhat mitigated forindividuals that have diverse commenting activity.These findings highlight some of the difficulties inmaintaining an ongoing supply of quality ideas fromthe crowd over time and emphasize the need for moreresearch on crowdsourcing communities.

AcknowledgmentsComments from participants in research workshops atthe University of North Carolina at Chapel Hill, theWharton School at the University of Pennsylvania, NorthCarolina State University, University of South Carolina,Yale University, Santa Clara University, Temple Univer-sity, Harvard University, and the University of SouthernCalifornia helped to improve this research during its evo-lution. The author thanks Teresa Amabile, Federico Rossi,Raffaele Conti, Tarun Kushwaha, Susan Cohen, KatherineBarringer, and the journal review team for their very helpfulcomments on earlier drafts. The author also thanks JunheeKim for his research assistance. The financial support of theMarketing Science Institute and Wharton Interactive MediaInitiative is greatly appreciated, as well as the Richard H.Jenrette Business Education Fund.

ReferencesAllison PD (2005) Fixed Effects Regression Methods for Longitudinal

Data Using SAS (SAS Institute, Cary, NC).Amabile T (1988) A model of creativity and innovation in organi-

zations. Staw B, Cummings L, eds. Research in OrganizationalBehavior, Vol. 10 (JAI Press, Greenwich, CT), 123–167.

Amabile T (1996) Creativity in Context (Westview Press, Boulder,CO).

Andrew JP, Manget J, Michael DC, Taylor A, Zablit H (2010)Innovation 2010: A return to prominence and the emergenceof a new world order. Report, Boston Consulting Group,Bethesda, MD. Accessed July 11, 2010, http://www.bcg.com/documents/file42620.pdf.

Audia P, Goncalo J (2007) Past success and creativity over time:A study of inventors in the hard disk drive industry. Manage-ment Sci. 53(1):1–15.

Backstrom L, Kumar R, Marlow C, Novak J, Tomkins A (2008)Preferential behavior in online groups. Proc. Internat. Conf. WebSearch and Web Data Mining (ACM, New York), 117–128.

Birch H, Rabinowitz H (1951) The negative effect of previ-ous experience on productive thinking. J. Experiment. Psych.41(2):121–125.

Bjelland O, Wood R (2008) An inside view of IBM’s “InnovationJam.” Sloan Management Rev. 50(1):32–40.

Bostwick W (2010) Starbucks sponsors coffee-cup redesign con-test. New York Times (March 16), http://www.fastcompany.com/1585666/starbucks-sponsors-coffee-cup-redesign-contest.

Boudreau K, Lacetera N, Lakhani K (2011) Incentives and uncer-tainty in innovation contests: An empirical analysis. Manage-ment Sci. 57(5):843–863.

Boutin P (2006) Crowdsourcing: Consumers as creators. Busi-ness Week (July 13), http://www.businessweek.com/innovate/content/jul2006/id20060713_755844.htm.

Bullinger A, Neyer A, Rass M, Moeslein K (2010) Community-based innovation contests: Where competition meets coopera-tion. Creativity and Innovation Management 19(3):290–303.

Burroughs J, Moreau CP, Mick D (2008) Toward a psychology ofconsumer creativity. Haugtvedt C, Herr P, Kardes F, eds. Hand-book of Consumer Psychology (Erlbaum, New York), 1011–1038.

Cameron A, Trivedi P (2009) Microeconometrics: Methods and Appli-cations (Cambridge University Press, New York).

Cardoso C, Badke-Schaub P (2011) Editorial—Fixation or inspi-ration: Creative problem solving in design. J. Creative Behav.45(2):77–82.

Chrysikou E, Weisberg R (2005) Following the wrong foot-steps:Fixation effects of pictorial examples in a design problem-solving task. J. Experiment. Psych.: Learn., Memory and Cognition31(5):1134–1148.

Conti R, Gambardella A, Mariani M (2010) Learning to beEdison? Individual inventive experience and breakthroughinventions. Presentation, DRUID Summer Conference, June16–18, Imperial College London Business School, London.Accessed June 10, 2011, http://www2.druid.dk/conferences/viewpaper.php?id=501970&cf=43.

Cramer M, Franses P, Slagter E (1999) Censored regression analysisin large samples with many zero observations. Research Report9939/A, Econometric Institute, Rotterdam, The Netherlands.Accessed October 7, 2011, http://repub.eur.nl/res/pub/1608/feweco19991109155424.pdf.

Czarnitzki D, Hussinger K, Schneider C (2011) Wacky patents meeteconomic indicators. Econom. Lett. 113(2):131–134.

Dahl A, Lawrence J, Pierce J (2011) Building an innovation commu-nity. Res.-Tech. Management 54(5):19–27.

Dahl D, Moreau P (2002) The influence and value of analogi-cal thinking during new product ideation. J. Marketing Res.39(1):47–60.

Dell_Admin1 (2007) Who makes decisions about ideas on Idea-Storm? How does is all work? Accessed March 12, 2010, http://www.ideastorm.com/ideaView?id=0877000000008o2AAA.

Di Gangi P, Wasko M (2009) Steal my idea! organizational adop-tion of user innovations from a user innovation community:A case study of Dell IdeaStorm. Decision Support Systems 48(1):303–312.

Di Gangi P, Wasko M, Hooker R (2010) Getting customers’ ideasto work for you: Learning from Dell how to succeed withonline user innovation communities. MIS Quart. Executive9(4):213–228.

Dugosh K, Paulus P (2005) Cognitive and social comparison pro-cesses in brainstorming. J. Experiment. Soc. Psych. 41(3):313–320.

Dugosh K, Paulus P, Roland E, Yang H (2000) Cognitive stimulationin brainstorming. J. Personality Soc. Psych. 79(5):722–735.

Duncker K (1945) On problem solving. Psych. Monographs58(5):1–110.

Elliott S (2010) Starbucks hopes there is no Java Jive from con-test. New York Times (June 17), http://www.mediadecoder.blogs.nytimes.com/2010/06/17/starbucks-hopes-there-is-no-java-jive-from-contest-results/?scp=1&sq=jovoto&st=cse.

Ernst H, Leptien C, Vitt J (2000) Inventors are not alike: The distri-bution of patenting output among industrial R&D personnel.IEEE Trans. Engrg. Management 47(2):184–199.

Fleming L, Szigety M (2006) Exploring the tail of creativity: An evo-lutionary model of breakthrough invention. Adv. Strategic Man-agement 23:335–359.

Franke N, von Hippel E, Schreier M (2006) Finding commerciallyattractive user innovations: A test of lead-user theory. J. ProductInnovation Management 23(4):301–315.


Friedman R, Fishbach S, Forster A, Werth L (2003) Attentional prim-ing effects on creativity. Creativity Res. J. 15(2/3):277–286.

Fuller J (2010) Refining virtual co-creation from a consumer per-spective. Calif. Management Rev. 52(2):98–122.

Garrison D, Anderson R, Archer W (2001) Critical thinking, cog-nitive presence, and computer conferencing in distance educa-tion. Amer. J. Distance Ed. 15(1):7–23.

George J (2007) Creativity in organizations. Acad. Management Ann.1(1):439–477.

German T, Barrett H (2005) Functional fixedness in a technologi-cally sparse culture. Psych. Sci. 16(1):1–5.

Girotra K, Terwiesch C, Ulrich K (2010) Idea generation and thequality of the best idea. Management Sci. 56(4):591–605.

Goldenberg J, Lehmann D, Mazursky D (2001) The idea itself andthe circumstances of its emergence as predictors of new prod-uct success. Management Sci. 47(1):69–84.

Guilford J (1967) The Nature of Human Intelligence (McGraw-Hill,New York).

Hargadon A, Sutton R (1997) Technology brokering and innovationin a product design firm. Admin. Sci. Quart. 42(4):716–749.

Harrison D, Klein K (2007) What’s the difference? Diversity con-structs as separation, variety, or disparity in organizations.Acad. Management Rev. 32(4):1199–1228.

Hauser J, Tellis G, Griffin A (2006) Research on innovation:A review and agenda for marketing science. Marketing Sci.25(6):687–717.

Hinsz V, Tindale R, Vollrath D (1997) The emerging conceptu-alization of groups as information processors. Psych. Bull.121(1):43–64.

Howe J (2008) Crowdsourcing: Why the Power of the Crowd Is Drivingthe Future of Business (Crown Business, New York).

Jansson D, Smith S (1991) Design fixation. Design Stud. 12(1):3–11.Jaruzelski B, Dehoff K (2010) The global 1000: How the top inno-

vators keep winning. Strategy+Bus. 61(Winter):1–14.Jeppesen L, Lakhani K (2010) Marginality and problem-solving

effectiveness in broadcast search. Organ. Sci. 21(5):1016–1033.Jervis C (2010) Personal email communication with the author on

May 3.Jouret G (2009) Inside Cisco’s search for the next big idea. Harvard

Bus. Rev. 87(9):43–45.Kavadias S, Sommer S (2009) The effects of problem structure and

team diversity on brainstorming effectiveness. Management Sci.55(1):1899–1913.

Killian V (2009) “IdeaStorm,” presentation during PDMA Carolinaspanel on “What the crowd can do for you: Crowdsourcing andwisdom of crowds,” April 23.

Kohn N, Smith S (2011) Collaborative fixation: Effects of others’ideas on brainstorming. Appl. Cognitive Psych. 25(3):359–371.

Kohn N, Paulus P, Choi Y (2011) Building on the ideas of others:An examination of the idea combination process. J. Experiment.Soc. Psych. 47(3):554–561.

Kornish L, Ulrich K (2011) Opportunity spaces in innovation:Empirical analysis of large samples of ideas. Management Sci.57(1):107–128.

Krishnan V, Ulrich K (2001) Product development decisions:A review of the literature. Management Sci. 47(1):1–21.

Levitt T (1963) Creativity is not enough. Harvard Bus. Rev.41(3):72–83.

Linsey J, Teng I, Fu K, Cagan J, Wood K, Schuun C (2010) A studyof design fixation, its mitigation and perception in engineeringdesign faculty. J. Mechanical Design 132(4):1–12.

Lindsley D, Brass D, Thomas J (1995) Efficacy-performing spirals:A multi-level analysis. Acad. Management Rev. 20(3):645–678.

Lotka A (1926) The frequency distribution of scientific productivity.J. Washington Acad. Sci. 16(12):317–323.

Magnusson P (2009) Exploring the contributions of involving ordi-nary users in ideation of technology-based services. J. ProductInnovation Management 26(5):578–593.

Maier N (1931) Reasoning in humans. J. Comparative Psych.12(2):181–194.

March J (1991) Exploration and exploitation in organizational learn-ing. Organ. Sci. 2(1):71–87.

Marsh R, Landau J (1995) Item availability in cryptomnesia:Assessing its role in two paradigms of unconscious pla-garism. J. Experiment. Psych.: Learn., Memory and Cognition21(6):1568–1582.

Marsh R, Landau JJ, Hicks J (1996) How examples may (and maynot) constrain creativity. Memory and Cognition 24(3):669–680.

Marsh R, Ward T, Landau J (1999) The inadvertent use of priorknowledge in a generative cognitive task. Memory and Cogni-tion 27(1):94–105.

Menchaca L (2007) Dell offers three consumer systems with Ubuntu7.04. Direct2Dell (May 24), http://en.community.dell.com/dell-blogs/Direct2Dell/b/direct2dell/archive/2007/05/24/15994.aspx.

Mumford M, Gustafson S (1988) Creativity syndrome: Integration,application, and innovation. Psych. Bull. 103(1):27–43.

Nijstad B, Stroebe W (2006) How the group affects the mind: A cog-nitive model of idea generation in groups. Personality Soc.Psych. Rev. 20(3):186–213.

Nijstad B, Stroebe W, Lodewijkx H (2002) Cognitive stimulation andinterference in groups: Exposure effects in an idea generationtask. J. Experiment. Soc. Psych. 38(6):535–544.

Osborn A (1953) Applied Imagination: Principles and Procedures of Cre-ative Thinking (Charles Scribner’s Sons, New York).

Park Y, Fader P (2004) Modeling browsing behavior at multiplewebsites. Marketing Sci. 23(3):280–303.

Perry-Smith J (2006) Social yet creative: The role of social relation-ships in facilitating individual creativity. Acad. Management J.49(1):85–101.

Perry-Smith J, Shalley C (2003) The social side of creativity: A staticand dynamic social network perspective. Acad. ManagementRev. 28(1):89–106.

Perttula M, Liikkanen L (2006) Exposure effects in design ideageneration: Unconscious conformity or a product of sam-pling probability? NordDesign (August 16–18), http://i.org.helsinki.fi/lassial/files/publications/071209-Exposure_effects_in_design.pdf.

Perttula M, Sipila P (2007) The idea exposure paradigm in designidea generation. J. Engrg. Design 18(1):93–102.

Poetz M, Schreier M (2012) The value of crowdsourcing: Can usersreally compete with professionals in generating new productideas? J. Product Innovation Management 29(2):245–256.

Preece J, Nonnecke B, Andrews D (2004) The top five reasons forlurking: Improving community experiences for everyone. Com-put. Human Behav. 20(2):201–233.

Purcell A, Gero J (1992) Effects of examples on the results of adesign activity. Knowledge-Based Systems 5(1):82–91.

Purcell A, Gero J (1996) Design and other types of fixation. DesignStud. 17(4):363–383.

Rietzschel E, Nijstad B, Stroebe W (2007) Relative accessibility ofdomain knowledge and creativity: The effects of knowledgeactivation on the quantity and originality of generated ideas.J. Experiment. Soc. Psych. 43(6):933–946.

Rietzschel E, Nijstad B, Stroebe W (2010) The selection of creativeideas after individual idea generation: Choosing between cre-ativity and impact. British J. Psych. 101(1):47–68.

Robinson A, Schroeder D (2005) Big results from small ideas.Indust. Management 47(3). Accessed April 12, 2012, http://www.findarticles.com/p/articles/mi_hb3081/is_3_47/ai_n29202280/.


Schellens T, Valcke M (2005) Collaborative learning in asyn-chronous discussion groups: What about the impact on cogni-tive processing? Comput. Human Behav. 21(6):957–975.

Schulze A, Hoegl M (2008) Organizational knowledge creation andthe generation of new product ideas: A behavioral approach.Res. Policy 37(10):1742–1750.

Shalley C, Zhou J, Oldham G (2004) Effects of personal and con-textual characteristics on creativity: Where should we go fromhere? J. Management 30(6):933–958.

Shah S (2006) Motivation, governance, and the viability of hybridforms in open source software development. Management Sci.52(7):1000–1014.

Silverberg G, Verspagen B (2007) The size distribution of innova-tions revisited: An application of extreme value statistics tocitation and value measures of patent significance. J. Economet-rics 139(2):318–339.

Silverman R (2011) For bright ideas, ask the staff. Wall StreetJournal (October 17), http://www.online.wsj.com/article/SB10001424052970204774604576631063939483984.html.

Simonton D (2003) Scientific creativity as constrained stochasticbehavior: The integration of product, person and process per-spectives. Psych. Bull. 129(4):475–494.

Simonton D (2004) Creativity in Science: Chance, Logic, Genius, andZeitgeist (Cambridge University Press, New York).

Smith S (2003) The constraining effects of initial ideas. Paulus P,Nijstad B, eds. Group Creativity: Innovation Through Collaboration(Oxford University Press, Oxford, UK), 15–31.

Smith S, Ward T, Schumacher J (1993) Constraining effects ofexamples in a creative generation task. Memory and Cognition21(6):837–845.

Strobe W, Nijstad B, Rietzschel E (2010) Beyond productivity loss inbrainstorming groups: The evolution of a question. Adv. Exper-iment. Soc. Psych. 43:157–203.

Sullivan E (2010) A group effort: More companies are turning tothe wisdom of the crowd to find ways to innovate. MarketingNews (February 28):22–28.

Tedjamulia S, Dean D, Olsen D, Albrecht C (2005) Motivatingcontent contributions to online communities: Toward a morecomprehensive theory. Proc. 38th Annual Hawaii Internat. Conf.System Sci. (IEEE Computer Society, Washington, DC).

Terwiesch C, Ulrich K (2009) Innovation Tournaments: Creating andSelecting Exceptional Opportunities (Harvard University Press,Cambridge, MA).

Terwiesch C, Xu Y (2008) Innovation contests, open innova-tion, and multi-agent problem solving. Management Sci. 54(9):1529–1543.

Torrance E (1966) The Torrance Tests of Creative Thinking (PersonnelPress, Princeton, NJ).

Toubia O (2006) Idea generation, creativity, incentives. MarketingSci. 25(5):411–425.

Vandenbosch B, Saatcioglu A, Fay S (2006) Idea management:A systemic view. J. Management Stud. 43(2):259–288.

von Hippel E (2005) Democratizing Innovation (MIT Press,Cambridge, MA).

Ward T (1994) Structured imagination: The role of conceptual struc-ture in exemplar generation. Cognitive Psych. 27(1):1–40.

Weiner B (1985) An attributional theory of achievement motivationand emotion. Psych. Rev. 92(4):548–573.

Weisberg R (1999) Creativity and knowledge: A challenge to theo-ries. Sternberg RJ, ed. Handbook of Creativity (Cambridge Uni-versity Press, New York), 226–250.

Weiss T (2006) The suggestion box 2.0. Forbes (December 12),http://www.forbes.com/2006/12/12/leadership-innovation-software-lead-innovation-cx_tw_1212ideas.html.

West M (2002) Ideas are ten a penny: It’s team implementationnot idea generation that counts. Appl. Psych.: An Internat. Rev.51(3):411–424.

Yang Y, Chen P, Pavlou P (2009) Open innovation: An empiricalstudy of online contests. Proc. 30th Internat. Conf. Inform Sys-tems (Association for Information Systems, Altanta).

crowdsourcing new product ideas over time: an analysis of the...

Documents