![Page 1: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/1.jpg)
Collective Intelligence in Human Groups
Anita Williams WoolleyCarnegie Mellon University
Network Science meets Team ScienceOctober 2013
![Page 2: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/2.jpg)
The Collective . . . • Ishani Aggarwal, Georgia Tech
• Christopher Chabris, Union College
• David Engel, MIT
• Nada Hashmi, MIT
• Lisa Jing, MIT
• Thomas Malone, MIT
Special thanks to NSF, the Army Research Office, and Cisco Systems for financial support of the research
![Page 3: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/3.jpg)
The Puzzle of Collective Intelligence
Ant Colony
Flock of Birds
Animal Herd
![Page 4: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/4.jpg)
The Puzzle . . .
• Does general collective intelligence exist in human groups?
![Page 5: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/5.jpg)
General Intelligence
• General intelligence is the inference one makes from the observation that people who do well on one task tend to do well on other tasks. – In addition to separate, non-correlated abilities associated
with each task, there is a more general ability that influences all tasks.
Source: Deary, 2000
![Page 6: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/6.jpg)
Individual Intelligence
• Spearman’s g
Charles Spearman (1904)
![Page 7: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/7.jpg)
Collective Intelligence: Questions
• Is there evidence of a general collective intelligence in groups?
• Can we isolate a small set of tasks that is predictive of group performance on a broader range of more complex tasks?
• Does c have predictive validity beyond individual intelligence of group members?
• How can we use this information to build a better science of collective performance?
![Page 8: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/8.jpg)
Study 1
• 40 groups spend five hours together in the laboratory
• Work together on a diverse range of tasks, plus a criterion computer game task
• Also measured individual intelligence
Woolley, Chabris, Pentland, Hashmi & Malone, Science, 2010
![Page 9: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/9.jpg)
Sampling Tasks
Choose
NegotiateExecute
Generate
Adapted from Larson, 2009; McGrath,1984
![Page 10: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/10.jpg)
Example TasksTask Description Scoring
Generate Brainstorming. Come up with as many uses for a brick as possible.
Scored on quantity and quality of ideas.
Choose Intellective. Members answer a set of Raven’s Matrices questions as a group.
Scored on correctness.
Negotiate Devise a shopping trip using a shared car so that all members can get as many of their items at the best places possible.
Cumulative score of all group members.
Execute Typing task. Members must collectively type difficult text into a shared online document.
Scored on number of words typed minus errors and skipped words in limited time period.
![Page 11: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/11.jpg)
Study 1
• Average inter-item correlation = .28• First principal component accounts for 43% of
variance
Woolley, Chabris, Pentland, Hashmi & Malone, Science, 2010
![Page 12: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/12.jpg)
Evidence for a c-factor
Task 1
Task 2
Task 3
Task 4
Task 5
Collective Intelligence
.32*
.36*
.72**
.57*
.69*
χ2 = 1.66, p = .89, NFI = .94, CFI = 1.00
![Page 13: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/13.jpg)
Task 1
Task 2
Task 3
Task 4
Task 5
.32*
.36*
.72**
.57*
.69*
Video Game Score
.50**
χ2 = 3.30, p= .95; NFI = .91, CFI = 1.00
Collective Intelligence
Woolley, Chabris, Pentland, Hashmi & Malone, Science, 2010
Evidence for a c-factor
![Page 14: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/14.jpg)
Task 1
Task 2
Task 3
Task 4
Task 5
Collective Intelligence
.32*
.36*
.72**
.57*
.69*
Video Game Score
.51**
Average IQ
.08
χ2 = 13.92 p = .45; NFI = .70, CFI = .99
Woolley, Chabris, Pentland, Hashmi & Malone, Science, 2010
Evidence for a c-factor
![Page 15: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/15.jpg)
Study 2
Task 1
Task 2
Task 3
Task 4
Task 5
Collective Intelligence
.72**
.79**
.10
.48*
.61*
Architectural Design Task
.36**
Average IQ
.05
107 groups of sizes 2, 3, 4, and 5
χ2 = 4.05, p=.13; CFI = .94; NFI = .91
Woolley, Chabris, Pentland, Hashmi & Malone, Science, 2010
![Page 16: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/16.jpg)
Evidence for a c-factor
0
5
10
15
20
25
30
35
40
45
50
1 2 3 4 5
Perc
ent
Varia
nce
Expl
aine
d
Study 1
Study 2 (five tasks)
Indiv Intelligence Test
Woolley, Chabris, Pentland, Hashmi & Malone, 2010
![Page 17: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/17.jpg)
Predictive value of c and g factors
Study 1: Video Game Study 2: Architectural Design0
0.1
0.2
0.3
0.4
0.5
0.6
Collective Intelligence Average Member Intelligence Maximum Member Intelligence
Woolley, Chabris, Pentland, Hashmi & Malone, 2010
![Page 18: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/18.jpg)
CI and Student Project Performance
• 49 MBA student teams at CMU, CI predicts:– Desert Survival Simulation 1 week later (r=.30, p=.01)– Change Pro Organizational Simulation 3 weeks later (r=.39,
p=.005)
• 114 groups of German Computer Science students– CI predicts peer-rated performance on final project two
months later (r=.21, p< .05)
![Page 19: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/19.jpg)
Learning & Collective Intelligence
• 98 teams• CI measured at beginning of session• Minimum-effort tacit coordination game (Van Huyck et
al.,1990). – Multiple rounds of individual decision making – Collective gains or loses money as a result of the decisions
made by team members without communication. – Provides a behavioral measure of learning across multiple
trials
Aggarwal, Woolley, Chabris, & Malone, under review
![Page 20: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/20.jpg)
Tacit Coordination Task
Minimum of the group’s choice
0 10 20 30 40
0 2400Member choice 10 2200 2800
20 1600 2600 3200
30 600 2000 3000 3600
40 -800 1000 2400 3400 4000
![Page 21: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/21.jpg)
CI and Learning
Aggarwal, Woolley, Chabris, & Malone, under review
![Page 22: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/22.jpg)
CI and Learning
Dependent Variable Rate of Learning(1) (2) (3)
Team Size -.003 -.003 -.01Initial performance -.04* -.04* -.04*Individual Intelligence .02* .01Collective Intelligence .02*
R2 .26 .32 .38
![Page 23: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/23.jpg)
CI and Learning in the Classroom
• 60 MBA student teams of 4-5 students each• Class conducted using Team-Based
Learning approach (Michaelson & Sweet, 2011)
– Individual students complete a “Readiness Assurance Test” at the beginning of each unit
– Teams complete same assessment immediately following
• All teams completed the CI battery at the beginning of the term.
![Page 24: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/24.jpg)
CI in Classroom Teams
Aggarwal, Woolley, Chabris, & Malone, in prep
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
Exam 1 Exam 2 Exam 3
Test
Sco
re
High CI TeamsLow CI TeamsMax Indiv/High CI TeamsMax Indiv/Low CI Teams
![Page 25: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/25.jpg)
What Predicts c??• Not group satisfaction (r = -.07) cohesion (r = -.12), or
motivation (r = -.01)
• Not personality• Proportion of females in group
![Page 26: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/26.jpg)
CI and Proportion of Women
Engel, Woolley, Aggarwal, Chabris & Malone, in prep
![Page 27: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/27.jpg)
Social Perceptiveness
Playful Comforting Irritated Bored
“Reading the Mind in the Eyes” Baron-Cohen et al., 2001
![Page 28: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/28.jpg)
CI and Communication
• Uneven distribution in speaking turns negatively predicts c (Woolley et al., 2010)
Sociometric Badge
![Page 29: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/29.jpg)
Effects of Cognitive Diversity
Large : BigTriumph: ___________ (1) Small (2) Success (3) Lose
Verbal Reasoning
Visual Reasoning
Kozhevnikov, Kosslyn & Shephard, 2005; Kozhevnikov & Blazhenkova, 2013; Woolley et al. 2008
![Page 30: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/30.jpg)
Cognitive Diversity & c
Aggarwal, Woolley, Chabris, & Malone, in prep
t2
![Page 31: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/31.jpg)
Slide 30
t2 Given the reaction when we displayed this graph at the MCI meeting last week, I wonder if we want to show this?tepper, 2010-07-20
![Page 32: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/32.jpg)
Online 60 minutes Collaborative interface Adaptable for studies Task groups: Typing, Matrix Problem Solving, Brainstorm, Unscramble, Sudoku
Our new online CI Battery
![Page 33: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/33.jpg)
vs.
68 groups of four people in two conditions Both conditions in the lab
Testing the new CI Battery
![Page 34: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/34.jpg)
Are the results comparable?
![Page 35: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/35.jpg)
Better groups chat more
Better groups participate more equally
Communication
![Page 36: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/36.jpg)
% Women and CI Online and F2F
Online
Face-to-Face
CI S
core
CI S
core
% Women
![Page 37: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/37.jpg)
Equally important in online and face-to-face groups (r=.57 and r= .55, p<.001)
Social Perceptiveness
![Page 38: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/38.jpg)
CI in Online vs. Face-to-Face Groups
Face-to-face Online
Avg. score on RME test for group members
0.57*** 0.55***
% women in group 0.20 0.41*
Amt. of communication 0.51** 0.47**
Std. deviation of communication among individuals
-0.29 * -0.41*
Std. deviation of individual contributions to task solutions
-0.47** -0.42*
(* = p<.0.05, ** = p <0.01 and *** = p < 0.001)
![Page 39: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/39.jpg)
General Conclusions
• Our studies supply strong evidence of a “c-factor” underlying collective performance that predicts future performance and group learning
• Factors that facilitate the transfer of information seem to facilitate CI– Equality of contribution– Social perception– Low or moderate cognitive diversity
![Page 40: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/40.jpg)
Collective Intelligence and Network Science
• Can networks be designed to produce a consistent level of performance across domains?
• What are the qualities of networks that yield a high level of collective intelligence?
• What is the relative contribution of individual capability versus network capability to the collective intelligence of networks?
![Page 41: Collective Intelligence in Human Groups · Collective Intelligence in Human Groups Anita Williams Woolley Carnegie Mellon University Network Science meets Team Science October 2013](https://reader033.vdocuments.us/reader033/viewer/2022052919/5b9c9dae09d3f2d6288d12d9/html5/thumbnails/41.jpg)
Future Directions
• Further explore what predicts CI• Use the CI battery to predict team performance in
other contexts– Larger groups online– Teams in organizational settings
• Experiment with tools that enhance the processes known to enhance CI