Andy J. Saltarelli, PhDandysaltarelli.com

A Walk Around Pasteur’s

Quadrant: Diverse Approaches to

Investigating the Effects of Social

Context on Educational

Outcomes

Background

Human

Development

Educational

Psychology

Educational

Technology

“It seems clear to me that we are

well past the quantitative–

qualitative debate and more

concerned with issues of providing

good, valid, and reliable evidence

to support our inferences and

conceptual models, regardless of

the nature of the general

methodology.” ~ Pintrich, 2000, p.

223

Goal of Usefulness

Goal of

Scientific

Understanding

YesNo

Yes

No

Basic

Research

(Bohr)

Use-inspired

basic

research

(Pasteur)

Pure applied

research

(Edison)

Pasteur’s

Quadrant (Pintrich, 2000; Stokes, 1997)

Goal of Usefulness

Goal of

Scientific

Understanding

YesNoYes

No

Lost Boys

Online

Cooperatio

nMultimedia

Learning

MOOC

Faculty

MOOC

Learners

Research

Research

Lost Boys of Sudan

Existential-phenomenological inquiry

(Gilgun, 2005; Giorgi & Giorgi, 2003)

Followed 19 youth over 7 years after being

resettled in US

Multiple semi-structured interviews with

youth, foster parents, and social workers

Lost Boys of Sudan

Cultural creation combining the good parts of American culture with the good parts of native Sudanese culture

“I’ve become like a hybrid between here, two cultures you know and these two cultures make me, I’m making good thing out of it.”

Lost Boys of Sudan(Qin, Saltarelli, et al., 2014, Journal of Adolescent

Research)

“Cultural appropriation”, integrative

adaptation

19 HS diplomas, 7 bachelor’s, 10

community college, 4 master’s

“Every single one of us has to go to

college because we need to go

back and help.”

“Education is my mother and education is my father” (Chanoff, 2005)

Lost Boys of Sudan(Rana…Saltarelli, 2011, Teachers College

Record)

Educational resilience

Research

Effects of Belongingness and

Synchronicity on Face-to-face

and Online Cooperative

Learning

Does adapting face-to-face (FTF)

pedagogies to online settings raise

‘boundary questions’ about whether

the same pedagogy stimulates

different psychological processes

under FTF and online conditions?

Social Interdependence

Theory(Deutsch 1949; Lewin, 1948; D. W. Johnson & Johnson, 1989,

2005)Interdependent Goal

Structures (Positive

Interdependence)

Promotive Interaction

Goal Achievement

+Motivation, +Achievement,

+Well-being, +Relationships

Constructive Controversy(Deutsch 1949; Lewin, 1948; Johnson & Johnson, 1998;

2009)

Argue incompatible views within a cooperative context

Seek agreement integrating the best evidence and

reasoning from both positions

5-step Procedure:

Constructive Controversy40 Years of Research — Meta-Analysis

(Johnson & Johnson, 2009)

(ES = Mean Effect

Sizes)

Constructive Controversy

v. Debate

Constructive

Controversy v.

Individualistic

Achievement .62 ES .76 ES

Perspective

Taking.97 ES .59 ES

Motivation .73 ES .65 ES

Self-esteem .56 ES .85 ES

In face-to-face

settings

MED

IA R

ICH

NES

S

SYNCHRONICITY

Face-To-Face

Vid

eoA

ud

ioTe

xt

Synchronous Asynchronous

Online Constructive Controversy

Test Constructive Controversy1 FTF x 2 Synchronicity (Sync, Async) x 3 Media (Audio, Video, Text)

(Roseth, Saltarelli, & Glass 2011, Journal of Education Psychology).

Previous Results(Roseth, Saltarelli, & Glass, 2011; Journal of Educational Psychology)

In Asynchronous CMC Achievement↓ Motivation↓ Belongingness↓

Theory: What are the mechanisms by which asynchronous CMC affects constructive controversy?

Practice: Can satisfying belongingness needs ameliorate the negative effects of asynchronous CMC?

Belongingness(Baumeister & Leary, 1995; Deci & Ryan, 2000; Walton et al., 2012)

Innate

Needs

Competence

Belongingness

Autonomy

Self-Regulation

Motivation

SYNCHRONICITY

Face-To-Face Synchronous Asynchronous

Belongingness

Test Constructive Controversy3 Synchronicity (FTF, Sync, Async) x 3 Belongingness (Acceptance, Control, Mild

Rejection)

(Saltarelli & Roseth, 2014, Journal of Education Psychology).

Belongingness Manipulation(Romero-Canyas et al., 2010)

Complete personality

profile

Belongingness Manipulation

Rank potential partners based on profile

Belongingness ManipulationPartner pairing

Synchronous Scaffold

Synchronous CMC Scaffold:

- WordPress, Google DocsTM

- Integrated text-based chat

Procedure:

- Complete initial

belongingness activity

- Dyads complete activity

over 70 min. class period

Asynchronous ScaffoldAsynchronous CMC

Scaffold:

- Custom WordPress,

BuddyPress

Procedure:

- Complete initial

belongingness activity

- Dyads complete activity

over 6 days

Dependent Variables

Operationalization

1. Time Time spent? (1-item), Time preferred?(1-item)

2. Social Interdependence

Cooperation (7-items, α=.89), Competition (7-items, α=.93), Individualism (7-items, α=.86

3. Conflict Regulation

Relational Regulation (3-items, α=.80), Epistemic Regulation (3-items, α=.82)

4. MotivationRelatedness (8-items, α=.88), Interest (7-items, α=.92), Value (7-items, α=.93)

5. AchievementMultiple-choice questions (4-items, α=.41), Integrative statement: # of arguments (κ=.95), use of evidence (κ=.90), integrative (κ=.87)

6. Perceptions of Technology

Technology Acceptance (4-items, α=.90), Task-technology Fit (2-items, α=.94)

DV

Summary of Findings

AsyncCMC

FTF and Sync CMC

▲ Cooperative perceptions

▲ Epistemic conflict

Led to…

▲ Motivation

▲Achievement

▲ Competitive perceptions

▲ Relational conflict

Led to…

▼Motivation

▼Achievement

Summary of Findings

Belongingnes

s

Met

▲Cooperative perceptions

▲ Epistemic regulation

▲ Intrinsic motivation

▲ Perceptions of

technology

Buffers but does not offset the deleterious effects of asynchronous CMC

Implications for Practice

▲ Satisfying belongingness needs can promote cooperation and motivation in online contexts

▲ Instructors may be able to monitor and enhance cooperative perceptions and epistemic regulation

▲ Varying synchronicity to match task demands may maximize affordances and minimize constraints

Looking Forward & DBR

Looking Forward & DBR

Looking Forward & DBR

Research

Psychological Interventions Close Global Participation Gaps

in Online Learning

Reasoning

together

http://lytics.stanford.edu/

Computer

Science

Learning

Science

EDM

LAK L@SSociology

Social

PsychologySociology

Design

“We engage in use-driven research and data-driven design

Data does not speak for itself. Rather,

people must actively make meaning of

the data… interpretation is a central part

of the data use process…noticing,

interpreting, and constructing

implications for action— are shaped by

individual beliefs, knowledge, and

motivation and are influenced by the

nature and patterns of social interaction

~ Coburn & Turner, 2011

MOCs may eye the world market, but

does the world want them?

Global Participation Gap

16 MOOCs (N~ 67,000):

- 50% less persistence (Kizilcec & Halawa, in press )

Previous run of course (N= 41,186):

- 50% less video lectures watched

- 40% less likely to take assessments

Current self-paced version (N = 60,000 enrolled, 4,562 did intervention)

- 75% less video lectures watched

- 82% less likely to take assessments

Why?

- Language Barriers

- Access to Technology

(Kizilcec, Saltarelli, & Cohen, Under Review,

PNAS)Western (Europe, Oceania, and Northern America) v. Non-

Western (Africa, Asia, and Latin America)

MOOCs may eye the world market, but

does the world want them?

Social Belonging & Identity

Threat to belongingness (Walton et al., 2014)

Threat to identity (Cohen et al., 2006, 2009, 2014; Steele, 1988)

Cultural mismatch (Harackiewicz et al., 2014; Stephens et al.,

2012)

Social Belonging & Identity

"I didn’t go to a very good university, and I

worried that my previous courses had not

prepared me well for this online course.

Honestly, when I first enrolled, I thought the

instructor was a bit scary. I thought the

grading was critical and hard, and I worried

about whether other students would

respect me. I was nervous about writing on

the discussion forum and I didn’t want to

ask people for help with quizzes. ~ Tom

Psychological Interventions

Intervention embedded in course survey

N = 4,562

Study Tips Self-Affirmation Social Belonging

1. Read Quotes2. Write reflection3. Write letter

1. Choose key values2. Write reflection3. Write letter

1. Read Quotes2. Write reflection3. Write letter

Perceived Beneficiary

Self Other Self Other Self Other

Results

Gaps completely closed by…

- Belongingness intervention if peer recipient

▲ 73% videos watched (gap: z = 0.98, P = 0.33)

- Self-affirmation intervention if self recipient

▲ 55% videos watch (gap: z = 0.85, P = 0.39)

▲ 100% assessments taken (gap: z = -0.72, P = 0.47)

No statistically significant effect on Western

learners

Research

“My Goal Is To Surf It, Not Just

Stand There”: Professors’

Sensemaking Strategies in

University Open Online

Learning Initiatives

Method

▲ Three-part semi-structured interviews with 16 MOOC

faculty

▲ Grounded theory (Corbin & Strauss, 1994, 2007,

2014)

▲Emotion and metaphorical language codes (Plutchik,

2011)

Sensemaking explores the social contexts in which

meaning is constructed; it is grounded in multiple

dimensions of a person’s identity; it emphasizes

retrospective meaning-making of events, choices, and

decisions; and it recognizes that people are in medias

Preliminary Results

▲ Incredible diversity of motivations, 13 distinct codes

▲ Role soft infrastructure – institutional values,

affirmations, ethos of intellectual generosity

▲ Expressions of agency and joy, cf. rhetoric of

disruption and unbunlding (Carey, 2015; Christensen &

Weise, 2014)

“And that is so empowering. It's such a great feeling to

be in a place where you can have a new idea and

somebody will support you in the exploration of that

idea.”

▲ Lost Boys – give voice to and understand lived experience, qualitative phenomenology

▲ Online Cooperation – theory testing and design-based research in authentic setting, quantitative experimental-control

▲MOOC Learners/Gaps – leverage big-er data, experimental-control, scale “wise” interventions

▲MOOC Faculty – give voice to, action research, program evaluation, qualitative grounded theory

Summary