inverting critique: emergent technologically-mediated critique practices of developing design...

INVERTING CRITIQUE: EMERGENT TECHNOLOGICALLY-MEDIATED CRITIQUE

PRACTICES OF DEVELOPING DESIGN STUDENTS

COLIN M. GRAY Purdue University

CRAIG D. HOWARD University of Tennessee—Knoxville

Gray, C. M., & Howard, C. D. (2015, June). “Why are they not responding to critique?”: A student-centered construction of the crit. In LearnxDesign: The 3rd International Conference for Design Education Researchers and PreK-16 Design Educators. Chicago, IL: School of the Art Institute of Chicago.

PRIOR WORK

CRITIQUEis central to design education

[Anthony, 1991; Dannels & Martin, 2008; Klebesadel & Kornetsky, 2009; Parnell, Sara, Doidge, & Parsons,

2012; Schön, 1987; Shulman, 2005]

instructor-centric

curriculum-centric

power-laden

CRITIQUEis central to design education

[Anthony, 1991; Dannels & Martin, 2008; Klebesadel & Kornetsky, 2009; Parnell, Sara, Doidge, & Parsons,

2012; Schön, 1987; Shulman, 2005]

[Anthony, 1991; Blythman, Orr, & Blair, 2007; Dutton, 1991; Gray, 2013, 2014; Gray & Howard 2014; Webster, 2007;

Willenbrock, 1991]

CRITICAL FRAMING

what if we looked at critique in a

[Anthony, 1991; Blythman, Orr, & Blair, 2007; Dutton, 1991; Gray, 2013, 2014; Gray & Howard 2014; Webster, 2007;

Willenbrock, 1991]

CRITICAL FRAMING

what if we looked at critique in a

contextualized with

PEER ASSESSMENT, DEVELOPMENT OF

DESIGN EXPERTISE & IDENTITY, CSCL, ETC.

[Easterday et al., 2014; Freeman & McKenzie, 2001; Lawson & Dorst, 2009;

Schön, 1990; Smith, 2015; Topping, 1998; Xu & Bailey, 2013]

CRITIQUE & CRITICAL PEDAGOGY

CRITIQUE & CRITICAL PEDAGOGY

FROM student as tabula rasa

TO student as capable,

emerging proto-professional

[Boling, Gray, & Smith, 2015; Gray, 2014; Freire, 1970]

CONTEXT

CONTEXT

Human-Computer Interaction Master’s program

Introductory project-based design course

Final project was based on the 2014 ACM SIGCHI Student Design Competition

CLASSROOM

=Physical Space [~60 student capacity; 8 screens; decentralized layout]

Virtual Space [Facebook; Google Docs; SMS]

+CLASSROOM

MULTIMODAL CRITIQUE

multiple critique conversations occur concurrently around a single designed

artifact (or presentation of that artifact), in both physical and virtual modes, with

multiple classes or groupings of interlocutors

FIRST-YEAR STUDENT

FIRST-YEAR STUDENT (PRESENTING)

INSTRUCTOR

PROJECTOR/SCREEN

WHITEBOARD

MENTOR (SECOND-YEAR STUDENT)

LIVE MENTOR CRITIQUE IN GOOGLE DOC

verbally

virtually

COMMUNICATION DURING PRESENTATION

virtually

verbally

verbally

COMMUNICATION DURING CRITIQUE

verbally

COMMUNICATION PATTERNS

PRESENTATION STUDENT CRITIQUE MENTOR CRITIQUE

INSTRUCTOR CRITIQUE

GOOGLE DOC

CRITIQUE STRUCTURE

1. What volume of critique did this instructional design support?

2. How did the volume of critique generated via digital means compare to the volume of critique in the physical classroom?

3. What was the nature of critique content that the advanced students generated?

RESEARCH QUESTIONS

What volume of critique did this instructional design support?

1

Team Collaborative Google Doc (GD) Backchannel (BC)

# Turns [CL*]

Average Words per turn (SD) Total words

# Turns


Team A 37 [10] 46.4054 (45.6369) 1717 13 5.1538 (4.4695) 67

Team B 26 [4] 25.8462 (21.6079) 672 69 7.3043 (5.7441) 504

Team C 38 [8] 37.1579 (28.9496) 1412 135 7.1852 (6.6968) 970

Team D 9 [1] 36.8889 (21.1332) 332 9 12.6667 (10.3923) 114

Team E 28 [5] 27.2857 (19.0879) 764 4 5.0000 (2.7386) 20

Team F 27 [1] 39.9259 (35.1683) 1078 116 10.3966 (9.1622) 1206

Team G 16 [1] 47.6875 (34.7234) 763 69 7.4058 (7.1431) 511

Team H 30 [4] 28.2667 (28.5668) 848 49 8.8367 (7.3105) 433

Team I 20 [1] 50.3000 (44.6946) 1006

Team J 19 [3] 31.6316 (20.0846) 601 60 10.6667 (8.3280) 640

Team K 16 [4] 50.4375 (49.0446) 807 61 6.8525 (5.7054) 418

TOTAL 266 [42] 37.594 10000 585 8.3470 4883

Team Collaborative Google Doc (GD) Backchannel (BC)

# Turns [CL*]


# Turns


Team A 37 [10] 46.4054 (45.6369) 1717 13 5.1538 (4.4695) 67

Team B 26 [4] 25.8462 (21.6079) 672 69 7.3043 (5.7441) 504

Team C 38 [8] 37.1579 (28.9496) 1412 135 7.1852 (6.6968) 970

Team D 9 [1] 36.8889 (21.1332) 332 9 12.6667 (10.3923) 114

Team E 28 [5] 27.2857 (19.0879) 764 4 5.0000 (2.7386) 20

Team F 27 [1] 39.9259 (35.1683) 1078 116 10.3966 (9.1622) 1206

Team G 16 [1] 47.6875 (34.7234) 763 69 7.4058 (7.1431) 511

Team H 30 [4] 28.2667 (28.5668) 848 49 8.8367 (7.3105) 433

Team I 20 [1] 50.3000 (44.6946) 1006

Team J 19 [3] 31.6316 (20.0846) 601 60 10.6667 (8.3280) 640

Team K 16 [4] 50.4375 (49.0446) 807 61 6.8525 (5.7054) 418

TOTAL 266 [42] 37.594 10000 585 8.3470 4883

Average Team Critique

8.0 interlocutors (SD=3.16; min=3; max=14)

15.53 critique acts (SD=11.51; min=1; max=34)

HOW MANY INTERLOCUTORS DO YOU NEED?

the top five commenters accounted for 81.70% of all unique critique acts in the

Google Docs

(top 3 = 60.71%)

How did the volume of critique generated via digital means

compare to the volume of critique in the physical classroom?

2

TeamPresentation Length Critique Length

# CL Critique Turns (1Y, 2Y, Inst.)

Average CL Critique Turn Length

Team A 00:05:35 0:23:54 20 (18/1/1)* 00:01:12

Team B 00:08:44 0:12:06 10 (8/1/1) 00:01:13

Team C 00:07:20 0:20:42 14 (12/1/1) 00:01:29

Team D 00:05:59 0:09:41 14 (8/3/3) 00:00:42

Team E 00:07:26 0:19:57 11 (6/2/3) 00:01:49

Team F 00:06:03 0:17:38 17 (15/1/1) 00:01:02

Team G 00:08:06 0:15:01 17 (11/3/3) 00:00:53

Team H 00:07:56 0:15:18 15 (10/2/3) 00:01:01

Team I 00:06:18 0:12:52 8 (6/1/1) 00:01:37

Team J 00:05:46 0:11:41 12 (9/2/1) 00:00:58

Team K 00:06:17 0:14:09 9 (7/1/1) 00:01:34

TOTAL01:15:30 (M=00:06:52; SD=00:01:05)

2:52:59 (M=00:15:44; SD=00:04:22)

147 (M=13.36; SD=3.57)

(M=00:01:14; SD=00:00:20)




Team A 00:05:35 0:23:54 20 (18/1/1)* 00:01:12

Team B 00:08:44 0:12:06 10 (8/1/1) 00:01:13

Team C 00:07:20 0:20:42 14 (12/1/1) 00:01:29

Team D 00:05:59 0:09:41 14 (8/3/3) 00:00:42

Team E 00:07:26 0:19:57 11 (6/2/3) 00:01:49

Team F 00:06:03 0:17:38 17 (15/1/1) 00:01:02

Team G 00:08:06 0:15:01 17 (11/3/3) 00:00:53

Team H 00:07:56 0:15:18 15 (10/2/3) 00:01:01

Team I 00:06:18 0:12:52 8 (6/1/1) 00:01:37

Team J 00:05:46 0:11:41 12 (9/2/1) 00:00:58

Team K 00:06:17 0:14:09 9 (7/1/1) 00:01:34

TOTAL01:15:30 (M=00:06:52; SD=00:01:05)

2:52:59 (M=00:15:44; SD=00:04:22)

147 (M=13.36; SD=3.57)

(M=00:01:14; SD=00:00:20)

Classroom Critique Acts

147(M=13.36; SD=3.57)




Team A 00:05:35 0:23:54 20 (18/1/1)* 00:01:12

Team B 00:08:44 0:12:06 10 (8/1/1) 00:01:13

Team C 00:07:20 0:20:42 14 (12/1/1) 00:01:29

Team D 00:05:59 0:09:41 14 (8/3/3) 00:00:42

Team E 00:07:26 0:19:57 11 (6/2/3) 00:01:49

Team F 00:06:03 0:17:38 17 (15/1/1) 00:01:02

Team G 00:08:06 0:15:01 17 (11/3/3) 00:00:53

Team H 00:07:56 0:15:18 15 (10/2/3) 00:01:01

Team I 00:06:18 0:12:52 8 (6/1/1) 00:01:37

Team J 00:05:46 0:11:41 12 (9/2/1) 00:00:58

Team K 00:06:17 0:14:09 9 (7/1/1) 00:01:34

TOTAL01:15:30 (M=00:06:52; SD=00:01:05)

2:52:59 (M=00:15:44; SD=00:04:22)

147 (M=13.36; SD=3.57)

(M=00:01:14; SD=00:00:20)


147(M=13.36; SD=3.57)

Google Docs Critique Acts

224(M=20.36; SD=6.62)




Team A 00:05:35 0:23:54 20 (18/1/1)* 00:01:12

Team B 00:08:44 0:12:06 10 (8/1/1) 00:01:13

Team C 00:07:20 0:20:42 14 (12/1/1) 00:01:29

Team D 00:05:59 0:09:41 14 (8/3/3) 00:00:42

Team E 00:07:26 0:19:57 11 (6/2/3) 00:01:49

Team F 00:06:03 0:17:38 17 (15/1/1) 00:01:02

Team G 00:08:06 0:15:01 17 (11/3/3) 00:00:53

Team H 00:07:56 0:15:18 15 (10/2/3) 00:01:01

Team I 00:06:18 0:12:52 8 (6/1/1) 00:01:37

Team J 00:05:46 0:11:41 12 (9/2/1) 00:00:58

Team K 00:06:17 0:14:09 9 (7/1/1) 00:01:34

TOTAL01:15:30 (M=00:06:52; SD=00:01:05)

2:52:59 (M=00:15:44; SD=00:04:22)

147 (M=13.36; SD=3.57)

(M=00:01:14; SD=00:00:20)


147(M=13.36; SD=3.57)

M = 1 minute, 14 seconds


224(M=20.36; SD=6.62)




Team A 00:05:35 0:23:54 20 (18/1/1)* 00:01:12

Team B 00:08:44 0:12:06 10 (8/1/1) 00:01:13

Team C 00:07:20 0:20:42 14 (12/1/1) 00:01:29

Team D 00:05:59 0:09:41 14 (8/3/3) 00:00:42

Team E 00:07:26 0:19:57 11 (6/2/3) 00:01:49

Team F 00:06:03 0:17:38 17 (15/1/1) 00:01:02

Team G 00:08:06 0:15:01 17 (11/3/3) 00:00:53

Team H 00:07:56 0:15:18 15 (10/2/3) 00:01:01

Team I 00:06:18 0:12:52 8 (6/1/1) 00:01:37

Team J 00:05:46 0:11:41 12 (9/2/1) 00:00:58

Team K 00:06:17 0:14:09 9 (7/1/1) 00:01:34

TOTAL01:15:30 (M=00:06:52; SD=00:01:05)

2:52:59 (M=00:15:44; SD=00:04:22)

147 (M=13.36; SD=3.57)

(M=00:01:14; SD=00:00:20)


147(M=13.36; SD=3.57)

M = 1 minute, 14 seconds


224(M=20.36; SD=6.62)

152.38% increase in critique capacity

What was the nature of critique content that the

advanced students generated?

3

GD CRITIQUE TURNS

M=37.59 words (SD=34.65)

135 design critiques (M=12.27; SD=5.56)

98 presentation critiques (M=8.91; SD=4.12)

Team C: “I think the “presentation preview” slide at the beginning was brilliant. You

could have very easily leveraged this to better include your research early on. Make

your research the equivalent of the “turn off your cell phones” announcement.

Whatever it may be, but briefly explaining that “the tech you are about to see is

fantastical, but be assured - it is feasible.” (TM: super awesome wicked great

suggestion)”

Team I: “Your presentation style (pace, confidence, etc.) is night and day between

the last time I saw you present. Nice job. (+1 UK, +1XE +1 DK)”

Team H: “Be careful about telling your stakeholders “I guess you guys aren’t aware

of this…” You might have a boss who gets upset at their assumed ignorance (+1

TQ, you don’t want to make them feel dumb or inadequate, you need buy in!)”

PRESENTATION CRITIQUE

Team C: "ok, textures? I am so confused, how on earth is this happening? Is this 3d glasses? I

may not have heard how this happens, but is this sort of like Disney that you have “robots” and

existing props that add this texture that you are talking about? ok, you talked about the

textured screens at the end. I would have liked to hear this as you go through your story

because it completely distracted me..."

Team F: “I’m concerned that your design is a bit magical. Your scenario is kind of perfect;

stressful person puts on magical jacket and suddenly everything is better! Stress and anxiety

doesn’t always arise, but can arise often from actual things going on in people’s life. Your jacket

doesn’t make less things due, or make people have less work, or make the trip suddenly over.

Just because you can pat people on the back doesn’t mean you’re going to make them less

stressed. (BX: I think of [other professor] and her issues with turbulence. She needs more than a

fake back pat)”

Team I: “The in-store has been handled decently enough, but my concerns are with the system.

How does this fit into my ecosystem? The stores? Food Companies? You say it is out of the

scope, but it is something that needs addressing as you are adding a brand new infrastructure

into stores.”

DESIGN CRITIQUE

NEGOTIATION THROUGH

BACKCHANNELING

“unfortunately I don't think it was stellar because

since it was backwards, the whole time I kept thinking

NO FUCKING WAY. WTF ARE YOU GUYS DOING

ARE OYU ON CRACK?”

“The message should be about the presentation

order. and i think the issues with scalability or how it

doesnt quite fit the body data stuff.”

DISCUSSION

DISCUSSION

mentors demonstrated an ability to manage anonymity and ownership effectively in digitally

mediated spaces

an expansion of critique allows for increased interaction and opportunity for crosstalk, even when

the physical space does not allow for it

multimodal critique encouraged socialization around design topics and negotiation of meaning—mirroring

the discursive quality of physically-mediated critique

mentors built skills as assessors and design professionals, clarifying their concerns in order to

reach consensus

colingray.me

THANK YOU