“spend another day in our class talking about this ...ahmerarif.com/papers/452.pdf · “spend...

ThisworkislicensedunderaCreativeCommonsAttribution-NonCommercial4.0InternationalLicense.

1

“Spendanotherdayinourclasstalkingaboutthisresearchplease”:Studentinsightsfromaresearch-baseddesignthinkingexercise

AtmanCynthiaJ.a,ArifAhmera,ShroyerKathrynE.a,TurnsJenniferA.a,Borgford-ParnellJimaaUniversityofWashington*Correspondingauthore-mail:{atman,ahmer,kshroyer,jturns,bparnell}@uw.edu

Abstract:Thispaperexploreshowguideddiscoverycanbeusedtoconnectinsightsfrom the ever-growingbodyof researchondesignprocesseswithdesign teaching.This paper focuses on a specific instance of a guided discovery activity in whichengineeringstudentswereinvitedtoengagewithselectedtimelinesfromastudyofdesignerprocesses;guidanceincludedpromptsattwopointsintime.Thegoalwasto see if the students could discovermeaningful insights about the design processandwhatfeaturesofdesignprocessescontributetoqualitysolutions.Thestudentsin this study succeeded in discovering six meaningful insights about the designprocess. The distribution of students’ insightswas not the same at the two time-points,suggestingthattheguidance is important inwhatstudentsdiscovered. Ourfindingsspeaktothevalueofthespecificguideddiscoveryactivitythatwestudied,andtheoverallideaofdevelopingactivitiesusingguideddiscovery.

Keywords:designpedagogy;guideddiscovery;designrepresentations;designexpertise

1.IntroductionAnever-growingbodyofresearchdocumentsinsightsintofeaturesofdesignprocessesthatcontributetosuccessfuldesignoutcomes.Intheirsynthesispaper“EngineeringDesignThinking,Teaching,andLearning,”Dymandhiscolleagues(2005)havecalledattentiontodivergent-convergentquestioning,thinkinginsystemsterms,anddecisionmakingasimportantaspectsofsuccessfuldesignprocesses.Inanotherexample,twooftheauthorsandtheircolleagueshaveinvestigatedhowtheamountoftimespentgatheringinformationandthefrequencyoftransitionsacrossdesignactivitiescontributetosuccessfuldesignprocesses(Atmanetal.,1999;Atmanetal,2007).Stillfurther,anexaminationofanyissue

AtmanCynthiaJ.,ArifAhmer,ShroyerKathrynE.,TurnsJenniferA.,Borgford-ParnellJim

2

inDesignStudiesinevitablyuncoversadditionalresearch-basedconsiderationsabouthowfeaturesofthedesignprocesscontributetosuccessfuldesignoutcomes.

Thispaperisconcernedwithhowsuchfindingscanbeleveragedtoinformdesignteaching.Onegeneralpathwaycouldbeembeddingthefindingsintheenvironment.Forexample,oneapproachwouldbetoprescribeadesignprocessthatdirectsstudentdesigners’attentiontowardstheveryactivitiesthathavebeenshowntobeimportantinthebodyofresearchondesignthinking.Anillustrationofthiscouldbeastudentdesignerbeinggivenasystemthathelpswitheffectivequestioning.Arelatedapproachwouldbetocreateresources,suchasvirtualagents,thatcouldremindadesignerwhentheyaredeviatingfromsuchdesirableapproachesinordertohelpthedesignergetbackontrack.Withthecurrentfocusonproblem-basedlearningandprojectworkforteachingdesign,theembedthefindingsintheenvironmentapproachmayseemfamiliar.Forinstance,muchoftheconversationaroundgettingprojectbasedlearningrightinvolvesnavigatingquestionsofhowtostructureanenvironmentsostudentssucceed.

Incontrast,anotherpathway(exploredinthispaper)involvesembeddingthefindingsintothedesigner’smentalmodelofthedesignprocess.Inotherwords,inthispathway,theideaistohelpthestudentdesignerunderstandresearch-basedfindingsaboutdesignthinkingsothatthefindingsarelateravailablewhentheyengageinmetacognitive-leveleffortssuchasplanningofdesign,reflectioninaction,andexecutivecontrol.Suchmetacognitive-leveleffortrelatestodesigners’abilitiesto“usedesignstrategieseffectively”—keyperformancedimensionsidentifiedintheextensiveliteraturereviewbyCrismondandAdams(2012,p.745).Withthispathway,ageneralquestionishowtohelpthestudentdesignercometounderstandandappreciatethesefindings.Astraightforwardstrategytoachievethisobjectivewouldbetopresentresearchfindingstostudentsoraskthemtoreadresearcharticles.However,thisstrategyisnotoftenreportedtobesuccessful.

Inthispaper,weexploreanalternativemethodofembeddingthefindingsintothedesignerthroughaformofguideddiscoverylearning.Discoverylearningisaninductivelearningapproachinwhichstudentsreceiveaproblemtosolvewithlittleornoguidancefromtheteacher.Guideddiscoveryisatypeofdiscoverylearninginwhichthestudentreceivesaproblemtosolvebuttheinstructoralsoprovidesfocus,coaching,feedback,andothersuchguidancetodirectthestudents(Mayer,2004).Guideddiscoveryattemptstoremediatethechallengesthathavearisenthroughcompletelyopendiscoverylearning(seeAlfierietal.,2011;Bruner,1961;DeanandKuhn,2007Kirschneretal.,2006;andMayer,2004)

Inthisguideddiscoveryactivity,studentsinteractwithresearch-basedvisualrepresentationsoffirst-year(freshmen)andgraduating(senior)studentdesignprocessestoseeiftheycandeveloppersonalinsightsaboutdesignthataresimilartofindingsdiscoveredbytheresearchers.Weprovideaworkedexampleforoneoftheconferencethemes,specificallyusingdesignresearchasanactiveforcethatallowsdesignstudentstorethinktheirideasaboutdesign.

“Spendanotherdayinourclassplease”:Studentinsightsfromaresearch-baseddesignthinkingexercise

3

Pragmatically,ourapproachdoesnotrequireachangingofprojectstructuresasanembedthefindingintheenvironmentapproachmight.Sinceprojectsarehardtostructure,itcanbechallengingtorestructuretheenvironment.Ourapproachprovidesstudentdesignerswiththeopportunitytodirectlyexploreresearchfindingssothattheycanlearnaboutimportantaspectsofdesignprocesses;aspectsthatmaycometopositivelyinfluencetheirownfuturedesignactivities.Giventhepotentialbenefitofsuchaneducationalactivity,wesetouttoanswerseveralquestions:

• Willstudentsdiscoveranyinsightsaboutdesign?Whatinsightsaboutdesignwilltheydiscover?

• Howcanwecharacterizethetypeofguidanceprovidedtothestudents?Whatwillbetheeffectoftheguidance?

• Whatwillbestudent’sreactionstotheirdiscoveriesandtotheexperienceofbeingaskedtodiscover?

Inthispaperweexplorethesequestionsthroughtheinstantiationof,andexperimentationwith,aguideddiscoveryactivity.Thepaperrepresentsanempiricalproof-of-concept,showcasingwhatsuchanactivitymightlooklikeandgatheringevidencethatdemonstratesthatsuchanactivitysuccessfullyhelpsstudentslearnaboutdesign.

Therestofthepaperproceedsasfollows.Firstwedescribeourspecificinstanceofguideddiscoveryofdesignprocesses.Thecontributionofthepaperisastudyoftheguideddiscoveryactivityinaction—themethodsandfindingsoccupythebulkofthepaper.Intheconclusionwediscussthesignificanceofthework.

2.TheResearch-BasedGuidedDiscoveryActivityResearchhasdemonstratedthatengineerswithdifferentlevelsofexperience(first-yearstudents,graduatingstudents,expertpractitioners)exhibitdifferentpatternsofdesignactivitieswhentheysolveadesignproblem(Atmanetal.,2007).Thesedifferencesaremadevisuallyapparentwhentheyaredisplayedastimelinesthatindicatethetimespentindifferentdesignactivitiesandthenumberoftransitionsamongdesignactivities.Timelinesfromsixengineeringstudents,threefirst-yearstudentsandthreegraduatingstudentsformthebasisoftheguideddiscoveryactivity(seeFigure1).


4

Figure1–Timelineactivityhandout.Timelinesrepresenttypicallow-performing,average-performing,andhigh-performingfreshmanandseniorengineeringstudents.

Thestepstotheactivityareasfollows(totaltimeis50to60minutes):

1. Aneducatorgivesabriefpresentationaboutthedevelopmentofthedesignprocesstimelinesusedinthetask(10minutes).

2. Studentsareeachgivenaworksheet(thetimelineactivityhandout)withquestionsonbothsides(Figure2).StudentsaregivenfiveminutestoindividuallyanalysethetimelinesandrespondtoPrompt1onthefrontofthesheet(Figure1).

3. Studentsthendiscusstheirresponseswiththeirprojectteams(10minutes).

4. Theeducatorthenleadsadiscussionwiththefullclassaboutstudentinsights.Additionalresearchfindingsarepresented,includingresultsfromasampleofexpertengineeringpractitioners,andthestatisticallysignificantresultsfromthecomparisonacrossthethreegroups(20to25minutes).

5. Studentsthenturnthepageoverandtake5to10minutestorespondtoPrompts2and3(seeFigure2foranexamplescanandtranscribedanswerfromastudentparticipantfromtheclassdiscussedinthispaper).


5

Figure2–ActivityworksheetandStudent22’sResponse


6

ThePromptsareasfollows:

Prompt1:ACTIVITY:Inthedesignprocesstimelinesshownabove,whatsimilaritiesanddifferencesdoyouseebetweenthefreshmenandseniorengineeringstudents?Dothesesimilaritiesalsoinvolvethequalityscores?Howso?

Prompt2:REFLECTIONACTIVITY:Whatwasthemostimportantthingyoulearnedtoday?Why?

Prompt3:REFLECTIONACTIVITY:Howcanweimprovethistalkforfutureaudiences?

Inthecaseofthisactivity,guideddiscoverylearningwasimplementedinsteadofpurediscoverylearningtoincreasethelikelihoodthatstudentswithlimiteddesignexperiencewouldbeabletouncoverinsightsinashortperiodoftime.Theamountofdata,representationofdata,andguidingquestionswerespecificallyscopedtodirectstudentsthroughtheactivitywhilestillallowingforautonomytodiscovertrends.Forexample,insteadofpresentingstudentswithalargeamountofrawdata,sixspecificdesignprocesseswerevisuallyrepresentedintheformoftimelines.

Ratherthansimplylisteningtoalectureorbeingprovidedwithageneralizeddesignprocessdiagram,studentswerepurposefullyguidedtoinductivelyuncovertrendsfromdesignprocessdata.ByexploringthetimelinesindependentlypriortorespondingtoPrompt1andsubsequentlywiththeirteammates,studentsdeveloprelevantinsightsintodesignprocesses.Followingthis,theeducatorthenfacilitatesadiscussionwiththefullclassandcomparesthestudentinsightstoresearchfindings.Students’finalperspectivesonimportantlessonsarethensolicitedwithPrompts2and3.Inotherinstanceswheretheauthorshavepresentedthisexercise,audiences(students,educatorsandpractitioners)haveidentifiedthestatisticallysignificantdifferencesacrossthetwostudentgroupsinthetimelinesjustfromthisguideddiscoverytask.Audiencememberobservationsarereinforcedwhentheylearnabouttheexperimentalresults,increasingbothconfidenceandexcitementwiththeirdiscovery.

Student22’sresponse,whichispresentedinFigure2,demonstratesboththebreadthanddepthofinsightsthatstudentscangainwiththisexercise.

3.Methods

3.1ParticipantsTwenty-fourmechanicalengineeringstudentsinathird-yearIntroductiontoDesigncourseatalargeresearchuniversityparticipatedinthisclassroomactivityinthespringof2015.

3.2CodeBookTheresponsesfromthetwenty-fourstudentswereanalysedbasedontheirwrittenresponsestoPrompts1and2.TheresponsestoPrompt3werenotincludedinthisanalysis(thepurposeofPrompt3istosolicitinputaboutteachingactivityratherthanstudent


7

learningaboutthedesignprocess).Theresponsestothefirsttwopromptswerecodedforpresenceorabsenceoftendesigninsightsineachresponse.Theseinsightswerebasedonninecodesusedinapreviousanalysisofthisactivity(Borgford-Parnell,2010).Anadditionalcode(Time)wasaddedandseveralcodeswereclarifiedresultinginafinalsetoftencodes.

Inourfindingsfromthisdataset,onlysixofthetencodeswereprevalent(Breadth,ProblemDefinition,GatheringInformation,Modelling,Iteration,andTime).Inthispaper,wethereforefocusonthosesixcodes.

CodeBook

Name Definition Examples–Prompt1 Examples–Prompt2

BreadthBreadth(accomplishingmoreofthedesignprocess)correlateswithhigherqualitydesign

“Highqualityscoreswerevariedinthedesignprocess,withallstepsprevalentthroughoutthetimeline”-17

“Toproduceagooddesignyoumustsplityourtimemoreevenlybetweenthedifferentstages.”-11

ProblemDefinition

Moretimespentonproblemdefinitionactivitycorrelateswithhigherqualitydesign

“Betterscoresreturnedtotheproblemdefinitionthroughouttheprocess…”-2

“Alwaysgobacktotheproblemandaskyourselfifyouarereallysolvingit.”-11

GatheringInformation

Moretimespentongatheringinformationactivityequatestohigherqualitydesign

“Somethingthatseemedtopromotehighqualityworkwastocontinuetokeepgatheringinformationandgeneratingideasthroughoutthewholeprocess.”-12

“Lastlyitisimportanttocontinuetoaskquestions&gatherinformationthroughouttheprocess.”-23

Modeling Everyonespendsthemosttimeinmodelingactivity

“Modelingtookupthemajorityofthetimeformostpeople.”-3

“…thoughevenexpertsspendthemosttime/effortintomodeling.”-16

IterationIterationand/ortransitionscorrelatewithhigherqualitydesign

“Thehighqualitydesignsbouncedaroundalot…”-19

“Thedesignprocessismosteffectivewhenundertakenasanonlinear,cyclicalprocess.Thedifferentactivitiesor‘phases’ofthedesignprocessshouldbeintegratedandrevisitedthroughoutdesigning.”-6

TimeTotaltimespentonthedesignprocesscorrelatestohigherqualitydesign

“Studentsthatspentlesstimehavelowerqualityscore.”-10

“I[learned]thatputtinginenoughtimeisvaluableuptoacertainpoint…”-15

Figure3-Descriptionsandexamplesofthesixcodes

3.3CodingProcessThecodingprocesswascarriedoutbytwotrainedresearchersthatindependentlyassigneddesigninsightcodesatthesentenceorsentencegrouplevelsforeachofthe24setsofstudentresponses.Sincewecodedresponsestotwoquestionpromptsforeachstudent,theactualnumberofresponsesthatwereanalysedwas48.Thiscodingprocessinitiallybeganusingtheaprioricategoriesthatweredevelopedusingabottom-upapproachinapreviousstudythatisextendedbythisresearch(Borgford-Parnell,2010).Theresearcherstrainedonaseparatesetofstudentresponses.Oncefinished,thetworesearcherscomparedresults,arbitratedanydiscrepanciesonthistrainingsettoagreement.Minorrefinementsweremadetotwoofthecodedefinitionsanda10thcode(Time)wasadded.


8

Aftertrainingwascomplete,theresearchersreplicatedthisprocessontheforty-eightstudentresponsesusedinthisstudy.Afterthisfirstiteration,theresearchersleveragedtheiremerginginsightstomakeminorrefinementstofourofthesecodesforthesakeofspecificityandinclusiveness.Additionally,theresearchersnoticedpatternsofdifferencesinthelevelofstudents’engagementandinsightsbeingdemonstratedwithinthetexts.Opportunistically,theresearchersbeganlookingforthosepatternsandannotatingforstatementsthatseemedtogobeyondtheacquisitionofdesigninsights.Statementsinthiscategoryhadtoindicatethatthestudentwasnowmorecuriousaboutdesignorwassomehowrelatedtotheirself-identityasadesigner.Theresearchersreported22agreementsoutof24judgementsforthisBDIcode(BeyondDesignInsights)andconsequently,wehavechosentoincludeitaspartofourdiscussionduetosomeoftheinsightsitsurfaced.

Uponcompletingtheiranalysis,theresearcherscomparedresultsandarbitrateddiscrepanciestoagreement.Twojudgements(outof480)requiredexternaladjudicationthroughamajorityvotebyathirdresearcher.Forthe6mostprominentinsightsdiscussedinthispaper(atotalof288judgements)26werebrieflydiscussed,10warrantedfurtherdiscussionorclarification,and1requiredadjudicationbyathirdresearcher.Priortothisarbitration,theinitialinter-raterreliabilityforthesesixcodeswasalsocomputedusingCohen’sKappa(1968)andfoundtobe0.78,0.59,0.88,0.85,0.83and0.86respectively.

4.FindingsDespitetherelativelyshortnatureofthedesigntimelineexercise,itprovokedavarietyofstudentreactions—manyofwhichwerecharacterizedbybuildingastrongerappreciationforconsideringafullerrangeofdesignactivities.Inwhatfollows,wefirstdescribethehighlevelresultsofouranalysis,includingattentiontohowstudentreactionsshiftedbetweenthetwoprompts.Wethendescribetheinsightsthatstudentstakeawayfromtheexercise.

Figure4showsthedistributionofthe89designinsightsthatresearchersobservedacrossthe48studentresponsesthroughouttheactivitybycategory.

Figure5providesadditionaldetailontheseresultsbyshowingthenumberofstudentsreportingeachinsightforPrompt1(thelightbars)andPrompt2(thedarkbars).RecallthatPrompt1capturesthestudents’initialinsightswhilePrompt2isdesignedtocapturefinaltakeawaysafterapresentationofresearchfindingsandagroupdiscussion.AllbutfiveofthestudentsaddedatleastonenewideatotheinsightstheyincludedintheirresponsetoPrompt2.Asthefigureshows,someinsightssuchasBreadthandGatheringInformationwerementionedwithsimilarfrequencyinresponsetoPrompt1andPrompt2.Otherinsights,suchasIteration,ProblemDefinition,Modelling,andTimeweredifferentiallyprominent.


9

Figure4-Thetotalnumberofstudentinsightsbycategory

Figure5-Thenumberofstudentinsightsbycategoryandprompt

Below,weprovideadescriptionofthestudentresponsesthatcomprisethesixcodedcategories.WealsopresentourfindingsfortheBDI(BeyondDesignInsights)codeinthissection.Thestudentquotesthatwehighlightinthissectionhavebeenselectedforbeingrepresentative,interestingandfortheirillustrativepower.

0 4 8 12 16 20 24

Modeling

ProblemDe9inition

Time


Iteration

Breadth

NumberofStudentswhoExpressedDesignInsight

DesignIn

sight

NumberofStudentswhoExpressedDesignInsight(forwholeexercise)

0 4 8 12 16 20 24

Modeling

ProblemDe9inition

Time


Iteration

Breadth

NumberofStudentswhoExpressedDesignInsight

DesignIn

sight

NumberofStudentswhoExpressedDesignInsight(forPrompt1andPrompt2separately)

Prompt1Prompt2


10

4.1StudentInsight:Timematters(Time)Oneoftheinitialinsightsreportedbyfourteenofthestudentsafterexaminingthedesigntimelinesrevolvedaroundequatingoveralltimespentwithgoodqualitydesign.Thiswasgenerallyexpressedinafairlysuccinctmannerbystudentswithstatementssuchas“themoretimeyouspend,themorelikelyyoucangetagoodscore”and“studentsthatspentlesstimehavealowerqualityscore”(students4&20).Interestingly,whilstfourteenstudentsconsideredthisanotableobservationtoshareinitially,onlytwoofthemchosetodiscussitasapointoflearninglateroninresponsetoPrompt2.Thiswouldseemtoindicatethatmanyofthesestudentsdidn’tconsiderwhattheycouldmakeofthisobservationtobeatakeawayimportantenoughforthemtoincludeastheirresponsetoPrompt2.

Threeofthestudentsalsofurtherqualifiedtheirobservationsbynotinghowtimespentstopsbeingafactorpastacertainpoint.Forinstance,student15wrotehowhe“learnedthatputtinginenoughtimeisvaluableuptoacertainpointbutthenamoreiterativedesignprocessbecomeshelpful”.Thesenuanceseffectivelyforeshadowtheotherthemeswhichthestudentswroteaboutregardinghowtimeshouldbespentindesignwhilestillpayinghomagetotheideathatacertainamountoftimeexpenditureisinevitableifqualityisdesired.

4.2StudentInsight:Timespentgatheringinformationraisesquality(GatheringInformation)Withinthethemeofhowtimeshouldbespent,twothirdsofthestudentswroteabouthowdesignerswith“goodscoresgatheredinfothroughouttheprocess”.Therelativelyevensplitofresponses(eleveninPrompt1andonlyanadditionaltwoinPrompt2)suggeststhatstudentscametothispointbythemselvesandwereattentivetoitssignificance.

Oneminordistinctionthatwecouldseeinthestudentresponsescamefromhowthemajorityofthemsawhighscorersandexpertsgatheringinformationcontinuouslythroughoutthedesignprocesswhileothersperceivedthattheinformationgatheredinanearlierphasecarriedmoreweight.Sowhilethreestudentswrotethattheylearned“towaituntilmoreinformationisgathered...tostartmodelingideas”(student8),otherswroteofhavinglearnedthatthereisnoendpointtotheinformationgatheringphase.Forexample,student5wrotethattheylearnedto“continuegatheringinformation,eventowards[the]end”,whilestudent7expressedthatshewould“trymyhardesttoavoidmodelingtooearly,andalsowhenIstartmodelingto‘comeupforair,’sotospeakandtakealookattheinformationgivenandmaybetheinformationnotgiven”.

4.3StudentInsight:Keeprevisitingtheproblem(ProblemDefinition)Interestingly,whileinformationgatheringandproblemdefinitionfrequentlygohandinhandasdesignactivities,therelationshipbetweendesignqualityandproblemdefinitionactivitieswastoucheduponbycomparativelyfewerstudents.OnlytwostudentsbroughtuptheproblemdefinitionactivityasafactorindesignqualityscoresinresponsetoPrompt1,whileanotherninedidsoafterdiscussioninPrompt2.Thissharpdifferencewouldindicatethat


11

observationalonewasnotenoughforstudentstomakealinkbetweenproblemscopingandfinaldesignqualityandthatdiscussionsbroughtthestudentstosurfacecertainkindsofinsights.Thiswasfurtherreinforcedbytherelativelyuniformtoneofstudentresponseswhichgenerallyexpressedthisthemeintermsofrevisitingproblemdefinition:“Itiseasytogetcaughtupinmodelingbutifyoudon'tgobacktotheproblem,youwillmostlikelyfailtomakeagooddesign”(student4).Thislackofvariationsuggeststhattheclassdiscussionpossiblyhelpedshapethestudentresponsesatthelanguagelevel.

4.4StudentInsight:Everybodyspendsthemosttimemodellingthesolution(Modelling)Significantly,manystudentresponsesseemedtorecognizethatmodelling—theactivitywhichconsumedthemostdesigntime—wasnegativelycorrelatedwithquality.Forinstance,student16offeredaconclusionthat“themostimportantthingIlearnedtodaywasthatwhenyoudesignaproduct,anidea,oranything,youhavetotrytonotonlyfocusonmodeling”.Sevenofthestudentswereparticularlyattentivetohowthetimelinesshowedmodellingasthemosttimeintensiveactivityformostdesignersinthesample.Forexample,students16and3respectivelynotedthat“evenexpertsspendthemosttime/effortintomodeling”and“modelingtookupthemajorityofthetimeformostpeople”.Thisobservation—thatnobodygetsawaywithnotdoingasignificantamountofsolutionmodellingduringdesign—canpotentiallyhelpstudentsdemystifyandbetteridentifywiththeworkprocessesofmoreexperienceddesigners.

4.5StudentInsight:Breadthisthekey(Breadth)Althoughstudentslinkedseveraldesignprocessactivitiestodesignquality,themostprolificinsightemergedfrom22ofthe24studentslinkinghighscoreswithaholisticdesignprocessintermsofactivities.Ineffect,studentswerenotinghowthemoreanactivitysuchasmodellingappearedasacontinuousblockonthetimelines,thelesstimewasthenavailableforotherimportantactivities.Theequalsplitandoverlapofstudentswhochosetocoverthisthemeintheirinitialobservationsversustheirchieftakeawayswouldindicatethatthisinsightwasbothreadilyapparentandveryimportanttothestudents.

Therichdiversityofexpressionusedaroundthiscodeisanotherinterestingfeaturethatwarrantselaboration.Forinstance,somestudentslikestudent12wroteaboutnot‘gettingstuck’on‘oneidea’orphasewhilereflectingupontheirowndesignprocess:“[Ilearned]thatspendingtimeonallaspectsofdesigniscriticalforaqualityproduct…Thisissoimportanttomebecauseitiseasytogetexcitedaboutacertainideaandforgetwhatthemainthingwasabout.”Otherstudentsofferedtheirconclusionsthroughideasregardingevendistribution,revisiting,balance,andintegrationwithrespecttodesignactivities.Itisdifficulttofirmlyconcludewhetherthiswidervarietyofphrasesindicatesadeeperormoregenuinelevelofengagement,butneverthelesswefoundthiscombinationoffrequencyanddiversitytobetelling.


12

4.6StudentInsight:Iterationandtransition(Iteration)Thesecondmostsystematicallyprevalentsetofinsightsarosefromnineteenofthestudentswritingabouthowgooddesignprocessesareiterativeinnature.Studentsexpressedthisconceptinavarietyofways.Someresponses,likestudent17’sreferredtocyclingorrepetition:“Thereseemstobeaclearcorrelationbetweenthequalityofthedesignandtheabilityofthedesignertokeepcyclingthroughallstepsoftheprocess”.Otherresponsesusedtermssuchas‘jumpingbackandforth’(student4),‘goingcircular’(student21),being‘nonlinear’(student19),etc.

Manystudentsalsoexplicitlycalledoutthepatternstheyobservedinhighqualityscoresasiterative.Forinstance,student3referredtoiterationdirectlyinthecontextofreflectinguponhisowndesignprocess:"Alsoseeingtheiterativestepsthatweremadewillmakemefeelbetteraboutsteppingbackandlookingeverythingover".SinceonlyeightresponsescontainedthiscodeinPrompt1versusseventeeninPrompt2,itseemsthatalthoughthiswasavaluabletakeawayformanystudents,itwasnotinitiallyapparent.Thatsomeamountoftimeordiscussionwasprobablyhelpfultosurfacingthisinsightcouldbeexplainedbythefactthattheiterationpatterncouldonlybeobservedafterabsorbingallsixtimelinesandsteppingbackforsynthesis.

4.7BeyondDesignInsights(BDI)Apartfromtheaboveinsights,morethanathird(10outof24)ofthestudentresponsescontainedelementsthatextendedbeyondlearningaboutdesignprocesses.Inmanyways,thesewerethestatementsthatthestudentswrotewhichdidnotdirectlyaddressthepromptstheyweregiven,butpointedtoreactionsthatweconsidertobesignificant.Unsurprisingly,mostoccurrencesofthisBDIcodeshowedupinthesecondresponsethatwassolicitedfollowedaclassdiscussionthatfeaturedresearchfindings,furthersupportingourfindingsconcerningthevalueofguideddiscovery.Inthisprompt,thestudentswereaskedtoengageinareflectiveactivitywheretheyselectedthemostimportantthingtheylearnedintheexercise.However,therewereanumberofinstancesevenduringthePrompt1responseswheresomestudentsmovedpastthedataobservationpromptandplacedthemselvesinthesituationofthedesignersintheexerciseorattemptedtogiveadviceonwhatnottodo.Forinstance,student13’sresponseendedwith“formeIwouldspendmoretimewith[the]decision[activity]thantheothersdid”.

OneofthemostfrequentlyexpressedBDItakeawaysatanoveralllevelwasanappreciationfor‘showdon’ttell’,whichtiedintoseveralotherthemes.Forinstance,thequoteforthetitleofthispapershowsthisparticipantexpressingthisappreciationwhiletyingitintowhatweinterpretasakindofpreparationforfuturelearning:“Supervaluable!Muchmorecompellingtoseerealdata,detail,makesmebelieve,insteadoftuningout'prescribed'info,can'ttrusthowtheyderiveditb/cdon’tknow.Spendanotherdayinourclasstalkingaboutthisresearchplease!”(student22).Elsewhere,studentresponsesalludedtohowsteppingoutsideoftheseprocessesandreflectinguponthemmightaffecttheirconfidenceorfuture


13

decisionmaking.Forexample,student3wroteofhavinglearned“howotherpeoplespendtheirtimeindesign.Ididn'thaveanyideahowothersdidit.SeeingwhatpartswerethemostimportantforthequalityscorewilldefinitelyshapethewayIdesigninthefuture”,whilestudent10noted,“Inowrealizethatspendingtimeatthisprocesswillpayoff”.

Figure6–Partialscansofstudent7and22’swrittenannotationsuponthedesigntimelinehandouts

Reviewingstudentresponsesfromthisbroaderperspectivealsohelpedusnoticehowstudentsweremakingmeaningofthetimelinesoutsideoftheprescribedspacesandpromptsthattheyweregiven.Someclearinstancesofthiscamefromthreeofthestudentsannotatingandsketchingdirectlyontheircopiesofthedesigntimelinestodiscoveradditionalinsights.Forinstance,intheimagescansabove(Figure6),wecanseestudent7attemptingtomapdesigntoashapebyconsideringthegeometricgradientorslopeofsenior3’sdesignprocess.Likewise,student22attemptstoprojectcertainpatternswithinthetimelinesontodesignbehaviourssuchaspivoting.Thesemorevisualattemptsatdiscoverycanhelpusappreciatethedifferentmodalitiesandbehavioursstudentswereengagingintoarriveattheirdiscoveries.

5.DiscussionThestudentsinthisstudysucceededindiscoveringsixmeaningfulinsightsaboutthedesignprocess,asdescribedinthefindingssection.Thestudents’abilitytoarriveatsuchinsightsconfirmsthatguideddiscoverywaspossibleandcorroboratespreviouswork(Borgford-Parnell,2010).Asalludedtointheintroduction,webelievethatstudentsdiscoveringandarticulatingtheirinsightswillbehelpfulfortheminfuturedesignactivities.Specifically,beingabletorecognizeeffectivestrategiesmeansthatthereispotentialforthemtomonitorandadjusttheirowndesignactivitywithincreasedconfidence.Forexample,seenfromahighlevelperspective,thestudents’insightsfocusonthesignificanceofspendingtimeandhowthattimeisspent.Implementingthatinsightmaypositionstudentstomonitorhowtheyarespendingtime,recognisewhentheyarenotspendingtimeeffectively(e.g.,spendingagreatamountoftimeinmodelling), andmakeexecutiveleveldecisions


14

abouthowtospendtimegoingforward(e.g.,decidingtorevisittheproblemdefinitionorgathermoreinformation).

Thedistributionofstudents’insightswasnotthesameforthetwoprompts.Prompt1guidedstudentstoarticulatetheirobservations,andPrompt2guidedthemtoreflectandarticulatewhatwasmostimportanttothem.Examiningthefindings,wecanseethattherewerethreepatternsofchange:(a)aninsightbeingprevalentforPrompt1andnotprevalentatPrompt2(e.g.,Time),(b)aninsightnotbeingprevalentforPrompt1andbeingprevalentforPrompt2(e.g.,IterationandProblemDefinition),and(c)abalancedpatterninwhichaninsightwasequallyprevalentforbothprompts(e.g.,BreadthandGatheringInformation).Surfacingthesepatternsofchangehelpsusbetterunderstandtheguideddiscoveryprocessovertimeintermsofwhatwaseasyforindividualstudentstodiscoverearlyonversuswhattheycoulddiscoverwithdiscussionswithpeersandtheeducator.Additionally,webetterunderstandwhatdiscoveriesstudentsrecognizeearlyonbutlaterreplacebymoreinterestinginsights,ascomparedtostudentsearlydiscoveriesthatcontinuetoretaintheirimportance.Thiswayofframingdifferenttypesofguidancecanhelpwiththedesignoffutureguideddiscoveryactivities,andalsoraisesquestionsabouthowdifferentpromptsmightfacilitatedifferentdiscoveries.

Asthedescriptionsofthestudentcommentsthatmadeupthecontentofthecodecategoriesindicate,therewasanindicationthatforsomestudents,theirinsightsweresomewhatpersonalinnature(forexample,insightsrelatedtoidentity,toimaginedfuturedesignactivities,andtoasenseofpersonalrelevance).Thisisinterestingbecausesuchpersonalpositivereactionmayincreasethelikelihoodthattheseinsightsarenotephemeral,butrather,becometheprinciplesthatstudentslaterusetogroundtheirowndesignprocesses.Interestingly,thepromptof“themostimportantthingyoulearned”doesnotnecessarilyfacilitatesuchpersonalreactions—learningforstudentscanoftenbeassociatedwiththetypeofknowledgethatistestedbytestsandexplainedintextbooks.Giventhatthepromptdidnotaskstudentstoprovideinsightintotheirpersonalreactions,theobservationthatone-thirdoftheresponsesfeaturedsuchapersonaldimensionmightactuallybeanunderestimationofthepersonalreactionsbeingfeltbythestudentsinthisproject.Thisleadsustowonderwhatwemighthaveobservedifthepromptshadbeendifferent.Forinstancewemighthavesurfacedevenmorepersonalinsightswithpromptssuchas“Whatwouldyoudodifferentlyifyouwereinafuturedesignsituation?”or“Doyouthinkofyourselfasadesigner?”or“Didtoday’sactivitiesinfluenceyourthoughtsaboutyourselfasadesigner?”

6.LimitationsandFutureResearchOnelimitationofthestudyisthatourunderstandingofwhatthestudentsdiscoveredismediatedentirelythroughwhattheywroteonthesheetofpapertheyweregiven.Seeingthatsomestudentsactuallymadeannotationsdirectlyonthetimelinesthemselvessuggeststhatwealsoneedtoconsiderotherartefactsandwaysthatstudentsmighthavebeen


15

engagingwiththetimelines.Onepossiblemeansofcapturingsomeoftheseadditionalinteractionswouldbetovideo-recordtheinstructionalevent.Thiswouldmakeitpossibletodiscernanyunscriptedquestionsandinteractionsthattookplacebetweenthestudentsandtheeducator.Additionalattentiontothoseinteractionscouldhelpinscriptingtheguidanceprocesssothatitisdoneinamoreexplainableandrepeatablemanner.Finally,giventhetime-constrainednatureoftheactivity,itwasnotpossibletofollowthestudentsintoanactualdesignactivityandseeiftheirnewinsightshelpedthembesuccessful.Futureresearchcanexplorewhetherthestudentinsightsfromtheshorterexercisedescribedinthispaperwillbehelpfulforstudentsoveralongerterm.

7.ConclusionInthispaper,wedocumentwhatwelearnedaboutusingguideddiscoverytohelpstudentdesignersdevelopinsightsaboutdesignprocessesthatarebasedondataandfindingsfrompriorresearch.Thispapercontributestotheevidence-baseforthespecificguideddiscoveryactivitythatislaidouthere.Ourfindingsleadustobelievethatdesigneducatorswhotrytorepeattheactivitywiththeirstudentsmaydiscoverthatadditionalaffordancesoccurwhenthereissharedvocabularyandsharedreferenceforideassuchasproblemsolvinganddesignprocessovertime.Thisactivityrepresentsoneexampleofcreatinganactivitywherestudentsinteractwithrealdataandhavetheopportunitytodiscoverpatternsthattheymaylaterleverageintheirowndesignwork.Seenasanexample,thiscanhelpothereducatorscreatesimilarguideddiscoveryactivitiesandsuggestfutureresearchdirectionsintotheuseofguideddiscovery.

Acknowledgements:WewouldliketogratefullyacknowledgeTerriLovinsforherassistanceonthispaperandthestudentswhoparticipatedinthisstudy.ThisworkwassupportedbytheNationalScienceFoundation(grantsRED-9358516andROLE-0125547),theBoeingCompany,theCenterforEngineeringLearning&TeachingattheUniversityofWashington,theMitchellT.andLellaBlancheBowieEndowment,andTheLeonaM.andHarryB.HelmsleyCharitableTrust.

8.ReferencesAlfieri,L.,Brooks,P.J.,Aldrich,N.J.,&Tenenbaum,H.R.(2011).Doesdiscovery-basedinstruction

enhancelearning?JournalofEducationalPsychology,103(1),1.Atman,C.J.,Adams,R.S.,Cardella,M.E.,Turns,J.,Mosborg,S.,&Saleem,J.(2007).Engineering

designprocesses:Acomparisonofstudentsandexpertpractitioners.JournalofEngineeringEducation,96(4),359–379.http://doi.org/10.1002/j.2168-9830.2007.tb00945.x

Atman,C.J.,Chimka,J.R.,Bursic,K.M.,&Nachtmann,H.L.(1999).Acomparisonoffreshmanandseniorengineeringdesignprocesses.DesignStudies,20(2),131-152.

Bruner,J.S.(1961).Theactofdiscovery.HarvardEducationalReview31(1):21–32.Borgford-Parnell,J.,Deibel,K.,&Atman,C.J.(2010).Fromengineeringdesignresearchto

engineeringpedagogy�:Bringingresearchresultsdirectlytothestudents.InternationalJournalofEngineeringEducation,26(4),748–759.


16

Crismond,D.P.,&Adams,R.S.(2012).Theinformeddesignteachingandlearningmatrix.JournalofEngineeringEducation,101(4),738.

Cohen,J.(1960).Acoefficientofagreementfornominalscales.EducationalandPsychologicalMeasurement,20(1),37-46.

DeanJr,D.,&Kuhn,D.(2007).Directinstructionvs.discovery:Thelongview.ScienceEducation,91(3),384-397.

Dym,C.L.,Agogino,A.M.,Eris,O.,Frey,D.D.,&Leifer,L.J.(2005).Engineeringdesignthinking,teaching,andlearning.JournalofEngineeringEducation,94(1),103-120.

Kirschner,P.A.,Sweller,J.,&Clark,R.E.(2006).Whyminimalguidanceduringinstructiondoesnotwork:Ananalysisofthefailureofconstructivist,discovery,problem-based,experiential,andinquiry-basedteaching.EducationalPsychologist,41(2),75-86.

Mayer,R.E.(2004).Shouldtherebeathree-strikesruleagainstpurediscoverylearning?AmericanPsychologist,59(1),14.

AbouttheAuthors:

CynthiaJ.AtmanisdirectoroftheCenterforEngineeringLearning&TeachingandprofessorofHumanCenteredDesign&Engineeringatthe University ofWashington. Her research focuses on engineeringdesign learning, reflection, and considering context in engineeringdesign.

AhmerArif isaPhDstudentat thedepartmentofHumanCenteredDesign&EngineeringattheUniversityofWashington.Hestudiesthedesignofdigitaltoolstosupportreflectivepractices.

Kathryn E. Shroyer is a PhD student in the department of HumanCenteredDesign&EngineeringattheUniversityofWashington.Herresearchinterestsincludeengineeringeducationanddesignthinking,teaching,andlearning.

Jennifer A. Turns is a professor of Human Centered Design &Engineeringat theUniversityofWashington.Her research interestsinclude engineering education, reflection, helping engineeringeducators, and application of ideas from complexity science to thechallengesofengineeringeducation.

Jim Borgford-Parnell directs the Office for Advancement ofEngineeringTeaching&LearningattheUniversityofWashington.Hetaughtdesign,research,educationtheory,andpedagogycoursesforover 35 years. He currently does research and instructionaldevelopmentinengineeringeducation.

“spend another day in our class talking about this ...ahmerarif.com/papers/452.pdf · “spend...

Documents