Upload File

the nm eval lab’s goal 1.1: “to support...

  • Home
  • Documents
  • The NM Eval Lab’s Goal 1.1: “To support studentsevallab.unm.edu/learning-center/16-17-focus-group.pdf1 Summary of Findings for Eval^2 Focus Group with Fellows May 11, 2017 Charla
8
1 Summary of Findings for Eval^2 Focus Group with Fellows May 11, 2017 Charla Henley Senior Fellow of the Evaluation Lab Background: The Eval^2 focus group was performed on May 4, 2017 with 12 of 12 students from this year’s cohort of Evaluation Lab Fellows. One student left early. The focus group was interactive in nature, so activities involved a variety of expression methods such as drawing, speaking, sharing with a partner, and individual writing. Questions were created to evaluate the following goal: The NM Eval Lab’s Goal 1.1: “To support students in developing the knowledge, skills, and tools they need to design and implement program evaluations that reflect organizations’ values and missions.” Collaboratively, we decided on the following: a) What did the course feel like for students? b) What did students experience in course/curriculum vs. in practicum? c) What did students experience in relationships with organizations? What were the challenges and benefits? d) What did students learn from their mentors? e) What was the experience of working as a team member? (Was everyone valued, did everyone contribute?) f) What skills can be attributed to the students’ participation in the lab? g) Would students recommend the program to other students? h) How could it be better? Findings: a) What did the course feel like for students? The answer to this question was achieved through a drawing activity, where students could draw what the course felt like on a gingerbread person. Overall, students either felt that the course was overwhelming and confusing (often represented with question marks over the eyes or brain, or the brain be “fried”), represented by Figure 1. or they felt empowered, engaged, and stimulated by the course (represented by smiles, hearts, and feelings of success) as shown in Figure 2. In some cases, illustrated in Figure 3., students felt confused at first, but empowered in the second semester. Most students mentioned stress and exhaustion.

Upload: others

Post on 05-Sep-2020

0 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: The NM Eval Lab’s Goal 1.1: “To support studentsevallab.unm.edu/learning-center/16-17-focus-group.pdf1 Summary of Findings for Eval^2 Focus Group with Fellows May 11, 2017 Charla

1

SummaryofFindingsforEval^2FocusGroupwithFellowsMay11,2017CharlaHenley

SeniorFellowoftheEvaluationLabBackground:TheEval^2focusgroupwasperformedonMay4,2017with12of12studentsfromthisyear’scohortofEvaluationLabFellows.Onestudentleftearly.Thefocusgroupwasinteractiveinnature,soactivitiesinvolvedavarietyofexpressionmethodssuchasdrawing,speaking,sharingwithapartner,andindividualwriting.Questionswerecreatedtoevaluatethefollowinggoal:TheNMEvalLab’sGoal1.1:“Tosupportstudentsindevelopingtheknowledge,skills,andtoolstheyneedtodesignandimplementprogramevaluationsthatreflectorganizations’valuesandmissions.”Collaboratively,wedecidedonthefollowing:

a) Whatdidthecoursefeellikeforstudents?b) Whatdidstudentsexperienceincourse/curriculumvs.inpracticum?c) Whatdidstudentsexperienceinrelationshipswithorganizations?Whatwerethe

challengesandbenefits?d) Whatdidstudentslearnfromtheirmentors?e) Whatwastheexperienceofworkingasateammember?(Waseveryonevalued,did

everyonecontribute?)f) Whatskillscanbeattributedtothestudents’participationinthelab?g) Wouldstudentsrecommendtheprogramtootherstudents?h) Howcoulditbebetter?

Findings:

a) Whatdidthecoursefeellikeforstudents?Theanswertothisquestionwasachievedthroughadrawingactivity,wherestudentscoulddrawwhatthecoursefeltlikeonagingerbreadperson.Overall,studentseitherfeltthatthecoursewasoverwhelmingandconfusing(oftenrepresentedwithquestionmarksovertheeyesorbrain,orthebrainbe“fried”),representedbyFigure1.ortheyfeltempowered,engaged,andstimulatedbythecourse(representedbysmiles,hearts,andfeelingsofsuccess)asshowninFigure2.Insomecases,illustratedinFigure3.,studentsfeltconfusedatfirst,butempoweredinthesecondsemester.Moststudentsmentionedstressandexhaustion.

Page 2: The NM Eval Lab’s Goal 1.1: “To support studentsevallab.unm.edu/learning-center/16-17-focus-group.pdf1 Summary of Findings for Eval^2 Focus Group with Fellows May 11, 2017 Charla

2

Figure1.Student’sexperienceincourse(confusingandoverwhelming)

Figure2.Student’sexperienceincourse(empowering&stimulating)

Page 3: The NM Eval Lab’s Goal 1.1: “To support studentsevallab.unm.edu/learning-center/16-17-focus-group.pdf1 Summary of Findings for Eval^2 Focus Group with Fellows May 11, 2017 Charla

3

Page 4: The NM Eval Lab’s Goal 1.1: “To support studentsevallab.unm.edu/learning-center/16-17-focus-group.pdf1 Summary of Findings for Eval^2 Focus Group with Fellows May 11, 2017 Charla

4

Figure3.Student’sexperienceofthefirstsemestervs.secondsemester

Thegingerbreadpersonactivitybroughttolighttheknowledgedifferencebetweenallparticipants,whichwasmentionedagaininthefocusgroupactivitiesthatfollowed.b) Whatdidstudentsexperienceincourse/curriculumvs.inpracticum?Studentsfrequentlyhighlightedguestpresenters,conferences,workshops,andpersonalfeedbackfromSoniaandMelissaonprojectsasbenefitstothecurriculumportionoftheEvalLab.Withinthecurriculum,studentscommentedthatthediscussionboardwasnotas

Page 5: The NM Eval Lab’s Goal 1.1: “To support studentsevallab.unm.edu/learning-center/16-17-focus-group.pdf1 Summary of Findings for Eval^2 Focus Group with Fellows May 11, 2017 Charla

5

helpfulandthathavingadiversecohortmeantstudentswereatdifferentlevelsofunderstanding(somethinkitwasboring,othersthinkitwasoverwhelming).Duringthepracticumportionofthecourse,studentsenjoyedworkingandbuildingrelationshipswithorganizationsandhavinginformationentrustedtothem.However,studentsfeltoneofthemoredifficultpartsofthepracticumcamefromorganizationwithintheorganizations(asintheorganizationscanbeorseemdisorganized).Overall,fellowsreportedcommunicationbetweenteamsandorganizationsasthenumberoneissueinthepracticum.c) Whatdidstudentsexperienceinrelationshipswithorganizations?Whatwerethe

challengesandbenefits?Byworkingwithcommunityorganizations,studentsgainedpractical/realworlduseforacademic/evaluationresearch,wereabletobridgetheiracademictrainingintothecommunity,andfeltproudtohaveanimpactonthecommunity.Studentsfeltthebiggestchallengeintherelationshipwiththeorganizationswasthetimeschedules(thefactthatorganizationsarebusyandschedulingwashard).Somealsonotedthatitwashardtofigureoutwhattheorganizationsreallywantedevaluated.d) Whatdidstudentslearnfromtheirmentors?Overall,studentsrespondedverypositivelyabouttheirmentors,mentioningthattheirmentorshelpedthemimprovetheirwritingskills,professionalism,timemanagement,andpresentationskills.Theirbiggestconcernsaboutthementoringprocesswerethatitwassometimesunclearwhowasresponsibleforwhatandsomefeltthatallmembersoftheirteamwerenottreatedequally.Onestudentmentionedfrustrationwiththementorbysaying,“[Iwas]frustratedbythetreatmentfrommentorinteam.[I]feltsheoversteppedmeeverytimeandtreatedmeassomeonewhowasunintelligent.”Therewasnocommentonhowtoalleviatethisinthefuture.e) Whatwastheexperienceofworkingasateammember?(Waseveryonevalued,did

everyonecontribute?)Inmostcases,responsesaboutworkingasateamwerepositive.Inthepositiveresponses,studentsmentionedlearningfromtheirteam,balancingthework,andfeelingliketheywerecontributingandsupported.However,therewereafewstudentswhoreportedthattherewasfrustrationbecausetheyfelttheyweredominatedbytheotherteammembers.Studentsalsomentionedthatthechangeinclassschedulesfromonesemestertothenextwasuncomfortable/inconvenient.f) Whatskillscanbeattributedtothestudents’participationinthelab?ThenumberoneskillstudentsattributetotheirparticipationintheEvalLabiscommunication.Studentsalsohighlighteddatavisualization,presentations,publicspeakingskills,excelskills,amongafewothersthoughcommunicationwasbyfarthemostrepeated.InFigure4,theWordlebelow,communicationisevidentasthegreatestskillstudentsattributedtotheirparticipationintheEvalLab.

Page 6: The NM Eval Lab’s Goal 1.1: “To support studentsevallab.unm.edu/learning-center/16-17-focus-group.pdf1 Summary of Findings for Eval^2 Focus Group with Fellows May 11, 2017 Charla

6

g) Wouldstudentsrecommendtheprogramtootherstudents?Everystudentwhoansweredthisquestion,answeredintheaffirmativethatyes,theywouldrecommendtheEvalLabtoafriend/fellowstudent.However,somementionedtheywouldonlyrecommendifthesuggestedchangesweremadeorwouldrecommendbutwiththedisclaimerthatthereisalotofwork,meetings,obligations,etc.Onefellowwrote,“Iwould[recommendit].However,Iwouldalsoletthemknowofallofthechallenges.Theorganizationsmaynotbeeasytoworkwith,workingwithateamcanbegreat,butalsochallenging.Allofthetimerequirements(sitevisits,posterpresentations,etc).Iwouldalsosaytonottakemorethan12credithoursandnottodoanotherinternshipwhiledoingtheEvaluationLab.Notonlyforyoursanity,butalsotoenableyoutobemoreflexible.

Meanwhile,otherswerethankfulfortheexperiencetheyhad:“Yes,IwouldrecommendtheEvalfellowshiptomostpeople.Itwasarichandhelpfulexperience.Thankyou.”“ThankyoufortheblessingandopportunitytoworkwithEvalLab.Theprogramandthefacilitator,professorsoftheEvalLabhavemadethisexperienceablessing.Pushingmehardertoworkandunderstandhowtodoresearchandorganizewithcommunityorganizations.”

Page 7: The NM Eval Lab’s Goal 1.1: “To support studentsevallab.unm.edu/learning-center/16-17-focus-group.pdf1 Summary of Findings for Eval^2 Focus Group with Fellows May 11, 2017 Charla

7

h) Howcoulditbebetter?Hereiswhatstudentsreallyliked:

• Havingachoiceinaligningwithorganization• Notfeelingconstrainedbyevaluationexamples• Sitevisits• Mentors• MelissaandSonia• Postersession• Money/stipend(laughter)"butreallyitwasoneofthereasonsIapplied"• Workingwithorganization(willcontinuetoworkwiththemhereafter)• Diversityofcohort• SeniorFellows(helpfulbutnottotallynecessary—otherswhodidn'thavethem

didn'tfeelliketheymissedout)• Happyhourswiththeorganizations• Guestpresenters• Datavisualization

Hereiswhatstudentssuggestbedonedifferently:

• Centralizegroupfeedbackonreports,maybeinachain(onepersonatatime)• Examplesofevaluationscouldofferoutlinestowrapheadaroundwhatit

[evaluation]couldbe• Clearlystateexpectationsforsitevisitsaheadoftimeinclass• Changetimeofsitevisits(hardtofitaroundclassschedules)• Calendarshouldbepublishedfrombeginningofsemestertoendwithoutside

events(sitevisits,workshops,presentations,etc.)• Shouldofferclasstimetodiscuss/introducemarginalizationasinstitutionalproblem

(notjustmentionedassurfaceissueordemographicissue)-->"justasimportantasIRBtraining"

• Discussionboardsneedtobemorespecificifcontinuingtouse"seemedlikesomepeoplejustrespondedbecausetheyhadto,wasn'tproductive"(thisisalsorepeatedfromthespeedroundofactivities)

• Giveoptiontochoosepartner/whoyouwanttoworkwith• Workflows/materialsoutlinestosupportalllevelsoflearninginclass(camefrom

discussionofsomestudentsthinkingthecoursewastooslowwhileothersthoughtitwastoofast/overwhelming)

ConclusionsandRecommendations:Returningtothegoalofthisfocusgroup,todetermineiftheEvalLabismeetingGoal1.1“Tosupportstudentsindevelopingtheknowledge,skills,andtoolstheyneedtodesignandimplementprogramevaluationsthatreflectorganizations’valuesandmissions,”Ibelievetheanswerisyes.Thoughthestudentsdidhaveafewrecommendationsforimprovement—theyclearlyfeltthefrustrationsofworkingwithoverworkedcommunityorganizationsandin

Page 8: The NM Eval Lab’s Goal 1.1: “To support studentsevallab.unm.edu/learning-center/16-17-focus-group.pdf1 Summary of Findings for Eval^2 Focus Group with Fellows May 11, 2017 Charla

8

teams—overall,theyfeltthattheEvalLabhelpedthemgrowincommunication,publicspeaking,workingwithExcel,andpresenting,amongothers.StudentsenjoyedtheirworkwiththeEvalLab.Inmanycases,studentsfeltstressedbutalsosupportedandbytheendtheywereveryproudoftheirproducts.ItappearsthatthestressandconfusiondidnotprohibitstudentsfromperformingwellintheEvalLaboverall,sincemanyreportedthattheirorganizationswereverysatisfiedwiththefinalproduct.Studentswerealsoverythankfulandverywillingtorecommendtheexperiencetotheirpeers.Thisisverypositivebecause,whilestudentsmayhavefoundthecoursechallenging,theyultimatelyfeltitwasapositiveexperiencethatcausedthemtogrow.Ontheotherhand,somestudentsdidn’tfeeloverwhelmed,andinsteadfeltthecurriculumofthecoursemovedtooslowlyandwasboringattimes.Studentsnoticedthattherewasalargedividebetweenthosewhoalreadyknewalotaboutevaluationandthosewhowerestartingfromscratch.Inthecasethatstudentsfeltunderwhelmed,theysuggestedthatperhapsworkflowsoradditionalmaterialcouldbesuppliedtothosethatarestrugglinginordertosupportalllevelsoflearningintheclass.Thiswasatopicthatsurfacedindrawing,writing,andspeakinginpairs.


Data brief - EVAL-TDA7803A - EVAL-TDA7803A evaluation board

Prelim eval

The Prehistory of Jefferson Lab’s SRF Accelerating

EVAL-AD7380FMCZ/EVAL-AD7381FMCZ User Guide€¦ · UG-1304 EVAL-AD7380FMCZ/EVAL-AD7381FMCZ User Guide Rev. 0 | Page 2 of 17 . TABLE OF CONTENTS Features ..... 1

KASPERSKY LAB’S AUTOMATIC EXPLOIT PREVENTION TECHNOLOGY (1).pdf · EXPLOIT PREVENTION TECHNOLOGY KASPERSKY LAB’S AUTOMATIC 2 Third party applications accounted for a massive 87

Data brief - STM32H743I-EVAL STM32H753I-EVAL - Evaluation ... · STM32H743I-EVAL STM32H753I-EVAL Description DB3235 - Rev 3 page 2/6. 2 Ordering information To order STM32H743I-EVAL

Claudia eval

Oak Ridge Lab’s SEOP R&D Efforts

 · Lumbar Puncture Microscope Eval -Amniotic Fluid Microscope Eval - Breast Discharge Microscope Eval - Post Coital Cervical Mucous Microscope Eval - Urine Microscope Eval - Vaginal

i-eval THINK Piece, No. 1 i-eval

Is Your Lab’s Competency Assessment a Competent Assessment? · PDF fileIs Your Lab’s Competency Assessment a Competent Assessment? ... A laboratory must evaluate and ... Is Your

THE JOY OF EVALUATION-2 - University of New Mexicoevallab.unm.edu/learning-center/summer-institute... · Evaluation Lab’s Eval^2 Team Mission time: one-hour meeting once per week,

EVAL-ADAU1962AZ/ EVAL-ADAU1966AZ User Guide

FOCUS ON manatee.disl.org • masgc.org facebook.com ... · The Florida Manatee: Biology and Conservation. University Press of Florida. Gainesville, FL. Dauphin Island Sea Lab’s

Media Eval

Gigadevice GD32 Cortex-M3 Evaluation Boards & Starter Kits Cortex-M3 Eval... · GD32103B-EVAL Part Number Board Features GD32103C-EVAL GD32103E-EVAL GD32105C-EVAL GD32107C-EVAL GD32150R-EVAL

EVAL-M1-6ED2230-B1 - Eval Board for 6ED2230S12T-1200 V

Stm3210c Eval

EVAL-AD5040SDZ/EVAL-AD5060SDZ to EVAL-AD5062SDZ (Rev. 0)

Eval revised

STM3220G-EVAL/STM3221G-EVAL demonstration firmware

(define applicative-eval (lambda (exp) (cond ((atomic? exp) (eval-atomic exp)) ((special-form? exp) (eval-special-form exp)) ((list-form? exp) (eval-list

Testing of beverages a testing lab’s perspective

Eval outcomesldp

EVAL 38060 PMAC1 / EV104 User Guide - Infineon Eval

Techniques for Successful Project Management – Lessons ... · each lab’s operating contract with DOE that could affect each lab’s annual fee. OAK RIDGE NATIONAL LABORATORY U

Autumn Conference - Core Laboratories€¦ · 17 - 19 September 2014 Le Collectionneur-Arc-de-Triomphe Paris Core Lab’s operational focus remains on crude oil-field development

Coursework eval

Digipak eval

The Eval that Men Dovfoley1/eval-that-men-do.pdf · What is eval? Question: What are some problems with eval? How does it affect static analysis? 13/62. What is eval? eval is the

EVAL FOLDER COSMETIC.pdf, page 11 @ Preflight ( EVAL ... - cosmetics.pdf · EVAL™ EVOH adds valuable function to cosmetic packaging 2 102976 EVAL FOLDER COSMETIC.indd 2 2/05/11

Data brief - STM32H743I-EVAL STM32H753I-EVAL - Evaluation ... · Data brief - STM32H743I-EVAL STM32H753I-EVAL - Evaluation boards with STM32H743XI and STM32H753XI MCUs Author: STMICROELECTRONICS

Eval powerpoint

STM3240G-EVAL evaluation boardcourses.engr.uky.edu/.../stm3240g_eval_user_manual.pdf · STM3240G-EVAL evaluation board Introduction The STM3240G-EVAL evaluation board is a complete

Eval Guide