1. Keeping it Personal:The Impact of Audio FeedbackMichele Raithby Lecturer in Social Work M.J.Raithby@swansea.ac.uk SALT Conference 2012

2. Themes Outline pilot project: Asynchronous audio feedback on assignment component Level 1 BSc (Hons) Social Work students Rationale Process Outcomes & what next 3. What do we Know Already? National Student Satisfaction Survey (HEFCE 2011) High level (32%) of dissatisfaction persists with assessment andfeedback Iceet al (2007) Higher levels of satisfaction with audio feedback compared to text only Audio feedback associated with increased retention of content Feelings of a personalised approach and increased involvement with the learning community Reduced staff time by 75%; increased feedback 225% Sounds Good (2009); King et al (2008) Not about saving staff time necessarily Audio increased quality of feedback 4. Rationale Level one is a crucial time for early establishment of student engagement Enhance quality and student experience of feedback Mix audio and typed comments to personalise the experience more Integrate into existing student reflective assessment component, with individualised dialogue with the instructor for further development of reflective skills Optimise staff satisfaction and time (a bonus) in providing feedback Combat repetitive strain injury Evaluate student opinion of audio feedback, including how they listen to it, any technological issues, and compare with written feedback component of module 5. StudentsCharacteristics 50 level 1 undergraduate Social Work students Range of experience; current practitioners to minimumrequirement of 210 hours experience Ages from 18 to mature return to learn (51) 6 males 6. Module ContentModule on Ethics & Values (20 credits) in first term Student social workers need to be reflective, academically and in practice Embed critical thinking & reflective practice Link to future Practice Learning Opportunities Challenge students personal & professional values 7. Blog Task Student task to write weekly blog analysing learning fromindividual teaching sessions Supported by seminars on reflective writing, values &practice Use Blackboard journal tool Blogs formative but basis for a summative component ofassessment (20%) 8. Online Dialogue Privacy settings not adiscussion board Only visible to bloggerand module convenor Module convenorcommentedbriefly/constructivelyon each blog entry Develop reflectiveregister of writingrequired for practice 9. Summative Assessment: 2 Parts Part 1 Essay (80%) typed feedback Part 2 Synthesise blog into academic submission of reflection on learning(20%) audio feedback Level 1 marks do not count to degree classification 10. Process Equipment: SALT small grant of 100 purchased Sony ICdigital voice recorder Save recordings as MP3 via USB to computer Keep technical demands on students to minimum Used own microphone headset 11. Practicalities Need quiet space - no hoovering With handset can be mobile Tolerate pauses and re-record if necessary 12. Timescale November 2011- December 2011: Module delivery: explanation ofaudio feedback at beginning of module. Upload group podcast toexplain technical procedure January 2012: Summative assessment submission in two parts February 2012: Return of feedback to students; combination of audioand text February/March 2012: Student survey to evaluate audio compared towritten feedback 13. Written Feedback Example: Page 1 14. Page 2: Marking Grid 15. Audio Feedback Example 16. Audio Comments 48 students out of 50 submitted assignments Recorded comments ranged from 1 minute 57 sec to 5.13 Tended to make comments longer nearer the end (lastone longest) Some correlation of longer comments for lower achievingstudents 17. Results: What Went Well Produced feedback ahead of deadlines Enjoyment & personal touch Encouraging tone of voice Extensive and specific feedforward remarks about futuredevelopment 18. What Didnt Go So Well Communication via Blackboard Unable to insert as podcasts for students to access So emails sent within Blackboard module site - could takea minute to send Large files potential to jam email accounts (includingown copy automatically sent to staff account) 19. Fenella FieldingEffect I lost my voice. 20. Evaluation Domains Questionnaire: Likert scales 1-5 (not useful- very useful): Usefulness of audio/ written components Ease of access, audibility, technical issues How many times listened to/read Use of feedforward Link to reflection Preference for type of feedback 21. Evaluation Results 39 responses (78%) Age range 18-51 (average 30) 4 males, 35 females Smiley faces 2 23 23 22. Usefulness Times listened to/Read Feed ForwardTyped 4.11.94.3Audio 4.824.5 23. Justify Explain Praise DevelopReferencing FutureMarksThinking WorkTyped 4 4.1 3.744.5 4Audio 4.7 4.6 4.94.64.4 4.5 24. 1st response disliked audio: Dont take much inlistening it. (1) All the rest liked it Personal used 18 times - most frequent adjective Students know their learning styles - (I dont) 25. Keeping it Personal Very personal. More so than written feedback. I listened more to thebreakdown of the feedback, but also valued the positive feedbackwhich I feels lacks in written feedback. It was more in depth than the written feedback. It came across morepositive than the written feedback even though the basic premise ofeach was the same. I think this may be because it used moreinformal language. It also felt more personal. Bit more human! (12) 5+ for encouragement. (6) 26. Tone of voice Your voice was very soothing. (6) Audio feedback could be better used by greater emphasis andutilising your personality. The current feedback, although it was you,lacked your personality. (23) A future beckons on late night radio! (24) 27. Accessibility Handwritten can be kept, referred to and easily found.Audio must listen to it over and over to find useful part.(16) I downloaded to my phone so was able to listen to itwherever I was, as used advice for other essay. (26) As Im dyslexic I found it easier to listen to rather thanreading it. (13) 28. Feed Forward and Reflection Fully explained what I did well (nice to hear) but also development of future work - very helpful (7) By far the most informative feedback to date. It ahs enabled me to distinguish my strengths and weaknesses and support my ongoing studies. (18) (...) personalised and therefore felt more like a reflective pice of work. (30) 29. Conclusions Students prefer a mix of audio and typed Typed good for specific guidance e.g. referencing Audio more like a tutorial: More personal, in depth, encouraging and tailored less formulaic or curt than typed On script or off message? Pencilled remarks on script also appreciated Audio more congruent with subject matter of reflection Audio suits my personal style Further work needed on transmission of audio files via Blackboard 30. References HEFCE (2011) NSS results http://www.hefce.ac.uk/news/hefce/2010/nssresult.htm Ice, P., Curtis, R., Phillips, P. & Wells, J. (2007) Using Asynchronous Audio Feedback to Enhance Teaching Presence andStudents Sense of Community. Journal of Asynchronous Learning Networks 11 (2) [online] Available at:http://www.adobe.com/education/resources/hed/articles/pdfs/v11n2_ice.pdf Higher Education Academy: Social Work and Social Policy (SWAP) (2010) Effective Assessment for the Degree in SocialWork. www.swap.ac.uk/docs/events/EEAsstdiscnotecol050310.pdf JISC (2009a) Effective Practice in a Digital Age [online] Available at:http://www.jisc.ac.uk/media/documents/publications/effectivepracticedigitalage.pdf JISC (2009b) Sounds Good: Quicker, better assessment using audio feedback. Final Report. [online] Available at:http://www.jisc.ac.uk/whatwedo/programmes/usersandinnovation/soundsgood.aspx JISC (2010a) Audio feedback [online] Available at: http://www.jiscdigitalmedia.ac.uk/audio/advice/audio-feedback JISC (2010b) Optimising Audio Feedback assisted Learning for Student and Staff Experience (AFAL). Final Report. [online]Available at: http://www.jisc.ac.uk/whatwedo/programmes/elearning/ltig/afal.aspx King, D., McGugan, S. & Bunyan, N, (2008). Does it make a difference? Replacing text with audio feedback. Practice andEvidence of Scholarship of Teaching and Learning in Higher Education. 3 (2) 125 163 31. Diolch yn fawr! Any questions?