comparative analysis of formative assessment apps

Observa(on Compara've Analysis of Forma've Assessment Apps

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges

TESOL Arabia Conference 11 March, 2016 HyaC Regency Hotel Dubai, UAE

James Buckingham

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


James Buckingham

Instructor Educa'on Technologist Digital Badge Specialist Zayed University Abu Dhabi, UAE

Compara(ve Analysis of Forma(ve Assessment Apps

Backchannel / list of links hBps://todaysmeet.com/TACON2016

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


increasing interest in the use of online applica'ons in K-‐20 driven by? •  increased availability of smartphones / table devices / laptops

•  increased reliability and availability of network connec'vity to support it

•  increased interest in using such apps to mo'vate students

•  perceived "new ability" to carry out low / no cost computer based assessment (Sharkey, 2005)

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


increasing interest in the use of online applica'ons in K-‐20 driven by? •  increased availability of smartphones / table devices / laptops

•  increased reliability and availability of network connec'vity to support it

•  increased interest in using such apps to mo(vate students

•  perceived "new ability" to carry out low / no cost computer based assessment (Sharkey, 2005)

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


increasing interest in the use of online applica'ons in K-‐20 today's buzzwords

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


increasing interest in the use of online applica'ons in K-‐20 today's buzzwords •  engaging students; •  promo'ng “interac(on”

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


increasing interest in the use of online applica'ons in K-‐20 today's applica(ons

•  ROI -‐ rela'vely low cost in 'me w/ rela'vely high speed of return

•  (me -‐ easy to use, create, collaborate and share

•  money -‐ low or no cost investment

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


increasing interest in the use of online applica'ons in K-‐20 today's applica(ons

•  ROI -‐ rela'vely low cost in 'me w/ rela'vely high speed of return

•  (me -‐ easy to use, create, collaborate and share

•  money -‐ low or no cost investment

What’s not to like?

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


But what is “forma(ve” assessment? not so easy to answer •  different defini'ons •  even different terms for it •  "informa've assessment" (Winkler, 2010)

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


But what is “forma(ve” assessment? not so easy to answer •  different focuses •  learner •  instruc'onal design •  course design •  program design

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


But what is “forma(ve” assessment? my focus •  learner •  instruc'onal design •  course design •  program design

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


But what is “forma(ve” assessment? my focus •  learner •  a student’s review of their learning •  check how well they have

understood •  use “data” to •  guide (focus) their next move •  improve their work

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


But what is “forma(ve” assessment? my focus •  learner •  to provide feedback that will •  build learner confidence •  promote learner improvement

(Wehlburg, 2011)

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


But what is “forma(ve” assessment? my focus •  learner •  theory •  helps make student learning processes more transparent (see what students know) so it benefits BOTH student and instructor

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


But what is “forma(ve” assessment? my focus •  learner •  research •  in top three of most effec've ways to impact on student learning

1.  make learning fun 2.  effec've PD 3.   forma(ve assessment (Winkler, 2010)

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


But what is “forma(ve” assessment? my focus •  learner •  methods •  feedback without grades •  self assessment •  peer assessment •  ques'oning

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


Key “forma(ve” assessment issues? Observa(on #1 -‐ lack of formal training •  oben guided by teacher's percep(on of what "works"

•  availability of formal instruc'on or PD on how to conduct effec've FA very low

(Volante & BeckeC)

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


Key “forma(ve” assessment issues? Observa(on #1 -‐ lack of formal training •  few chances to learn / explore FA •  high dependency on HE courses to

direct assessment methods •  …. even then, few focus on FA ..

choosing instead to focus on SA

(Volante & BeckeC, 2011)

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


Key “forma(ve” assessment issues? Observa(on #2 -‐ assessing vs grading •  Tempta(on •  to "grade / measure” •  valued so highly by school admin

(Whelburg, 2011) (Volante & BeckeC, 2011)

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


Key “forma(ve” assessment issues? Observa(on #2 -‐ assessing vs grading •  Tempta(on •  to "grade / measure” •  overemphasis on SA =

environment of high accountability •  excessive tes'ng = neglect of

cri'cal thinking (far more difficult and 'me consuming to test?)

•  more SA = less feedback (?) lower quality feedback (?) (Whelburg, 2011) (Volante & BeckeC, 2011)

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


Key “forma(ve” assessment issues? Observa(on #3 -‐ limited methods

(Torrance & Pryor, 2001)

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


Selec'ng “FA” applica'ons criteria? •  Key traits •  easy to use •  easy to realize results •  low cost / no cost = accessible to all

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


Selec'ng “FA” applica'ons criteria? •  Key affordances (design supports ac'on) •  provides feedback to both instructor / student •  data produced ... downloadable .. for interpreta'on

•  supports instructor authored feedback •  supports collabora'on / sharing with peer/instructors

•  supports both convergence and divergence ques'oning

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


Selec'ng “FA” applica'ons criteria? •  popular among educators •  social cura'on •  currently ranked highest in educator

based surveys

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


Selec'ng “FA” applica'ons criteria? •  popular among educators •  Edshelf •  Graphite •  Jane Hart's 100 tools for 2015

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


Selec'ng “FA” applica'ons criteria? •  Edshelf

social review / cura'on by educators

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


Selec'ng “FA” applica'ons criteria? •  Graphite

social review / cura'on by reviewers & educators

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


Selec'ng “FA” applica'ons criteria? •  Graphite

social review / cura'on by reviewers & educators Ranking Kahoot 3 / 4 Socra've 3 / 4 Nearpod 4 / 5

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


Selec'ng “FA” applica'ons criteria? •  Jane Hart's 100 tools for 2015

nomina'ons & reviews by educators

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


Selec'ng “FA” applica'ons criteria? •  Jane Hart's 100 tools for 2015

nomina'ons & reviews by educators Ranking 17 – Kahoot 32 – Socra've 50 – Nearpod

Observa(on

Defini(on

Issues

Applica'ons

Sugges(ons

Future

Challenges


Challenges facing “FA” adop'on Address training needs •  educators may lack skills on •  how to interpret student data results •  offer feedback that will lead to improved student

results Unclear next steps? •  lack of incen(ves for teachers to invite learning

how to measure effec'veness of feedback •  lack of consensus among academics on how to

move forward (Sharkey & Murnane, 2006)

Observa(on

Defini(on

Issues

Applica'ons

Sugges(ons

Future

Challenges


Sugges'ons – My “current” review Kahoot #17 •  supports “convergence” type ques'ons •  does NOT support divergence type ques'ons

•  maximum 4 choices – “right / wrong” feedback •  does NOT support authored feedback •  could be adapted to realize peer feedback • 

Observa(on

Defini(on

Issues

Applica'ons

Sugges(ons

Future

Challenges


Sugges'ons – My “current” review Socra(ve #32 •  supports BOTH “convergence” and “divergence”

ques'ons •  does support authored feedback •  could be adapted to realize peer feedback

Observa(on

Defini(on

Issues

Applica'ons

Sugges(ons

Future

Challenges


Sugges'ons – My “current” review Nearpod #50 •  supports BOTH “convergence” and “divergence”

ques'ons •  does support authored feedback •  could be adapted to realize peer feedback

Observa(on

Defini(on

Issues

Applica'ons

Sugges(ons

Future

Challenges


Sugges'ons -‐ General Focus on the learner’s needs •  nature of feedback needs to be •  more suppor've (in small groups?) •  more informa've

leads to •  less convergence in FA design •  True / False •  Right / Wrong

Observa(on

Defini(on

Issues

Applica'ons

Sugges(ons

Future

Challenges


Sugges'ons -‐ General Focus on the learner’s needs •  nature of feedback needs to be •  more suppor've (in small groups?) •  more informa've

leads to •  more divergence in FA design •  open ques'ons / open answers

Observa(on

Defini(on

Issues

Applica'ons

Sugges'ons

Future

Challenges


The Future? •  more variety in FA strategies (growing segment of app market)

•  more personalized & self directed instruc'on

Observa(on

Defini(on

Issues

Applica'ons

Sugges'ons

Future

Challenges


The Future? •  more variety in FA strategies (growing segment of app market)

•  more personalized & self directed instruc'on

•  Current examples?

Interac've video applica'ons

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges

List of references


Abuid, B. A. (2014). A student par'cipa'on assessment scheme for effec've teaching and learning. Learning and Teaching in Higher Educa'on: Gulf Perspec'ves, 11(1). Retrieved from hCp://lthe.zu.ac.ae/index.php/lthehome/ar'cle/view/109 Foggo, L. (2007). Using Blogs for Forma've Assessment and Interac've Teaching. Ariadne, (51). Retrieved from hCp://www.ariadne.ac.uk/issue51/foggo/ Laborda, J. G., Sampson, D. G., Hambleton, R. K., & Guzman, E. (2015). Guest Editorial: Technology Supported Assessment in Formal and Informal Learning. Journal of Educa'onal Technology & Society, 18(2), 1–2. l’Anson, S. (2016, February 24). Forma've Assessment Archives. Retrieved from hCp://interac'veachievement.com/category/forma've-‐assessment/

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges

List of references


Sharkey, N. S., & Murnane, R. J. (2006). Tough Choices in Designing a Forma've Assessment System. American Journal of Educa'on, 112(4), 572–588. hCp://doi.org/10.1086/505060 Torrance, H., & Pryor, J. (2001). Developing Forma've Assessment in the Classroom: Using ac'on research to explore and modify theory. Bri'sh Educa'onal Research Journal, 27(5), 615–631. hCp://doi.org/10.1080/01411920120095780 Volante, L., & BeckeC, D. (2011). Forma've assessment and the contemporary classroom: Synergies and tensions between research and prac'ce. Canadian Journal of Educa'on/Revue Canadienne de L’éduca'on, 34(2), 239–255. Wehlburg, C. (2011). A scholarly approach to assessing learning. Interna'onal Journal for the Scholarship of Teaching and Learning, 5(2), 2.

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges

List of references


Winkler, C. C. (2010). Examples of Effec've Forma've Assessment. Mar'n Luther College. Retrieved from hCps://www.mlc-‐wels.edu/library/search-‐find-‐2/special-‐collec'ons/pdf-‐files/LitReviewWinkler2010.pdf

Observa(on

Defini(on

Issues

Applica(ons

Sugges(ons

Future

Challenges


James Buckingham

Instructor Educa'on Technologist Digital Badge Specialist Zayed University Abu Dhabi, UAE


Backchannel / list of links hBps://todaysmeet.com/TACON2016

comparative analysis of formative assessment apps

Education