Strategy Examples

eLearning & eAssessment

Dr Mathew Hillier • Lecturer in Higher Education, TEDI, University of Queensland• Visitor, CLPD, University of Adelaide

Web edition update June 2011

Contact: mathew.hillier at gmail.com

EnvironmentProblemsSolutionsExamples

eLearning Strategiesand e-Assessment Examples

• Consideration of the multifaceted environment in which eLearning is to take place.– Multiple Perspectives (Linstone)– Systems Thinking (Checkland, Ackoff, Churchman)– Worldviews (Weltanschauung) / Point-of-view

• Paul Ramsden: A focus on several different levels of the system“To achieve change in the quality of teaching and learning, we ought

rather to look carefully at the environment in which a lecturer works and the system of ideas which that environment represents.” Ramsden (2003) http://bit.ly/8XSV0I

– Multi leveled: Lecturer, Program teams, School/department group, Head, Faculty/division, Institution leaders…

eLearningEnvironment

Prezi: http://bit.ly/9Jl8Ir

eLearningEnvironment

Prezi: http://bit.ly/bG066g

eLearning Potential Problems from an Academic’s Perspective

Prezi http://bit.ly/bmXA7F

eLearningPotential Solutions

• Build capacity– Local discipline groups – Networked across the institution– National/international– Face to face and/or online

• Possible CoPs.... – Mentoring– Teaching with virtual worlds– First year co-ordinators– Sessional staff– RHD supervisors– Work-integrated Learning– “Blended Learning / e-learning” CoP– ALTC winners, grant writing?...

Communities of Practice

An example: http://transformingassessment/community/

Social NetworksA range of services can be used and set-up to be internal only or externally accessible. Crosses functional boundaries.

Facebook

YouTube

Wordpress Blog

Twitter

Yammer messenger

LinkedIn

Social Networking

Facebook:For teaching – Where the students are!For PDIntegration with LMS

First YearUse ‘groups’. Students don’t need to ‘friend’ each other. For local / foreign student integration.Assessed on quality of critiques.

Transforming Assessment FB http://on.fb.me/ta-fbImaging Our World 2010 http://on.fb.me/cv0AqC

Acknowledgement: Josh McCarthy (Architecture UofA)Public facebook group.

E-Marking / Rubrics

Each student has a marking sheet for individual feedback.Breakdown. Grad quals.

Based on MS Excel (not tied to net). Efficient rubric based assessment. Each assessment set-up using templates. Scripted generation of student sheets. Auto print functions to PDF.

Comment banks can be shared between markers. Insert auto comments - editable on-the-fly.

Evaluate by sliders. Enter data only once.

Each assessment item has a work book listing all students in the course. Names source from course summary workbook.

Course workbook collates marks and calculates final grades.

Main site http://www.transformingassessment.comSecondary http://transformingassessment.adelaide.edu.au (SBLi / Open Sim)YouTube http://www.youtube.com/user/transformassessment

AcknowledgementsDirector:Professor Geoffrey Crisp Project Officer:Shamim Joarder

Statistics9705 visits Jan 2010 to June 2011

69 CountriesAustralia (AU) 4,062United Kingdom (GB) 1,503United States (US) 1,134New Zealand (NZ) 512Saudi Arabia (SA) 433Hong Kong (HK) 277Singapore (SG) 183South Africa (ZA) 154Canada (CA) 141India (IN) 102Spain (ES) 81Germany (DE) 68Ireland (IE) 61Netherlands (NL) 59Malaysia (MY) 54Italy (IT) 47Portugal (PT) 44United Arab Emirates (AE) 41Russian Federation (RU) 36Bahrain (BH) 36Mexico (MX) 32France (FR) 32 and more!

e-Assessment Examples

Learning Management Systems

http://www.transformingassessment.com/moodle/

Common Features:ActivitiesResourcesQuestionsAssignmentsWikiBlogs

Uni wide standard

Question types are extensible. Requires installation of scripts into the Moodle server.

Integration with...Applets (Molecule Editor / Jmol )Social Network (Facebook)Virtual Worlds (Sloodle / SL)Audio (Nanogong)Mind Maps (Compendium)Remote LabsVirtual Reality View (QuickTime VR)Augmented Reality (Cam markers)Virtual Classrooms (BigBlueButton)

Assignment Types

Voice submission

Multiple TypesSingle file submissionOnline textOffline activityPeer reviewVoice submission

Nanogong without Save and Speed Buttons

The NanoGong applet can also load a pre-recorded voice file from a website.

Wimba Voice Tools in Blackboard

Requires installation on Moodle server.

Once installed teachers can add it via the ‘add activity’ menu

Students use the NanoGong applet to record an audio response and submit it for assessment.

http://gong.ust.hk/nanogong/

Quizzes: Applets

http://www.transformingassessment.com/moodle/file.php/27/jmol/jmol01.html

Jmol: Open-source Java viewer for chemical structures in 3D

Moodle Quiz

Students interact with online tools to obtain data to construct an answer.

Quizzes: Linked Java Applet

Read question

Submit Answer

Open applet using link provided in the question.

Use tools to construct answer

Answersrecorded in grade book (Moodle / Blackboard)More applets http://chemmac1.usc.edu/bruno/java/javaap.html

To create this type of question:1.Create question text as normal in the LMS2.Create another web page containing the applet. <applet code=TitratePh.class width=450 height=370> 3.Add link to applet in the question text.

Student work flow

Remote LabsPhysical hardware can be connected to the internet. Students can use the equipment 24/7 thus increasing use time (although not to replace actual in lab time). Increases return on equipment investments.

Quizzes: Augmented Reality

Mix physical and virtual. Use for complex visualisations. Overlay of data on the real world. Interactive activities and quizzes.

Quizzes: Augmented Reality

Examples of augmented reality using markers.

Music – drum sequences. Music – band / instruments

Engineering exploded models

Quizzes: Assertion-Reasoning

Multiple choice quizzes can be much more than recalling facts.For example, using an assertion and reasoning format we can test understanding of why an answer is right or wrong.

Quizzes: Certainty-Based, or Confidence-Based Marking (CBM)

University College London

http://bit.ly/asa9mu

Confidence based approaches penalise guessing. Students need to choose a response and declare their level of certainty.

Certainty v Mark ExpectedCertainty levels and consequences

3D Virtual Worlds

http://www.transformingassessment.com/secondlife.php (instructions)http://slurl.com/secondlife/transforming%20assessment/254/254/23/

SimulationsInteractivityRole Plays3D ModellingReconstructionData visualisation

3D Virtual WorldsSLOODLE

As if the student was doing the activity in the LMS

A set of scripts for Moodle and SL that acts as a bridge between the two.

or

Student undertakes assessment in the virtual world

Set up Quiz in the LMS. Results are stored in the in grade book.

Modules: Quiz, Tracker, Backpack, Blogger, Presenter, Glossary, Drop box...QuizHUD....

3D Virtual WorldsInteractivity

Interactive Q&A example.

3D Virtual WorldsInteractivity by branching

Integration using web form to set-up.

Branching Example

Scripted prims / avatars for interactive Q&A, role plays and simulations.

QuizHUD facilitates touch to answer activities/quiz, standard MC Quiz, touch to explore (extra info).Creates interest for students beyond a static text book! Able to test knowledge in context.

‘Wear’ the HUD object. This controls and presents the quiz. HUD communicates from inside the virtual world to a web server.

Touch objects in the environment to answer the questions.Basic feedback is provided in the HUD

http://www.youtube.com/watch?v=ZgkUtSSEpBc

Web component stores quizzes, results and information pages.

HUD object

Additional scripted visual feedback after touch

3D Virtual Worlds: Touch to answer activities

3D Virtual Worlds: Touch to answer activities

QuizHUD examples for Chemistry, Colour Wheel, Midwifery, Model Solar system. Showing ‘explore’ mode.

3D Virtual Worlds Spaces and Equipment

Virtual Lab and Equipment OrientationReduces time required to orientate to real world physical layouts and equipment.Processes can be established using scripting to assess steps and procedures.Beakers, flasks, crucible, watch glass, mortar and pestle, evaporating dish, pipette, funnel ring, stand, Bunsen burner, test tubes, microscope .... In fact any equipment can be simulated from the basic to the very complex.

(Another QuizHUD example).

See also iGEM lab in SL: http://maps.secondlife.com/secondlife/LINDSAY%20Virtual%20Medicine/63/71/57

Audience Response Systems

http://www.urvoting.com/

VotApedia (Urvoting.com)Needs an account.1. Set up via webform:2. Choose phone numbers for each response (avoid first 5).3. Start survey4. To vote: a. mobile phone: - unanswered call (free) - SMS (costs) b. Web form (free)5. Stop survey and view results

Audience Response Systems: Team Based Learning / Large Classes

TBL uses a mix of procedures some of which involve students using ‘scratchies’ to uncover answers (with reducing marks the more they scratch on a given question). This provides instant feedback to students as they work towards a group response and stimulates in-group discussion.A ‘scanner’ is used to collate marks from students in the room to be feedback to the lecturer for further attention (JIT lecturing style).Problem! – collating takes time in large classes interrupting the flow of the session.

Potential solution: Replace physical elements with digital ones that leverage what students already have (mobile phones, laptops, ipads etc) to provide more efficient and faster collation/feedback to the lecturer saving ‘admin’ time.

TBL process for lecture sessions:Pre-reading -> readiness test -> team test -> discussion/mini lectures.

Wiki

MediaWiki is a free software wiki package, originally for use on Wikipedia

Some e-assessment examples using Media Wiki http://www.transformingassessment.com/wiki/

Collaborative document editing, discussion, detailed audit trail of edits.

Manual marking.Potential for auto summary and data mining.

Integrated e-assessments can be computer marked.

Dates, versions and authors. Versions can be compared.

Note: Wiki built into LMS are often rudimentary!

e-Portfolios

http://cterfile.ed.uiuc.edu/mahara/view/view.php?id=327 http://www.transformingassessment.com/mahara/

Build a portfolio over time.Demonstrate competencies.Create groups.Critique and reflect.Can be integrated with Moodle.

Blogs

Note: blogs built into LMS are often rudimentary! WordPress is one of the best blogging platforms. See an example http://transformingassessment.com/wordpress/

Blogs can be used by students and by teachers alike. Students can record their reflections and experiences about a particular issue, question, topic, event or objects that you may have made available. Entries are date/time stamped and can be commented upon by others.

Assessments can be embedded into a blog entry too. You could use these as discussion points so that students could comment on the questions and the associated feedback. You could also use this format to encourage students to reflect on their level of understanding of key concepts. Students could comment on their approach to solving a problem or how they went about tackling a task that you set.

Social Video Media

http://www.youtube.com/user/transformassessment

Statistics (May 2010 – June 2011)Channel Views: 2,120Video views 17,464

For Studentsor for PD

Multimedia•Generate, •Share •Communicate•Argue•Instruct•Express•Critique

Game based learning / Serious games

Simulations, role plays. Business, science, history, language/communication.

Scenario Based Learning

Location Window

Action Window

Content Window

Environment Window

Scenarios entail branching, contingencies, multiple resources. Data is collected on time taken, path taken, resources used (efficiencies) which can be contrasted against the effectiveness of answers.

http://www.sblinteractive.org/http://transformingassessment.adelaide.edu.au/SBLiServer/

Scenario Based Learning

Location Window

Action Window

Environment Window

Content Window

Timer

Clickable Image Part

Configurable by TeachersQuestion Release dateQuestion Feedback Release dateMoodle CompatibleMaximum Attempt Number

The scenario presented on this slide is from SBLi.Acknowledgement : Shamim Joarder

Scenario Assessment

Many resources viewedand miss the solution

Many resources viewedand problem solved

Few resources usedand problem solved

Few resources usedAnd missed solution

Efficient, Effective

Guessing

Not Efficient, Effective

Not Efficient, Not effective

0 100

5

Resources used

Qu

est

ion

Sco

re

(many)

(high)

Acknowledgement : Shamim Joarder

Scenarios for Diagnostic Assessment

Example: Evaluating a student’s prior knowledge of a topic.

Provide further descriptive informationProvide one or more examples

S – StartE – EndQ – Multiple Question(s) to be answeredNumbers represent weight of resources used.

S EDefinition

Definition+ Description

Definition+ Description

+ Example

Q

Q

Q

20

3040

Instructions to student: “Use the least amount of information as you can to answer the questions”.If a student is able to successfully answer the (reasonably sophisticated) questions without needing additional information then we can say (all else being equal – i.e not guessing or cheating) that their prior knowledge or familiarity with the topic is high.

Acknowledgement : Shamim Joarder

Scenario data reporting

Acknowledgement : Shamim Joarder

Questions?

For further information or a copy of these slides:mathew.hillier at gmail.com

End!

• ALTC Projects relating to eLearning• ALTC Fellows relating to eLearning• Researchers of interest

Appendix

Australian Learning and Teaching Council Projects of interest (as at 2010) relating to e-learning• (2010) – Newcastle: The eOSCE: advancing technology to improve students• (2010) – UNSW: An adaptive e-learning community of practice for mechanics courses in engineering• (2010) – UNSW: Extending the science curriculum: teaching instrumental science at a distance in a global laboratory using a collaborative electronic

laboratory notebook• (2010) – UNSW: Learning and teaching technical competence in the built environment using serious video game technology• (2010) – UQ: A national approach to education in advanced microscopic characterisation through integrated learning tools • (2009) – Adelaide: Getting a MUVE on: developing Web 2.0 curricula in the humanities• (2009) – Adelaide: The Medici project: developing a multi-disciplinary, sustainable resource for blended learning initiatives in tertiary medical education• (2009) – Newcastle: Facilitating WIL through skills-enabled e-portfolios in the disciplines of construction and nursing• (2009) – Melbourne: Web 2.0 authoring tools in higher education learning and teaching: new directions for assessment and academic integrity• (2009) – Murdoch: Remix, mash-up, share: authentic Web 2.0 assessment scenarios and criteria for interactive media, games and digital design• (2009) – UNE: e-Teaching leadership: planning and implementing a benefits-oriented costs model for technology enhanced learning• (2009) – UNE: The ReMarks PDF - Mark up Editor Stage 2• (2009) – UNSW: Learning to teach online: developing high-quality video and text resources to help educators teach online• (2009) – UNSW: New media to develop graduate attributes of science students• (2009) – UQ: IS-IT learning? Online interdisciplinary scenario-inquiry tasks for active learning in large, first year STEM courses • (2008) – Deakin: Building academic staff capacity for using eSimulations in professional education for experience transfer• (2008) – Newcastle: Examining the impact of simulated patients and information and communication technology on nursing students' clinical reasoning• (2008) – UniSA: Enriching student learning experience through international collaboration in remote laboratories• (2008) – UNSW: Physclips II - Waves and sound: an integrated set of multi-level multimedia resources and laboratory experiments• (2008) – UQ: An integrated system for online clinical assessment of practical skills (eCAPS) for web-based courses • (2007) – LaTrobe: Encouraging benchmarking in e-learning• (2007) – Newcastle: Assessing and improving spatial ability for design-based disciplines utilising online systems• (2008) – UniSA: Facilitating flexible, enquiry-based experiential learning through an accessible, three-dimensional virtual learning environment (3DVLE)• (2007) – CSU: On-line student supervision training - accessible and cooperative learning in social work• (2007) – UniSA: Peer Review of Online Learning and Teaching• (2007) – UQ: Developing and disseminating TEAM SKILLS capacities using interactive online tools for team formation, learning, assessment and mentoring • (2007) – UQ: A national curriculum for entomology: Capacity building through collaborative, web-based delivery• (2007) – UQ: The virtual slidebox • (2007) – UTS: Embedding peer review of learning and teaching in e-learning and blended learning environments

Appendix

ALTC Fellows of interest re e-learning

Appendix

Professor Des Butler 2009 ALTC Teaching FellowQueensland University of TechnologyUsing cost-effective multimedia to create engaging learning experiences in law and other disciplinesProfessor Geoffrey Crisp 2009 ALTC National Teaching FellowThe University of AdelaideRethinking assessment in the participatory digital world – assessment 2.0Professor Geoffrey Meyer 2009 ALTC Teaching FellowThe University of Western AustraliaBuilding a network of academics who use, contribute to and disseminate an online, cost-effective histology learning and teaching resource for students in Australia and overseasProfessor Matthew Allen 2008 ALTC Teaching FellowCurtin University of TechnologyLearning in Networks of Knowledge (LINK) - improving student educational outcomes in online learning, using Web 2.0 concepts and a knowledge-networking approach

Professor Michael Christie 2008 ALTC National Teaching FellowCharles Darwin UniversityIncreasing the participation of Indigenous knowledge holders in tertiary teaching through the use of emerging digital technologiesDr Sandy O'Sullivan 2008 ALTC Teaching FellowBatchelor Institute of Indigenous Tertiary EducationPromoting strategies and creating opportunities for inter/multimedia practice as a culturally appropriate dissemination tool for Indigenous postgraduate research trainingProfessor David Boud 2007 ALTC Senior FellowUniversity of Technology, SydneyStudent assessment for learning in and after coursesProfessor Peter Goodyear 2007 ALTC Senior FellowThe University of SydneyTeaching, technology and educational design: the architecture of productive learning environmentsProfessor Ron Oliver 2006 ALTC Associate FellowEdith Cowan UniversityPromoting the uptake of re-usable ICT based learning designs

Researchers of interest• Caroline Steel. UQ AU. VP Ascilite, Res Fellow Lang & culture, Fmr Lecture Higher Ed, TEDI. Current and emerging Ed technologies.

Teacher beliefs about tech, pedagogy, teaching and affordances theories.• Catherine McLoughlin. ACU AU. A/Prof Sch Ed. Social media/web 2.0 for T&L. Cross cultural issues. Learning spaces, e-assessment.• David Boud. UTS AU. Assessment futures. Student assessment in and after courses. Assessment elements. Longer term. Self

assessment of performance, capabilities, work and others. • David Kennedy. Lingnan HK. Director T&L centre. T&L technologies. Mobile learning. Problem based learning. Visual and

information literacy. Outcomes based approaches. ePortfolios (customised).• David Nicol. Strathclyde UK. Fmr Director Centre Academic practice and learning enhancement. eLearning. Graduate qualities.• Diana Laurillard. London Knowledge Lab UK. Conversational approach. Interaction between T & SS. Multifaceted learning needs

an array of technical capabilities.• Geoffrey Crisp. Adelaide AU. Director Centre for Learning and Professional Development. Assessment, e-assessment, ed

technologies, web 2.0 and social media. Exmplars. Interactive assessments.• Gráinne Conole. OpenU UK. Chair elearning. e4innovation.com Learning design via designed approach v beliefs based approach.

Open course design (George Siemens, Stephen Downer itu.umanitoba.ca)• James Dalziel. Macquarie AU. Director of eLearning Centre of Excellence. Learning activity management system (LAMS),and

Research AMS.• Paul Ramsden. Consultant AU/UK. Fmr PVC T&L USyd. Higher Ed Academy UK, Uni London, Maquarie. Teaching and student

experience. Teaching quality. Policy. Role of leadership. Learning to teach in higher ed: learning from our students. Argues for holistic, multi leveled approach top T&L improvement – lecturer, department, head, program teams, faculty, uni management.

• Peter Goodyear. USyd AU. CoDirector Centre Computer Supported Learning & Cognition (CoCo). Learning technology and new media. Learning cognition and motivation. Learning through inquiry. Productive learning environments. Teaching, technology and learning design.

• Ron Oliver. Edith Cowan AU. PVC T&L. Share/reuse of learning designs. Toolbox learning object repository. Authentic learning in web environments. Online learning communities (Chris Brook, Curtin).

• Terry Anderson. Athabasca CA. Chair distance ed. Social networking, online conferences, self paced learning (non-schedule based). Terrya.edublogs.org

Appendix