effective slo assessment in the online environment
Post on 04-Jun-2018
222 Views
Preview:
TRANSCRIPT
-
8/13/2019 Effective SLO Assessment in the Online Environment
1/61
Effective SLO Assessment
in the Online Environment
Jim Julius, MiraCosta College
Kevin Kelly, Wiley Learning Institute
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
Jim Julius, MiraCosta College, & Kevin Kelly, Wiley Learning Institute
-
8/13/2019 Effective SLO Assessment in the Online Environment
2/61
Effective SLO Assessment
in the Online Environment
Before we get started
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
Jim Julius, MiraCosta College, & Kevin Kelly, Wiley Learning Institute
-
8/13/2019 Effective SLO Assessment in the Online Environment
3/61
What is your role at your institution?
Faculty
Professional Development/Instructional Design
Academic Technology/eLearning
Student support staff
AdministrationAcademic Affairs
Administration - Student Affairs
Administration - Other
3Effective SLO Assessment in the Online Environment - Module 1
-
8/13/2019 Effective SLO Assessment in the Online Environment
4/61
-
8/13/2019 Effective SLO Assessment in the Online Environment
5/61
What are your SLO responsibilities?
SLO and/or assessment committee member
Department coordinator for SLO/assessment
Course author/creator
Instructorconduct assessment of SLOs
Staffsupport the systems where SLO data lives
SLO lead for department
Assessment officer
5Effective SLO Assessment in the Online Environment - Module 1
-
8/13/2019 Effective SLO Assessment in the Online Environment
6/61
Overall agenda
Choose appropriate assessment strategies
Choose appropriate assessment tools
Managing consequencesof online SLOassessment
Analyze, report, and act upon online SLO data
6Effective SLO Assessment in the Online Environment - Module 1
-
8/13/2019 Effective SLO Assessment in the Online Environment
7/61
Effective SLO Assessment
in the Online Environment
Module 1:
Choose appropriateassessment strategies
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
Jim Julius, MiraCosta College, & Kevin Kelly, Wiley Learning Institute
-
8/13/2019 Effective SLO Assessment in the Online Environment
8/61
-
8/13/2019 Effective SLO Assessment in the Online Environment
9/61
(Re)imagine Assessing
Learning Outcomes
-
8/13/2019 Effective SLO Assessment in the Online Environment
10/61
(Re)imagine Assessing Learning Outcomes
Key factors to keep in mind:
Consider the level(s) of learning you want students to
achieve or demonstrate Make the objectives measurable
Avoid common misconceptions
TRUE or FALSE:onlinemeans all learning or assessment must take
place behind a computer screen.
TRUE or FALSE:distanceeducation means students are located far
away
Effective SLO Assessment in the Online Environment - Module 1 10
-
8/13/2019 Effective SLO Assessment in the Online Environment
11/61
-
8/13/2019 Effective SLO Assessment in the Online Environment
12/61
(Re)imagine Assessing Learning Outcomes
Consider Level(s) of Learning
Psychomotor (skills)
Observing: Active mental attending of a
physical eventImitating: Attempt to copy a physical behavior
Practicing: Try a specific physical activity over
and over
Adapting: Fine tuning, make minor adjustmentsin a physical activity
12Effective SLO Assessment in the Online Environment - Module 1See http://ets.tlt.psu.edu/learningdesign/objectives/psychomotor
Image source:
http://www.sharjah.ac.ae
http://ets.tlt.psu.edu/learningdesign/objectives/psychomotorhttp://ets.tlt.psu.edu/learningdesign/objectives/psychomotor -
8/13/2019 Effective SLO Assessment in the Online Environment
13/61
(Re)imagine Assessing Learning Outcomes
Consider Level(s) of Learning
Affective (attitudes or feelings)
Receiving: Shows awareness.
Responding: Shows attention and motivation.
Valuing: Demonstrates commitment.
Organizing: Integrates and prioritizes new
values.
Characterizing by value: Behavior reflects new
values.
13Effective SLO Assessment in the Online Environment - Module 1http://www.uwsp.edu/education/lwilson/curric/affectiv.htm
Image source:
Karin Kirk, SERC
http://serc.carleton.edu
http://www.uwsp.edu/education/lwilson/curric/affectiv.htmhttp://www.uwsp.edu/education/lwilson/curric/affectiv.htm -
8/13/2019 Effective SLO Assessment in the Online Environment
14/61
Thought prompt
To what extent is it
possible for students
to achieve higherlevel outcomes in
online/distance
environments? Harder
Roughly the same
Easier14Effective SLO Assessment in the Online Environment - Module 1
Creative
CommonsAttribution:
billsoPHOTO
http://www.flickr.com/photos/billsophoto/5243121852http://www.flickr.com/photos/billsophoto/5243121852http://www.flickr.com/photos/billsophoto/5243121852http://www.flickr.com/photos/billsophoto/5243121852http://www.flickr.com/photos/billsophoto/5243121852http://www.flickr.com/photos/billsophoto/5243121852http://www.flickr.com/photos/billsophoto/5243121852http://www.flickr.com/photos/billsophoto/5243121852http://www.flickr.com/photos/billsophoto/5243121852 -
8/13/2019 Effective SLO Assessment in the Online Environment
15/61
(Re)imagine Assessing Learning Outcomes
Consider Environment
Tech-enabled F2F classes
Assessing outcomes might reflect
enhanced capabilities with tech
Flipped classes
Assessing outcomes might reflect
action or performance focus
Online classes
Assessing outcomes might include
student interaction with the world
15Effective SLO Assessment in the Online Environment - Module 1
-
8/13/2019 Effective SLO Assessment in the Online Environment
16/61
-
8/13/2019 Effective SLO Assessment in the Online Environment
17/61
Select Appropriate
Assessment Strategies
-
8/13/2019 Effective SLO Assessment in the Online Environment
18/61
-
8/13/2019 Effective SLO Assessment in the Online Environment
19/61
Assessing performance
Different strategies used for class assessments
Written: tests, quizzes, essays, written assignments,
case studies Oral: recitations, student presentations, oral exams
Manual: lab practicum, individual and team projects,skills observation
What other types of assessments do you use?
Effective SLO Assessment in the Online Environment - Module 1 19
-
8/13/2019 Effective SLO Assessment in the Online Environment
20/61
Reasons to assess learning differently
Traditional F2F Methods
Standardized testing
(multiple choice, etc.)
Written work (essays,research reports, etc.)
Student presentations
Lab work
Team projects
Possible F2F Issues
Does not support all
learners
Hard to require studentfeedback in classroom
Not enough time in class;
usually not recorded
Hard to make up if youmiss the lab exam
Hard to determine who
did what, and when
Effective SLO Assessment in the Online Environment - Module 1 20
-
8/13/2019 Effective SLO Assessment in the Online Environment
21/61
Thought prompt
Does one assessment provide enough
information to show that a student has
learned something?
21Effective SLO Assessment in the Online Environment - Module 1
-
8/13/2019 Effective SLO Assessment in the Online Environment
22/61
Apply Universal Design for Learning principles
Universal Design for Learning (UDL)
The core goal is to Teach every student
Principles guide instructors to provide multiple pathways forstudents to succeed
Representation: Share content in more than one format
Engagement: Use multiple ways to make learning meaningful
Expression: Provide more than one way for students to show whatthey know
See full presentation about UDL and online assessment
22Effective SLO Assessment in the Online Environment - Module 1
http://prezi.com/1rm9g73wkic_/udl-and-online-assessment/http://prezi.com/1rm9g73wkic_/udl-and-online-assessment/http://prezi.com/1rm9g73wkic_/udl-and-online-assessment/http://prezi.com/1rm9g73wkic_/udl-and-online-assessment/ -
8/13/2019 Effective SLO Assessment in the Online Environment
23/61
Providing assessment choices and levels
Provide alternative questions
Provide different assignments
Let students submit in different formats
Use two or three of these strategies together
23Effective SLO Assessment in the Online Environment - Module 1
-
8/13/2019 Effective SLO Assessment in the Online Environment
24/61
-
8/13/2019 Effective SLO Assessment in the Online Environment
25/61
Effective SLO Assessment
in the Online Environment
Module 2:
Choose appropriateassessment tools
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
Jim Julius, MiraCosta College, & Kevin Kelly, Wiley Learning Institute
-
8/13/2019 Effective SLO Assessment in the Online Environment
26/61
-
8/13/2019 Effective SLO Assessment in the Online Environment
27/61
Select tools for online activities
-
8/13/2019 Effective SLO Assessment in the Online Environment
28/61
Using technology for assessment
Low-stakes quizzes/High-stakes exams
Quizzes through a Learning Mgmt System (e.g., Moodle)
Written assignments - Iterative/peer-review process
Calibrated Peer Review
Online paper submission tools (e.g., Turnitin)
Electronic portfolios
Written assignments - Grading/feedback process Screencast (e.g., Screenr, Echo360)
Tablet Apps (iAnnotate)
Effective SLO Assessment in the Online Environment - Module 2 28
-
8/13/2019 Effective SLO Assessment in the Online Environment
29/61
Using technology for assessment
Individual or small group presentations
Online presentation tools (e.g., VoiceThread)
Labs
Commercial lab kits sent to studentshomes
Kitchen sciencelab
Virtual labs/simulations
Intelligent tutors (e.g., ALEKS)
Effective SLO Assessment in the Online Environment - Module 2 29
-
8/13/2019 Effective SLO Assessment in the Online Environment
30/61
Using technology for assessment
Individual or small group projects
Google docs or wiki pages for team organization/coordination/workflow
Blogs or ePortfolios to share and assess final products
Performance-based assessment (science, creative arts)
Upload observation logs by trusted expert(s)
Audio/video recording
Other?
Effective SLO Assessment in the Online Environment - Module 2 30
-
8/13/2019 Effective SLO Assessment in the Online Environment
31/61
Choosing technology for assessment
Effective SLO Assessment in the Online Environment - Module 2 31
synchronous
asynchronous
physical digital
collaborative
individual
x: where
y: when
z: who
-
8/13/2019 Effective SLO Assessment in the Online Environment
32/61
Choosing appropriate technology
Questions to consider
How will using this technology help students learn?
Do students have access to this technology?
Do students have access to training and support to
use this technology?
Is this technology accessible to students with
disabilities?
Effective SLO Assessment in the Online Environment - Module 2 32
-
8/13/2019 Effective SLO Assessment in the Online Environment
33/61
-
8/13/2019 Effective SLO Assessment in the Online Environment
34/61
Choosing appropriate technology
Resources to investigate
Virtual resource site - teaching with technology
http://www.umuc.edu/virtualteaching/
Discussion-based online teaching
http://cgi.stanford.edu/~dept-ctl/tomprof/postings.php (#561)
How technology influences student success
MERLOT & Pachyderm (CSU tools) http://www.merlot.org; http://www.pachyderm.org
Effective SLO Assessment in the Online Environment - Module 2 34
http://www.umuc.edu/virtualteaching/http://cgi.stanford.edu/~dept-ctl/tomprof/postings.phphttp://caret.iste.org/index.cfm?fuseaction=answers&QuestionID=1http://www.merlot.org/http://www.pachyderm.org/http://www.pachyderm.org/http://www.merlot.org/http://caret.iste.org/index.cfm?fuseaction=answers&QuestionID=1http://cgi.stanford.edu/~dept-ctl/tomprof/postings.phphttp://cgi.stanford.edu/~dept-ctl/tomprof/postings.phphttp://cgi.stanford.edu/~dept-ctl/tomprof/postings.phphttp://www.umuc.edu/virtualteaching/ -
8/13/2019 Effective SLO Assessment in the Online Environment
35/61
-
8/13/2019 Effective SLO Assessment in the Online Environment
36/61
Activity
1. Make a teaching-with-technology plan
2. Share elements of your plan with the group
Objectives Resources Activities Assessments
Effective SLO Assessment in the Online Environment - Module 2 36
-
8/13/2019 Effective SLO Assessment in the Online Environment
37/61
The role of rubrics
Tie rubric criteria to specific outcomes
Repeatedly assess across multiple assignments
37Effective SLO Assessment in the Online Environment - Module 1
-
8/13/2019 Effective SLO Assessment in the Online Environment
38/61
Consider support for assessment tools
Campus apps (e.g., LMS featuresquizzes or
tests, assignment submissions)
3rdparty apps (e.g., Turnitin, Google Picasa)
Publisher tools (e.g., WileyPLUS)
Digital feedback tools (e.g., digital rubric withinePortfolio, audio/video/screencast feedback)
38Effective SLO Assessment in the Online Environment - Module 1
-
8/13/2019 Effective SLO Assessment in the Online Environment
39/61
Put it together with
ePortfolios
-
8/13/2019 Effective SLO Assessment in the Online Environment
40/61
Gathering evidence over time
Electronic portfolios allow different stakeholders to view
progress of
ePortfolio resource site:
http://eportfolio.sfsu.edu
Effective SLO Assessment in the Online Environment - Module 1 40
http://eportfolio.sfsu.edu/http://eportfolio.sfsu.edu/http://eportfolio.sfsu.edu/http://eportfolio.sfsu.edu/http://eportfolio.sfsu.edu/http://eportfolio.sfsu.edu/http://eportfolio.sfsu.edu/http://eportfolio.sfsu.edu/http://eportfolio.sfsu.edu/ -
8/13/2019 Effective SLO Assessment in the Online Environment
41/61
Align with Outcomes at Multiple Levels
Effective SLO Assessment in the Online Environment - Module 1 41
-
8/13/2019 Effective SLO Assessment in the Online Environment
42/61
ePortfolios enable
42Effective SLO Assessment in the Online Environment - Module 1
-
8/13/2019 Effective SLO Assessment in the Online Environment
43/61
Assessment via ePortfolios
ePortfolios to document:
Self-assessment
Peer review
Instructor
feedback (rubric)
43Effective SLO Assessment in the Online Environment - Module 1
-
8/13/2019 Effective SLO Assessment in the Online Environment
44/61
Assessment
examples
Rubric
comments
can be sharedin addition to
automated
systems
44Effective SLO Assessment in the Online Environment - Module 2
-
8/13/2019 Effective SLO Assessment in the Online Environment
45/61
provide
feedback
& rubrics
encourage& share
exemplars
cultivate
critical
thinking
elevate
self
evaluation
reward
peer
review
use game
& UDL
elements
provide
guidelines
& grades
foster
learning
improvement
require
attention
to purpose
make
collaboration
a priority
Synthesis
45Effective SLO Assessment in the Online Environment - Module 2
-
8/13/2019 Effective SLO Assessment in the Online Environment
46/61
Added benefits of ePortfolios
Demonstration of student achievement
Accreditation
Program Assessment
Student Bridge to Transfer/Workforce
Student Recruitment
46Effective SLO Assessment in the Online Environment - Module 1
-
8/13/2019 Effective SLO Assessment in the Online Environment
47/61
Effective SLO Assessment
in the Online Environment
Module 3:
Managing consequencesof online SLO assessment
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
Jim Julius, MiraCosta College, & Kevin Kelly, Wiley Learning Institute
-
8/13/2019 Effective SLO Assessment in the Online Environment
48/61
-
8/13/2019 Effective SLO Assessment in the Online Environment
49/61
Potential Benefits
Students may develop showcase
resources beneficial to their next steps
Thinking about improving online SLO
assessment may result in improvements to
on-ground courses
Other?
Effective SLO Assessment in the Online Environment - Module 1 49
-
8/13/2019 Effective SLO Assessment in the Online Environment
50/61
-
8/13/2019 Effective SLO Assessment in the Online Environment
51/61
Discuss:
Authentication & Academic Integrity
Do you (should you) require use of an
institutional course management system
for online assessment?
What do you do about third party
systems?
Under what circumstances should
proctoring be encouraged? Required? And
what should proctoringinclude?
Effective SLO Assessment in the Online Environment - Module 1 51
-
8/13/2019 Effective SLO Assessment in the Online Environment
52/61
Effective SLO Assessment
in the Online Environment
Module 4:
Analyze, report, and actupon online SLO data
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
Jim Julius, MiraCosta College, & Kevin Kelly, Wiley Learning Institute
-
8/13/2019 Effective SLO Assessment in the Online Environment
53/61
Potential Concerns
Institutions may not be collecting/
aggregating assessment data in useful
ways related to outcomes
Institutions may not be disaggregating
assessment data into DE and non-DE
Institutions may not be seriously looking at
DE vs. non-DE results and following
through with improvement plans
Effective SLO Assessment in the Online Environment - Module 1 53
-
8/13/2019 Effective SLO Assessment in the Online Environment
54/61
Activity
In your table, look at the data packet
provided. Assume you are all part of an
academic discipline/department.
What does the data tell you? What does it
not tell you? What questions does it cause
you to ask?
What is required to get buy-in for this kind
of review process?
Effective SLO Assessment in the Online Environment - Module 1 54
-
8/13/2019 Effective SLO Assessment in the Online Environment
55/61
-
8/13/2019 Effective SLO Assessment in the Online Environment
56/61
-
8/13/2019 Effective SLO Assessment in the Online Environment
57/61
Effective SLO Assessment in the Online Environment - Module 1 57
-
8/13/2019 Effective SLO Assessment in the Online Environment
58/61
Effective SLO Assessment in the Online Environment - Module 1 58
-
8/13/2019 Effective SLO Assessment in the Online Environment
59/61
Points to Ponder
Do tools and processes support meaningful
collection & analysis of course SLO data by
course modality? Is there faculty buy-in for this? Are DE courses defined, tracked, and reported
in a consistent manner?
Is it possible/reasonable to make comparisonsacross departmental, college-wide and
statewide DE data? What are the limitations?
Effective SLO Assessment in the Online Environment - Module 1 59
-
8/13/2019 Effective SLO Assessment in the Online Environment
60/61
Points to Ponder
Do you have a way of tracking DE
program outcomes? (Do you have a way
of tracking DE programs?)
What will the ultimate outcome be of SLO
mania combined with ratcheting up DE
standards?
Effective SLO Assessment in the Online Environment - Module 1 60
-
8/13/2019 Effective SLO Assessment in the Online Environment
61/61
top related