iacbe conference presentation-2015
TRANSCRIPT
![Page 1: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/1.jpg)
Measuring Program Objectives Via Online Assessment
Julia Cronin-Gilmore, Ph.DTracy Gies
Bellevue University
International Assembly for Collegiate Business Education
![Page 2: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/2.jpg)
About your Presenters…Julia
![Page 3: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/3.jpg)
About your Presenters…Tracy
![Page 4: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/4.jpg)
We are from…
![Page 5: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/5.jpg)
Some think of it as…
![Page 6: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/6.jpg)
Some think of it as…
Gates - $81.6 Billion, Buffett $74.8 Billion
![Page 7: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/7.jpg)
Some even think of it as…
![Page 8: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/8.jpg)
Background• New Degree, M.S., Marketing• 12 Courses
![Page 9: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/9.jpg)
Create Program Objectives• Demonstrate problem solving and critical
thinking skills in order to be successful in the business world
• effective communication skills and professionalism when working with others in a business setting.
• Demonstrate effective interpersonal skills and an appreciation of business ethics.
• Demonstrate the ability to apply content knowledge to create value as business leaders in an organization.
![Page 10: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/10.jpg)
Two Direct & Two Indirect Measures
Direct measures:Entrance & Exit Exams: 80% of students will score at least 73% or greater, which is at a level of proficiency.
![Page 11: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/11.jpg)
Two Direct & Two Indirect Measures
Indirect Measures:Mid-term and End of Program Surveys: At least 80% of the students' will note the ability to apply program learning outcomes as noted by their responses of "Agree" or "Strongly Agree".
![Page 12: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/12.jpg)
Measure Program ObjectivesTwo Direct Measures• In other programs, we used a
business plan and a group of professors would evaluate their specialty area (marketing, finance,…).
• Found to be a time consuming process.
• Needed something manageable for one person
• Decided to try analyzing a case study
![Page 13: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/13.jpg)
Case AnalysisFour Cases Total• All four are equal in length – about ¾
page.• Each has multiple things going on
Three Central Questions• What are the relevant facts of the case?• What are the ethical issues?• What are the ethical alternatives?
![Page 14: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/14.jpg)
Exam #1 ResultsStudents didn’t score so well…
12.21 16.59 23.65
30 Possible Points
![Page 15: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/15.jpg)
Test ResultsTest Results were Skewed…• Students are used to answering once
on a multiple choice exam.• There were several answers that were
correct.• If students answered correctly, they
received a point• If students missed a correct answer, it
deducted points from ones they earned.
![Page 16: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/16.jpg)
Going Forward• Questions reflect how many correct
answers are possible.• Students will earn one point per
correct answer• Students will still loose points earned
for incorrect answers (couldn’t fix in the system).
![Page 17: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/17.jpg)
Test ResultsTest Results were Skewed…• Exam #1 was extra credit• Eliminated the issue with it being a
major portion of their grade or• Something I had to prepare them for.• Most were encouraged to take• Not every student completed the
exam.
![Page 18: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/18.jpg)
Measuring Program Objectives Via Online Assessment
Part IISetting It Up in the LMS
![Page 19: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/19.jpg)
Agenda
• Why use cases online?• Mapping• Creating the tests in the LMS
![Page 20: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/20.jpg)
Why Use Cases
• Authenticity (Suskie, 2009)
• Effectiveness and student acceptance of CBL (Barnes et al, 1994; Srinivasan et al, 2007)
• Program curriculum favors CBL learning
• CBL in assessment good tie-in
![Page 21: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/21.jpg)
Issues
• Grading responses to cases can be very time consuming
• There can be a wide range of correct answers
• Setting up auto graded tests that capture a wide range of correct answers can be difficult
![Page 22: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/22.jpg)
Our Approach
• Use Multiple Answer questions• Try to capture as broad of a range as
possible of correct answers in terms of ethical considerations, and resolutions
![Page 23: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/23.jpg)
What We’ve Learned So Far
• Generally well received by students• Refine instructions
![Page 24: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/24.jpg)
Mapping
• Provides a common document to prompt discussion for all parties involved in course design, development and review
• Keeps the two main components of learning in a program (program outcomes and learning objectives) a the forefront of design and development discussions
• Helps ensure alignment of program outcomes and course objectives
![Page 25: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/25.jpg)
How to use the Excel Mapping Tool
• Some cells are linked so you can enter info on the Data Entry worksheet, and it will fill the appropriate cells on the other worksheets.
• Demo
![Page 26: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/26.jpg)
Setting up the Test
• Blackboard 9.1.201404.160205• Fairly standard Multiple Answer set
up• Partial Credit• Group cases and MC questions
![Page 27: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/27.jpg)
Set Up in Blackboard
![Page 28: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/28.jpg)
Questions?
![Page 29: IACBE Conference Presentation-2015](https://reader030.vdocuments.us/reader030/viewer/2022032504/55c3e9dfbb61eb52458b46fb/html5/thumbnails/29.jpg)