from learning outcomes to testing

7/27/2019 From learning outcomes to testing

1/23

Quality Assurance Netherlands Universities

From learning outcomes to testing


2/23

Steps

Intended learning outcomes of the programme

Intended learning outcomes of the course

Develop a teaching and learning strategy to enable students

to achieve learning outcomes Design a assessment method to test if the learning outcomes

have been achieved


3/23

From curriculum to course

Intended learning outcomes for the programme

|

VIntended learning outcomes for courses or modules

How to assess LOs for each course

How to assess if the students have reached the LOs of the

whole programme


4/23

When the student has passed each course can you be sure

that she also achieved the LO of the programme?


5/23

Smart

Specific

Measurable

Acceptable

Realistic

Time limited or defined

Smart formulated LOs are easier to assess


6/23


7/23

Tests and learning outcomes

LO of a module in Economics

On successful completion of this module students should be

able to recognise the main indicators of stock market timing

What kind of test would you use to assess this LO?


8/23


9/23

LO and testing

Another LO of the same module is:

Criticise budgetary decisions using economic criteria

On what level in Blooms taxonomy is this question?


10/23

What kind of questions or tests

This LO is on the highest level of the taxonomy it concerns

evaluation

What kind of test is appropriate?

What would you prefer?


11/23

Validity and representativity

Tests have to be valid. What does that mean

There are several kinds of validity:

Content validity or representative- The test questions need to be representative for the LO and

content of the course

It has no use to ask a question about stock markets for an

Information Sciences course

Criterium validity- The results of a student on a test should be coherent with the

results/ scores of the same student on the same kind of tests


12/23

validity

Concept validity

- When a student scores 60% on a test,does this mean that she in

reality achieved 60% of what was aimed with the course?


13/23

More about different kind of tests

The choice for a certain kind of test can be determined by:

The intended learning outcomes (Blooms taxonomy)

The number of candidates

The amount of knowledge,skills and competences you want totest

Size of the course

Time available for the test

- To answer one mc question takes < 1 minute- To answer one essay question 10-60 minutes


14/23

objectivity

Construction

- The construction of multiple choice tests takes a lot of effort

when you want to do it right

- You have to make sure that all the intended learning outcomes

are covered by the tests ( and not only what you like most about

the course)

- Can you test insight, or application,skills with a mc test

Assessment- The assessment of essay questions, papers etc. takes a lot of

effort when you want to do it right


15/23

How to assess?

Quality assurance principles related to assessment

- Validity, I discussed this before

- Objectivity, we want to be sure that you will not favour your

favourite students- Reliability, is the test distinguishing between the good and the

bad students


16/23

How to?

- Statistics (test the tests)

- Four eyes- applied to construction of the test as well as the

assessment

- External reviewers, not the teachers themselves butindependent reviewers

- Answer models that describe what kind of answer you expect

on an essay question, what elements should be part of it,

which concepts should be named and how you weigh these

aspects in the final mark- Assessment forms to be used for theses, reports etc.


17/23

Tools

Test matrix

- a refined list of all LO of the course or the programme. This can

be useful when there are a lot of items a student has to know

and when you can not test them all in one exam. A test matrix

can categorize these items and guarantee that all items have afair chance to come into the test.

Rubrics

- Rubrics can help to when it is not easy to establish a final mark

e.g. for scoring a presentation or a report. They can be seen as

refined assessment forms or answering models.


18/23

More about Test Policy

Back to the question: How to assess if the students have

reached the LOs of the whole programme?

What if all the courses are assessed by multiple choice

exams?

What if all the courses ask the students to give an oral

presentation and never a paper or another kind of written

report.

What about group work how to assess that?


19/23

Some issues of test policy in NL

Both bachelor and master programmes require a kind of final

thesis by the student, which should indicate that the graduate

has achieved the LO of the programme

bachelor theses: group work with guarantees that the individual

input of every group member is visible; specific assessment

forms are used for this.

Programmes aim at a variety of test forms in the curriculum andmake sure that all kind of academic skills are tested during

the programme


20/23

Responsibility

Boards of Exams are responsible for the quality of the exams,

they monitor the tests, and regularly review the graduation

theses to check if they fulfill the requirements


21/23

How to ensure quality

Did the graduates achieve the intended learning outcomes

Is the institute in control

Does the programme reach its objectives


22/23

Any questions?

Literature: Marjorie Devine & Nevart Yaghlian:

Test Construction Manual,Construction of Objective Tests.

http://www.cte.cornell.edu/documents/Test%20Constructio

n%20Manual.pdf

John Biggs: Aligning teaching and assessing to course

objectives. Teaching and Learning in Higher Education:

New Trends and Innovations. University of Aveiro, 2003
http://www.cte.cornell.edu/documents/Test%20Construction%20Manual.pdfhttp://www.cte.cornell.edu/documents/Test%20Construction%20Manual.pdfhttp://www.cte.cornell.edu/documents/Test%20Construction%20Manual.pdfhttp://www.cte.cornell.edu/documents/Test%20Construction%20Manual.pdf


23/23

Test policy in your university

What kind of tests are used in your degree programme?

(Name at least three)

How do you know that the graduates have achieved theintended learning outcomes?

Who is responsible for the quality of exams?

from learning outcomes to testing

Documents