müller-benedict: grade inflation and other challenges to grading grade inflation and other...

Müller-Benedict: grade inflation and other challenges to grading

Grade inflation and other challenges facing grading

An analysis from a long-term perspective

Volker Müller-Benedict

Universität Flensburg

Oslo, Oktober 2015


German: 1 2 3 4 5 6

Value: best -----------------------worst | failed

No central authority to rule the system of higher education: 16 federal states, each with autonomy to rule the educational

system Freedom of investigation and teaching as a basic law (Art 5.3): each

university can regulate teaching in its own way

The German grading scale

Norwegian: A B C D E F

2 of 24


Diversity of examination regulations

The final grade may be calculated by different weights of the

component testings The final grade may be calculated by a different number of

component testings

The subdivisions of the final grades may be different:

sometimes only whole numbers 1, 2, 3, 4;

sometimes half numbers 1, 1.5, 2, 2.5,..

the majority allows thirds:1, 1.3, 1.7, 2, 2.3, 2.7, 3.0, 3.3, 3.7, 4.0

Leads to a diversity of examination regulations:

3 of 24


Types of final degrees: • „Staatsexamen“ (state examinations: lawyers, clergymen,

teachers, physicians)• „Diplom“ (natural and social sciences), • „Diplom (FH)“ (universities of applied sciences)• „Magister“ (humanities)

Since Bologna:• BA Bacchelor• MA Master

For the admission to MA the grades of the BA exam are crucial.

Types of examinations (oral, written, possible repetitions etc.) may

be very different and depend on the specific examination regulations

of a university

Types of degrees and exams

4 of 24


Previous investigations

Investigations in the U.S. state „grade inflation“ (increase of better grades without better performance). Some universities prescribe proportions of A‘s and B‘s.

Empirical findings:

In 2003, 2007 and 2012 the German „Wissenschaftsrat“published grades of all German universities. Remarkable differences in mean grades between the universities were detected, also a trend to grade inflation.

Betterment of grades can be• compliant to performance • independent of performance ( only then: grade inflation)

Leading question:

How comparable are grades, between disciplines, universities and over

time spans?

5 of 24


Betterment of grades independent of performance

If the proportion of better performance, i.e. A‘s, increases, other performances being equal, the standard deviation remains the same. But:

How to detect grade inflation?

D

C

B

A

Grades in ‘math. teach.‘

Standard deviation of grades in ‘mathematics for teaching‘

3,33,0

2,6

2,2

1,8

1,0

Grades No Ge

6 of 24


Research questions, theoretical distribution of grades

Research questions:• Is there grade inflation of German exams?• Are there stable differences in grades between disciplines and

universities?• How can we explain these facts? Are these trends independent of

performance?

Theoretical distribution of grades• ‘Absolute‘ anchor: the actual horizon of knowledge and competencies,

no relations to individual or group levels• Not ‘too simple‘ or ‘too difficult‘: distribution over whole scale

Necessary data:• Dense in the time dimension• Long-term, minimum 25 years

7 of 24


Data and methods

Data sampled• about 1960 to 1997: of archives of 7 universities (by Thomas Gaens) • since 1996: electronic database of federal statistical bureau (distatis)• diploma, master of arts or state examination (BA‘s not before 2000)• group discussions with examiners (mostly university professors) (by

Elena Tsarouha)

Two parts:1. Descriptive results: long-term development

methods: graphics2. Analysis: Explanation of cycles and grade inflation

methods: time-series regression

Additionally:• Quotations from group discussions• Some possible recommendations

8 of 24


grades – long-term trends of different disciplines (1)

(Smoothed by lowess (0.2))

grade inflation

cycles

1950

1952

1954

1956

1958

1960

1962

1964

1966

1968

1970

1972

1974

1976

1978

1980

1982

1984

1986

1988

1990

1992

1994

1996

1998

2000

2002

2004

2006

2008

2010

1.00

1.50

2.00

2.50

3.00

BWL

VWL

Deutsch LA

Mathematik LA

Chemie

Mathematik

Psychologie

Biologie

Busi. econ.Macroecon.Germ. teach.Math. teach.ChemistryMath. dipl.PsychologyBiology

9 of 24


grades – long-term trends of different disciplines (2)

„[…] There are also scientific cultures, we didn’t yet speak about that, if for example I had to examine together with these actual colleagues her in [ univ. A], then I would probably, we would quarrel during the exam, about evaluations. Because they have other scientific paradigmas, which I can’t – say, handle, and reversely they not mine […]“

1959

1961

1963

1965

1967

1969

1971

1973

1975

1977

1979

1981

1983

1985

1987

1989

1991

1993

1995

1997

1999

2001

2003

2005

2007

2009

1

1.5

2

2.5

3

3.5

Jura Staatsex-amen

Sozi-ologie Magis-ter

Law: grades restricted by specific laws

Sociology

10 of 24


grades – long-term trends of different universities (1)


Mathematics for teaching secondary level

1960

1962

1964

1966

1968

1970

1972

1974

1976

1978

1980

1982

1984

1986

1988

1990

1992

1994

1996

1998

2000

2002

2004

2006

2008

2010

1.4

1.9

2.4

2.9

Göttingen

Braunschweig

Karlsruhe

Berlin

Tübingen

GöttingenBraunschweigKarlsruheBerlinTübingen

11 of 24


grades – long-term trends of different universities (2)


German language Magister (not teaching)

„[…] What worries me most although is the difference of quality between [univ. A] and [univ. B]. There is no way. Who gets a 2 in [univ. A], recieves a 1 in [univ B] […]“

1970

1972

1974

1976

1978

1980

1982

1984

1986

1988

1990

1992

1994

1996

1998

2000

2002

2004

2006

2008

2010

1.4

1.9

2.4

2.9

Göttingen

Berlin

Tübingen

Heidelberg

Münster

Saarbrücken

GöttingenBerlinTübingenHeidelbergMünsterSaarbrücken

12 of 24


grades – long-term trends of different types of exams


„I would say – in state examinations I was more severe, always more severe than in the old Magister. Why? I don’t know exactly but I would think I was more severe because I had a clearer impression of the demands on teaching German in school than I had of the demands on a Magister who can aspire such a lot of different careers “

1960 1970 1980 1990 2000 2010 20201

1.5

2

2.5

Math. dipl

Math.teach

Mathematicsstate examina-tion (for tea-ching in sec.schools)

Mathematics diploma

13 of 24


Descriptive results - summary

1. Over four decades grade inflation is proceeding in most disciplines.

2. Remarkable stable differences of mean grades between the disciplines.

3. Considerable differences of mean grades between universities (same discipline), stable over one ore more decades.

4. Stable considerable differences between state examinations and universitarian exams (diploma, magister)

5. Mean grades are fluctuating with periods of different bandwidth around 20 years.

Part 2: Explanation of cycles and of grade inflation

14 of 24


Causes and influences on grading

Differentiation between one-time and long-term influences:

• One-time: change of examination regulations, evaluations, options of choicing courses with „good grades“, change of examiners,…

• Long-term: relation student / professor (quality of teaching), intention of grading harsh or soft-gloved, social combination of the students, change of „teaching generations“,…=> are correlated with varying number of tested students (leading indicator)!

Many imaginable influences !

Cycles of the number of a) first semester, b) students in general, and c) of exams have same shape, but time-delayd by half a study length.

15 of 24


Synchronous cycles of frequency

Frequency of mathematics state examinations (for teaching in sec. schools)

5 universities, absolute numbers 5 universities, standardized

Synchronous cycles: reaction to a common national job market for teachers with business cycles of about 25 years.

16 of 24


Asynchronous cycles of frequency

Frequency of exams in German language (magister, not for teaching in schools, 5 universities standardized)

No common cycles, frequencies don‘t represent a common cause.

17 of 24


Two distinct groups of academic careers

• Common national job market (teachers: only one employer, the state)• Clear job profile• Shortage and abundance of applicants are a theme of political and

public discussions• The perception of the job market can influence the examiners

1. Job market - dependent careers (e.g. mathematics for teaching)

• Diverse jobs possible • No common job market cycles• Shortage or abundance of applicants are notperceived by the public• Situation of the job market has no influence on examiners

2. Not job market - related careers (e.g. German language magister)

18 of 24


Relation fequeny - grades (1)

Mathematics state examinations (for teaching in sec. schools), all universities

grades

frequency (shifted backward 3 years)

Frequency indicator of actual job market conditions

19 of 24


Statistical results

Method: Prais-Winston-Regression

German language magister: same direction of counts and grades

Mathematics state examinations for teaching: reverse direction of counts and grades

Grade_Math_teach Coef. P>|t

Freq_Math_teach (Lead 3)

- 0.208 0.002

time - 0.030 0.061

constant +58.27 0.063

Grade_Germ_MA Coef. P>|t

Freq_Germ_MA +0.580

0.000

time - 0.105 0.000

constant +208.8 0.000

20 of 24


Explanation of grade inflation

Why grade inflation? How cycles are related to growth?

Assumption: grades are more elastic in direction „better“ than in direction „worse“

Method:• First differences of data• Disaggregation of the timeseries of frequencies into two

timeseries „only growth“ and „only decrease“

Corollary:If the assumption holds, long term grade inflation follows as a consequence.

21 of 24


Different elasticity of grades with respect to counts

Diff_Grade_Math_teach Coef. SE P>|t

Pos growth (shortage) - 0.377 0.057 0.000

Neg growth (abundance)

+0.023 0.067 0.731

Mathematics for teaching

Different significancies according to different effects of shortage or abundance to grades.

1970

1973

1976

1979

1982

1985

1988

1991

1994

1997

2000

2003

2006

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

In phases of shortage (red curve above the x-axis, frequency increasing) grades are improving (blue curve, marks decrease); in phases of abundance (green curve below the x-axis) grades remain stable or worsen only a bit.

22 of 24


Summary

„[…] However my inner grade always was a little bit worse than the grade I gave, because particularly in the exam Magister…Because you have such big doubts, o god, how could those people get along? And if they were thrown into life with a bad grade of Magister additionally, then at the end they will become nothing[…].“

Student frequency is an „overloaded“ indicator, points to various conditions, recomendations are not obvious.

The long-term German data show• Relation of grades to the cyclical movement of student frequency.• Grades have tendency to improve because of less elasticity to

worsen than to improve.

23 of 24


Summary

Because grades are exposed to many influences, which do not reflect the

performance, they should generall not be treated as that exact

number/letter, which appears on the certificate!

Thank you very much für your attention!

24 of 24

müller-benedict: grade inflation and other challenges to grading grade inflation and other...

Documents