Student Learning Study Status of Student Learning across 18 States of India in Urban and Rural Schools

SUMMARY

Student Learning Study (SLS) is a benchmarking study of student achievement conducted by Educational Initiatives (EI), with the support of Google.org, USA. Tests were conducted with the permission of the state governments.

Salient Features of the Study

Test Administration:

?

?Standardization through master trainings and evaluator trainings

?Quality control with detailed manuals, checklists and field audits

?Performance data captured through OMR sheets, coded by evaluators

Carried out by EI’s team of trained evaluators

The length of the line in the figure above is 4 cm. How long is the pencil shown in the picture below?(Use the ruler shown in the picture.)

Video studies in 3 states to analyse misconceptions through student interviews

Coverage:?18 states and 1 Union Territory?2,399 Urban & Rural Govt. Schools?About 1.6 lakh students tested ?2 Stage Stratified Cluster Sampling

?Additional oral reading test administered one-on-one to students

?Scoring rubrics with detailed granular codes to capture different answers

?Separate questionnaires for collecting background information about the student, school, teacher and head of the school

Class 4 Maths Matrix Plot

1 - 7

8 - 14

15 - 21

22 - 28

29 - 35

36 - 37

Sophisticated analysis using Item Response Theory

A Class 4 student explaining how he understood the question.

Misconception and common error

analysis to diagnose learning issues

Which rational number does the pointer on the number line show?

A. - 1.2

B. - 0.4

C. 1.2

D. - 0.8

Tools:

?Students of classes 4, 6 and 8 tested in Language and Maths

?Scientifically designed, full length tests covering all the competencies

?Multiple choice and free response items

?Questions check for deeper understanding of concepts apart from procedural learning

?Comparison on common questions from international tests, such as, TIMSS and PIRLS

Multivariate analysis to identify factors associated with students learning

Availability of granular data that can be drilled to question level

Andha Pradesh 1 Hyderabad 2 Kadapa 3 Mahabubnagar4 West GodavariAssam5 Dhubri6 KamrupBihar7 Araria8 Patna9 Saharsa 10 Saran

Chandigarh11 ChandigarhChhattisgarh12 Raipur13 RajnadgaonDelhi14 Delhi Gujarat15 Ahmedabad 16 SurendranagarHaryana17 Gurgaon 18 Rewari

Jammu & Kashmir 19 Jammu20 Kathua Jharakhand 21 Chatra 22 RanchiKarnataka23 Bangalore 24 Gulbarga Kerala25 Thiruvananthapuram 26 Waynad

Orissa 35 Khordha36 Koraput Punjab37 Ludhiana38 SangrurRajasthan39 Dungarpur40 Jaipur41 Jodhpur42 Tonk

Madhya Pradesh 27 Betul 28 Bhopal 29 Hoshangabad30 PannaMaharashtra 31 Nagpur 32 Nanded 33 Nashik34 Yavatmal

Tamil Nadu 43 Chennai44 Dharmapuri 45 Kanniyakumari 46 Pudokottai Uttarakhand 47 Dehradun 48 Tehri Garhwal

Scale Anchoring to provide benchmarks of what students know and are able do at the Low, Intermediate, High and Advanced ability levels

0 20 8060 10040

10.1 26.0 55.1 76.5

15.0 27.6 51.0 76.3

14.2 28.0 52.1 77.4

6.9 21.1 47.2 73.2

10.5 30.0 59.2 84.3

16.6 32.1 58.9 82.1

Language 6

Language 8

Language 4

Maths 4

Maths 6

Maths 8

Percentage of Students Reaching SLS 2009 National Benchmarks

Percentages of Students Reaching National Benchmarks Advanced Benchmark

High Benchmark

Intermediate Benchmark

Subject Low Benchmark

5. Percentage of students below Low Benchmark.

1. Percentage of students at or above Advanced Benchmark.

2. Percentage of students at or above High Benchmark.

3. Percentage of students at or above Intermediate Benchmark.

4. Percentage of students at or above Low Benchmark.

0

100

12

34

5

%

Graph above shows the cumulative percentage of students reaching each benchmark.

The heaviest box is ________________.

The lightest box is _______________.

Sample Question 8: The question tests if the child is able to understand the concept of weights and then reason out based on the information given.

© Educational Initiatives Pvt. Ltd.

The union territory of Chandigarh, although tested, has not been included in this comparison. Delhi, is also not included in this comparison as only class 4 was tested here.

1. In the lower classes, a fair amount of 'rote-based' or 'procedural' learning is evident in very basic numeracy skills

such as number sequencing, operations of whole numbers involving 1 or 2 digits; naming of numbers; reading clock time, understanding currency,

etc. Even among procedural questions, students are able to comparatively handle only 'straightforward' questions that are closer to what one would

practise from a typical textbook and not when they are slightly atypical. In higher classes, students are falling behind in all learning, even procedural.

Responses to some questions suggest that students are probably coping through learning happening outside the class.

2.

The learning that is happening seems to be procedural or rote-based and not one of 'Learning with Understanding' as students find it difficult

to answer questions that require a deeper understanding of the concept.

3. Student performance in common questions that were used to

check learning gains across classes showed that performance increased as students move from class 4 to 6 to 8 in both language and maths. However,

the extent of improvement was oft n slightly incremental and not a large jump as one would expect. In most cases, nearly 40% of students in class 8

do not seem to have acquired class 4 competencies. Many misconceptions that students have in lower classes still continue in higher classes and in

some cases were found to even become stronger.

4. Their writing does not go beyond the most basic, tried and

tested formulaic sentences they probably trained for while in their lower classes. Their writing shows that they are learning language more as a

subject and less as a means of natural communication. The writing has a number of errors in spelling and grammar, and punctuation marks are

conspicuous by their absence.

5. For e.g., in the oral reading test, about 87.1% of students

in Gujarat could read a simple word, but only 40.3% could read a short passage and only 22.6% could comprehend the information implicit in the

passage they read.

6. Based on the relative performance/rank of the states in

different classes and subjects, an attempt has been made to consolidate the performance of 17 states, as seen in Graph 1. Kerala, Maharashtra, Orissa

and Karnataka are clearly performing overall better than the national average. Jammu and Kashmir, Madhya Pradesh and Rajasthan were among the

states that ranked among the bottom three overall. Bihar performed the same as national average.

Learning levels are extremely low.

Learning taking place is not 'Learning with understanding' and a number of misconceptions exist among students on the concepts

learnt.

Learning gains seen across classes is slightly incremental and not a large jump.

Students find it difficult to express their thoughts in their own words in writing.

In all the states tested, fewer students were found to comprehend what they read.

There are significant state-wise differences in student performance across states.

For example, in class 6, while

more students could add the fractions 2½ and 1 ½ in a word problem using a real life context, fewer of them could add the same when asked as a

straightforward addition question as '2½ +1 ½= __'.

e

Maths Class 4

Write the answer.

43

× 2

Fill in the appropriate number in the box.

3 × = 3 + 3 + 3 + 3

Sample Question 25: This is a procedural question that checks for the process of multiplication of students answered this correctly.

67.1%Sample Question 25: This is a conceptual question that checks whether the student is able to link multiplication with repeated addition of a number. of students answered this correctly. 30.4%

AP - Andhra Pradesh, AS -

HR - Haryana, JK - Jammu & Kashmir, JH - Jharkhand, KA - Karnataka, KL - Kerala,

MP - Madhya Pradesh, MH - Maharashtra, NT - National, OR - Orissa, PB - Punjab,

RJ - Rajasthan, TN - Tamil Nadu, UK - Uttarakhand

Assam, BR - Bihar, CG - Chhattisgarh, GJ - Gujarat,

Graph 1: Composite performance index of schools of different states Graph 2: Students scoring zero in the classes and subjects tested

Composite Performance Index of Indian States for all Classes and Subjects

States

Co

mp

osi

te P

erf

orm

an

ce I

nd

ex

10

8

6

4

2

0

-2

-4

-6

-8

-10

States

Extent of Zero Scorers Among the Indian States

%

KL

MH

OR

KA

PB

HR

UK

TN

NT

BRJH GJ

AP

AS

CG RJ

MP

JK

10

8

6

4

2

0

KL

MH

OR

KA

PB

HR

UK

TN

NT

BR

AP

AS

CG

RJ

MP

JKJH GJ

Main Findings

9. The comparative performance of boys and girls is similar to many international studies – boys seem to do better in maths, although the SLS study

reveals that it is a meaningful difference that matters in class 8 only.

10. Analysis of Background factors showed that when students perceive themselves as being good at studies and think of the school as a place of fun and

learning, they show good performance. Students who said that they read material other than textbooks for at least 30 minutes a day showed higher

achievement. This reveals the importance of inculcating reading habits in students. Teacher training is important especially for mathematics teachers,

for better achievement of students in the subject. Unexpectedly, analysis shows that students whose teachers have an academic degree developed

lower scores in both language and mathematics than those who did not have these.

1. Orient policies to focus on learning outcomes

2. Make low stakes diagnostic assessments – a regular feature of the state programs

3. Extensively use Benchmark data from the study to build reform

4. Implement a student progress tracking system

5. Create a large scale awareness campaign to redefine attitudes towards learning a movement against rote learning and for learning with understanding

6. Widely disseminate the findings of this report among teachers and others

7. Provide effective teacher support based on the feedback from the assessment

–

Graph 3: Composite performance index of schools of different states

Is it fair to compare school

systems that are known to have

wide differences in resources and

types of students it caters to?

It is well known that schools in other

countries as well as the Indian private

elite schools may have richer

resources with students from better

socio-economic strata, which by

themselves may contribute to

differences in student learning.

Hence the purpose of these

comparisons should not be to

measure or rank the school systems

in terms of performance. However,

given the goal for our government

school system to achieve the best

quality education for students, such

comparisons do indicate the

performance level students in these

schools could aspire for.

Maths 8Maths 4Lang 4

International Versus Indian Public School Students

International India

80

70

60

50

40

30

20

10

0

Perf

orm

an

ce in

Co

mm

on

Qu

est

ion

s

A.

He had always hated mice.

B.

There were too many of them.

C.

They laughed too loudly.

D. They ate all his cheese.

Why did Lakhan want to get rid

of the mice?

Class 4 Language

Passage Excerpt:

When Lakhan discovered that he had mice in his

house, it did not bother him much at first. But the

mice multiplied. They began to bother him. They

kept on multiplying and finally there came a time

when even he could stand it no longer.

Question involves retrieving stated information in the text and making straight forward

inferences from it.

Sample Question 3: Internationally, of students answered this correctly, while

41.8% of students from Indian schools could get this correct.

79.0%

Overall Recommendations

A high correlation of 0.85 and above was observed in the performance across the classes and subjects tested in each state. This means that if a state

does well in a class or subject compared to other states then it more or less tends to do well compared to other states in other classes and subjects

too.

7. A comparison of information in Graph 1 with that

in Graph 2 reveals that Andhra Pradesh, Assam, Chhattisgarh and Jharkhand, although below the national average, have fewer students scoring zero

in the test compared to better performing states such as Haryana and Karnataka. This indicates that while the states’ efforts could be addressing the

lowest ability students, it does not provide adequate support for overall improvement of all students. Jammu and Kashmir, Madhya Pradesh and

Rajasthan which ranked among the bottom 3 in overall performance also had the highest number of students scoring zero indicating that these

students are being left behind in these states.

8. The levels of learning of Indian students in government schools in class 4 and class 8 tested is much lower than the international average as represented

by studies like Trends in International Maths and Science Study (TIMSS) and Progress in International Reading and Literacy Study (PIRLS). Similarly, on common questions used from an extensive study for private schools in India catering to the elite and upper middle class, students in government

schools showed a much lower performance.

Extent of students scoring zero and the overall performance of a state showed differences.

The mean difference significant at 0.05 level. Effect size >0.2 Small >0.5 Medium, >0.8 – Large – –

Note:- Union Territory

Statistically Significant and Large Effect

Statistically Significant and Medium Effect

Statistically Significant and Small Effect

Difference Above National Average

Statistically Significant and Large Effect

Statistically Significant and EffectMedium

Statistically Significant and EffectSmall

Difference Below National Average

*

Maths Class 4

70

.3

69

.9

66

.0

65

.1

63

.8

57

.9

%

National: 56.5%

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

Rajasth

an

Delh

i

Oris

sa

Mah

aras

htra

Kerala

Karna

taka

Chand

igarh

*

Tam

il Nad

u

Uttara

khan

d

Punjab

Harya

na

Bihar

Andhr

a Pra

desh

Mad

hya P

rades

h

Jhar

khan

d

Chhat

tisga

rh

Assam

Gujara

t

Jamm

u an

d Kas

hmir

58

.9

58

.6

53

.7

55

.4

53

.2

53

.1

52

.8

52

.2

50

.2

48

.9

45

.2

29

.1

42

.5

Language Class 6

72

.9

60

.5

58

.0

57

.8

55

.5

55

.1

51

.2

37

.4

35

.9

14

.4

Jamm

u an

d Kas

hmir

Kerala

Mah

aras

htra

Chand

igarh

*

Punjab

Oris

sa

Karna

taka

Tam

il Nad

u

Uttara

khan

d

Harya

na

Gujara

tBiha

r

Jhar

khan

d

Andhr

a Pra

desh

Chhat

tisga

rh

Assam

Mad

hya P

rades

h

Rajasth

an

%

National: 48.5%

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

50

.2

47

.6

49

.4

46

.8

46

.0

45

.7

42

.6

41

.5

National: 44.4%

Tam

il Nad

u

Chhat

tisga

rh

Language Class 8

74

.7

64

.5

61

.5

59

.0

58

.7

57

.4

57

.2

19

.0

Jamm

u an

d Kas

hmir

Kerala

Harya

na

Chand

igarh

*

Karna

taka

Punjab

Oris

sa

Gujara

t

Uttara

khan

d

Mah

aras

htra

Bihar

Tam

il Nad

u

Rajasth

an

Assam

Jhar

khan

d

Mad

hya P

rades

h

Chhat

tisga

rh

Andhr

a Pra

desh

%

National: 53.5%

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

56

.8

53

.255

.8

52

.5

51

.2

50

.1

50

.0

48

.2

47

.9

47

.5

Language Class 4

73

.7

70

.0

64

.1

63

.1

61

.2

60

.7

58

.5

58

.1

57

.4

41

.9

40

.9

19

.0

Jamm

u an

d Kas

hmir

Delh

i

Kerala

Oris

sa

Mah

aras

htra

Tam

il Nad

u

Chand

igarh

*

Karna

taka

Uttara

khan

d

Punjab

Andhr

a Pra

desh

Chhat

tisga

rh

Gujara

t

Harya

naBiha

r

Jhar

khan

dAss

am

Mad

hya P

rades

h

Rajasth

an

%

National: 54.8%

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

52

.1

50

.9

50

.8

50

.5

49

.5

48

.2

47

.1

Maths Class 8

45

.6

45

.4

43

.8

41

.7

40

.2

%

0.0

10.0

20.0

30.0

40.0

50.0

National: 35.3%

Chhat

tisga

rh

Kerala

Karna

taka

Oris

sa

Mah

aras

htra

Chand

igarh

*

Harya

naBiha

r

Punjab

Uttara

khan

d

Jhar

khan

d

Rajasth

an

Tam

il Nad

u

Gujara

t

Assam

Andhr

a Pra

desh

Jamm

u an

d Kas

hmir

Mad

hya P

rades

h

80.0

60.0

70.0

38

.9

37

.2

37

.1

35

.2

36

.5

35

.0

33

.8

32

.8

31

.6

29

.7

28

.3

27

.5

19

.5

Performance of the States for all Classes and Subjects

© Educational Initiatives Pvt. Ltd.

e-mail us at: Website: assessment@ei-india.com www.ei-india.com

Bangalore Office Hyderabad Office Delhi Office Mumbai Office Head Office: Ahmedabad

613-615 JB Towers, Opp. Doordarshan Tower, Drive in Road, Ahmedabad – 380054, INDIAFax: 079 – 26841400

Tel: 079 – 40269696, 40269625

No.97, 2nd Floor, Robertson Road, Frazer Town, Bangalore – 560005, INDIA

Tel: 080 – 41657715, 64567049, 41237162

3rd Floor, Lakshey Deep Plaza, A-252A, Sant Nagar, East of Kailash, New Delhi – 110065 INDIA

Tel: 011 – 26462264, 40584952

Flat No 401/A, Shri S.K.Vihar, H.No:3-4-578/1,Beside Sri Chaitanya College, Narayanaguda, Hyderabad – 500029, INDIA

Tel: 040 – 32419534

Row House Number 1,Sea Coast 2, (Near Cidco Guest House), Kille Gaothan, Belapur, Navi Mumbai – 400614, INDIA

Tel: 022 – 32686329, 27561430

CONTACT DETAILS: Educational Initiatives Pvt. Ltd.