september 2011alternative system alternative system for admission into engineering programmes report...
TRANSCRIPT
September 2011 Alternative system
Alternative System for Admission into Engineering Programmes
Report of the CommitteePresented to the Council of
Indian Institutes of TechnologyA proposal for consideration
September 2011 Alternative system
Acknowledgement
Prof Acharya and his Committee members Dr BK Gairola; Sri V Joshi; Sri H Bhartia; Sri M
Tuli Directors of all IITs 2063 people who participated in opinion poll Director and Experts from ISI and their
students Colleagues from MHRD, DST and NIC Chairpersons of school boards and Chairman and members of the IIT Council Some Media representatives and all those
who helped in the exercise
September 2011 Alternative system
Decision of the Council in its 41st meeting
Prof Acharya committee had been commissioned to study the present system of conducting examinations for admission into engineering progammes of the country and suggest alternatives
The committee presented its interim report in the 41st meeting
There was unanimity that the present system required a change as proposed
The committee was enlarged to address the issue of the need to recognize diversity of learning
September 2011 Alternative system
Major findings of Acharya Committee
Screening based on normalized Board scores at Standard X and/or Standard XII and Multiple Choice examination replacing the two stage JEE from 2006.
Entry barrier to be raised to 60% in the +2 examinations.
Factors, other than the Standard XII marks and AIR based on PCM testing, such as raw intelligence, logical reasoning, aptitude, comprehension and general knowledge need to be considered.
Need to factor in school performance more significantly into the selection process.
September 2011 Alternative system
Major findings of Acharya Committee
Decision based on one time test needs to be re-examined. Opportunities to improve must be built in.
Students must be relieved of the pressure of multiple JEEs.
Influence of coaching for JEE needs to be minimized. Urban-rural and gender bias has to be eliminated or
at least minimized. The objective type of examination lends itself to
undue influence of coaching. The conventional pen and paper examination with well designed long and problem solving oriented questions should be revived by keeping numbers in any JEE within reasonable limits.
September 2011 Alternative system
Work of the present committee
1. Study of Acharya committee work2. Made Non formal survey among hundreds of school
students, parents, employers, faculty and media person3. Met and decided on General approaches4. Held with Chairmen and nominees of all school boards5. Collected data on scores of school board examinations6. Wrote down a philosophy document and placed in public
domain7. Carried out survey of public opinion poll8. Enrolled ISI for carrying out exploring statistical methods
for normalization of school board scores 9. Met with Directors and Senior Faculties of IITs10. Prepared a draft report for comments and advice of the
Council
September 2011 Alternative system
Consultation with states and School boards
First stage Gained access to data on scores on school
board examinations through formal mechanisms
Enrolled participation in development of NTS Second Stage
Assistance in harmonization and electronic access to data
Building trust and development of a process with safeguards and integrity
September 2011 Alternative system
Public participation through opinion poll
On-line opinion survey among the people of India For multi parametric grading system as
against single test models of JEE Screening out as against selection
strategies With Responder profile, opinion polls,
suggestions for alternative national test systems; risk mitigation strategies
Survey time slot open for three weeks
September 2011 Alternative system
Responder profile to the poll
2063 participated 59% students; 8% teachers, 5.5% parents, 23.4%
non-teaching professionals, less than 1% coaching
~80% of respondents had taken entrance examinations in their lives
1220 students had participated 80% students were from engineering stream and
95% of them had taken entrance examination ~160 teachers had participated
~90% of them are engaged in tertiary education
September 2011 Alternative system
Analysis of the current system of admission into engineering
960 of 2063 commented on the multiplicity of entrance examinations with different sociological implications 715 agreed with the view expressed
947 of 2063 commented on need for reform 85% voted for reform in admission
system
September 2011 Alternative system
Inputs for reforms fo admission systems
1. Reduction in the number of examinations to one
2. Testing knowledge intensity, alignment to the 12th class syllabus
3. Reduction of dependency on coaching and pressure on students
4. Emphasis on aptitude….. (More than 90%)5. Transparency in processes6. Removal of negative marking7. Online processes8. Multiple centers, better scheduling
September 2011 Alternative system
Responses to reform directions
Factor-in performance in school boards 66% in favor and 34% not in favor
45 of those disfavor fear that board examinations do not assessing capability and 30% fear non-uniformity
620 responded to question on Indian equivalent of SAT type 73% voted in favor of aptitude type test
646 responded to types of tests 70% prefer a mix of aptitude and advanced type
tests 629 responded questions on test features
Dominant support is for high-filter type and SAT type tests
September 2011 Alternative system
On summary
More than 85% supported the concept of a single entrance test for admission into engineering programmes and voted for reforms
70 % voted in favor of one test with provisions for testing both aptitude and advanced knowledge
66% of people favor factoring in school board scores
Of 34% People who disfavor fear primarily the problem of non-uniformity. This could be addressed.
September 2011 Alternative system
Some Important suggestions for the committee from opinion poll
National Test in place of multiple competitive examination is generally welcomed
Concerns expressed about the process integrity and fairness of testing
methodology Normalization methodologies across school
boards Multiple chances for candidates for
improving scores A single screening examination with a mix
of aptitude (like SAT type) and advanced (like JEE type)
September 2011 Alternative system
Study for normalization of scores of school board examinations
Committee gained access to some relevant past data with a view to examine Stability of scores of the same school board
over time Potentials for normalization of scores across
various boards Enrolled ISI into carrying out statistical
studies for normalization of board scores
September 2011 Alternative system
Work carried out Indian Statistical Institute
For exploring normalization methodologies for school board scores
September 2011 Alternative system
Pilot testing
Selected four boards for pilot testing Central Board, TN Board, WB Board, ISE Evaluation years 3-4 years for each board
Variations in Density of population CBSE (5-6lakhs), TN(5.6-7.3 lakhs), WB (3.0 -4.6
lakhs, ICSE 25000-56000 Evaluation of stability of scores over time for
the same board Potentials for mapping the profiles of several
boards onto one selected board through monotone transformations
September 2011 Alternative system
Models tested
Select a percentile score (P) for all boards and determine the scores (X1) for P across boards
MappingModel 1: Y1= {Xn – X1}/{Xm-X1}Model 2: Y2 = Xn/X1
where Xn , X1 , Xm are scores obtained by any candidate, marks corresponding to percentile P, and maximum scores obtained by any candidate in each board. Y1 will range from 0 to 1.0; while Y2 will be ratios in the range from 1.0 to Xm/X1
September 2011 Alternative system
Observed Relationships of Scores
For the four boards over timeModel 1: Y1= {Xn – X1}/{Xm-X1}Where aggregate score percentages are used
percentile rank
no
rma
lize
d s
core
50 60 70 80 90 100
0.0
0.2
0.4
0.6
0.8
1.0
CBSE 2007CBSE 2008CBSE 2009ICSE 2007ICSE 2008ICSE 2009ICSE 2010
TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009
Normalized score vs. percentile rank: cutoff 50 %
percentile rank
no
rma
lize
d s
core
60 70 80 90 100
0.0
0.2
0.4
0.6
0.8
1.0
CBSE 2007CBSE 2008CBSE 2009ICSE 2007ICSE 2008ICSE 2009ICSE 2010
TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009
Normalized score vs. percentile rank: cutoff 60 %
percentile rank
no
rma
lize
d s
core
75 80 85 90 95 100
0.0
0.2
0.4
0.6
0.8
1.0
CBSE 2007CBSE 2008CBSE 2009ICSE 2007ICSE 2008ICSE 2009ICSE 2010
TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009
Normalized score vs. percentile rank: cutoff 75 %
percentile rank
no
rma
lize
d s
core
85 90 95 100
0.0
0.2
0.4
0.6
0.8
1.0
CBSE 2007CBSE 2008CBSE 2009ICSE 2007ICSE 2008ICSE 2009ICSE 2010
TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009
Normalized score vs. percentile rank: cutoff 85 %
aggregate score as fraction of max score
pe
rce
ntil
e r
an
k
0.5 0.6 0.7 0.8 0.9 1.0
50
60
70
80
90
10
0
CBSE 2007CBSE 2008CBSE 2009ICSE 2007ICSE 2008ICSE 2009ICSE 2010
TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009
Percentile rank vs. aggregate score: cutoff 50 %
aggregate score as fraction of max score
pe
rce
ntil
e r
an
k
0.6 0.7 0.8 0.9 1.0
60
70
80
90
10
0
CBSE 2007CBSE 2008CBSE 2009ICSE 2007ICSE 2008ICSE 2009ICSE 2010
TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009
Percentile rank vs. aggregate score: cutoff 60 %
aggregate score as fraction of max score
pe
rce
ntil
e r
an
k
0.6 0.7 0.8 0.9 1.0
75
80
85
90
95
10
0
CBSE 2007CBSE 2008CBSE 2009ICSE 2007ICSE 2008ICSE 2009ICSE 2010
TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009
Percentile rank vs. aggregate score: cutoff 75 %
aggregate score as fraction of max score
pe
rce
ntil
e r
an
k
0.7 0.8 0.9 1.0
85
90
95
10
0
CBSE 2007CBSE 2008CBSE 2009ICSE 2007ICSE 2008ICSE 2009ICSE 2010
TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009
Percentile rank vs. aggregate score: cutoff 85 %
percentile rank
no
rma
lize
d s
core
50 60 70 80 90 100
0.0
0.2
0.4
0.6
0.8
1.0
CBSE 2007ICSE 2007ICSE 2008ICSE 2009ICSE 2010
TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009
Normalized score vs. percentile rank for PCMB: cutoff 50 %
percentile rank
no
rma
lize
d s
core
60 70 80 90 100
0.0
0.2
0.4
0.6
0.8
1.0
CBSE 2007ICSE 2007ICSE 2008ICSE 2009ICSE 2010
TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009
Normalized score vs. percentile rank for PCMB: cutoff 60 %
percentile rank
no
rma
lize
d s
core
75 80 85 90 95 100
0.0
0.2
0.4
0.6
0.8
1.0
CBSE 2007ICSE 2007ICSE 2008ICSE 2009ICSE 2010
TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009
Normalized score vs. percentile rank for PCMB: cutoff 75 %
percentile rank
no
rma
lize
d s
core
85 90 95 100
0.0
0.2
0.4
0.6
0.8
1.0
CBSE 2007ICSE 2007ICSE 2008ICSE 2009ICSE 2010
TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009
Normalized score vs. percentile rank for PCMB: cutoff 85 %
September 2011 Alternative system
Standardized (Normalized) score
Normalized percentile rank
Percentile rank of student – 75
100 – 75X 100.=
September 2011 Alternative system
Some general conclusions emanating from the ISI study
Percentile scores are relatively stable over the periods studied for each board when aggregate percentage scores are analyzed
It is possible to normalize percentile ranks across boards
Correlations of normalized percentile ranks against percentile ranks of various boards map on to common linear relationship
September 2011 Alternative system
percentile rank
no
rma
lize
d p
erc
en
tile
ra
nk
75 80 85 90 95 100
02
04
06
08
01
00
Normalized percentile rank vs. percentile rank: cutoff 75%
Transformations onto one theoretical board
ISI Experts are certain that this transformation will be the same for all boards for all years
September 2011 Alternative system
It appears that for normalization of school board scores
A statistical method is feasible after all. ISI may be encouraged to develop the methodology further and extend it to all boards and reconstruct past scenario for present IIT and NIT students over the last four years for revalidation of the method
September 2011 Alternative system
A pilot test among a select group of students: A suggestion
A group of statistically significant number of some volunteers from current student population may be enrolled into a pilot test for evaluating the utility of different models and suggestions. This pilot test has to be designed with care and confidentiality based on informed consent of all involved. The merit or otherwise of the approach will be discussed within the committee before decisions are taken
September 2011 Alternative system
Summary of work done so far
Opinion poll reveals support for reforms and favor single examination with provisions for both aptitude and advanced
while making provisions for factoring-in scholastic performance in school board examinations
ISI study presents a methodology for scientific normalization of scores across school boards Percentile scores are stable over each board and it
is possible to carry out monotone transform board scores and accomplish normalization across boards
Weighting options for school and entrance tests’ performance are considered
September 2011 Alternative system
Two approaches considered Approach 1
weighing consistency of performance in school board examinations and employ them for testing ability to write solutions and
One objective screening test with two sections; one for testing the aptitude and the other advanced knowledge in domain areas.
Approach 2 weighing consistency of performance in
school board examinations and employ them for testing ability to write solutions and
one objective aptitude test based on multiple choices and computer based correction systems
September 2011 Alternative system
Considerations of six different options
Option 1: Deployment of Scores as criteria based on class XII performance only Equal weighting of school board scores A1and A2
Equal weighting of aptitude scores A4 and advanced scores A5
Normalized score = {A1 + A 2+A4 +A5 }/4 Option 2: Deployment of Scores as criteria
based on class XII performance only Equal Weighting of board score A3 which is {A1+
A2}/2 Equal Weighting of Aptitude scores A4 and A5
Normalized score ={A3 +A 4+A5}/3
September 2011 Alternative system
Considerations of six different options
Option 3: Deployment of Scores as criteria based on consistency of performance at class X and Class XII levels as well as in National Level Aptitude and Advanced Tests Equal weighting for aggregate as well as subject performance at
class X and Class XII levels where ) 0.1X (normalized score at class X in aggregate + normalized score at class X in subjects of choice + normalized score at class XII + normalized score at class XII in subjects of choice)
One third weighting of aptitude score 0.3 A4 One third weighting of advanced score A5 Normalized score =0.1{ Normalized aggregate class X +
normalized class X subject score + Normalized class XII aggregate + Normalized class XII subject score} + 0.3 A3and 0.3 A5
Option 4: Deployment of School Board Performance as screening but not as determinant for National ranks Specify a Cut-off normalized percentile rank score for school
performance say as 80 or 85 percentile rank 50% weighting of National Level Aptitude score A4 for candidates
passing the cut off of percentile rank 50% weighting of National Level Advanced Score A5 for candidates
passing the Normalized score = 0.5 A4 +0.5A5
September 2011 Alternative system
Considerations of six different options
Option 5: Deployment of School Board performance as subject score and National Level Aptitude Test as a combination and avoid the Advanced Testing system according freedom for the individual institutions to select mixing proportions within a pre-specified guideline
Option 6: Equal weighting of School Board performance as subject score and National Level Aptitude Test as objective test system whereNormalized score =0.5 A2+0.5A4
September 2011 Alternative system
Recommended preferences of the committee Option 2: Deployment of Scores as criteria based on
class XII performance Option 6: Equal weighting of School Board
performance as subject score and National Level Aptitude Test as objective test system 0.5 A2+0.5A4
Option 5: Deployment of School Board performance as subject score and National Level Aptitude Test as a combination and avoid the Advanced Testing system with freedom for the individual institutions to select mixing proportions within a pre-specified guideline
Option 4: Deployment of School Board Performance as screening but not as determinant for National ranks; Specify a Cut-off normalized percentile rank score for school performance say as 80 or 85 percentile rank; and rank by 0.5 A4 +0.5A5
September 2011 Alternative system
A Suggestion
A committee of experts from engineering institutions could be assigned the task of interacting with ISI Group for internalization of methodology for normalization of board scores
IITs could be assigned the task of setting up a question paper for National Screening Test based on objective examination models and conduct the examinations for the year 2012-13
For Aptitude examination like SAT, we may take the help of NTS or any other global agency