Significance of Evidence in Assessing 21st Century Skills
Dr. Priyanka SharmaResearch Officer, CAER
CBSE- CAER International Conference28-29 April 2014
CBSE-CAER International Conference, 28-29 April 2014
CBSE-CAER International Conference, 28-29 April 2014
21st Century Contexts
Global citizenship Knowledge-and-intelligence-based economyDominance of tertiary or service sector in global economy Unprecedented growth in ICTsSustainability issues
Significance of 21st Century Skills
Cont.
2005
CBSE-CAER International Conference, 28-29 April 2014
Skill sets deficient in school pass out % response
Written communication 81 %Leadership 73%Work ethic 70%Critical thinking and problem solving 70 %Self direction 58%
Significance of 21st Century Skills
(source: Are they really ready to work. 2006. P21)
Skill sets required for the workforce Employers’ priority
Critical thinking and problem solving 78 %Information technology 77%Creativity and Innovation 74%Collaboration 74%
THE
GAP
CBSE-CAER International Conference, 28-29 April 2014
DECISION
MAKING
CBSE-CAER International Conference, 28-29 April 2014
21st Century Systems to Nurture 21st Century Skills
What is needed
• 21st Century Standards and Assessment
• 21st Century Curriculum
• 21st Century Teaching Learning Methods
• 21st Century Learning Environments
• 21st Century Professional Development
CBSE-CAER International Conference, 28-29 April 2014
Standards and Assessments - Starting Point for 21st Century Systems
CBSE-CAER International Conference, 28-29 April 2014
What Kind of Assessment is Needed
• Encourages students to monitor their own progress and demonstrate evidence of their learning
• Provides sufficient information to teachers
• Enables concerned stakeholders to track the current position of the student against the destination
• Helps students, teachers and parents plan the developmental journey
Technical Criteria- Validity, Reliability, Fairness, Scalability, Comparability
EVIDENCE
Basic Steps of Evidence Based AssessmentDefining the construct and describing developmental
continuum for 21st Century Skills
Collection of evidence
Systematic recording of observations and evidence
Using evidence for professional decision making and estimating student’s achievement
Reporting the progress or location of a student on the developmental continuum
CBSE-CAER International Conference, 28-29 April 2014
Framework of 21st Century Skills
Partnership for 21st Century Skills (P21 task force)
• Learning and Innovation Skills• Life and Career Skills
National research Council, USA• Cognitive Skills• Interpersonal Skills• Intrapersonal Skills
Assessment and Teaching of21st Century Skills (ATC)
• Ways of thinking• Ways of working• Tools for Working• Living in the world
CONT.CBSE-CAER International Conference, 28-29 April 2014
Critical Thinking & Problem Solving
Creativity & InnovationFlexibility/AdaptabilityCollaboration & Social
SkillsMedia Literacy & IT SkillsCommunication/
presentationSelf awareness & self
regulationMetacognition
The National Curricular Framework, IndiaIndividual Well Being Social Well being
Independence of thought and action
Learning to learn and problem Solving
Aesthetic and physical values
Commitment to democratic values
Participation in democracy and economic and social changes
Sensitivity to other’s well being
CBSE-CAER International Conference, 28-29 April 2014
Life Skills (co-scholastic skills)
Thinking Skills Social Skills Emotional SkillsSelf awareness, Problem Solving, critical thinking, creativity, decision making
Empathy, Effective communication, Interpersonal relationships , Collaboration and ICT- literacy
Managing feelings and emotions, Stress management
Work Education Dispositions
CBSE-CAER International Conference, 28-29 April 2014
Challenges of Collecting Evidences of 21st century Skills
• Several of these skills embody non cognitive and metacognitive domains of learning
• Inadequate understanding of manifestation of non cognitive and metacognitive domains
What can we do to ?
• Describe what it means to grow in each of the domain/criteria
• Develop exemplars of appropriate behaviour- how do students of a specific age group demonstrate these behaviour
CBSE-CAER International Conference, 28-29 April 2014
Source: How Students Learn Science
CBSE-CAER International Conference, 28-29 April 2014
Misinterpretation of evidences, statements, graphics, questions, information, or other’s view point; Fails to identify relevant propositions and arguments; Ignores alternative strategies and other view points; Draws invalid conclusions and defends based on pre-conceptions; Indifferent to other’s views
Biased/Inappropriate interpretation of evidences, statements, graphics, questions, information, or other’s view point; Identifies propositions-arguments and reasons logically; analyses obvious alternatives, but hardly/superficially evaluates others view points; draws evidence based conclusions and but seldom justifies findings and results; values ideas and opinions of others
Accurate interpretation of evidences, statements, graphics, questions, information, or other’s view point; Identification of propositions-arguments and logical reasons; Thoughtful analyses and evaluation of alternative strategies and view points; Drawing evidence based valid conclusions and justifying findings and results; Considering other’s views for decision making
A Segment of Developmental Continuum of Critical Thinking
CBSE-CAER International Conference, 28-29 April 2014
Collection of Evidence- Sources
Strategies/methods Evidence
Questioning paper pencil test, oral questions, quizzes standardised or classroom based in real or hypothetical situations
Observation Rubrics, anecdotes, checklist, observation sheetPerformance Assessment
Performance in simulated real life like situations, laboratory tasks, role play, group discussions, debate, interviews,…
Product Assessment Models, students made objects like art, portfolios; working documents like note books, concept mapsAssignments- Project, reports, essays, write-ups,..
Self-assessment /Peer review/Conference
Self reports (narrative/questionnaire/checklist), Reflection journal, logs
CBSE-CAER International Conference, 28-29 April 2014
PROBLEM SOLVING ASSESSMENT- Large Scale Paper Pencil Test
• Real world• Personal and social life• School life• Interest and attitude• Societal and national issues
Context
Language Conventions Qualitative Reasoning Quantitative Reasoning
Collection of Evidence - Questioning
Collection of Evidence - Observation
Observing students who are engaged in learning
• Provides continuous information about students’ strengths and challenges and help to locate a child by assimilating these evidences
• Observation might be – direct/indirect - formal/informal
• Suitable for individual or group behaviour
CBSE-CAER International Conference, 28-29 April 2014
Cont.
CBSE-CAER International Conference, 28-29 April 2014
Collection of Evidence - Performance AssessmentForm of assessment that requires students to construct a response, create a product or demonstrate application of knowledge and skills in real life like situations
Involve• activities for which there is no single correct answer• assessing groups rather than individuals• tasks that would continue over an extended period of time• self evaluation or performances• higher level cognitive skills, social skills and intrapersonal skills• real world contexts and constraints• assessment of directly observable behaviour• immediate feedback to students about their strengths and
areas of improvement
Might be product or process assessment
CBSE-CAER International Conference, 28-29 April 2014
• Scored responses - dichotomous/partial credit• Checklist • Rating scales• Rubrics• Anecdotes• Progress Map• Categorisation of qualitative responsesImportant Consideration
Data must be sufficient, but manageable and directly tied to decision-making
Information needs to be strong enough to support professional judgement about student’s achievement level and further journey
Recording Evidence
Recording Evidences- Checklists
Checklists and Observation Schedules are assessment tools that help teachers gather information about pre determined learning
behaviours through observation.
CBSE-CAER International Conference, 28-29 April 2014
Makes efforts to understand the task before attemptingYes/No
Relates task to my knowledge and understanding Yes/No
Draws a mind map of all information, requirements Yes/No
of the task and possible solutions before attempting the task Yes/No Determines steps of a tasks, before starting the work Yes/No Carefully plans steps of action Yes/No Thinks through the steps while working
Yes/No Tries more than one way while working
Yes/No Evaluates how much work is left, how much has been achieved
Yes/No Questions – what I need to know in order to accomplish a task
Yes/No Discovers the main ideas in a task Yes/No Any other
Check list for assessment of problem solving skills
CBSE-CAER International Conference, 28-29 April 2014
Recording Evidence- Rating Scale
Self regulation 1 2 3 4 5
Is able to set own goals
Identifies learning opportunities and strategies to meet learning goalsMonitors own progress against set goals
Assesses and reflects upon own strengths and weaknessesSeeks feedback from others and works upon them
(Source of evidence- Portfolio or anecdotes)
CBSE-CAER International Conference, 28-29 April 2014
Recording Evidence - Rubrics
Information literacy skills Content standard: Researches and Evaluates Information Measurable Learning Outcome: uses a variety of appropriate tools to note observations and represent information
Skills Level 1 Level 2 Level 3Uses tools of IT … … …Records observation … … …Represents information … … …
are guidelines for recording evidence, in which assessment criteria are combined with rating scale
• lists a set of criteria that define and describe the important components of the skill• specifies the level of performance for each of the criteria• focus on the quality of the product/process and not the quantity
CBSE-CAER International Conference, 28-29 April 2014
Criteria Beginner Intermediate AdvancedAnalysis of question and inquiry
Sees only superficial aspects of the problem
Identifies key aspects of the problem, but not interaction
Identifies key aspects and interaction components, too
Collecting, analysing and evaluating information
Follows prescribed guidelines to collect information; fails to distinguish between relevant & irrelevant
Assesses collected information for relevance and accuracy
Critically investigates sources to identify relevant information;
Using evidence and criteria
Accepts arguments without questioning and reasoning
Acknowledges need for valid reasoning, fails to evaluate evidences
Evaluates arguments for validity of reasons
Decision making Subjective decision making, not based on the most appropriate criteria
Uses the most appropriate criteria for decision making, but not able to justify
decision based on objective analysis of all criteria for decision making
Alternative choices and implications
Tries only one way Recognizes alternative solutions, but not able to justify implications
Can solve a problem in more than one way and explains implications
Analytic rubrics (Problem solving)
CBSE-CAER International Conference, 28-29 April 2014
Communication
Level 1 Level 2 Level 3
Reading information Interprets information from passage, table, picture, graph, timeline, flow chart,…
Identifies relevant information; Assesses the validity of the reasoning; Identifies false statements and fallacious reasoning
Delineates and evaluates the arguments and specific claims in a text
Speaking and Listening Engages effectively in collaborative discussions
Initiates and participates effectively in collaborative discussions
Builds on others’ ideas and expresses own very clearly
Expressivity …………… …………….. …………….
Student 1 April 13 Feb 14Student 2 April 13 Feb 14Student 3 April 13 Feb 14
Student 4 April 13 Feb 14
Student 5 April 13 Feb 14
Using Evidence - Estimating student’s achievement
CBSE-CAER International Conference, 28-29 April 2014
Closing the LoopAssessing with Purpose
Meaningful Reporting and Feedback
Focus on Enablers and Drivers
CBSE-CAER International Conference, 28-29 April 2014
Thank You