ALIS for age 16 and over

ALISGetting the measure of

Getting the measure of ALIS 3

3

Getting the measure of ALIS 2

2

“I wish to share some good news following our OSTED inspection. Our department was judged to be outstanding and the body of evidence submitted to OFTSTED included our use of the P scales information you send us each year. I would like to take this opportunity to thank you for your continued support and I hope we will have many more years of working together.”

Evelyn Gardner, Head of ASD Learning Centre, Castle Hill School, Croydon.

How likely are students to achieve particular subject grades as they set out on the journey to post-16 qualifications? And as the school/college itself seeks to improve performance, how can it evaluate the effect of any new approach?

ALIS (Advanced Level Information System) has the answers. ALIS gives teachers the accurate and reliable data they need to predict exam outcomes and help students fulfil their abilities. For school/college leaders it provides support for policy changes.

Evidence basedALIS is one of a suite of monitoring and assessment systems from CEM (Centre for Evaluation and Monitoring). It is designed for A/AS level and other graded qualifications such as BTEC. A variation, CEM IBE, is a bespoke system which focuses entirely on the International Baccalaureate.

ALIS generates prompt, detailed and objective feedback which can be analysed at multiple levels - student, subject and whole school. You can be sure ALIS information is valid, because it compares your school/college’s performance with others belonging to ALIS. These account for around 50% of all A-level entries from a UK-wide representative sample. ALIS caters for non UK based schools and has a growing International market. ALIS has nothing to do with league tables.

CEM is a research organisation within Durham University and all analysis is underpinned by robust statistics and 30 years of research. Data from all CEM systems is confidential, designed to help schools to evaluate purely for themselves what works and to make their practice evidence based.

IntroducingALISThe complete solution to assess ability, predict performanceand plan improvements for post-16 qualifications There is, of course, more to education than exam

results. Since the introduction of Self-Evaluation, reporting on attitudes has become increasingly important. That is why ALIS also provides a survey to record your students’ views on their learning experience and to monitor their health and welfare.

Rounded PictureThe rounded picture presented by ALIS allows teachers to use objective data to learn more about their students’ ability and to review past performance. Teachers are empowered to take practical action to enable all students to reach their individual potential.

Data in depthALIS offers two parallel methods of measurement. One uses (I)GCSE results as a baseline. The other is a unique Computer Adaptive Baseline Test (CABT) which assesses developed ability – what students have learned overall, not just in formal teaching. Both baselines generate predictions and value-added analyses specific to each student and subject, in recognition of performance differences across the curriculum.

Although (I)GCSE is a powerful indicator of future attainment, some students’ grades can be affected by factors such as prior teaching styles, health and family. For them and students with no (I)GCSEs (such as adult or overseas learners) the baseline test alongside (I)GCSE data produces an alternative insight, levelling out apparent ability differences due to exam preparation.

Getting the measure of ALIS 4 Getting the measure of ALIS 5

4 5

The Centre for Evaluation and Monitoring (CEM) enables schools to measure how effectively they deliver learning. Our chain of related assessment systems links the education of children and young people from age 3 onwards. The data produced has strongly correlated with performance in external exams since 1983.

The forerunner of ALIS stemmed from research by our founder Carol Taylor Fitz-Gibbon. On being approached by a governor of a north east England school, she responded to their concern that Maths A level results were weaker than those for English.

You can rely on our independence as a not-for-profit organisation. We have no political agenda. Our mission is solely to help children and young people reach their potential. That is why we have continued to innovate to improve the range, ease of use and feedback of our systems and are one of the world’s leading providers of computer adaptive systems. These automatically adjust the assessment process to a student’s level of knowledge so that learning can be targeted at individual ability.

Unrivalled Support To help you run ALIS assessments and enable you to derive the most benefit from your data we offer free support by phone and email. We also organise tailored in-school training and teacher training events both in Durham or at schools and centres around the world.

Through this quality of partnership with schools we ourselves learn from their experience of using CEM systems. This refreshes our fund of knowledge and helps our team - who include experienced teachers in the tradition of both our founder and our present Director Professor Robert Coe - to develop even more cost-effective ways to make a positive difference to students’ prospects.

CEM & ALISYour proven partner in helping to improve performance

She set up a pilot project with nine other regional schools. It revealed a consistent relationship between GCE O level scores and A level outcomes. Thus our value-added approach was born.

Independence Today CEM has grown to assess the progress of over a million learners in over 70 countries every year. As part of Durham University, a World Top 100 university, we hold fast to the principles of providing high quality research information through a scientific and evidence-based approach.

Getting the measure of ALIS 6 Getting the measure of ALIS 7

6 7

You have a choice of two versions of ALIS – Basic ALIS and Full ALIS. Basic ALIS works from two sources of information about the student, (I)GSCE results and a computer-adaptive baseline test (CABT). Full ALIS includes an Additional Survey which comes in two parts – a subject specific Teaching and Learning Survey and a general Welfare Survey. The more data you can gather about the student using ALIS, the more extensive is the feedback as a guide for future teaching and planning.

Measuring what matters

Computer Adaptive Baseline Test (CABT)This assesses individuals who have not taken (I)GCSE recently or at all, such as students from overseas, those attending international schools, adults or vocational qualification holders. Teachers also use CABT as an alternative opinion on the (I)GCSE-based diagnosis. Prior qualifications do not always tell the true story. Students coached in GCSE exam techniques can get high scores leading to high ALIS predictions, but may struggle with Post-16 learning and ‘apparently’ underperform. In contrast, some students have the potential to do better than (I)GCSE results indicate because of circumstances at exam time or their natural rate of development.

The CABT is taken under exam conditions and consists of vocabulary, maths and a non-verbal section. Together they provide a measure of developed ability – the student’s potential.

• Vocabulary measures fluency and the use of words picked up from the student’s background, not necessarily from English lessons.

• Maths assesses logical thinking, manipulating numbers and numerical concepts without being linked to the Maths syllabus.

• Non-verbal asks students to recognise shapes, match pictures and use visual intuition as a measure of perceptual speed and accuracy

Basic ALIS (I)GCSE baseline When students have received their (I)GCSE results and have settled on their Post-16 options, these details are submitted to ALIS. The average (I)GCSE score is the main baseline for AIis, having consistently proved to be the best single indicator of post-16 performance.

Getting the measure of ALIS 8 Getting the measure of ALIS 9

8 9

For each section students start with a question suited to average A level entry student ability. Those who get it right are moved to a more difficult question from a bank of hundreds. Those who get it wrong go to an easier question. Everyone settles at a level where they feel comfortable and go as far as they can until the questions are too difficult. The total time for completing the test lasts between 45 minutes and -1 hour. This computer adaptive aspect of ALIS is unique for post-age 16 tests.

ComparisonWhen the results of both (I)GCSE and CABT agree, that is cause for confidence in predictions. If they diverge, that is cause for concern. The student may well need extra individual attention.

Full ALISFull ALIS gives you everything Basic ALIS provides plus an Attitudinal Survey. This is usually taken during the second term of the second year of study when the students have enough experience on which to base their opinions. The survey takes an hour to complete. The results are anonymised to allow honest answers. There are two sections:

• 22 questions about subjects and the way they are taught together with the student’s aims and expectations

• a questionnaire inviting comments about the school or college

The profile that emerges is valuable in its own right, but becomes especially significant when compared to the performance other similar schools/colleges taking the Full ALIS. If the value-added scores from Basic ALIS show, for example, that Biology scores are lower than the national average, it is worth relating this to the way the subject is delivered in the opinion of students in the Full ALIS feedback. Such comparisons can lead to positive changes in classroom practice, such as paired working.

Getting the measure of ALIS 10 Getting the measure of ALIS 11

10 11

Feedback for the future

Feedback is fast, detailed and definitive. You get exactly what you need when you need it most. Delivered online via the ALIS+ secure website, it takes the form of tables and charts for easy comparison of performance and identification of trends. There are three varieties of feedback which arrive at different stages in the student’s post-16 career. Together they make up a platform for planning the future direction of both student and school:

1. Baseline and predictive feedback is based on (I)GCSE results submitted on the Student Registration Spreadsheet in September and any CABT taken. The predictive feedback with or without CABT scores will arrive from early October.

1. Baseline and Predictive Feedback

• Individual Student Predictions and Chances Graphs in each chosen subject showing the distribution of possible predicted grades for a student of that ability

• Subject-Level Prediction Reports

• Prediction Reports based on either average students (50th percentile), Top 25% of students (75th percentile) and Top 10% (90th percentile)

• ‘PARIS for ALIS’ interactive software enabling you to analyse feedback

• Intake Profiles of the cohort, including summary year by year comparisons of how your students performed in relation to a nationally representative sample of ALIS schools/colleges

• Regression Equations

For students who have completed the Computer Adaptive Baseline Test (CABT) you will also receive:

• All the above based on students CABT scores

• Detailed Individual Student Predictions – a breakdown of each student’s performance on the assessment, illustrating their abilities in each of the Vocabulary, Mathematics and Non-Verbal areas of the test.

• In addition, early predictions are available online within hours of taking the CABT as a subject specific predictions spreadsheet.

2. Value Added Reports: When your students have completed their A levels, other qualifications, or IB diplomas, and you have submitted their final grades, we will provide comprehensive Value Added reports which detail progress made at the student, subject, institution and consortium level.

3. Attitudinal Reports: for schools/colleges registered for Full ALIS

Getting the measure of ALIS 12 Getting the measure of ALIS 13

12 13

2. Value-added reportsHaving received students’ final A level grades, other qualifications ,and IB, in August each year; we aim to publish preliminary Value Added feedback by 1st September with final Value-Added reports available by the end of September. You will receive:

• Executive Summary of your students’ Value Added performance compared to others in ALIS

• Year by Year comparisons by Subject, Qualification and Overall Institution performance.

• Summary and student-level reports based on (I)GCSE and/or CABT baselines as appropriate. Subject level analysis includes individual student performance, gender and ethnicity analysis, syllabus level analysis and institution type analysis.

• Chances Graphs and Regression Equations.

• ‘PARIS for ALIS’ interactive software.

3. Attitudinal reportsIf you opt for Full ALIS, in addition to all the above you will also receive feedback from the Attitudinal Survey:

• Anonymised students’ answers to questions such as : “How do you now feel about having decided to do the courses you are now completing?”; “How do you now feel about having chosen to do the courses at this particular school/college?” “What if anything should be improved on the programme or in the school/college?” and “What if anything would you praise about the programme or the school/college?”

• Attitudinal summary, also containing information on the use of private tutors, a breakdown of students’ attitudes, aspirations and expectations, and views on the availability of resources in the school/college.

• The teaching and learning process – showing for each subject the style of teaching perceived by students compared to a background drawn from the ALIS cohort.

• Comparative information to enable you to view the performance of your school/college in the perspective of other (anonymised) similar schools/colleges in ALIS.

Getting the measure of ALIS 15

15

Getting the measure of ALIS 14

14

Benefiting from the power of ALIS It is the combination of tried and trusted assessments with the richness and variety of evidence–based feedback that gives ALIS its distinctive value. ALIS can be used by all schools, colleges and academies. Easy to implement and cost-effective, ALIS generates the information which can help aid performance improvement.

Easy to administer Baseline assessments and surveys are administered by computer, and delivery of data and feedback is via the web at a timescale to suit each school/college

No teacher marking

Two Baselines (I)GCSE data is used with the computer-adaptive baseline test (CABT) to complement or instead of (I)GCSE

Flexible ALIS works with students of any age, including adult learners, those with no (I)GCSE qualifications, those from abroad and those at international schools

Wide range of predictive data based on either average students (50th percentile), top 25% of students (75th percentile) or top 10% (90th percentile)

Wide range of qualifications e.g. A/AS level, International Baccalaureate (assessed by CEM IBE), BTEC Nationals, OCR Nationals (level 3)

Wide range of value-added • Subject and syllabus level (comparing

students to others in the ALIS cohort studying the same course)

• Institution type (comparing your students to others studying the same subject in schools/colleges of the same type)

• Data provided at student, subject, institution and consortium levels

Interactive software for on-course monitoring, custom group analysis (gender, ethnicity, teaching set) to drill down into your own data

Statistically robust We analyse around 50% of all A-level entries in the UK annually from a nationally representative sample of schools/colleges

Accurate measure of performance, progress and attitudes • For self-evaluation and school/college

improvement

• To help identify students who need help or to spot high fliers

• To support your professional judgement

• To monitor student voices/views about the school/college and welfare

Objective evidence To focus on key areas and support inspection process.

Confidential and impartial Controlled access and distribution of data not to be released to other organisations without permission. ALIS is based on research and evidence and is not intended to for use in rankings

www.cem.org