evaluation of alternate exams presentation · web view“flipping- pages gets annoying, especially...

Logo above: University of Birmingham

Evaluation of alternative exam presentation methods for pupils with low vision

Report for RNIB

September 2010

Authors:

Dr Graeme DouglasDr Mike McLindenAnnette Weston

Visual Impairment Centre for Teaching and Research (VICTAR)School of EducationUniversity of BirminghamBirmingham, B15 2TT

Contents

Acknowledgements................................................................................4

Executive summary................................................................................5Summary of findings.........................................................................5Recommendations............................................................................7

1 Introduction and context................................................................9

2 Student performance and views of alternative examination papers............................................................................................102.1 Introduction..............................................................................102.2 Method....................................................................................102.3 Results....................................................................................13

3 Case examples: Students who prefer very large hardcopy print (N24+).............................................................................................233.1 Introduction..............................................................................233.2 Method....................................................................................233.3 Results....................................................................................263.4 Discussion...............................................................................37

4 In-depth interviews with staff involved in material modification........................................................................................................404.1 Introduction..............................................................................404.2 Method....................................................................................404.3 Analysis of results...................................................................43

5 Group Interviews: Students who are non-print readers............685.1 Introduction..............................................................................685.2 Method....................................................................................685.3 Results....................................................................................715.4 Discussion...............................................................................82

6 Technical analysis of the RNIB E-formatting software..............836.1 Modifying the examination paper............................................83

Evaluation of alternative exam presentation methods for pupils with low vision 2

6.2 Using the software to format the examination paper and quality of the output....................................................................................856.3 ‘Accessible’ files......................................................................87

7 Discussion.....................................................................................897.1 Summary of findings................................................................897.2 Classroom practice versus public examination system...........917.3 Possible solutions....................................................................927.4 Recommendations..................................................................947.5 Functional specification of future software..............................95

8 References...................................................................................100

9 Appendix 1 Student Questionnaire............................................101

10 Appendix 3 Font size questionnaire..........................................104

11 Appendix 3 Professional Interview Schedule................................106


AcknowledgementsWe would like to thank all our participants for generously giving us their time and views. We thank RNIB for funding this research. We thank the steering group for their helpful advice throughout this project. Thank you to Rory Cobb for general advice and help with the group interviews with students. Thank you to Sue Keil and Paul Nisbet for their general advice.


Executive summaryThis is a report commissioned by RNIB in relation to examination access by blind and partially sighted learners. The research particularly focussed upon the evaluation of a prototype piece of software – ‘RNIB E-formatting software’. RNIB E-formatting software was designed to allow the teacher/school to modify examination papers in terms of font size and style, text and background colour, and the line width inside diagrams. Once it has been modified to meet the individual requirements of the candidate, a paper can be printed for them to have as a paper copy or it may be read on-screen.

The research team carried out five related pieces of research: Student performance and views of alternative examination papers Case examples: Students who prefer very large hardcopy print

(N24+) In-depth interviews with staff involved in material modification Group Interviews: Students who are non-print readers Technical analysis of the RNIB E-formatting software

Summary of findings1. How do the Modified Large Print (MLP) versions of papers produced by RNIB E-formatting software compare to traditional MLP papers? (both process and outcome)

a. Findings from a trial involving 21 students with low vision suggest that examinations that are formatted using the RNIB E-formatting software neither significantly advantages or disadvantages student performance compared with existing MLP approaches. Familiarity with examination format appears to be a more important variable.

b. Professionals interviewed were broadly positive about the RNIB E-formatting software: They were positive about having a greater range of options for

examination formats which the software could offer. They particularly identified control over page breaks and formatting

of pictures/diagrams as key areas for further development. There were mixed feelings about how the process of formatting

and production could be incorporated into the short one hour


period before the examination. Some had concerns about technical problems and working under time pressures.

c. There is evidence that appropriate and consistent modification of examinations is of key concern to professionals irrespective of the method of delivery. The process of producing the software versions of the examination papers (from four different examination boards) appears to have an important associated benefit, namely standardising of the modification process and the general format of the examination papers.

d. Case examples of five students who preferred very large print (greater than 24 point) demonstrated that they appear to benefit from hard copy large print formats which are different from those available through existing MLP approaches. An estimated 9% of GCSE-aged students with a visual impairment in England and Wales use font sizes greater than 24 point.

2. What methods of accessing examinations are useful for the future, and can the RNIB E-formatting software (or a developed version of it) play a useful part? (e.g. electronic copies, interactive papers)

a. When reflecting on the ‘perfect scenario’, many professionals reported that students should have access to examinations which had been modified appropriately and could then be formatted in a flexible way to meet individual needs. The use of technology was seen as an important part of the formatting solution by all participants (teachers, students with low vision and non-print readers). This suggests a distinction between appropriate modification of a paper ‘at source’ and the formatting options that could potentially be provided by the software.

b. Access to on-screen interactive examinations was seen by both students and professionals as offering a range of opportunities for improving access. Nevertheless, further work is required to enable these opportunities to be developed further. Key challenges identified through discussion with students (blind and low vision) as well as professionals included: technical work is required to ensure computer-based material is

fully accessible; appropriate examination modification is carried out ‘at source’;


training of students in the use of assistive technology to access exams;

training of staff to teach students how to use this assistive technology for exams access.

c. Future development needs to be flexible enough to offer a range of on-screen formats (e.g. a range of fonts, sizes, colours, and zoom facilities). They also need to include options for hard copy versions of questions and/or supporting information such as diagrams and tables to be produced in a variety of formats.

RecommendationsWe present two broad recommendations based upon the research findings. The recommendations are made in relation to the project research questions, i.e. particularly linked to the potential of the RNIB E-Formatting software (or similar) in the provision of accessible examination papers. It is important to note that the authors advise that the recommendations are implemented as part of a more general review of examination and curriculum access approaches for visually impaired students in England and Wales, specifically in relation to: The procedures and quality control of examination provision (including

the process of modification); Curriculum access strategies adopted in the classroom (particularly in

relation to the use of low vision aids and computers).

Recommendation 1:

A software application for supporting the production of an increased range of examination formats for students with visual impairment should be developed. The application would be used by teachers and visually impaired students, and it should have the following functions:

a) a range of formatting options;b) printing facilities for students who require hardcopy large print

examinations;c) onscreen presentation facilities for students who require onscreen

access to examinations.

The report provides a draft functional specification for the software application. The RNIB E-formatting software evaluated in this research project provides a very useful prototype of such an application.


Recommendation 2:

The future development of mainstream interactive computer-based examinations presents significant opportunities for improving access to examinations amongst visually impaired students. To ensure that visually impaired students can benefit from these opportunities the following general recommendations are made:

a) Examination boards and designers of computer-based examinations technology should work with access technology experts (for example, within RNIB) to ensure that inclusive design principles are incorporated into the development of this technology.

b) Developments in computer-based examinations will mean that educators must continue to ensure that visually impaired pupils are taught appropriate access skills so that they can independently access these examinations.


1. Introduction and contextThis is a report commissioned by RNIB in relation to examination access by blind and partially sighted learners. The research was designed in 2008 in response to the Research Brief prepared by RNIB Corporate Research Team: “Passing the test: how accessible are public examination systems for blind and partially sighted pupils in the UK”. The research was carried out in two phases of work carried out between April 2009 and July 2010:

Phase 1: International comparison of approaches Phase 2: Trialling and functional specification of the RNIB ‘E-

formatting’ software

Phase 1 report (entitled ‘Summary report on international systems of exam access for visually impaired pupils’) was presented in September 2009. This report presents the findings from Phase 2. Following some reconfiguration of the project following the Phase 1 report, Phase 2 had the following research questions:1. How do the modified large print (MLP) versions of papers produced by

RNIB software compare to traditional MLP papers? (both process and outcome)

2. What methods of accessing examinations are useful for the future, and can the RNIB software (or a developed version of it) play a useful part? (e.g. electronic copies, interactive papers)

The research team carried out five related pieces of research which are presented in turn:

Student performance and views of alternative examination papers (linked to research questions 1 and 2)

Case examples: Students who prefer very large hardcopy print (N24+) (linked to research questions 1 and 2)

In-depth interviews with staff involved in material modification (linked to research questions 1 and 2)

Group Interviews: Students who are non-print readers (linked to research question 2)

Technical analysis of the RNIB E-formatting software (linked to research question 1)


2. Student performance and views of alternative examination papers2.1 Introduction

The research team carried out an experiment which compared paper-based formats prepared using existing large print approaches (modified within the existing large print ‘access arrangement’ policies) to formats prepared using the E-formatting software. The aim was to examine whether either format gave any measurable advantage to the visually impaired pupils (in terms of time taken). The participants also gave their opinions about the different formats through a short questionnaire and group interviews.

2.2 Method

Overall design

A repeated measures design was employed so that each participant tried out two alternative presentations of an examination – modified large print (‘MLP’) and E-formatting software (‘Software’). Different versions of the examination papers and order of presentation were balanced. Time taken to complete the tasks was measured, and opinions about the different formats were gathered through a questionnaire.

Participants

Twenty one participants took part in the study in three separate sessions. All were taking part in a study week at Worcester New College during their Easter holiday in 2010. All had low vision, were print readers and expected to be using print-based examination papers in the GCSE examinations. The majority of participants attended mainstream schools (except one mainstream school with resource base and one ‘private’). A breakdown of the participants is presented in Table 1.


Table 1. Characteristics of the participants taking part in the trial (N=21). Note: frequencies based upon available data.Characteristic NAge group

Year 10 9Year 11 12

GenderMale 13

Female 8Font size (provided prior to testing)

12 point 218 point 1024 point 9

Font size (questionnaire response)16 118 824 836 1

LVA used (questionnaire response)Yes 8No 12

Preferred font (questionnaire response)Arial 9

Impact Bold 1Comic Sans 3

Any 1TOTAL 21

Materials and variables

Content for two ‘pretend’ examinations was created (version A and B). Both examinations had the same structure and length and were matched for content difficulty. Each paper consisted of ten questions. The question styles were drawn from existing papers and included:

Multiple choice with tick box (‘Tick the BEST answer’) Sorting statements into categories Multiple choice (Put a ring around the correct answer) Reference to a picture, single word written answer on answer line Reference to a table, multiple choice Labelling a diagram


Reference to a graph, single word written answer on answer line Reference to text and a picture, single word or two word answers Linking of associated statements

As part of the introduction given in each session, it was emphasised to participants that we wanted them to complete all questions and that we were testing the quality of the material and not testing them. For this reason the questions were very easy (containing relatively simple general knowledge and logical questions). The key measured variable was time taken to complete each examination paper.

After completing the two versions of the examinations (as described in the procedure below), participants completed a questionnaire (see Appendix). The questionnaire contained questions in relation to:

Personal working preferences (font style and size, use of LVA) Preferences for different formats of examination and explanations Ideal examination format and potential of onscreen examinations.

On completion of the questionnaire participants took part in semi-structured group discussions. The topics explored were:

Reflections upon the versions of the examinations just taken General experience and expectations in relation to taking

examinations Potential of computer-use in examinations.

Procedure

Participants were allocated into three groups and each data collection session took place over approximately a 1.25 hour period. At the beginning of the session the broad purposes of the research was explained and it was emphasised that the material was being tested rather than the participants themselves. Participants were then asked to sign a consent form declaring that they were willing to take part in the research, but it was noted that they could withdraw at any time. Participants were each given a £10 gift voucher irrespective of whether they chose to take part in the research or not. It is worth noting participants were taking part in a study weekend in preparation for GCSE examinations and taking part in this session was seen as a useful and relaxed way of practicing and considering different formats of examination papers.

Participants were asked to carry out the testing in ‘exam-like conditions’, i.e. in silence and working independently. Each participant took two


versions of the examination – one modified large print (‘MLP’) and the other E-formatting software (‘Software’). The order of presentation was balanced. The following procedure took place:

1. Instructions to participants were given2. Examination Paper 1 was handed out (all pre-prepared and named for

each participant)3. When the researcher said ‘start’, participants started timers on their

desk and began Examination Paper 14. Once Examination Paper 1 was completed, participants would stop

the timer, note the time on the front of the paper, and indicate to the researcher (who also noted time taken as a check).

5. Participants then sat in silence until everyone had completed the Examination Paper 1 (a total maximum time of 15 minutes).

6. All the papers were collected, and Examination Paper 2 was distributed.

7. The process for Examination Paper 1 was repeated for Examination Paper 2.

8. On completion of Examination Paper 2, the questionnaire was distributed and completed by participants.

9. On completion of the questionnaire all participants took part in a group discussion.

To control for effects of order and exam content, the experimental design was balanced, i.e. participants received different versions of the paper (A or B), prepared in different ways (MLP or Software), in different orders (first or second).

2.3 ResultsThroughout the report all the available data is used. For example, analysis of performance (based upon time taken) includes 19 of the 21 participants because two participants were not able to complete the examination papers in the time available.

Performance – time taken

A key aim of the research was to establish whether there was a difference in performance when using examinations prepared using the existing large print approaches (modified within the existing large print ‘access arrangement’ policies, ‘MLP’) compared with formats prepared using the E-formatting software (‘Software’). In terms of mean time


taken for a sample of 19 students with low vision, very little difference was found (mean time MLP 422 seconds cf. mean time Software 418 seconds) and unsurprisingly this was not found to be statistically significant. This suggests that examinations formatted using the E-formatting software neither significantly advantages or disadvantages student performance compared with existing modified large print approaches.

Analysis of ‘control variables’ used to balance the experimental design was also carried out. No significant difference was found between the times taken across the two versions of the examination paper confirming that they appear to be matched in terms of difficulty. However, a highly significant difference was found in terms of order of presentation – examinations taken first were significantly slower than examinations taken second (first mean = 486 seconds cf. second mean = 354 seconds). In general terms this is not surprising as participants learned about the structure of the examinations from the first examination to the second and this was reflected in faster answering (and to this extent vindicates balancing the design). Perhaps more helpfully, this strongly suggests that encouraging students to practise examination papers, allowing them to get used to question styles and structures, is of enormous benefit.

Table 2. Summary of repeated measures t-test: (a) MLP vrs Software, (b) First vrs Second, and (c) Version A vrs Version B. N=19Comparison Mean

(seconds)Standard Deviation

Significance

MLP vrs SoftwareMLP 422 147

Software 418 162 t (18) = 0.12; p>0.05First vrs Second

First 486 146Second 354 132 t (18) = 7.15; p<0.0005

A vrs BVersion A 409 147Version B 431 162 t (18) = 0.64; p>0.05

Note: 420 seconds = 7 minutes

Another interesting observation is linked to the two participants who did not complete either examination paper within the available 15 minutes. Given the examination papers were less than 700 words long (plus diagrams) these two participants clearly had a particularly slow reading


speed. Again, this general point is discussed within the overall discussion.

Preferences

Participants were asked which examination format they preferred overall. Table 3 shows that answers were fairly evenly distributed, although the largest proportion of participants thought the Software version was the best. Perhaps of greater interest were the comments and explanations for what participants liked and disliked about the different formats (Table 4 and Table 5).

Table 3. Which examination format did you prefer? N=21Format Frequency

MLP 6Software 8Both the same 5Unsure 2

Table 4. MLP – explanations of what participants liked and disliked. Answers grouped by authorsLikes (frequency)Generally positive about font, font size, or use of underlining (7)Generally positive about the layout (5)Generally positive about use or graphics (4)Layout of pages / page breaks (3)Liked using large A3 paper (1)

Dislikes (frequency)Disliked using large A3 paper (3)Wording confusing (2)Poor multiple choice format (2)Poor pictures (2)Poor general layout (2)


Table 5. Software – explanations of what participants liked and disliked. Answers grouped by authorsLikes (frequency)Generally positive about the layout (5)Generally positive about use or tables and pictures (4)Generally positive about font, font size (4)Liked using A4 paper with larger font (3)Good multiple choice format (1)

Dislikes (frequency)Poor page breaks / split questions (6)Wording confusing (2)Poor / lack of pictures (2)

Participants’ likes and dislikes of the two formats overlapped considerably. This is unsurprising given that many expressed no preference between either formats. For example, there were positive views about general formatting, fonts, graphics, and layouts for both MLP and Software versions. Similarly, there were negative views regarding some graphics, wording and multiple choice formats. These negative views (and some of the positive ones) broadly fall within the modification process rather than the formatting process per se. That is, irrespective of format (MLP or Software) a modifier will have made a decision about particular graphics and wording modifications and structure of multiple choice questions (e.g. tick boxes aligned left may be easier for the visually impaired pupil to find rather than scanning over white space to a right aligned box). This distinction between the modification and formatting (including enlargement process) is raised elsewhere in the report.

Nevertheless, some of the comments did link to particular features of the formats (and the process of producing them). The Software version use of page breaks was identified as a problem by six participants, i.e. having questions split over pages. This is discussed in section 6 ‘Technical analysis of the RNIB E-formatting software’ and confirms the points made there. It was also raised in some detail in the group discussions. This highlights the importance of navigating documents (examination papers in this context) and the importance of considering this in the exam modification process, e.g. as illustrated by the following quotes:


“It gets annoying when you have to turn the pages. And sometimes the question’s on one side and information is on another page.” Discussion group 1.

“I preferred the large print one it was easier to read -more compact, less page turning.” Discussion group 2.

Positively, three participants (who preferred 24 point font) specifically said they liked having the examination on A4 paper (which is possible using the E-formatting software, but not within the MLP procedures). Perhaps in keeping with an emerging theme (and general ‘truism’) of ‘everyone is different’, one participant described preferring having bigger A3 paper. It’s worth noting that this unexpected preference is achievable using the E-formatting software (by using an A3 printer or photocopier enlargement). Again, this was raised in the group discussions:

“I think the software was better because, it's on smaller paper but with the same size writing.” Discussion group 1.

Page breaks and paper size both link to the process of navigating the examination paper. Some participants made links between this and the potential of technology.

Ideal format and use of computers

Participants were asked to ‘explain how you would have liked the examination papers formatted for YOU’. The purpose of the question was to get some sense of how they would ideally like examination papers formatted. Most of the responses made references to specific formatting strategies (and often referring to the examination papers they had just used). Some of the responses were quite full, while others made specific single points:

“Font size to be 18 all way through. Clear pictures. Question numbers in bold. Information tables on same page as questions to prevent having to keep flicking back and forward.” Questionnaire.

“Size 24 one question per page bold, comic sans, simplified diagrams and maps.” Questionnaire.

“The resources/questions and answers on the same double page spread.” Questionnaire.


The points raised tend to highlight the wide variety of individual formatting preferences, and some appreciation of the modification process. In contrast, one respondent implied that she did want any modification to the formatting:

“The same as everyone else.” Questionnaire.

Only one participant appeared to consider possibilities beyond options which are currently available, in this case having the examination paper accessed using a computer:

“On a computer screen.” Questionnaire.

Linked to this last point, the questionnaire contained a question “Consider if the examination papers were presented ‘on-screen’. What would be the advantages and disadvantages for YOU?” Responses are summarised in Table 6.

Table 6. Advantages and disadvantages of onscreen examination presentation. Answers grouped by authorsAdvantages (frequency)General positive access (e.g. ‘easier to read’) (7)Easier navigation (e.g. no page turning, wouldn’t miss out questions) (5)Individualised format (4)

Disadvantages (frequency)General negative access (e.g. ‘difficult to see’) (5)Visual fatigue (eye would tire over time) (4)Difficult navigation (e.g. scrolling) (2)Computer crash (1)

As might be expected, the identified advantages and disadvantages seem slightly contradictory – some participants identifying advantages in the accessibility / navigation afforded by the computers, some identifying disadvantages. It’s difficult to be certain when drawing upon small written quotes, but perhaps the positives can be summarised as a view that the computer enables individuals to change the formatting according to their own preferences and use the computer functions to navigate through examinations. These points were raised in group discussions:


“Flipping- pages gets annoying, especially when it's stapled in certain places. It stops you getting to certain bits of the booklet. So tapping the keys getting up and down to get to earlier questions later questions would help.” Discussion group 1.

“It’s brighter so you can see clearer. And you can change the font size, the background colour, and the font colours...” Discussion group 2.

“I don't want to sound lazy or like I would take advantage, but my hands do get tired and my neck gets craned when I write a lot like in English. [..] In English exams that would help if I could do it on a laptop instead of just [laughs] crouching down for 2 1/2 hours!” Discussion group 3.

“[Using a laptop in an exam would be a good thing because] it would be bigger I would not have to keep flipping pages. Annoying.” Discussion group 1.

In contrast, others were less convinced by this, feeling that their preferred format could not be accommodated in this way and navigating (e.g. scrolling) was difficult. Again, these and other related issues were raised in the group interviews:

“No, I wouldn't use it in an exam. [..] I just prefer to write in an exam. [..] Plus if I write, I can write it out in a certain way... But if you do it on a laptop you have to do in a document... it's hard to do things like spider diagrams, when you're planning.” Discussion group 2.

“It depends on what kind of exam paper because it was just questions it wouldn’t be too bad. But if it was like science, with diagrams or maps, I think you would be quite tricky to look at the whole thing at the same time. Because of using zoom text...” [Researcher: the paper enables you to see ‘the whole’] Discussion group 2.

Perhaps some of these problems could be overcome with training and practice in the use of tools which allowed participants to use computers more flexibly and skilfully.


However, more challenging is the issue of ‘visual fatigue’ which was raised by four participants independently in the questionnaires (all of whom also recognised advantages to using a computer, so they were not all together dismissive of such a strategy), e.g.

“My eyes would start to ache from staring at a screen for a long time. It would set off my nystagmus.” Questionnaire.

“Eyes would get more tired. [..] The half-hour, yes. But, say, for a two hour exam, no... where you're looking at it constantly.” Discussion group 2.

Similarly, although only one participant raised concerns about computer crashes (and therefore loss of work) this is also an important point which is raised in the interviews with professionals. Some strategies can overcome some of the dangers (e.g. periodic print outs and backups), but this adds to time pressures and general worry at a stressful time.

The potential of computer-based examinations and preparing young people to access information are explored throughout this report as key themes.

Additional themes raised in the group discussions

During discussions about laptops it was encouraging to note that many used laptops in class, and most described being able to touch type. The use of computers was also raised in the interviews with professionals highlighted elsewhere in the report. Even so, some participants were anxious about appearing to be treated and looking different (whether in exams or the school generally), or identified other barriers to using computers:

“I have a laptop but I don’t use it at School. [Because] it’s a mainstream school and I don’t feel comfortable using it in classes. It doesn’t seem right.” Discussion group 2.

“I don’t have [a laptop]. Not allowed. Girls in the school were jealous of it so they stopped me using it.” Discussion group 2.

“The head teacher said I could bring [the laptop] in, but most teachers said they did not want me to use it. Got angry because I was typing on a laptop. [..] I can look at the board


at the same time as writing. So it’s quicker…” Discussion group 2.

The vignettes are poignant as they suggest access to information for these students is being compromised. These points also have similarities to studies which have found that low uptake of low vision aids (LVAs) in schools is partly explained by students not wanting to appear different (e.g. Mason, 1999). In keeping with this over half of the participants in this study (12) said they did not use an LVA yet all of them described preferring a large font size.

Discussions suggest that there was a lack of awareness amongst the participants about the examination modification process (even though many of them would be taking examinations in the following months, and had taken examinations in the past). This was most clearly illustrated by participants sometimes having an expectation of having examination formats which are not available within the current modification system. Linked with this, participants had mixed experiences of practicing examinations in the relevant format:

“I’ve tried mock exams but they were not the right format.” Discussion group 2.

“In mock exams I never ever have an enlarged paper like that [referring to test material]. But in exams [I do]. [..] The resource base makes the decision for me.” Discussion group 2.

There was also some (limited) discussion about where the modification system had not worked in previous exams participants had taken:

“Some reason it always seemed to be me who gets the wrong one... or they only photocopy half the paper ... so have to stop my exam and everyone else can carry on. It's rubbish.” Discussion group 2.

[Participant] gave an example of exams not arriving until a day late and the LSA having to photocopy the normal paper onto A3 sheets, “The size is quite annoying because it was covering most of the desk.” Discussion group 3.

As this report develops we identify a tension between normal classroom practice (in terms of curriculum access) and options available in


examinations. There are clearly multiple reasons for these tensions, but it suggests that students may be taking examinations in relatively unfamiliar formats. The experiment described in this section illustrates the significant impact that familiarity can have on performance.

This general point about difference between normal classroom and examinations practice exists irrespective of whether the current examination modification process works smoothly. Even so, some of the examples given by students suggest that examination modification process does not always work smoothly. Both these themes are developed further in the interviews with professionals.


3. Case examples: Students who prefer very large hardcopy print (N24+)3.1 IntroductionThe research team carried out semi-structured interviews with five young people who were identified as preferring to use enlarged hardcopy print in a font greater than 24 point (i.e. greater than the maximum sized print provided by examination boards through existing access arrangements). During the interview participants were asked to comment upon different font sizes in terms of their preference using a ‘font size questionnaire’, and then carried out matched ‘examination-style’ tasks which compared paper-based formats prepared using existing large print approaches (enlarged to 24 point bold Arial, with a LVA if required) to a format based upon their own preference. The aim was to examine whether either format gave any reported advantage to the participants. The participants also commented upon onscreen presentations of the same examination questions.

3.2 Method

Participants

Five participants took part in the study. All had low vision, were print readers and were interviewed separately. Three of participants attended a special school for visual impaired students (and had not yet taken any GCSE examinations). Two of the participants had attended mainstream schools in the past and were currently attending mainstream colleges. A summary of the participants is presented in Table 7.


Table 7. Characteristics of the participants taking part in case examples (N=5).

Participant number

Name Age Preferred font (based upon information given prior to interview)*

Preferred font (based upon information given at interview)*

1 Keith 12 36 point Arial Bold 36 point Arial Bold2 Ruth 14 36 point Arial Bold 36 point Arial Bold3 Yvonne 12 36 point Arial 36 point Arial4 Danny 19 40 point Arial 28-30 point5 Sarah 19 42 point Verdana Unsure/ 36 point

Arial Bold* Note: this information was not always consistent between prior and interview information.

Materials and procedure

The researcher asked the participants about the size of print they liked and used by administering the ‘Font size questionnaire’ (see Appendix 2). Information gathered included: preferred font style and size and also some details about LVA and computer use for accessing print. This was followed by asking participants to read through alternative print sizes (Arial bold 18, 24, 28, 32, 36 and 40 point) and for each providing their opinions about ease of access (generally and in an examination). At the end of the questionnaire, students were asked again about the font they preferred to use overall.

Content for two ‘pretend’ examination-style tasks was created (version A and B) taken from materials developed for the student performance tasks (previous section). However, the tasks were much shorter using only three multiple choice questions taken from those materials. Both versions A and B had the same structure and length and were matched for content difficulty. The format of version A was based upon the students preferred hard copy large print format (see Table 7) which had been prepared prior to the interview. Version B was based upon Arial bold 24 point (on A4 paper). It is worth noting that the research team created the materials using Microsoft Word (the RNIB E-formatting software proved too difficult for the process as it did not give the control regarding line spacing and page breaks – this is discussed in a later section of the report, “Technical analysis of the RNIB E-formatting software”).


As part of the introduction given in each session, it was emphasised to participants that we wanted them to complete all questions and that we were testing the quality of the material and not testing them. For this reason the questions were very easy (containing relatively simple general knowledge and logical questions).

The researcher first asked participants to answer question 1 of version A. The researcher noted observations including the time taken and asked the participants for their opinion of the task (“How did you find that?”). Once question 1 was completed, the same took place for question 2 and 3 (together). After completing version A, a similar procedure was followed for version 2 questions. In most cases, participants could not access this version without the use of a handheld LVA or CCTV (participant 2 could read the version 2 text without an LVA).

A final part of the structured interview asked participants about advantages and disadvantages of each of the format versions, and which they preferred. In addition, alternative onscreen versions of the questions were also presented to the participants using the RNIB E-formatting software (which gave control over font and background colour as well as font style and style). Participants were encouraged to read from the screen, and comment on the different presentations available. [Note: The laptop computer used for the demonstration had a relatively small screen (approx A4 landscape sized). The font sizes selected using the RNIB e-formatting software were not reproduced exactly to the chosen font size on the screen and the sizes reported in the report below are estimates based upon visual inspections of the screen presentation.] A final question asked participants about the advantages and disadvantages of onscreen presentation.

Each interview took between 45 minutes and one hour. At the beginning of the session the broad purposes of the research was explained and it was emphasised that the material was being tested rather than the participants themselves. Participants were then asked to sign a consent form declaring that they were willing to take part in the research, but it was noted that they could withdraw at any time. Participants were each given a £10 gift voucher.


3.3 Results

Case Example: Keith

Discussion about fonts

Keith described his preferred font size and style as 36 Arial (bold) without the use of an LVA. He also said he used this in his day-to-day classroom work. Even so, he described usually using a computer with JAWS and a large font in most lessons. His profile using the font size questionnaire is in the table below. Keith felt he would prefer size 40 for an examination.

Table 8. Keith – Font size questionnaire responsesSize Can you read? Could you read for a long piece

of text?18 OK No24 OK No28 Easily No32 Easily Don’t know36 Easily Yes40 Easily Yes

Observation of Keith reading questions

Table 9. Keith – Observations when reading hardcopy print questions

Version Time NotesVersion 1: Arial (bold) 36 point

120 seconds Worked very close to material. Described finding it ‘easy’ and ‘like he does in the classroom’.

Version 2: Arial (bold) 24 point with CCTV (white on black)

130 seconds The CCTV enlargement was larger than 36 point and Keith appeared to have a greater working distance. He navigated the questions without obvious difficulty and skilfully used the movable reading table.

Keith used a thick black pen to write his answers. Writing was very slow, and Keith described that he would usually write using a computer.


Overall, for the two multiple choice questions, Keith was marginally slower when using 24 point with the CCTV. Nevertheless the difference in working speed for this short multiple choice task did not seem obvious.

Preferences and screen use

Keith found it quite difficult to explain his preferences in the short interview. However, when asked his preference between 36 point or 24 point and CCTV he clearly chose 36 point, saying he was “used to it” (referring to classroom practice). Keith also noted that 24 point without the CCTV would be impossible for him.

When asked, given any choice, what format he would like for examinations, Keith said 40 point Arial Black (on paper). When prompted, Keith did say he would like to use a computer (and later having seen the examination paper onscreen felt that the computer presentation was his overall preference - again this was prompted).

When reading the examination questions from the computer screen, Keith worked very close and used approx 48 point1 on screen and felt he would prefer bigger). Advantages Keith felt the computer gave him were greater control of font size and style, as well as colour combination (Keith chose white font on black background). Disadvantages were seen as screen reflection. When prompted whether he thought onscreen would be tiring he replied ‘not really’.

Case Example: Ruth


Ruth described her preferred font size and style as 36 Arial (Bold), without the use of an LVA. However she said her usual font size in her day to day work was 24 (without an LVA). She did have an LVA, but usually had large print prepared. She also said she usually used a computer in class. Her profile using the font size questionnaire is presented in the table below.

1 Note: the onscreen fonts are approximate because they are a product of the selected font size and monitor size of the laptop computer used. Estimates were made afterwards based upon comparison with standardised printed material.


Table 10. Ruth – Font size questionnaire responsesSize Can you read? Could you read for a long piece

of text?18 OK No (“eyes hurt”)24 OK No28 Easily No32 Easily Yes36 Easily Yes40 Easily Yes

Observation of Ruth reading questions

Table 11. Ruth – Observations when reading hardcopy print questionsVersion Time Notes

Version 1: Arial (bold) 36 point

50 seconds Used a ‘tilt-top’ table. Seemed to find it easy. Described it as “straightforward”. Good posture, maintaining a relatively normal reading distance.


35 seconds Worked quickly, partly linked to anticipating the question structure. Worked closer to the point: “I struggled - had to go closer than the 36…” Ruth also wondered about getting tired.

Preference and screen use

Ruth preferred the 36 point compared to the 24 point (although she worked through the 24 point quicker, most likely she was anticipating the question structure having done it second). She felt the 36 point was big and easier to read, although it was “too bold”. She felt the 24 point was too small, although she thought it was “set out nicer - less room”.

Ruth looked at the onscreen version and discussed different sizes of font. She felt the screen might be easier to access, and of all the presentations discussed she felt the computer was the best choice for her. Even so, when prompted about whether it might be tiring, she felt her eyes did get tired when using a computer.

Case Example: Yvonne

As well as her visual impairment Yvonne also had a hearing impairment.



Yvonne described her preferred font size and style as 36 Arial (Bold) without an LVA. She used 24 point with a CCTV. She had an LVA which she used with books, and described using a computer ‘sometimes’. Her profile using the font size questionnaire is presented in the table below. Following this discussion Yvonne felt that she would prefer font size 32 for her day to day work, and 28 point for examinations (the smaller font would mean that there would be “less page-turning”).

Table 12. Yvonne – Font size questionnaire responsesSize Can you read? Can you read for a long piece of

text?18 No No24 OK No28 Easily Yes32 Easily Yes36 Easily Yes40 Easily (“too big”) Yes


Observation of Yvonne reading questions

Table 13. Yvonne – Observations when reading hardcopy print questions


90 seconds Used a ‘tilt-top’ table. Worked slowly and deliberately. Described she was “happy” with the format. Yvonne needed help with one of the questions (‘Who was the first person to walk on the moon?’).

Version 2: Arial (Bold) 24 point + CCTV (white on black)

125 seconds Seemed to work obviously slower than when using 36 point, particularly navigating the paper (Yvonne did not use the CCTV movable reading table, but moved the paper), although she skilfully held her place with her finger when answering a question. Described the format as “easy”. Yvonne needed help with one of the questions (‘Which person is a footballer’).


Yvonne did not express particular preferences for any use of the formats she used in the activities (although she was very shy and reluctant to speak very much). When she tried the onscreen format (Arial bold, approximately 48 point) she worked at a working distance of about 25cm with a comfortable posture. She felt that working from a computer might be easier to see because it can be made bigger. When prompted about eye strain, Yvonne said “maybe” noting that when she worked on her computer at home for a long period her eyes did water. When reflecting upon all the formats she had tried (36 point, 24 point with CCTV, an onscreen computer), Yvonne felt all were the same and she had no preference.


Case Example: Danny


Danny described his preferred font size and style as 28-30 without the use of an LVA (different to the Arial 40 point we had anticipated). Ideally this would be 30 without the LVA but smaller if he was to use a magnifier. He mentioned Calibri as his preferred font but clarified this by saying that as long as the font was clear he could read it. He used a range of font sizes in his day-to-day College work. He reported his vision had deteriorated recently and he was now learning Braille. He presented the three LVAs he used at home – a hand magnifier; one with an internal light and an Amigo portable video magnifier (CCTV). He uses a computer with Zoomtext in his College and at home. His profile using the font size questionnaire is presented in the table below. Of all the fonts presented Danny felt he would prefer 32 point for day to day work and for examinations – although he thought 30 point would still be appropriate for him.

Table 14. Danny – Font size questionnaire responses

Size Can you read? Could you read for a long piece of text?

18 No No24 No No28 OK/Hard No “uncomfortable for a long piece

of text”32 Yes Yes36 Yes Yes “Maybe too big for long piece

of text”40 Yes “Definitely too big”

Observation of Danny reading questions

Given that there was some ambiguity about Danny’s preferred font, the researcher showed Danny various sizes of the examination questions (Arial bold 28, 36, 40, and 24 points). Danny preferred not to write down answers but just to verbalise them (he was used to doing this with a ‘reader’ in previous exam situations). He tended to use a computer to write (only using a pen for writing his signature).


Table 15. Danny – Observations when reading hardcopy print questions


210 seconds Read slowly and quite hesitantly – tended to lean forward slightly to view paper on table did not appear to have a comfortable posture. Commented that did not find the Roman numerals easy to read or locate on the page. Commented that found “the text too small to read comfortably”.


105 seconds Much more fluent reading – more comfortable body position (Danny noted this in discussion afterwards). Commented that he found the paper “easy” to read and had no problems in accessing the information. Also commented that he found this more comfortable for “longer reading”.


110 seconds Similar to 36 – more fluent than 28 and comfortable viewing position. Commented that he found this paper to be very clear and could see it well.

Version 2: Arial (bold) 24 point (no magnifier)

N/A Could not read – brought paper close to his eyes but gave up trying “Sorry I can’t see this”.

Version 2: Arial (bold) 24 point (with Amigo CCTV)

130 seconds (not including pauses to adjust device)

Appeared to be competent using the CCTV independently– he read fluently once he had made adjustments. Able to adjust the magnification independently to alter the viewing size (started large with only 1-2 words on the screen then moved down so could approx 3-4 at a time). Was comfortable using the device although he was still getting used to using it for extended reading (it was relatively new to Danny).


He had a go at the paper with a hand magnifier and could read this (almost word by word) albeit slowly. Danny felt he would struggle to use this for extended reading (tended to use this for seeing instructions on tablet boxes and similar).

Preferences and screen use

The CCTV allowed Danny to access the paper and he could read this relatively fluently once he had found the correct magnification for the font size. He assumed a comfortable sitting position when using the CCTV although he was not sure whether it would be comfortable when using it for a long exam. Danny did not use the CCTV much at present as more reliant on the computer for access – and for hand magnifier for small amounts of text. On balance he felt he would prefer the larger font (i.e. 36 in this case) rather than 24 plus CCTV (mainly due to comfort in using the device for extended period and the fact that he was still becoming familiar with the CCTV).

Danny was not sure about preference of font for exams for him but felt it had to be larger than 30 and no bigger than 40. As indicated above, the font style itself was not considered an issue as long as it was clear. On balance he preferred bold if this was available.

Discussion about onscreen access was interesting in that he seemed much more comfortable reading from a screen rather than from paper (both in terms of seating position and in reading fluency, and seemingly with a slightly smaller font). We explored several options with the software (Danny selected various Arial font sizes he wanted to try out):

24 Arial bold – seemed very comfortable with this –good seating posture and fluent reading. Felt that the tables (which had low contrast lines) could be improved.

22 Arial (first not bold then with bold) – without bold leant forward to get closer to screen – with bold resumed a similar distance to that used for 24 Arial (bold).

34 Arial – could do this but felt it was too big and would be too slow.


On balance, Danny felt that 24 Arial bold “stood out best” and was “clearest” for him. Alternatively, he felt he could use 22 Arial bold but not for long periods of reading on screen.

In terms of advantages of onscreen access, Danny felt: that the text was clearer compared to paper (he made reference to contrast), it’s possible to do all changes to the exam format before starting in a short period of time (here Danny was making specific reference to the RNIB E-formatting software, or similar software to it), and he would be more comfortable sitting the exam using a computer than paper (in terms of familiarity).

In terms of disadvantages, again Danny referred to the feature of the RNIB E-formatting software which was used and noted that the writing/input of answers was problematic. While he felt he could toggle between documents, he would ideally prefer if he could input directly into exam document.

On balance he felt that he would prefer to access an exam through this route rather than a paper version (irrespective of software). Danny was impressed by features of software (trying out various options in terms of colour/contrast) and felt it could be “very useful to people like me in the future”.

Case Example: Sarah


Sarah was not sure about her preferred font size but thought it was Arial bold 48 point. Her mother made reference to a report that said it was Verdana 42 without the use of an LVA. With an LVA she thought she used 36 point. Both noted that Sarah had a deteriorating condition and she was learning Braille. She has recently started using an Amigo compact CCTV. She was about to start college so did not yet know how she would access information – in school text had been blown up for her and presented on A3 paper (no access to LVA nor did she know what font had been used). She used Zoomtext software on a desktop computer at home and was hoping to purchase a laptop prior to starting college. Her profile using the font size questionnaire is presented in the table below.

Table 16. Sarah – Font size questionnaire responses


Size Can you read? Could you read for a long piece of text?

18 No No24 No No28 No No32 No No36 Hard (very slowly and

with paper close to eyes - <3cm)

No

40 OK (a bit easier to read, but still held very close and not

comfortable reading this)

No

After the font size questionnaire discussion, in another discussion about font size Sarah felt she would prefer a font of between 42-48 point (“anything bigger than 40 would be OK”), and for examinations ideally 48 point.

Observation of Sarah’s reading questions

As with Danny, there was some ambiguity about Sarah’s preferred font. The researcher showed her the largest printed material which had been prepared (Verdana 40, with and without bold), followed by Arial bold 24 point. Sarah preferred to read out her answer rather than write them down (she tended not to write using a pen apart from signing forms).


Table 17. Sarah – Observations when reading hardcopy print questionsVersion Time Notes

Version 1: 40 point Verdana (then with bold)

Approx 240 seconds

Brought paper close to her eyes – very slow (read word by word). Lost track of where question ended and response was required. Did not see response boxes initially – asked where answer would go. Ring in Q2 not clear – and had problems with Roman numerals.

We tried the same paper in bold Verdana – a lot clearer and more fluent (though could have been the result of practice) and much more confident in tackling the questions. Commented the ring (‘circle the correct answer’) in Q2 was not bold. Not timed in bold as it was same paper but generally appeared a lot more accessible

Version 2: Arial (bold) 24 point (with and without Amigo CCTV)

N/A Without CCTV: Struggled to read – very slow – read half of Q1 then gave up.

With CCTV: Could access the paper but had difficulties in finding correct magnification size (started on maximum giving only one word on screen at a time). Once Sarah had found appropriate settings, she was able to read through the paper to the end and answer the questions – seemed to enjoy the experience (after previous attempts when she had got quite dispirited). We did not time this as she was struggling to use the technology (which was new to her) – but generally much more confident in her reading once she had found the appropriate setting.


The Arial bold 24 point was a real struggle for Sarah, but with the CCTV she could access the text and was far more confident in her reading. This raises issues regarding familiarity with the technology – Sarah required time to find the appropriate adjustment for the 24 point text.


However, once this had been done she was much more comfortable with using the technology and we tried using it with other font sizes for comparison. Even so, Sarah felt she would not want to use the CCTV for a long paper (though she was happy using it with the short three question paper that was used in the study).

We explored the onscreen version and discussed different sizes of font. Sarah tried a format in Verdana 32 point – could read this fluently on screen – able to independently scroll down the page – and could answer the question without any prompting. Next, Sarah tried Arial bold 32 point which was read fluently (but she had difficulties in locating the table lines, which were not bold). She was happy to explore other fonts – Arial 40 bold was equally accessible. Contrast of some of the pictures may have been problematic.

Overall, Sarah was very confident and comfortable with onscreen access and she felt she would prefer her exams to be presented in this way if possible (CCTV fine for small amounts of text). She felt the advantages were: more independent; less frustrating; easier to access. Disadvantages identified were: more time as need to scroll down the page (in terms of amount of text on screen); contrast in diagrams and tables “needs to be sorted”; hurts eyes if looking at computer for too long.

3.4 DiscussionThe aim of these additional case examples was to gain an insight into the print access challenges faced by students who prefer to use hardcopy large print in font size greater than 24 point. In terms of the number of students this applies to in the UK, findings of a recent survey of 21 teaching services carried out in England and Wales gives some insight. Table 18 illustrates the range of preferred reading formats these services report they currently provide with over 9% of supported pupils reported to prefer font size greater than 24 point. This suggests that the five participants who took part in these case examples are by no means unusual in their preference for 24+ point hardcopy large print.


Table 18. Visually impaired students (aged 13-16 years) preferred reading formats as reported by 21 education services in England and Wales (N=287 pupils).

Preferred reading format

Total (N) Percentage

12 point 57 19.86%14-18 point 113 39.37%20-24 point 69 24.04%>24 point 27 9.41%Braille 18 6.27%Electronic 3 1.05%Total 287 100.00%

Although the case examples provide relatively limited qualitative data, they do allow some useful comparisons and observations which are relevant to the current study:

A comparison between bespoke hardcopy large print (in the student’s preferred size, greater than 24 point) and Arial bold 24 point (as is available through existing access arrangements).

In four of the five case examples participants felt they would find it too hard to access the Arial bold 24 point material without additional magnification. Ruth was an exception and was able to access the material with relative ease, although she felt it was too small for sustained work. This suggests that for all these participants, with the possible exception of Ruth, the use of bespoke hardcopy large print (of various sizes greater the 24 point) was needed if they were to access the print without the use of additional magnification devices. This leads to a follow-up comparison:

A comparison between bespoke hardcopy large print (in the student’s preferred size, greater than 24 point) and Arial bold 24 point with additional LVA if available.

All four of the participants who required additional magnification were able to access the Arial bold 24 point when using an LVA. In all four cases they used a type of CCTV (Keith and Yvonne used a large desk-based system, while Danny and Sarah used a portable device). It is encouraging that all four could access the material in this way (and it may be possible for them to access smaller sized print in this way for


particular tasks). Nevertheless, Keith, Danny and Sarah all said they would prefer the bespoke large print for an examination (Yvonne did not express a preference). Although given the structure of the activities it is difficult to be categorical, participants were probably slower when working with the CCTV and 24 point compared with the bespoke hardcopy large print. It is a delicate and speculative point which is discussed elsewhere, but potentially all participants would have improved their speed of access using the CCTV with practice. Both Danny and Sarah were new to using their portable CCTVs and even within the space of the short interview they improved speed and confidence as they learnt to use the magnification facilities more efficiently.

The case examples also provide insight to a third area of interest:

How do participants respond to the on-screen computer presentation as an alternative option?

Four of the five participants expressed an overall preference for the potential of onscreen computer presentation (Yvonne expressed no overall preference). It was the control of presentation the screen gave them which seemed to be main advantage (and this was expressed in more detail by the two older participants, Danny and Sarah). In all the cases the participants were using computers in their studies to some extent. Interestingly, the computers offered potential for writing answers as much as reading the questions – an important observation was the difficulty Keith, Yvonne, Danny and Sarah had writing their answers and, perhaps unsurprisingly, this seemed as big a problem as reading the print. For these participants, answering examination questions had required (or would in the future) the support of an amanuensis (scribe) or computer. Danny raised this issue directly, suggesting that while he could toggle between electronic question and answer documents on the computer, he would ideally prefer if he could input directly into an examination document.

Given the advantages of the onscreen presentation, and the common use these participants make computers anyway, the use of onscreen formats for examinations seems a better option for many of these students. Even so, some of the students did mention (although generally following prompts from the interviewer) the potential of visual fatigue associated with using a computer screen.


4. In-depth interviews with staff involved in material modification

4.1 IntroductionThe research team carried out semi-structured interviews with education professionals who support young people with visual impairment. The aim of the interviews was to explore the potential of the E-formatting software with explicit reference to existing modified large print (‘MLP’) procedures. The interviews also considered how the use of the E-formatting software could be implemented in different contexts. In addition the interviews discussed the potential of on-screen presentation of examinations.

4.2 MethodParticipants

Eleven participants took part in the study in nine separate interviews. A list of the participants is presented in Table 19. Participants were recruited in a variety of ways: contacts offered by the project advisory group, contacts sought through head of services in the midlands regions, and a trainee QTVI who was interested in project work in the area of examinations access. All the participants had experience of working with visually impaired students who were taking examinations as well as prior knowledge of modified large print (‘MLP’) procedures.


Table 19. Characteristics of the professionals taking part in the semi-structured interviews (N=11).

Interview and format Job title / settingA - face-to-face Senior practitioner for a visiting teaching

service (VTS)B - face-to-face QTVI in VTSC - face-to-face QTVI in VTSD - face-to-face Senior Technician for resource base and

VTSE - face-to-face Teaching Assistant (TA) in mainstream

school (supporting one child)F - telephone Trainee QTVI in VTSG - telephone QTVI in VTS and resource baseH - face-to-face QTVI in VTS and resource base H - face-to-face Teaching assistant in resource baseI - face-to-face Lecturer in specialist FE collegeI - face-to-face Lecturer in specialist FE college

Interview schedule and procedure

The semi-structured interview schedule was designed around the two project research questions:

1. How do the Modified Large Print (MLP) versions of papers produced by RNIB E-formatting software compare to traditional MLP papers? (both process and outcome)


The sections of the interview schedule were designed to provide a framework in which the researcher and interviewee could ‘step through’ the process of converting a standard GCSE paper to a MLP paper, and consider the possible role the RNIB E-formatting software could have in this process. The interview drew upon a 2009 GCSE examination paper (OCR, Twenty First Century Science, Biology A, A221/02). This paper was selected as it was a complex paper that incorporated a variety of question types, and a MLP and E-formatting software (‘Software’) version had been produced for visually impaired pupils in 2009.


The interview schedule (presented in full in Appendix) was structured around three sections. Section 1 explored hard copy examinations (focussing upon the primary research question). The interviewer presented the hardcopy original paper and then discussed this in terms of student familiarity and ease of access. A similar format was used for the MLP version of the paper. For the Software version two hardcopy versions were shown as examples (18 point Comic Sans, and 28 point Arial) .A demonstration of the pilot software was then provided by the interviewer on a laptop to illustrate its potential to create versions of the examination in a variety of font sizes and styles (and colours of paper).

Section 2 explored the potential of onscreen examinations (focussing upon research question 2). The interviewer presented the original, MLP and Software examination papers onscreen (the original and MLP were PDF files and used Adobe Acrobat). It was emphasised that students would be able to use access tools to format the onscreen presentations.

The focus of section 3 was on the ‘perfect exam scenario’. Within this section, participants were asked to:

Reflect on the type of examination provision for visually impaired pupils that they would you like to see in the future (one of the prompts included as part of this was a distinction between ‘students controlling the access’ versus ‘materials being prepared to his/her specification’).

Discuss the specific quality of the formatting and modifications of the example examination paper.

Discuss the ‘principles’ under discussion, i.e. ‘the proof of concept’ of the RNIB E-formatting software (and potential of the onscreen examinations).

Procedure

Seven of the interviews were carried out in person in a quiet room. The interviewer brought hardcopies of the examination papers and demonstrated the RNIB E-formatting software and onscreen papers using a laptop. In the case of the two interviews carried out over the telephone, materials were sent to participants in advance and the participants had a computer available during the interview.

At the beginning of the interview the broad purpose of the research was explained and the structure of the interview described. Participants were


then asked to sign a consent form declaring that they were willing to take part in the research and that they could withdraw at any time.

With permission from the participants, all the interviews (with one exception) were audio recorded. The researcher worked though the interview schedule and took key notes. Each interview took between 1 and 1.5 hours. When the interview was complete, the researcher wrote up the notes (drawing upon the audio recordings as an aide memoire and the source of verbatim quotes when helpful). These interview transcripts were analysed as described below.

Analysis of interview transcripts

The qualitative data analysis tool NVivo 8 was selected to help the research team make ‘sense’ of the data. The transcripts were coded in order to draw out themes. The coding structure was constructed by:(1) Ensuring it was grounded in the responses provided by

participants as captured in the transcripts of the interviews.(2) Drawing upon the project aims and research questions (and

how these were reflected in the interview schedule).(3) Drawing upon discussions within the research team following

the interviews.(4) Drawing upon broader literature in relation to curriculum access

and visual impairment.(5) Drawing upon feedback from the advisory group.

NVivo software was used by author 1 to scroll through the transcripts, highlight text and to ‘code’. These codes were then described and illustrated with quotes within a first draft of the report. This draft was shared with other authors and modified and refined further. Such a process of analysis is necessarily subjective but seeks to: provide an authentic representation of the interviews and professional

views; connect with a broader literature and policy context; develop practical and useful recommendations.

4.3 Analysis of resultsUnsurprisingly the identified themes partly reflect the interview structure. Further, as described in the method, some participants placed more attention on a detailed analysis of the quality of the modified papers


while others emphasised the principles and potential of different approaches. The themes are arranged under the following headings:

1. ‘The devil is in the detail’: Technical analysis of MLP and Software papers

2. The potential of RNIB E-formatting software3. Managing the examination process using the software4. Tensions and contrasts: classroom practice, examination options and

the ‘real world’5. Access to the curriculum: student skills and service policies6. The potential of onscreen examination presentation7. Different views of a ‘perfect world’

An emerging ‘story’ running through the analysis is an interesting distinction between two higher level approaches to exam modification and access, namely ‘a student-centred approach to curriculum material preparation’ verses ‘an independence-skills approach to curriculum access’. This distinction was not always explicitly articulated by participants but provides a useful means of understanding the challenges of examination access faced by young people with visual impairments, and hopefully ways of providing solutions.

1. ‘The devil is in the detail’: Technical analysis of MLP and Software papers

The interviews generated detailed technical analyses of the quality of the examination papers. Key features of included:

Precise opinions about how things should be done in relation to exam modification. For some participants this overlapped with a feeling of dissatisfaction with the existing MLP procedures and the quality and consistency of some of the examinations they had received.

A number of participants made reference to the JCQ Best Practice Guidance (JCQ, 2008), and had detailed understanding of its contents.

Commonly (though not exclusively), participants who were not teachers raised these issues in greatest detail. This perhaps reflects their close work with preparation of curriculum materials for visually impaired students.

Each analysis tended to be child-centred in nature (often making reference to particular children with whom participants supported).


A selection of responses for the MLP and the Software paper respectively is analysed below.

Modified Large Print (MLP)All participants offered feedback on the quality of the MLP paper with a number providing a detailed page by page analysis. Three examples are included below to illustrate the nature of this detailed feedback:

“Too much on a page – [child] would not cope with it all. [..] If pictures are not relevant I would miss them out completely.” Participant 5.

“Weight of lines may be a bit thicker - this is a common problem. Why capitalisation and underline? ….seems over the top and unnecessary, and possibly confusing. But it is part of the guidelines...” Participant 7.

All questions could be on one page (Q2); Better clarity on box needed (Q3); Diagram (Q5) still problematic – lines are thicker and numbers better but line merges into shading. Overall an improvement in terms of access and navigation “but there could be some more improvements.” Participant 1

Participant 4 also identified aspects of the modification which was described as being ambiguous. She also passed on a precise documented list of errors from the previous year’s exams that made reference to JCQ Best Practice Guidance (JCQ, 2008).

The level of detail and knowledge displayed by the participants was impressive. They reflected upon the difficulties individual students they work with might have. This detail is illustrated with an example identified by at least three participants when discussing the modification and formatting of a ‘sorting’ question (in particular the capitalisation and underlining). Figure 1 presents a scanned image of the question. This example is particularly interesting as a similar format was used in the performance task in our work with visually impaired young people (see . 2. Student performance and views of alternative examination papers). Figure 2 presents a scanned image of the answered question in which the participant made an error.


Figure 1. Example of a MLP question which several participants identified as being possibly problematic (Source: OCR, Twenty First Century Science, Biology A, A221/02). Note: original font size 18 point, reduced to fit on page.


Figure 2. Example of MLP question used in the trial in which a participant made an error. Note: original font size 24 point (18 point A4 paper, photocopy enlarged to A3), reduced to fit on page.


It is useful to consider likely factors which led to the participant making this error. Primarily, the underlined capital F (F) appeared to have been read as a capital E. Many people without a visual impairment may also have made such a visual error, except the context of having a list of letters in the question would make the error unlikely (so much so that people with normal vision would probably not even notice the visual ambiguity). However, because this participant has very poor visual acuity, he held the paper very close. This in turn meant that his field of view and therefore his contextual cues were reduced.

The example is helpful as it serves to support the points raised by the professionals with an actual error made by a visually impaired student. In the broader policy sense it might be argued that the individual needs of the student are impossible to accommodate within any centralised system. Alternatively, it could be argued that the system is adequate but in this case it was not implemented in accordance with modification guidelines (or the guidelines need to be improved).

We develop this theme elsewhere in the report, including the discussion. Within this analysis it is useful to note however that participants raised many detailed points about this particular MLP paper – one which was complex, but had been considerably, and presumably expensively, modified from the original. In that sense it appears not to result from any lack of ‘investment’ or ‘effort’ on the part of the examination board, modifier or producer.

Participants in fact raised many general points about dissatisfaction with the quality of examinations they had received in the past (e.g. Participant 2, 4, 7, and 8). This also links with research by others. Cobb (2008) inspected 33 MLP examination papers and concluded ‘a lack of consistency between different awarding bodies in the style and quality of the modified papers produced’ (p.ii).

RNIB E-formatting softwareAs with the analysis of the MLP paper, participants provided detailed points about the Software version of the paper. These are described in more detail in the next section (Technical analysis of the RNIB E-formatting software) and provide some possible areas for development. Points raised include:

Line spacing Pagination (page breaks, splitting of questions over page) Limited control over diagrams (including sizing and size of text) Better choice of colours for onscreen presentation


It should be noted that for many participants these points were raised in the context of a generally positive reaction to the software discussed in the next section (particularly in relation font size). There is however an important relationship between ‘formatting' and 'modification' that needs to be unpicked carefully. As an example, a number of the positive points raised by participants about the software were in fact linked to modifications which had been made independent of the E-formatting software itself. As observed by some participants, the modifications to the Software and MLP versions were not consistent, suggesting that they had been designed by different modifiers. As a result, different modification choices had been made including for example the position of the checkbox in multiple-choice questions (left or right aligned), the inclusion or exclusion of pictures. This meant that in practice the comparison between software and MLP versions was not ‘like with like’. A similar point was highlighted in the section on student performance (Student performance and views of alternative examination papers).

This again highlights the critical role of the modifier and the challenges of ensuring quality, reliability and consistency in the modification process A possible strength of the E-formatting software therefore is that it could serve to ‘force’ a particular style of presenting particular questions. For example, a modifier preparing papers for the E-formatting software may always choose an agreed method of presenting multiple choice questions (left aligned check boxes which do not require visual scanning over space), numbering, text alignment, etc. Such a system may encourage greater consistency of work (‘within’ and ‘between’ modifiers).

2. The potential of RNIB E-formatting software

Under this second heading, we briefly consider the potential of the RNIB E-formatting software. As noted above, participants were broadly very positive about the software, and in particular to the ease within which a range of font sizes / ‘bespoke’ formats could be achieved. This is particularly relevant to students who prefer print size greater than the standard 18 point, and crucially, do not like exam papers presented in A3 format (who were seen as being excluded from current procedures). This point is illustrated by the following quotes:

“Arial Point 28 paper – much nearer to [child D’s] preferred font size. [..] Generally is very similar to what I produce for D… hours and hours of work for one person.” Participant 5.


“This is much more the bespoke. This looks much more like the sort of work that our access support assistants would produce for people. [..] It would be ideal if we could do this. [..] I think that [the RNIB E-formatting software]... To be able to do it for yourself would be great. That is pretty near the perfect world.” Participant 3.

“We always recommend A4 but some examination boards provide A3 which we, and most students hate!” Participant 6.

Many of the participants (1, 2, 7, 10, 11) referred enthusiastically to the potential of the software:

“First impressions of it, I think it's much, much better. [..] Brilliant [ potential]. We could definitely use it.” Participant 2.

“In the ideal world they would get what they want at the press of a button...[laugh] I don't think we're there yet. But it would be nice to be able to pick up a file to convert it quickly into what they like.” Participant 7.

There were relatively few negative comments beyond the technical points described previously (page breaks, formatting of diagrams). There was however some anxiety about whether the software really could cope with the formatting required for specialist topics, which for example required subscripts and superscripts (science and maths), or a range of fonts (English and drama):

“If you just have something that just automates print size and background there are other sorts of things that crop up… [..] For example with poetry once you start modifying you start to get issues with line breaks, stanzas etc. For drama you couldn’t distinguish between stage direction and words...” Participant 10, 11.

Similarly, one participant (8) noted that the flexibility of the software might lead to unexpected / additional required modifications, for example cross references to other pages; “circle the correct answer” may be inappropriate if the size of font means that examination will most likely be answered using a word processor.


3. Managing the examination process using software

Participants were asked how they might use the RNIB E-formatting software in their context and how this process might be managed. This is particularly relevant because under current arrangements services may have one hour before the exam starts in which to use the software to prepare and print the examination paper. There were mixed views on this. Some participants were optimistic about the prospect believing that the software would serve to simplify a complex and difficult procedure:

“That would be brilliant.” It would also help the service because it would give them the flexibility producing materials on the day. Participant 3.

“The point of you going in on the morning of an exam, and then potentially in the hour [before] being able to access a program like this and provide the student with size 36 Arial bold wouldn't be a problem from that point of view. We have three [support teachers] [..] The benefits are fantastic. It wouldn't be a problem arranging to go to school on that morning. Liaising with the exams officer etc.” Participant 2

The same participant (2) felt that they had to go through this complex process anyway describing an example of how a paper had not arrived as requested, and frantic photocopying had taken place i.e. working in this way was typical. Even so, some participants may have been overly optimistic about the flexibility of the early opening procedures for GCSEs, drawing only upon their experience of working with SATS papers:

“When we did SATS papers – had to apply for early opening. [..] Problem you have, is if you make a mistake on the paper. We had a copy of the modified SATS papers – not big enough [font] for child. Now have software to scan documents and text but then had to type it all up. [..] If using this software could use this the day or hour before – haven’t got the worry of having to check if you have made any mistakes.” Participant 5.

Several other participants highlighted the pressure and stress involved in preparing papers in the short time before the exam suggesting that the introduction of technology which could go wrong may serve to heighten this stress:


“I hate using computers for exams! It adds to the stress level ... when you have to prepare something under stress it’s not very good.” Participant 7.

“I cannot begin to tell you the stress involved in the exam centre. If you put the disc in, and the student is waiting, and it comes up and the diagrams haven't been enlarged, the page breaks all over the place. [..] What if it goes wrong, what if it doesn't work, what if you... it's horrendous!” Participant 4.

Some nervousness about being at the mercy of equipment laptops not working, networks going down and that worries her a little bit. Do you need a backup plan of having a hard copy as well? Participant 3.

A number of participants who had extensive experience working within a resource base felt that it may be more of a challenge in mainstream schools without resource bases. Nevertheless, others argued that with adequate preparation and practice these problems could be overcome:

Sees no problem with the use of the software one hour before the exam. It could be done by the subject teacher or a teaching assistant from the service - felt that probably best managed by the service. The key thing is being aware of the process and having training in the use of the software. Important to have a practice CD. Participant 3

User interface is quite straightforward. You will need to have had a chance to practice with it beforehand. Participant 8.

Participant 3 described additional policies the service had in terms of ensuring exam provision is arranged: they write the examination concessions into annual reports; they have a form which is sent to school (SENCO) at key points during the year (which details the recommended options for exam access); they have a course/workshop at the beginning of each year for professionals working in mainstream schools with visually impaired pupils. This participant predicted that discussion about the use of such software could easily be fitted into this process.

Only one participant reported negative views about the potential of using the software. In part this was related to a belief that the practical reality of


using the software tool would prove to be very difficult, and was possibly linked to the participant’s reportedly negative experience of the exams modification procedure in the past. The participant also expressed a view that it was the exam boards’ responsibility to ensure that the modifications were made and that it was unfair that they should pass on this responsibility to individual schools and professionals. The position is succinctly summed up through the following quotation:

“If [using this software] is as simple as pressing buttons here, then it's simple for somebody to press buttons somewhere else and send it in the post. If it's not as simple as just pressing buttons, then it's going to cause chaos in schools.” Participant 4.

4. Tensions and contrasts: Classroom practice, examination options and the ‘real world’

A fourth theme which emerged from the interviews was an apparent contrast between normal classroom practice in relation to curriculum access and formats which are available in examinations. This theme can be illustrated by the following quotes which suggest that some students with visual impairments have their examinations presented in a format that does not mirror their classroom practice:

“It would be helpful if students could use software [in examinations] that they are familiar with in their everyday lessons.” Participant 6.

“The problem with the exam boards is they are doing us no favours if they produce material in A3 [..] it goes against what we as QTVIs are trying to encourage in schools.” Participant 3.

“It's pointless producing beautifully adapted papers and questions if when it comes to exams they've still got one of the layouts that’s given. [..] I don't think it allows them to demonstrate their potential. All their schoolwork, all the text in school, generally through their years is presented to them in the same format and then to end up with the exam which is different…” Participant 2.

Cobb (2008) identified a similar phenomenon in his research noting ‘modified large print papers produced by the current system do not


match the classroom experience of many pupils in mainstream schools’ (p.ii). The general point was also extended further in the interviews with some participants contrasting the educational experiences of visually impaired students with the challenges of information access they may face beyond school in the ‘real world’:

“When they leave school they’ve got to try and cope with the real world and find strategies of dealing with it – magnifiers, CCTV cameras or whatever. They need to deal with all sorts of print styles.” Participant 7.

“What we tend to do is get [student who prefers 40 point print] to achieve a balance because we know that when she goes out into [pause] the real world, not everything is going to be modified. So I suppose you’re doing her no favours by saying here’s everything modified. So you’ve got to use low vision aids and CCTVs as well.” Participant 3.

Given the tensions reported above, we consider briefly how professionals and services rationalise these in practice. For some there seemed to be an acceptance that examinations are ‘exceptions’ or ‘peculiarities’ that must be worked around, i.e. professionals must work within the system as best they can and, crucially, help students practise. As illustrated by the following quotes, creating practice exams and getting hold of practice exams serves as an important strategy:

“Boards are particularly reluctant to let go of practice papers – which is exactly what VI candidates need. The format is more a problem to them rather than the content. They need more practice not less but is much harder to get practice papers for VI rather than sighted candidates.” Participants 10 and 11.

“They would have them for practice papers so they would be familiar with it.” Participant 1.

“The students we have they all have very different requirements. [..] We tailor make... When we do mock exams we prepare them how the exams are going to come in which is not what they want but it's what they have to get used to using...” Participant 4.


Perhaps the most strategic approach was adopted by participant 7 who described providing curriculum material in either Arial 12, 18 or 24 point. The choices available were directly linked to the examination procedures. Assessments of students are carried out to establish which presentation appeared to give the optimum accuracy / speed / working distance balance. Students who were not able to access 24 point print were encouraged to use large print combined with a low vision aid, or alternatively Braille may be considered as the route to literacy.


5. Access to the curriculum: student skills and service policies

The previous discussion (in relation to theme 4) leads into an analysis of participants’ views of general policy on curriculum access and how this relates to access to examinations. We consider first policies and then discuss the related topic of student access skills (in particular the use of LVAs and computers).

PoliciesParticipants discussed their broad approaches to curriculum access for students with low vision. Sometimes these appeared to reflect service policies explicitly, while in some cases participants focused exclusively on their own practice. Views can be usefully categorised as either ‘a student-centred approach to curriculum material preparation’ or ‘an independence-skills approach to curriculum access’. The former was characterised by curriculum material being prepared specifically for individual young people. For example:

The service has no general policy, decisions are based ‘individual needs’ basis. Participant 6.

“The policy is, ‘if they need it in an enlarged style then they get it.’ ” Participant 4

Range of sizes prepared to students: standard 14 point - 48 point bold. Majority have around N24. Also described a student who had material presented to them in 24 but with 1.5 line spacing - i.e. very highly specifically designed for each student. Participant 2

For many, the quality of the bespoke materials provided for the students was a matter of professional pride (and to some extent a key function of their service):

“The [support teachers] are fantastic and they wouldn't let the students [have standard 12 point]. I can quite confidently say, hand on heart, that a student would not have to access this. [referring the 12 point text]” Participant 2.

“The students we have all have very different requirements. We produce for them to learn, week in week out, what they need. Which can be anything from size 14 up to 34... They


might want it on white paper or yellow paper, bold or not bold. Some children find one type of font easier to read than another. We tailor make.” Participant 4.

A smaller number of participants discussed broader policy decisions regarding access, that reflected perhaps a more pragmatic (and arguably a less student-centred) approach:

“If they can't use 18 to 24 we think of other things - CCTV or magnifiers.” Participant 7.

“At present we say you can have 18 or 24 point that’s it.. you’re going to have a reader if you want something different [..] It would be reasonable to set boundaries – I think RNIB has supported the idea that is 18 or 24. We have to accept this is not a perfect world as we cannot tailor make exam paper to each individual student.” Participant 10 and 11.

Participant 7 reflected that the policy of providing material in options which matched those available in examinations was as much about pragmatism as it was about philosophy. She noted that if the RNIB E-formatting software was available then students could have a broader choice of curriculum materials (in particular the range of font styles, noting that she thought Arial font could be problematic):

“If this [RNIB E-formatting software] was available we wouldn’t need to be so fussy and we could let them use the font I wanted.” Participant 7.

From the descriptions provided by participants there is evidence that a significant amount of professional time is currently devoted to the preparation of curriculum materials:

Huge amounts of team time resource is given up to preparing materials – “[some staff] seem to spend 99% of the time doing it.” Participant 2.

“Generally [the MLP paper] is very similar to what I produce for [student]… hours and hours of work for one person.” Participant 5.


Student accessFor those participants adopting ‘independence-skills’ approaches to curriculum access, a key strategy is to encourage students to use low vision aids (LVAs) and computers as a means of accessing print. Although the use of LVAs was raised in many interviews, they were not central to the discussions or the interview schedule. Most reference to LVAs was in passing, although on a number of occasions participants described how they used them in relatively negative terms. In this regard LVAs were often characterised as a ‘second best’ in comparison to specifically prepared printed material. This was argued from the point of view of the professionals and the students:

“But LVA's and CCTV's are visually more tiring which is why we often go to be modified large print.” Participant 3.

They don't have a child who uses a magnifier all the time - though they have had some who use a magnifier with normal print, or with large print. Participant 7 also tests reading speed when using a magnifier and CCTV. Interestingly although speeds matched (N18 print with magnifier versus very large print) the large print version still chosen by the child. Participant 7.

“Anything like this we would move heaven and earth to make sure it was a format they could read on an A4 piece of paper. So they can be in their class with paper the same as everybody else. [..] In other places in the county, where it is possibly not so easy as they haven't got the backup the same as they have here, children do choose their vision aids more than they do here.” Participant 4.

In line with the findings presented earlier report, some participants (e.g. 3, 7, 4, and 2) highlighted that many young people did not want to use LVAs because it made them look and feel different, e.g.:

“But when you get to this age you have also got to rely on young people using their low vision aids because it can be a bit of a problem.” Participant 3.

The interviews discuss the role of computers (and in particular onscreen access for exams) much more explicitly. The interview schedule asked participants directly about the use of computers and laptops in the classroom and discussion around onscreen access to examinations


made the assumption students access to information using computers. Unsurprisingly then, the use of computers as a method of curriculum (and exam) access was discussed in more detail. Most participants reported that laptops were relatively common in classroom practice:

“Students are very used to using laptops in their lessons and so would be familiar with them by GSCE time. Three-quarters of students used computers in lessons on a daily basis.” The [service] would provide laptops if the student was statemented, but if not, the supply of equipment such as laptops is the responsibility of the school. Participant 6

A lot of students use laptops all the time in classrooms. At the resource base it’s “pretty common”: all 8 have access to a laptop and 3/8 use a laptop all the time, the remaining five use laptops often. [..] Many use laptops anyway in the exam for word processing. Participant 7.

It’s becoming more common. “A lot of our large print users have their own laptop in school.” If students are using laptop frequently they will have curriculum material in e-format “This is happening… It’s the way we are pushing for this to go.” Participant 1.

In terms of barriers to greater use of computers, participant 4 felt that many students chose not to use laptops because they did not want to stand out as different in the classroom (i.e. similar to the argument made in relation to LVAs). Further, as illustrated by the following quote, a number of participants argued that it was important to consider the training needs of classroom teachers:

“Training of teachers is an issue. The mindset of teachers... we are trying to get the teachers to take responsibility for producing materials in appropriate font size or to provide e-resource for student to modify themselves.” Participant 1.

Further, as reported by participants 5, developing the skills of students themselves is needed, particularly if computers are to be used in exams:

[Specific child] currently uses Arial 30 point. She is reliant on laptop for writing. [..] She uses a computer regularly but would need to practice accessing in this format. It would be


beneficial as it would cut down on all the modification. Participant 5

Nevertheless, in the interviews participants made links between the use of computers in the classroom and the potential of using them in an examination:

“We have secondary-age students using laptops, a couple of them for 100% of the time. For them to be able to have an exam paper on a computer and to be able to answer it on a computer…” Participant 2.

“Could be very powerful if [that way of working] was built in to all their teaching materials not just for their exam at the end.” Participant 8.

6. The potential of onscreen examination presentation

All participants were shown on screen versions of an original examination paper and its MLP equivalent on screen. This involved presenting the two PDF files on screen, using Adobe Acrobat reader. Presenting the files in this way provided very flexibility in the presentation options – the researcher was able to zoom in and out (enlarging the presented font) and scroll up and down. Importantly, it was emphasised that this method of presentation was being used to demonstrate the principle of onscreen examinations and to encourage reflection. Participants were also shown the RNIB E-formatting software to demonstrate an alternative version of onscreen presentation.

In line with the interview schedule, the discussion around the potential of onscreen presentation of exams focused upon: an initial general reaction to the idea, how students used computers in the day-to-day classroom (reported

earlier), and potential advantages and disadvantages of using computers in

examinations in this way.

Many interviews generated broadly positive responses to the notion of onscreen presentation of examinations. While there were caveats and caution about the implementation of such an idea, many welcomed the general principle. For example, the following two quotes capture broad


positive views about student familiarity with computers and the idea of ‘mainstreaming’ assessment (‘everyone uses computers’):

“Brilliant. Absolutely brilliant. Yep, I think it's the way to go. [..] We have secondary-age students using laptops, a couple of them for 100% of the time. For them to be able to have an exam paper on a computer and to be able to answer it on a computer…” Participant 2.

“It’s a ‘move forward’ for visually impaired [people] – everyone uses computers. [..] That’s what [student] will have to do … whatever job he does in the future, it will all be computer-based.” Participant 5.

Caution was often constructive, particularly focussing upon training, the importance of developing students’ access technology skills, and the development of technology that works:

“For some students I can see that an [onscreen] version would be an option. But you need to train students.” Participant 4.

“Touch typing becomes far more important.” Participant 3.

“It does depend on how good the technology is…the answering onscreen is great if the technology works.” Participant 10 and 11.

Technical development that would enable students to answer on screen was highlighted by most participants, and to this extent the potential of onscreen technology depended upon interactive and accessible software. As illustrated by the following quote, a number reported that this offered hidden advantages of providing students access to ‘meta-level’ information which would help students navigate (e.g. move from question to question, answer boxes could be linked to questions and available marks).

If it was based upon form-filling this has a lot of possibilities e.g. the use of Tab, space for checkboxes. There also may be other hidden advantages if this was done well – e.g. using navigation tools to step from question to question. If they could answer on the screen the students “would take it in their stride.” Participant 7.


Table 20 presents a summary of advantages and disadvantages identified by participants. Many of the disadvantages identified are reflected in the caution described above. To this extent these cautionary points are a reminder of theme 1: ‘the devil is in the detail’. Many of the potential advantages of onscreen presentations of examination have little meaning if care is not taken in preparing the material appropriately, i.e. the process of modification.

Table 20. Onscreen examinations – potential advantages and disadvantages. Answers grouped by authorsAdvantages (including positives)Control of the font.Control of the size of font.Control of the colour combinations.Familiarity, e.g. links to classroom practice; use of existing access technology; students can create a format which they use to.Independence.Quicker preparation time for the teacher.Assessment reflects the ‘real world’ in which the student will eventually have to operate.Less stress and anxiety.This is the likely direction of future developments for all students (e.g. moving towards the ‘paperless school’).

Disadvantages (including concerns and challenges)Navigation, e.g. scrolling, moving from question to question.Unfamiliarity, e.g. with unusual specific examination software, unusual students who are not familiar with computers.Training would be required (students and staff).Non-text will need consideration, e.g. diagrams.Specific topic requirements, e.g. specialist characters, drawings and graphs.Technical breakdown and lost work.Anxiety about using computers (particularly staff).Visual fatigue.

7. Contrasting views of a ‘perfect world’


At the end of the interview participants were asked “what type of examination provision for visually impaired pupils would you like to see in the future?” Importantly, the interviewer predicated the question by reviewing the different formats that being discussed (MLP, RNIB E- formatting software and onscreen presentation). It is important to recognise that the interviews were not an attempt to ‘survey’ opinion, but rather a conversation in which ideas were generated and opinions and views were formed. Therefore, participants’ responses to this question captured their emerging views at the end of the interview drawing on the other issues that had been discussed.

Table 21 presents a summary of how different interviews concluded what would be their ‘perfect world’. For each we have provided quotes to illustrate some of the key points that were made and attempt to categorise these into different ‘ideals’: ‘Existing system (done well)’ in which print copies of examinations are

delivered to the exam centre. ‘Formatting tool’ in which a bespoke printed version could be

produced in the exam centre (similar to the RNIB E-formatting software).

‘E-version’ in which an electronic version of the examination is presented.

This is, of course, a simplified categorisation. Nevertheless, it offers some notion of the emerging opinions of the participants and their general views of how the examination process could potentially function in the future. Most participants considered that an improved examination system (a ‘perfect world’) could incorporate a new way of working which involves the use of either formatting tools (such as a developed version of the RNIB E-formatting software) or computer-based examinations. No participants dismissed either idea.


Table 21. ‘In a perfect world’: Views about the future of examination modification which emerged from nine interviews with professionals.Interview Key points and example quote(s) ‘Ideal

category’A Access to e-version of examination which can

be provided to the student on a computer in the format they require.

“The easiest way for student to do the exam. [..] For me, I go with the laptop option [..] I would be using the technology.”

E-version

B E-versions of examinations.

“I think electronically presented and electronically completed. [..] [Why not say], ‘right, you manipulate into the format that you want rather than I give you something that I think you should use.’ ”

E-version

C Having a tool to prepare examination in the students preferred format.

“You would like to order your exam paper from the board and say ‘right, I want point 24 in Verdana, bold, double spaced by next Friday!’ [Laughs] [But using something like the RNIB E-formatting], to be able to do it for yourself would be great. That is pretty near the perfect world.”

Formatting tool


Interview Key points and example quote(s) ‘Ideal category’

D The current system improved and working well. Examinations should b delivered in the appropriate format.

“I would like an exam paper to come in prepared in a tailor-made way for an individual child. [..] That would be my starting point to the students I work with here. The paper, hardcopy, suitably prepared in terms of size, colour, all rest of it. And I would like it to be properly quality controlled so we are confident it.”

Some students may prefer an e-version of examination.

“For some students I can see that an [onscreen] version would be an option. But you need to train students.”

Existing system done well.

Possibility of E-version

E Having a tool to prepare examination in the students preferred format.

“Papers to be supplied in e-format (on disc/memory stick) – so if [we] need to adapt them we don’t have to retype them. [..] If we were supplied with modified paper but had RNIB software so we could enlarge it again and have the ability to print it off.”

Formatting tool

F Greater clarity, consistency and quality in the examination modifications procedure and more flexibility about breaks and opening times.

Existing system done well.

Unclear.


More variety of examination layouts:

“Students would have more choice of format, layout and paper size.”


Interview Key points and example quote(s) ‘Ideal category’

G Personal preference for having a tool to prepare examination in the students preferred printed format, but a realisation that E-versions of examinations is probably what the students will want.

“Having printed bespoke paper would probably be my preferred option. But I doubt if the children would feel like that... they seem to rely so much more on technology. [..] In a perfect world and have all the children learning typing skills when they were little instead of finger pecking all the way through.”

Formatting tool and E-version

H The ideal world would be ‘modify on an individual basis’. RNIB software takes you a long way down that route (with additional features). Accessible e-versions of exams has possibilities but there are many unknowns.

Formatting tool and possibility of E-version

I Having a tool to prepare examination in the students preferred printed format and onscreen.

Still a role for paper copy – in a perfect world would have pack which has an e-version which they could use on the computer but also includes a paper version.

Formatting tool and E-version


5. Group Interviews: Students who are non-print readers

5.1 IntroductionAlthough the main remit of the project concerns exam access for students with low vision who use print as their primary literacy medium, an opportunity was provided to interview Years 10 and 11 students who were not print readers (i.e. they use Braille or technology to access literacy). This is valuable because it explores the possibility of a single accessible examination solution which could include both print and non-print readers. Two group interviews discussed: participants’ experiences of taking exams to date and the preparation involved; the potential use of screen reading software in examinations; and a reflection upon what the perfect examination scenario might be.

5.2 MethodParticipants

Seven participants took part in the semi-structured group interview. All participants were taking part in a study weekend at Worcester New College during their Easter holiday in 2010. All were blind and did not read print, either accessing literacy through Braille or technology (or in most cases both), and all described receiving 100% extra time allowances in examinations. A breakdown of the participants is presented in Table 22.


Table 22. Description of the participants taking part in the interview (N=7)Partic-ipant

School type Primary literacy medium

Use of ICT

A Mainstream school

Does not use Braille or print. Main access to literacy is through a laptop with screenreader.

Mainly computer with screenreader*

B Mainstream school with resource base

Grade 2 Braille Braille Notetaker and Perkins Brailler

C Mainstream school with resource base

Grade 2 Braille Mainly through computer with screenreader

D Mainstream school with resource base

Grade 2 Braille Mainly computer with screenreader

E Mainstream school with vi room

Grade 2 Braille Braille for Maths and Science. Computer with screenreader for other subjects

F Mainstream school with enhanced resource provision but not a resource base

Grade 2 Braille Uses laptop with Jaws for most lessons. Music use ‘Braille notetaker’

G Mainstream with resource base

Grade 2 Braille Braillenote for all subjects except for Maths and Science (Perkins Brailler used). Does not use laptop in class

* Note: a ‘screenreader’ is access technology that attempts to re-present information presented on the computer screen in an alternative format (usually speech).


As shown, six of the participants were Braille readers and were expecting to be using Braille-based examination papers in their forthcoming GCSE examinations. One participant (A) did not read Braille or print and used technology for accessing examinations (i.e. mainly a laptop with a screen reader). All participants attended mainstream schools, a number of which were described as having enhanced resource provision.

Materials and procedure

The seven participants took part in the semi-structured interview in two separate sessions (five in session 1; two in session 2). The interviews were recorded and a transcript made to aid analysis. Given the small numbers of participants in each group the results from both sessions are discussed together with individual quotes provided to support the key themes raised in the interviews.

The interview schedule followed the following structure:1. Introductions2. Sharing experiences of taking exams to date3. Preparation of exams4. Use of a screenreader to access exams (pros and cons)5. Reflections on screenreading activity6. Perfect exam scenario.

The screenreading activity involved participants working through examination style questions presented on the computer in Microsoft Word.

Each interview took approximately one hour. At the beginning of the session the broad purposes of the research was explained. Participants were then asked to sign a consent form declaring that they were willing to take part in the research, but it was noted that they could withdraw at any time. Participants were each given a £10 gift voucher.


5.3 ResultsSharing experiences of taking exams to date

Participants were asked to share their experiences of taking public exams to date, and provide brief information about: how exams were accessed, the nature of additional support provided, how the process has worked from their point of view: evaluative

comments requested

A summary of their responses is presented in Table 23.

Table 23. Participants experiences of taking exams

Part. How exams are accessed

Additional Support Other comments

A For year 6 SATS paper participant typed answers using a computer. Uncertain how exams will be accessed in the future.

Reader (individual subjects not specified)

Reported that the arrangements have worked well to date and feels ‘well supported’

B Braille with a reader for certain subjects.

Perkins Brailler used for Maths and Science. Braille notetaker used for RE and English

Reader for Maths (layout of diagrams) and English (to help with referencing back to text)

Reported that felt well supported.

C Braille with a reader for Maths and Science

Reader for Maths (reads the equations) and Science (if running out of time)

Reported that felt generally well supported. Particular problems had arisen in accessing bar charts in a past GCSE Maths paper.

D Braille with a reader for Maths and Science

Reader for Science and Maths. Worked through Maths diagrams before the exam starts.

Reported that was generally well supported.

E Braille with a reader for Maths and Science.

Reader for Maths and Science (to talk through diagrams). A laptop is

Reported that was well supported.


used for Science. For multiple choice questions in science a scribe completes the form. Maths is recorded on a Perkins Brailler.

F Braille with a reader for Maths.

English - separate hard copy booklets are produced in Braille (questions in one booklet and text in a second booklet). Answers are written on either a laptop or Braillenote

French- listens to tape and writes answers using a laptop. Can ask ‘reader’ to pause the tape.

Reported that was well supported

G Braille with a reader for Maths

Types answers on a Perkins. Diagrams are in a separate booklet.English – all textbooks in Braille (hard copy). Writes answers on Braille notetaker.

No evaluative comments provided

As shown in Table 5.2 all participants reported that they had access to a reader for at least one subject, the most common being Maths and Science. Whereas participant A was not able to specify which exams a reader was available for, other participants were very precise about in which particular parts of the exam support was needed. As an example, participants B and E reported that specific support was required in Maths to help in understanding the layout of diagrams. Similarly Participant C reported that support was required to read equations and tables noting that in a past GCSE Maths paper she had experienced problems in accessing bar charts. One participant (B) reported that particular support was required with ‘referencing back to the text’ in English and Welsh exams. In elaborating this point she explained:

‘In English you get a passage and sometimes with the questions I get confused as you have to refer back to the paragraphs… with Welsh exam you get a long passage with just a few questions.’ (Participant B)


A similar point was made by participant G in relation to the use of English exams in Braille. She reported that she found it ‘difficult to flick from text to question book’ and during her last English literature exam ran out of time:

‘I knew what I wanted to do but didn’t have time to finish the exam’. (Participant G)

As an alternative to a reader she wondered instead whether a Braillenote could be used to allow her to ‘speed read’ through the text.

A number of participants described particular strategies they used with the reader to access exams. As an example participant C reported that she would only need the services of a reader for Science if she was running out of time. A similar point was made by participant G, who noted that in Maths she begins independently with the reader only helping her if she is behind schedule:

‘I try to read as much as possible – sometimes if I’m behind they will start reading it to me’. (Participant G)

One participant spoke of particular ‘advance’ strategies she used to prepare for interpreting the diagrams in Science and Maths exams:

‘I usually go through the diagrams before the exams start…. my exam board does not modify diagrams. Say there’s a motor in my physics exam they put it in 3-D rather than 2-D’. (Participant D).

One participant reported that when she had a test ‘out of the blue’ her reader would write down the answer for her and for multiple choice questions in science a ‘scribe’ will complete the form. In comparison another participant was keen to emphasise her independence in exams reporting that:

‘I have never required anybody to write down answers’. (Participant E).

The majority of participants reported that they took their exams in a separate room from their peer group, usually the resource base.


Preparation of exams

Participants were asked to provide information on who would be involved in the preparation of exams within their schools. Although none were directly involved in the process, a number of participants had some knowledge of the various stages involved in obtaining exam papers in an appropriate format including the role of the School exams officer. This is illustrated through the following quotes:

‘If it is a GCSE paper the school orders the paper through the exam officer. If it is a past paper my teaching / support assistance orders them’. (Participant D)

‘For internal exams it is up to class teachers to give in paper to the LSA to adapt in time. For external exams I tell my assistant what exams I have coming up who then liaises with the school exams officer who orders from the board’. (Participant C)

‘My helpers send off paper to exam board. If it is a mock exam it will be produced in the school…and will be laid out in the same format as modified papers’. (Participant F)

Other participants spoke in more general terms about the process and either made no reference to the exams officer or did not know about this role:

‘The teachers get the paper and give it to the support worker who embosses it into Braille and make diagrams themselves. I am not involved at all in this process and am not aware of an exams officer in the school’. (Participant E)

‘The school orders the Braille paper through the exam board. My support worker opens the paper on the day but cannot view them in advance’ For internal exams the support worker ‘embosses them off for me’. (Participant G)

Other participants were less sure about the procedures involved:

‘The exam preparation is done in advance and generally works well - but I am not sure who is involved’. (Participant B)


‘I don’t know how it’s done in my school’. (Participant A)

Participants were also asked to provide information about who they would have a conversation with in their school about access to an exam paper (eg whether it should be accessed through Braille or a screenreader). One participant highlighted the role of the QTVI in this process:

‘My QTVI would be important…. we’ve done mocks and then made a decision about the real exam. I did a mock maths and didn’t finish it so they said that for the maths exams we are going to read to you’. (Participant E)

The majority of participants reported that either they did not have a conversation with their teachers at all about access or they did not consider there to be need to have such conversations with their teachers. This latter point is illustrated by the following quotes from two participants who reported that as Braille was their medium of instruction this would be carried through to the exams:

‘They know me by now so I just receive it in Braille’. (Participant F)

‘I would only use Braille as I’m not very good at using screenreaders’. (Participant C)

Although access was not discussed, two of the participants emphasised the opportunities they were provided with to practise papers and ensure familiarity with the format of exams. This was considered to be a particular issue with understanding diagrams in Maths and Science as illustrated by the following quotes:

‘It can sometimes be an issue in understanding diagrams – or when asked to draw something onto German film….you’re spending most of your time looking at the diagrams all the time and you’re not getting the time to look back over…not having the time to go back over the questions you can do’. (Participant F)

‘Also they are getting quite strict in what you can and can’t do… for a diagram they might ask you do something but say you have to do it in a certain way. You can’t cut shapes out to show other people. My helper cannot cut out the diagram


for me to show me a 3-D version of a net. Instead of asking me to draw sometimes I am asked to describe what I think it looks like. Sometimes I get 3-D shapes but not often.’ (Participant G)

Use of a screenreader to access exams

Participants were asked to consider the potential advantages and disadvantages of using a screenreader to access exams. A summary of the key points is presented in Table 24.

Table 24. Summary of potential advantages and disadvantages of accessing exam through a screen readerAdvantagesIndependence – provides increased independence in comparison with a readerConcentration – may require less concentration than reading the same text in BrailleSpeed – could be quicker than reading the same text in BrailleResources – Frees up support worker/reader to do other tasksNavigation – Could be helpful for scrolling from line to line Time – could save time if could type in answer directly to document as it won’t have to flick from one hardcopy book to the nextReference – Can mark your place to make it easier to cut and paste (e.g. if you wanted to insert a quote into your answer)Familiarity – more similar to way of working if currently use screenreaders

DisadvantagesTechnical problems – e.g. could lose the file if in electronic formatAccuracy (semantics) – the screen reader might not read the meaning of words accurately (example given of accessing a foreign language through a screenreader)Accuracy (technical) – the screenreader may not read technical diagrams and graphs accuratelyTime – could take more time to listen to the paper all the way through. Issue raised about how one might use ‘skim listening’ in the same way as ‘skim reading’ to scan the text. Some make write Braille faster than they type.Personalised – Screenreader would need to be set up for individual requirements (speed, voice etc).


Reflections on screenreading activity

Participants were then invited to complete an onscreen exam accessed through a screenreader (Dolphin Supernova). The material was based upon the question styles developed for the student performance tasks (in a previous section). The material were not designed in great detail, rather an attempt was made to make the questions accessible to a screenreader (e.g. removal of pictures) and to use answering styles suitable for that an electronic format (e.g. for multiple choice questions, replacing ‘circle the correct answer’ with ‘type the letter of the correct statement here’). The activity was purposefully informal because of the exploratory nature of the work, and some of the participants were used to using other screenreading packages. Following this task, participations were asked to talk through their experiences. A summary of the key points raised in the discussion is presented below.

Orientation to technology

Two participants emphasised that it would be important to ensure that the technology is working prior to the exam starting. It was suggested by one participant that candidates are provided with a period of ‘orientation time’ before the exam to ensure the technology is in good working order (10 minutes was proposed by the participant as a minimum). A similar point was made by one participant who reported that it would be helpful to provide time in advance to become familiar with the speed and voice and alter this if necessary to suit personal preferences:

‘You need orientation time prior to the exam starting – to set up speech programme. You also need to be familiar with the particular type of software’. (Participant B)

Accuracy (technical)

All participants reported that tables were found to be the main limitations of using a screenreader to access the exam. This is illustrated by the following quotes:

‘The tables really confused me. I found it difficult to make sense of. I managed to answer question but it was difficult – I had to keep going up and down to find information’. (Participant F)


‘I didn’t do anything on the tables….it just babbled at me’. (Participant B)

‘I liked it. It is less tiring and quicker than Braille – except for the table. It didn’t give all the information you require re. percentages’. (Participant C)

Three participants reported that they would like to have referred to hard copy Braille versions of all tables/diagrams if they were to take an online public exam.

‘I prefer to have a hard copy in front of me with tables / diagrams’. (Participant B)

Different possibilities were discussed including the option of having the exam paper presented as a Braille display on a laptop or Braillenote (and whether a hard copy of the tables in Braille would be required if available on Braillenote).

Personalised settings

Four participants reported that the settings of the screenreader provided them with too much information (which could have been modified with different settings). As examples it was reported by one participant that she did not require information about commas and full stops but that it was useful to know about brackets around the mark indicators at the end of questions (e.g. [10]). Similarly, another participant reported that she was ‘confused’ by brackets around certain types of information, and she would have welcomed modification so she did not receive all the punctuation:

‘I have [screen reader setup] so it doesn’t read any punctuation – this becomes really annoying’. (Participant C)

Confidence with technology and Training in appropriate skills

Two participants reported that candidates could be at a disadvantage as they did not feel confident in using screenreaders. As with discussions in relation to low vision readers requiring training in the use of access tools (computers and LVAs), these participants were anxious about the access technology skills they would require if examinations were presented electronically.


One participant reported that different types of skills are required for using a screenreader in comparison with other ICT tasks (e.g. web browsing). This was supported by another participant who reported that she was used to using Braille so may find it difficult to adjust to an onscreen version for exams.

Subject specific use

Two participants reported that screenreaders may be more helpful in some subjects than others in accessing exams.

‘It depends on which subject. For maths/science I would use a Perkins. For English I would prefer everything in Braille’. (Participant C)

In comparison, another participant reported that this type of approach could be helpful in English based subjects where there is a lot of text to scroll through, but queried whether a screenreader would be helpful (or allowed) for a comprehension test in English language.

Perfect exam scenario

As a final question, participants were asked to reflect on what they would consider to be a perfect exam scenario for them. A key theme raised was in relation to the range of access technology students wanted. It was staggering the range of combinations that the students said they would prefer in a ‘perfect scenario’: different methods of reading the examination (screen reader, hardcopy Braille, refreshable Braille), and different methods of writing answers (computer, Braille), and all the combinations of these.

‘I would rather do it written than spoken – I can get across what I want to say better.’ (Participant C)

‘I would read in Braille and record using a Braille notetaker’. (Participant F)

‘Would record on Braillenote and read as a hardcopy. Technology - no role to play in this scenario’. (Participant E)

‘When I use the computer I have speech on and my Braille display on as well – so I can listen back to what I have


written – if I’ve got spelling mistakes I can correct it.’ (Participant G)

‘Being read to and I write it down – worked well so far’. (Participant A)

‘Refreshable Braille as an option? [But it’s] really temperamental - don’t want to depend on it’. (Participant C)

‘Available on Braillenote – I’m faster at Braille than typing. Papers to be available on Braillenote – and then could answer on the Braillenote’ (Participant F)

‘Screenreader on a word document where you can input text directly onto document’ (Participant C)

‘If I was using screenreader would still like to write answers in Braille’ (Participant E)

Unsurprisingly given the range of materials participants anticipated using (including speech software), participants discussed the environment in which they would want to take the examination. They particularly highlighted that they needed a quiet room, probably with no other candidates present:

‘Would prefer to be in a place where there is no background noise’ (Participant D)

‘I like what I do now. In my VI room which is isolated and quiet. Can be distracting if support worker is Brailling while I’m doing my exams. I’d like a reader with me if I need them – just in case’. (Participant E)

‘On my own – I can think better.’ (Participant F)

‘Environment would include a lot of desk space’ (Participant G)


5.4 DiscussionThe interviews with these students who do not read print proved very interesting. For all except one participant, Braille was a key route to literacy. Nevertheless, all recognised the role that screenreading / computer technology played in their life: all used a screenreader and all appeared to anticipate using this technology in some way in their examinations. Perhaps as a result of this, all engaged with interest at the possibility of screenreading technology having a greater role in examinations. Some key issues raised in the interviews were: Some topic/curriculum areas appear to lend themselves to being

‘screen read’ more than others. Particular presentations are problematic for some people when

accessed electronically, e.g. tables were identified by many participants. It may be that training and practice would help with this, or that hardcopy Braille in a separate booklet would be a preferable alternative or addition for some. Refreshable Braille was also highlighted as an option by some.

Familiarity with technology was identified as important – plus the requirement for additional support for certain types of questions (e.g. those involving tables and/or figures). Training needs were identified as important.

This has implications for the people who support – skill in the use of the technology was also felt to be required by teachers.

Child voice would be important in process of making decisions – it was clear that the participants had very different ideas to one another about the combinations of technology that would suit them.

Selective use of screenreaders may be a way forward for exam access. Participants identified many potential advantages from their point of view, and relatively few (non-technical) disadvantages. In principle, some exams which are available electronically for students with low vision could also be suitable for students who do not access print, and vice versa (particularly if answers can be given electronically too). However, the very different modifications required for some examinations may mean that this is often not possible.


6. Technical analysis of the RNIB E-formatting software

The invitation to tender described the RNIB E-formatting software as being designed to allow “the teacher/school to modify the exam paper in terms of font size and style, text and background colour, and the line width inside diagrams. Once it has been modified to meet the individual requirements of the candidate, the paper can be printed for them to have as a paper copy or it may be read on-screen.” In this section of the report we review how the current version of the software operates. We focus our discussion upon two stages of the software's use, as well as a short discussion about file structure:1)Modifying the examination paper.2) Using the software to format the examination paper and quality of the Output.3) ‘Accessible’ files

In the proposal to carry out this work, the term ‘proof of concept’ was used. This term was used because the team were commissioned to establish whether the concept of the RNIB E-formatting software could, in principle, usefully improve students’ access to examination papers. In other sections of this report we report upon interviews with professionals in which we explored the broad principles of using a tool of this type. The interviews offered evidence that software of this kind would be welcomed by many. In this and the next section we argue that the development of similar software could be part of a broader strategy of working towards computer-based examinations. How the software should be developed is dependent upon these broad strategic directions.

6.1 Modifying the examination paper

The RNIB E-formatting software as used by a classroom teacher would be presented on a CD-ROM. The teacher would run the software on the CD-ROM, choose an XML ‘exam’ file, select key formatting features (such as font style and size), and then generate the printable version of the examination paper. Prior to the teacher receiving the software in this format, a lot of preparation must go into the creation of the XML file. Importantly, this preparation must incorporate any modifications which are required, e.g. the removal or simplification of diagrams, the


implementation of standard formats such as capitalisation, and the modification of language. The guidelines for this modification process are presented within the JCQ Best Practice Guidance (JCQ, 2008).

The modifications made in the equivalent MLP and Software examination papers were different. It appears that the MLP version of the examination paper was passed onto RNIB and then further changes were made during the creation of the XML file. In some cases these differences were quite marked (e.g. the exclusion of pictures). Table 25 presents a comparison of the modifications made for one example examination paper (the same paper which was used in the professional interviews). Analysis of other examination papers reveals a similar difference.

Table 25. A summary of differences in modifications made in the MLP and software versions of the same examination paper (OCR, Twenty First Century Science, Biology A, A221/02)Question Modification1(b) Change of the position of the diagram key.2 Removal of pictures and speech bubbles.3(b), 5(e), 8 Matching task changed from connecting two boxes to

choosing a statement from each of the two lists.5 Graph changed, e.g. different arrowheads, use of the key.5(c) Vertical list of options (compared with horizontal).10, 10(c) Slightly different schema diagram, e.g. different

arrowheads.General Positioning of tick boxes in multiple-choice questions (left

aligned in software, right aligned in MLP).General The use of CAPITALISATION (software) and

CAPITALISATION underlined (MLP).General The use of matrix in some tables.General Available marks following the question (software) and right

justified (MLP).

It is unclear why these different modifications were made. Some of the changes related to diagrams would have been necessary because the software required some vector diagrams in a particular format to enable scaling. Changes involving the removal of diagrams may also be linked to this. Other general changes (such as multiple-choice and capitalisation) may have just been alternative interpretations of the JCQ guidelines.


A positive consequence of this process was that all of the exams produced using the RNIB E-formatting software had a similar ‘look and feel’. Part of the reason for this was that the RNIB software used to produce the XML files did so in a standard way. Part of the reason also is that a single team was involved in the process of modifying all the examination papers.

6.2 Using the software to format the examination paper and quality of the output.

The software was demonstrated to participants relatively quickly. Nevertheless they got an idea of the principle of the software and the basic process and quality of the output (and this was the main focus of the interviews with the professionals).

In terms of the process of using the software, those who did comment expressed that it appeared to be relatively straightforward. Participants were asked directly about whether they thought they would be able to use the software in the context of preparing a student for an examination. Many participants felt this was possible, while others were more hesitant and were particularly nervous about working under pressure one hour before the examination (within the early opening regulations). It is useful to consider the task of producing a quality large print paper using the software within the time limit of one hour. This period of one hour must also include time taken for printing. Some long papers which are printed in a large font may take a considerable time. Given this, it may be safer to think that the task of opening the software and creating a file ready for printing should take a maximum of 30 minutes.

The simple interface has the potential for this. However, based on trials carried out by the first author, this is problematic (both in terms of time taken and getting the material to look as required) because of the clumsy process of inserting page breaks and modifying diagrams. The issue of page breaks is perhaps best illustrated by considering the creation of the materials for tasks used in the ‘case examples of students who prefer very large hardcopy print’ (greater than 24 point). The material needed for the participants taking part in those case examples included a variety


of styles font in sizes 24, 28, 36 and 40 point. While the material was relatively simple in structure (only three multiple choice questions), the creation of the materials effectively gave an opportunity to experience how easy the RNIB E-formatting software was to operate in extreme situations. In practice, it proved very difficult to format the material exactly as wanted and in the end it was prepared using Microsoft Word. The key problem was controlling the page breaks as we wanted the materials to have one question per page. At relatively low point sizes this proved impossible for us using the RNIB E-formatting software. Unsurprisingly, the direct interaction with the material using Microsoft Word was easier because we could make use of subtle techniques such as margin width changes and variable line spacing, as well as the line break functions offered. As a result we were able to get the material to look as required (even at large font sizes).

If these problems could be addressed then the issue of time would probably be solved. However, providing an interface to control these functions would be difficult within the current software model which does not allow the user to directly interact with the document (the print ready file is presented as ‘read only’ within Internet Explorer). In fact, the only way of finding out where page breaks will appear is using the print preview function within Internet Explorer.

The interface for modifying the line widths of the vector diagrams is technically impressive and complex. However, it may be too difficult for the software to automatically render diagrams with the precision that users will require (the result being a large choice of possible diagrams, but none of them being ‘quite right’). It may be more productive to create a range of diagrams and simply give the user a choice. In practice, it may be that users of the software print out diagrams and then make additional modifications (including enlargements) to the hard copy. The process of photocopy-enlarging part of MLP examinations appears to be relatively common practice within the current procedure.

In terms of the quality of the output, participants were largely impressed with the quality of the printed papers. Unsurprisingly, they particularly liked the control over font style and size. Nevertheless some features were identified as possible useful additions. Additional features raised in the interviews were: Control over line spacing (3 participants) Control over page breaks (5 participants) Control over diagrams (size, positioning and fonts) (6 participants) Landscaping diagrams (1 participant)


As already described, several other comments were made in relation to modification issues (e.g. arrowheads on diagrams, quality of graphs). One other point raised was related to the colours available for onscreen presentation. The current choices are all saturated and bright colours and it would be useful to have options for a greater range of colours, particularly ones which are off-white shades for reduced glare.

Other points noted by the first author when using the software for the creation of materials was that it would be useful to be able to control the line format in tables.

6.3 ‘Accessible’ filesAs already described, an XML file structure underpins the RNIB E-formatting software. It is some of the features of XML which enables the user to be able to finely control the format of the output files. Also embedded within the XML files created for the RNIB E-formatting software is information about the structure of the examination paper. This ‘meta-data’ is similar to a style sheet associated with a word processor document (for example produced by Microsoft Word). Therefore meta-data about headings, questions, sub-questions and no doubt other structural information is (or could be) contained within the XML file.

Although beyond the remit of this report and the technical knowledge of the authors, this meta-data could be used further if the software were developed. The development of interactive onscreen examinations would require an interface which enables users to be able to navigate the examination paper with ease (an issue raised in the interviews with professionals and students). For example, the user might want to move from question to question with keystrokes, to be able to clarify which answer box was associated with which question, and to be able to check how many marks were allocated to a given question. Importantly, XML is not the only format which contains this meta-data. Future work with examination boards would usefully involve discussion about the importance and potential of this meta-data for visually impaired students. This might be in the relative short-term if PDF versions of examinations were to be offered to visually impaired students as in Scotland (by adding form fields and potentially ‘tagging’ PDF documents to make them more accessible and navigable). Either way, it is very likely to be


relevant in the longer term as examination boards develop onscreen and interactive examinations.


7. DiscussionIn this final section of the report we present a summary of the key findings before considering what this reveals about access to examinations by visually impaired pupils in England and Wales. This leads to a short discussion in which we consider how the current examination modification and formatting procedures for these pupils could be improved. The recommendations include reference to the further development of software for the preparation of hard copy and onscreen versions of examinations. In the final section a broad ‘functional specification’ for this potential software is presented.

7.1 Summary of findings

Below we present the findings from this phase of the research in relation to the two research questions.

1. How do the Modified Large Print (MLP) versions of papers produced by RNIB E-formatting software compare to traditional MLP papers? (both process and outcome)

a. Findings from a trial involving 21 students with low vision suggest that examinations that are formatted using the RNIB E-formatting software neither significantly advantages or disadvantages student performance compared with existing MLP approaches. Familiarity with examination format appears to be a more important variable.

b. Professionals interviewed were broadly positive about the RNIB E-formatting software: They were positive about having a greater range of options for

examination formats which the software could offer. They particularly identified control over page breaks and formatting

of pictures/diagrams as key areas for further development. There were mixed feelings about how the process of formatting

and production could be incorporated into the short one hour period before the examination. Some had concerns about technical problems and working under time pressures.

c. There is evidence that appropriate and consistent modification of examinations is of key concern to professionals irrespective of the method of delivery. The process of producing the software versions of the examination papers (from four different examination boards)


appears to have an important associated benefit, namely standardising of the modification process and the general format of the examination papers.

d. Case examples of five students who preferred very large print (greater than 24 point) demonstrated that they appear to benefit from hard copy large print formats which are different from those available through existing MLP approaches. An estimated 9% of GCSE-aged students with a visual impairment in England and Wales use font sizes greater than 24 point.


a. When reflecting on the ‘perfect scenario’, many professionals reported that students should have access to examinations which had been modified appropriately and could then be formatted in a flexible way to meet individual needs. The use of technology was seen as an important part of the formatting solution by all participants (teachers, students with low vision and non-print readers). This suggests a distinction between appropriate modification of a paper ‘at source’ and the formatting options that could potentially be provided by the software.

b. Access to on-screen interactive examinations was seen by both students and professionals as offering a range of opportunities for improving access. Nevertheless, further work is required to enable these opportunities to be developed further. Key challenges identified through discussion with students (blind and low vision) as well as professionals included: technical work is required to ensure computer-based material is

fully accessible; appropriate examination modification is carried out ‘at source’; training of students in the use of assistive technology to access

exams; training of staff to teach students how to use this assistive

technology for exams access.

c. Future development needs to be flexible enough to offer a range of on-screen formats (e.g. a range of fonts, sizes, colours, and zoom facilities). They also need to include options for hard copy versions of


questions and/or supporting information such as diagrams and tables to be produced in a variety of formats.

7. 2 Classroom practice versus public examination systemIn formulating project recommendations, the findings listed above can be considered within a broader national context with a particular focus on: classroom practice and the public examinations system.

Classroom Practice

In considering classroom practice, a commonly expressed view amongst those professionals interviewed was that ‘good’ service provision should provide bespoke hardcopy materials in order to meet the individual needs of visually impaired pupils. These interviews took place in the context of their general dissatisfaction with currently available exam formats, in particular the limited range of options available for students with visual impairment. In this context therefore it is perhaps not surprising that such ‘student-centred’ views were espoused.

However, in a separate literature review carried out by some of the authors (see Douglas et al, 2009), it was argued that these views are not in keeping with literature which promotes the use of assistive technology and low vision aids (LVAs) to enable students to access information independently. For example, a review of studies carried out by Lussenhop and Corn (2002) concluded that the eight key studies they identified “point strongly toward a conclusion that reading standard print with optical devices is as effective a literacy medium as large print—and perhaps a more effective one” (p. 67). The authors note that LVAs are not always the appropriate solution, but even so they feel it is important for teachers and students to re-examine “assumptions and traditional reliance on large print” (p. 68). In their review Douglas et al (2009) argue that the dilemmas teachers face perhaps “highlights a tension between educational approaches and contemporary policy directions in relation to disability; arguably the former would emphasise teaching students access skills (i.e. how to use LVAs), while the latter would emphasise providing materials to optimise access (i.e. the provision of bespoke large print)” (p. 101).


In practice, the take-up of handheld LVAs by pupils with low vision in the UK is low (e.g. Mason, 1999), many preferring to use bespoke large print. As presented in section 3 of the report, this large print is sometimes, but not always, 18 point Arial Bold. However, the research also suggests that many pupils with visual impairment use computer-based assistive technology to access print.

Public Examinations System

In considering the public examination system, there is some evidence that current examination modification and formatting process is implemented inconsistently and with limited quality assurance procedures. This is based upon the interview data and analysis of MLP papers presented in this project as well as findings of Cobb (2008).

Putting aside the quality of the implementation of the current examination modification process, interviews in this project also suggest that the current examination process does not appear to reflect classroom practice for students with visual impairment who read print. The most obvious examples from the project of this ‘disconnection’ are students with low vision who do not routinely use 18 point Arial Bold in their day-to-day education must still use it in their examinations.

7.3 Possible solutionsSuch a ‘disconnection’ is clearly not helpful to students seeking to gain qualifications. Students without visual impairment do not experience such a difference between their way of working in the classroom and their experience of assessment2. Solutions must seek possible ways of encouraging greater harmony of practice.

The first report produced as part of this research project (‘Summary report on international systems of exam access for visually impaired pupils’) noted that providing visually impaired students with access to examinations is challenging across all the ten surveyed countries. The report also noted that approaches used in England and Wales are typical of approaches used across most of the surveyed countries (albeit to differing degrees), e.g. hardcopy large print, Braille, content modifications, extra time allowance, and use of scribes. However, the

2 Although it is interesting to note that the current chief executive of Ofqual, Isabel Nisbet, has recently speculated that teenagers are more comfortable typing answers on a keyboard than writing them down and raised concerns, “We need to make sure [the way pupils are tested] isn’t overtaken by the modern world and doesn’t become a relic of the early 20th century.” See Shepherd (2010).


availability of electronic versions of examinations was relatively common across the surveyed countries and this is in contrast to the approach in England and Wales.

It was argued that where electronic versions of examinations do not exist, visually impaired students’ access to examinations appears to be threatened because the range of formats they require is difficult to provide. Therefore, the importance of electronic versions of examinations seems a critical approach to enable students to gain access to their preferred format.

This approach to accessing electronic versions of examinations is adopted by the RNIB E-formatting software which has been evaluated in this research. The software as currently developed aims to provide a method for teachers to create a range of hard copy formats of examinations while maintaining the security of the examination content. It offers good control over print style and print size which in principle could provide a wider range of presentation options than currently available in England and Wales. The process of producing the software versions of the examination papers also appears to have an important associated benefit, namely standardising of the modification process and the general format of the examination papers.

However, evidence suggests that while printed hardcopy examination papers are important, student access to electronic versions of the examinations is also important. It offers several potential advantages: Some students seem to prefer it, particularly those who require very

large print (greater than 24 point); It reflects the way many students with visual impairment work in the

classroom; It is a pragmatic solution for those students who require fonts bigger

than 24 point (because it is difficult to produce hardcopy versions of exam papers in these font sizes);

It has the potential of offering other computer-based functionality which is of use to visually impaired students.


7.4 RecommendationsWe present two broad recommendations based upon the research findings. The recommendations are made in relation to the project research questions, i.e. particularly linked to the potential of the RNIB E-Formatting software (or similar) in the provision of accessible examination papers. It is important to note that the authors advise that the recommendations are implemented as part of a more general review of examination and curriculum access approaches for visually impaired students in England and Wales, specifically in relation to: The procedures and quality control of examination provision (including

the process of modification); Curriculum access strategies adopted in the classroom (particularly in

relation to the use of low vision aids and computers).

Recommendation 1:

A software application for supporting the production of an increased range of examination formats for students with visual impairment should be developed. The application would be used by teachers and visually impaired students, and it should have the following functions:

a) a range of formatting options;b) printing facilities for students who require hardcopy large print

examinations;c) onscreen presentation facilities for students who require onscreen

access to examinations.

The report provides a draft functional specification for the software application. The RNIB E-formatting software evaluated in this research project provides a very useful prototype of such an application.

Recommendation 2:

The future development of mainstream interactive computer-based examinations presents significant opportunities for improving access to examinations amongst visually impaired students. To ensure that visually impaired students can benefit from these opportunities the following general recommendations are made:

a) Examination boards and designers of computer-based examinations technology should work with access technology experts (for example, within RNIB) to ensure that inclusive design principles are incorporated into the development of this technology.


b) Developments in computer-based examinations will mean that educators must continue to ensure that visually impaired pupils are taught appropriate access skills so that they can independently access these examinations.

7.5 Functional specification of future software

It is difficult to write a functional specification for a piece of software which is only part of an examination modification process (and this process is not necessarily fixed). Nevertheless, we suggest the following functional specification based upon our interpretations of the research findings. If developed, the software may not incorporate all the proposed functions presented because it will depend upon stakeholders’ decisions about how they wish to respond to the project recommendations. The key stakeholders are the examination boards in England and Wales, and decisions must be made as to whether the software should be a tool for preparing hardcopy print, a tool for presenting onscreen examinations, or both. The proposed software functions are therefore grouped into several broad areas which are presented in turn.

General formatting facilities Printing facilities Onscreen presentation facilities Security Interactive facilities Contextual requirements

General formatting facilities

A central function of the software will be the formatting facilities. This is true whether the software is designed to prepare material for printing hardcopy exams or for onscreen presentation. The functions are divided between formatting facilities related to text (e.g. font style and size), structure / layout (e.g. line spacing), and tables, pictures and graphs (e.g. size manipulation). It is also noted that the software must have the facilities to accommodate the particular formatting requirements of some curriculum areas (e.g. poetry indents and line breaks, mathematical equations).


Table 26. Specification in relation to general formatting facilitiesSpecificationText

Font sizeFont styleFont weight (bold/not bold)Colour (foreground / background). Large colour range. Note: only relevant for onscreen.

Structure / layoutLine spacingPage breaks. Note: only relevant for printing.

Tables / Pictures / diagramsLine width (diagrams/graphs) Line width (tables)Size (diagrams/graphs/pictures)

Specific curriculum requirementsFacilities to accommodate the particular formatting requirements of some curriculum areas (e.g. poetry indents and line breaks, mathematical equations).

Printing facilities

Many students and professionals felt that hard copy versions of exams were still a preferred format. Given this, the preparation of bespoke printed materials is a key function of the software. The current version of the RNIB E-formatting software has particular difficulties with some aspects of print preparation. Three areas of the functional requirements have been identified in relation to printing. Firstly, easy formatting and layout is required: most notably in relation to page breaks, line spacing and sizing of graphics. Secondly, it is useful to be able to view onscreen what will be printed on paper. This ‘What-you-see-is-what-you-get’ (or WYSIWYG) facility is extremely useful and efficient when preparing printed material. Thirdly, teachers were concerned about the time that would be taken for the production of material when under time pressures before an examination. For this reason it may be sensible for some popular formatted versions of examinations to be ‘print ready’, i.e. when printed they will have appropriate layout without the need to re-format. Arial bold 18 and 24 point appear to be traditionally popular.


Table 27. Specification in relation to printing facilitiesSpecificationEasy formatting of paper spacing and layout – page breaks, line spacing for specific paragraphs, picture / diagram / table sizes.WYSIWYG - ‘What-you-see-is-what-you-get’Standard formats prepared as print ready (suggest in Arial bold 18, Arial bold 24)

Onscreen presentation facilities

Many students and professionals identified onscreen versions of exams as a preferred format. If this presentation method was developed, then examinations may need to be designed / modified specifically for this purpose. For example, some exam question structures assume that the examination is presented on paper (e.g. ‘label the diagram’, ‘circle the correct answer’). A detailed analysis of the required modifications for onscreen presentation would need to be considered. It would also need to account for the student response mode (e.g. using a word processor).

There are a range of functions which were identified which would support onscreen presentation. The most important are the general formatting options described above. However, there may be a variety of ways of implementing these formatting facilities, e.g. some functions may be dynamic, enabling users to change them in real time (not offered by the current version of the RNIB E-formatting software). Whether these functions are embedded within the design of software, or require the use of additional operating system / access technology (e.g. a screen magnifier) would need further consideration.

Additional facilities identified in the research might include some navigation and search facilities. For example, this might enable users to use key strokes to move from question to question, or search for text expressions.

This model of onscreen examination presentation would most likely involve the user typing the answers to the examination in a word processing package (and submitting a printed version of the answers to the examination board at the end of the assessment). In this case, the software would require that the user could ‘toggle’ between software applications using standard operating system routines, or view the applications on the same screen if desired. Making use of other standard facilities may also be helpful, for example being able to ‘copy’


text from the examination document and ‘paste’ into the answer document may be very helpful when constructing answers.

Finally, using the software in combination with screen magnification and/or screen reading access technology may be helpful for some users.

Table 28. Specification in relation to onscreen presentation facilitiesSpecificationFormatting facilities as described above (general formatting facilities).Keyboard navigation (e.g. up, down; left, right; page up, page down; question by question)Search and cross-referencing facilities‘Copy and paste’ facilitiesEasy toggle between examination software and word processing (answering) softwareCompatible with magnification software, or inbuilt facilities.Compatible with screenreading software

Security

It is important that the software is able to maintain the security of the examination content. In practice this means that the software should not allow accidental or malicious changes to the wording or graphical content of the examination (e.g. the removal or addition of a word, removal or changing of graphics).

Interactive facilities

A more speculative facility which was identified by the research was the potential of ‘interactive examinations’ which allow for answers to be entered into appropriate ‘answer fields’. Some online assessments are emerging in the UK and elsewhere which have these qualities, and the PDF versions of examinations offered in Scotland by the SQA also have some of these facilities. Specifications are not presented because of the speculative nature of the idea. Nevertheless, things to consider would be the availability of extensive keyboard facilities for navigation, and access for non-print users (including linking question content and answer boxes and other ‘meta data’ including available marks).


Contextual requirements

The specifications outlined above are linked to the direct functioning of potential software. An important part of the interviews with professionals was in relation to contextual issues: i.e. how and when such software might be used, and the process of requesting access arrangements for given visually impaired pupils.

Table 29. Specification in relation to contextual requirementsSpecificationSoftware is delivered on time.The software is in a format which can be routinely run on standard equipment (the RNIB E-formatting software arrived in the form of a CD and this serves as a useful example).Appropriate early opening time allowances are possible.Past papers are available for practice.There is a clear protocol (including realistic timescale) for requesting access arrangements.Modification and formatting is applied consistently for all examinations.


8. ReferencesCobb, R. A. (2008). Exploring systems for the provision of modified large

print examination papers to partially sighted candidates in England and Wales. MPhil, School of Education, University of Birmingham.

Douglas, G., McLinden, M., McCall, S., Pavey, S., Ware, J. Farrell, A. M. (2009). International review of the literature of evidence of best practice models and outcomes in the education of blind and visually impaired children. Report for National Council for Special Education (NCSE), Ireland. Accessed 17th June 2010:http://www.ncse.ie/research/researchreports.asp

Joint Council for Qualifications (JCQ) (2008). Best practice guidance for the modification and production of examination papers for candidates with a visual impairment. Accessed 13th June 2010:http://www.rnib.org.uk/professionals/education/schoolbasedlearning/mainstream

Lussenhop, K. and A. L. Corn (2002). “Comparative studies of reading performance of students with low vision.” RE:view 34(2): 57–69.

Mason, H. (1999). “Blurred vision: a study of the use of low vision aids by visually impaired secondary school pupils.” British Journal of Visual Impairment 17(3): 94–97.

Shepherd, J. (2010). Should computers replace pen and paper in the exam room? Discuss. The Guardian, Thursday 19th August, 2010, page 8.


http://www.rnib.org.uk/professionals/education-old/schoolbasedlearning/mainstream

http://www.rnib.org.uk/professionals/education-old/schoolbasedlearning/mainstream

http://www.ncse.ie/research/researchreports.asp

Appendix 1Student Questionnaire

Name: _______________________

What type of school do you attend? (Choose one)MainstreamResource baseSpecialOther:

For the LARGE PRINT version, list any things that you liked and disliked:

Liked Disliked

The SOFTWARE version, list any things that you liked and disliked:

Liked Disliked


Which version of the examination do you think was easier to access? (Choose one)

LARGE PRINTSOFTWAREBoth the sameUnsure

Please explain your answer:

Given any choice, what is your preferred print size and font?

Print size: ________Font: ________

Do you use an LVA when reading? Yes/ No

Explain how you would have liked the examination papers formatted for YOU?


Consider if the examination papers were presented ‘on-screen’. What would be the advantages and disadvantages for YOU?

Advantages Disadvantages


Appendix 2Font size questionnaire

This is 18. A Ford is a kind of car.

This is 24. A carrot is a kind of vegetable.

This is 28. An orange is a kind of fruit.

This is 32. A cat is a kind of animal.

This is 36. The earth is a planet

This is 40. The sun is a star.Evaluation of alternative exam presentation methods for pupils with low vision 104

Questions about size of print you like and use

Name: _______________Date: ______________

GeneralDo you ever use an LVA when reading? Yes/ NoGiven any choice, what is your preferred print size and font?

Print size: ________Font: ________- Is this with or without and LVA?

What font size do you usually use in the classroom / day-to-day work? __________- Is this with or without and LVA? (also ask about computer use)

18 point(answer 1 to 4, 1 is easily, 2 is OK, 3 its hard, 4 is No, or TOO BIG)Can you read this? ____(answer yes, no, maybe, don’t know)Could you read it for a long piece of text (e.g. an examination paper)? ___

24 pointCan you read this? ____Could you read it for a long piece of text (e.g. an examination paper)? ___





OverallFor you, which font size would you prefer for classroom/day-to-day work? ___For you, which font size would you prefer for examinations? ___


Appendix 3Professional Interview Schedule

Section A - Hard Copy Examinations

1) Original paper

This is an example of standard examination papers (not modified, not enlarged). Most students without visual impairments would receive this version of the exam paper. Explore the paper and give your thoughts under the following headings.

Thinking about students with low vision (who are print readers):

a) Many students work with a variety of material (including unmodified, ‘normally sized’ print) – including but not exclusive to exams. For the students you know, how familiar would they be accessing information in this format? Prompts: Give examples; Service policy?

b) How easily would the students you know access this material? Prompts: Give examples.

What about navigate this material? That is, being able to move from question to question, link between question and source material, etc.


2) Modified Large Print (MLP) paper

This is an example of modified large print (MLP) examination papers (point size 18, and some of the content modified). Many students with visual impairments would request and receive this version of the exam paper. It is acceptable for papers to be photocopy-enlarged in the one hour before the examination (giving a font size of approximately 24 point). Explore the paper and give your thoughts under the following headings.


a) Many students work with a variety of material (including MLP – that is modified to a ‘standard’). For the students you know, how familiar would they be accessing information in this format? Prompts: Give examples; Service policy?

b) How easily would the students you know access this material? Prompts: Give examples.

What about navigate this material? That is, being able to move from question to question, link between question and source material, etc.


3) Software formatted paper

DEMONSTRATE THE SOFTWARE

This is an example of software formatted examination papers (in the hardcopy examples given the font is Arial point size 28, and Comic 18). As demonstrated, other versions of the examination paper can be prepared using the software (different font styles, different font sizes). Potentially, students with visual impairments could request and receive a version of the exam paper which could be formatted using this kind of software in the hour before the examination. Explore the paper and give your thoughts under the following headings (remembering that the software can be used to create the same examination paper in different formats).


a) Many students work with a variety of material (including material prepared in a format of their choosing). For the students you know, how familiar would they be accessing information in this format? Prompts: Give examples; Service policy; Software / electronic formats used?

b) Using this type of software, consider its potential for giving students access to examinations? [Note: the example printed is Arial point 28, but any other font / size could have been chosen]

Prompts: How might it be improved?What about navigate this material? That is, being able to move from question to question, link between question and source material, etc.


c) Think about managing the process. Given that the use of this type of software would need a member of staff preparing material before the exam started – how do you think you / your service / the school could manage this?


Section B – Onscreen Examinations

Original and MLP papers onscreen

Although it is currently not an option for examinations, we are interested in your views about the possibility of onscreen presentation.

In the examples we’ll give the standard and MLP examination papers are presented onscreen through Adobe Acrobat Reader. This onscreen presentation of the examination paper enables the user to use software tools to enlarge or reduce the onscreen display. DEMONSTRATE THIS.


a) What are your initial reactions to this onscreen method of presenting examinations?

b) Thinking more broadly than examinations, students can access their school work using computers. We’re trying to understand how common this is for the students you know. How common would you say it is? Remember, we want you to think about students who read print.

Prompts: How many students use laptops in class?How many students are given curriculum material in electronic format?Please give examples (of students who do and students who do not)Does the service have a policy on this way of working?


c) Consider if examination papers were presented ‘on-screen’. What would be the advantages and disadvantages for the pupils you work with?

Advantages Disadvantages

Section C – In a perfect world

1) Think about what we have discussed, including the exam options which are currently available (modified large print papers in Arial 18 point) and options which could one day be available (e.g. software for more bespoke formatting, onscreen examinations). What type of examination provision for visually impaired pupils would you like to see in the future?

[Prompt:distinction between the student controlling the access, or materials being prepared to his / her specificationWhat about your role / the role of staff in this ‘perfect scenario’]


evaluation of alternate exams presentation · web view“flipping- pages gets annoying, especially...

Documents