writing mentors guiding student...

Ghent University

Faculty of Arts and Philosophy

Writing Mentors Guiding Student Writers:

Evaluating Mentoring Efficacy

through Student Reflection and Essay Analysis

Paper submitted in partial fulfilment of the requirements for the degree of

“Master in de Taal- en Letterkunde: afstudeerrichting Engels-Nederlands”

Thijs Gillioen

Academic year 2015 - 2016

Supervisor: Prof. Dr. Mieke Van Herreweghe

English Studies Group - Linguistics Section

Co-Supervisor: Dr. Sarah Haas

English Studies Group - Linguistics Section

The author grants the permission for this master’s dissertation to be made available for

consultation. Every other use is restricted by the limitations imposed by copyright, in

particular with regard to the requirement to cite the source explicitly when referring to

results from this thesis.

Thijs Gillioen,

May 2015

Acknowledgements

I would like to offer due recognition to the following people, for their effort,

motivation, and support during the writing of this thesis.

Firstly, I would like to thank Prof. dr. Mieke Van Herreweghe, for her

willingness to support this research, and for supporting the peer writing mentoring

programme since its inception.

Secondly, I would like to extend my gratitude to dr. Sarah Haas, my co-

supervisor. It is only thanks to your excellent counsel, and boundless patience in the

face of countless questions and drafts, that this paper could become what it is now.

Likewise, it is only thanks to your guidance that I have become a better writer, and

researcher, than I could ever have hoped to be when I started studying English four

years ago. You have gone above and beyond the call of a supervisor, and for that, I am

eternally grateful.

Thirdly, I would also like to thank Alexander De Soete, who was kind enough

to dedicate much of his time to aid me in my research, by joining me and dr. Haas as

the third rater for my analysis of student writing. Aside from this, our many

conversations, discussions and laughs have helped shape this paper in more ways

than one, for which I am very grateful.

Fourthly, many thanks go to the UGent writing mentors, without whom this

research would not have been possible. Their continuous, and selfless, dedication of

their free time, energy, thoughts and contributions are what make the programme

work: it is my sincere pleasure to be part of such a wonderful group.

Finally, I would like to thank my parents, for giving me the opportunity to

study, for supporting me unconditionally, and for being there for me, even when I

was often too occupied to return your kindness.

Table of Contents

Abstract i

List of Tables ii

List of Figures iii

List of Abbreviations iv

I Introduction 1

II Research Situation: Methodology & Context 3

1 Methodology: Action Research 3

2 Context: Literature Review & Research Setting 5

2.1 Literature Overview: Teaching Writing Skills 5

2.2 Writing Instruction for English Majors at UGent 7

2.3 Literature Overview: Automated Essay Scoring: A Solution? 9

2.4 Writing Mentoring Programme at UGent: Inception and Grounding 10

2.5 AR Cycle 1: Setting up a Peer Mentor Programme 13

2.6 AR Cycle 2: Lowering Barriers for Engaging with Mentoring 14

III AR Cycle 3: Evaluating the Efficacy of the Writing Mentor Programme 16

1 Literature Overview: Evaluating Writing Mentor Programmes 16

2 Evaluating the Current Programme: Two Perspectives 20

3 Perspective 1: Analysis of Pre- and Post-Mentoring Texts 22

3.1 Data Collection: Student Papers 22

3.2 Text Analysis Framework: “Reader Engagement” (Haas 2016) 23

3.2.1 Overview of Engagement Framework 23

3.2.2 Developing a Rating Scale Based on the Framework 28

3.2.2.1 Pilot Test I: Devising an Initial Rating Scale 29

3.2.2.2 Pilot Test II: Refining the Rating Scale 30

LOCs vs. L-LOCs: Temporarily De-Emphasising Spelling and Grammar 30

Marking Each Instance of a HOC 33

Marking Each HOC Equally 34

Marking Essays That Do not Meet “General Responsibilities” 35

3.3 Analysing Student Essays with the Rating Scale 36

3.4 Overview of Results 37

3.5 Discussion of Results 41

3.5.1 General Remarks 41

3.5.2 Specific Areas of Discussion: Data Trends 44

3.5.2.1 ‘Apparent’ Number of HOCs Retained vs. ‘Actual’ Number of HOCs retained 44

3.5.2.2 Essays with ‘Complex Issues’ 50

3.5.2.3 Specific HOC Trends 52

3.6 Recommendations for Programme Improvements 56

4 Perspective 2: Analysis of Student Reflections 59

4.1 Data Collection: Student Reflective Writing 59

4.2 Methodology: Content Analysis 60

4.3 Overview of Results 67

4.4 Discussion of Results 68

4.4.1 General Discussion 68

4.4.2 Discussion of Reflections per Category 70

4.4.2.1 Practical Reasons 70

4.4.2.2 Confidence and/or Motivation Boost 74

4.4.2.3 Extra Explanation Offered and/or Helps to Understand Theory 75

4.4.2.4 Appreciation of Non-Directive Approach 76

4.4.2.5 Student Advice on Mentoring (Programme) 78

4.4.2.6 Attending Multiple Mentoring Sessions 79

4.4.2.7 Appreciation of Approachability 81

4.4.2.8 Students Who Did not Feel Helped 83

(27.449 words)

4.4.2.9 Writer Development 85

4.5 Recommendations for Programme Improvements 87

IV Limitations, Conclusion, and Future Research 91

1 Limitations of the Current Research 91

2 Conclusion of the Current Research 93

3 Suggestions for Future Research 97

V References 101

VI Appendices 110

Overview 110

Appendix 1: ETTVI Scribing 3 Assignment 111

Appendix 2: Overview of General Responsibilities 117

Appendix 3: Analysed Student Essays 118

Appendix 4: Samples from Reflective Writing 214

Abstract

At UGent, a major Belgian university, students of English are expected to attain a

high level of academic writing skills. However, due to the constraints of university

funding, a single teacher was responsible for teaching 250+ students how to write,

resulting in a situation where students could wait for weeks before getting any

feedback on their work. To combat this issue, a peer writing mentoring programme

was established during the academic year 2013 - 2014. Employing students as near-

peers, this programme aims to provide guidance in students’ development as writers

by presenting them additional opportunities for non-directive support. Through an

Action Research-based investigation, this paper aims to evaluate this programme in

its current form, and make suggestions for improvements in its conduct and

practices. It did so by assessing both students’ written product, and their reflection on

the programme’s influence on their development as writers. Students’ pre- and post-

mentoring texts were analysed using Haas’ “framework for teaching and assessing

writing” (2016), thus looking into the textual manifestation of the effects of writing

mentoring. This unveiled several areas where students are greatly helped by a

mentoring session, and some areas where further improvements could be made, both

by specific training for the mentors, and by more (focussed) sessions with students.

Secondly, it analysed students’ reflective writing in order to understand students’

perspective on the mentoring process and on their development as writers. Since the

concept of writing mentoring (and, by extension, a writing centre) is alien to Belgian

students, their perceptions of the mentoring process were vital to understanding how

they can be assisted more efficiently in their development as writers. The points of

discussion in this paper may be expanded to be applicable in other mentoring

situations, especially those in similar low-budget, students-as-peers-contexts.

!i

List of Tables

!ii

Table 01. HOCs and LOCs in Reader-Friendly Writing (Haas 2016: 27) 27

Table 02. Overview of Marks, L-LOCs/LOCs/HOCs in Pre- and Post- Mentoring Essays

39

Table 03. Overview of HOCs in Each Essay 40

Table 04. Apparent Retained HOC Issues vs. Actual Retained HOC Issues

45

Table 04. Apparent Retained HOC Issues vs. Actual Retained HOC Issues (second part)

46

Table 05. Overview of Complex Issues in Each Essay 50

Table 06. Number of HOC Issues 53

Table 07. Overview of Reflective Writing Categorisation Process 61

Table 08. Overview of Categorised Reflective Writing 67

List of Figures

!iii

Figure 01. Action Research (Lewin 1946) 3

Figure 02. Alphabetical List of “Building Blocks of Engaging Text” (Haas 2016: 26)

26

List of Abbreviations

AES Automated Essay Scoring

AR Action Research

AR cycle Action Research Cycle

AWE Automated Writing Evaluation

BA Bachelor of Arts

ESL English as a Second Language

ETTV Engelse Taal- en Tekstvaardigheid

GO! Onderwijs van de Vlaamse Gemeenschap

HOC Higher-Order Cognition

LOC Lower-Order Cognition

L-LOC Lower-Lower Order Cognition

VVSKO Vlaams Verbond van het Katholiek Secundair Onderwijs

!iv

!1

I Introduction 1

This applied linguistics dissertation aims to conduct an evaluation of the efficacy of

the peer writing mentoring programme that has recently been established at the

linguistics section of the English Studies Group at UGent. To do so, it uses the Action

Research methodology (hereafter referred to as “AR”), which seeks to “bring together

action and reflection, theory and practice, […] in the pursuit of practical solutions […]

and, more generally the flourishing of individual persons and their

communities” (Reason & Bradbury 2001: 1). By engaging in the AR reflective cycle

(Lewin 1946) and making decisions based on both literature and data, this paper will

use the result of this inquiry to propose practical recommendations for improving the

programme’s efficacy.

The writing mentoring programme at UGent was established during the

academic year 2013 - 2014; it was modelled on comparable mentoring programmes

in writing centres, which are present at most universities in the US, and are

appearing increasingly frequently at universities throughout Europe, particularly in

the UK, Ireland and Germany. This programme functions within the context of the

Engelse Taal- en Tekstvaardigheid (abbreviated ETTV) classes, which are

compulsory for students wishing to obtain an academic Bachelor’s degree in English

at UGent. In these classes, mentors have been employed to provide additional

opportunities for students to receive support for their written work.

The first section of this paper will introduce the AR framework, which

underpins this dissertation. To orient the reader, the second section will then

Due to the similar subject matter and research methodology of this paper and the research 1

conducted in the author’s Bachelor paper, elements of the methodology and literature review may appear in familiar guises in both that paper and this thesis, for the sole purpose of orienting the reader towards the nature of the programme and the current state of academic research into writing mentoring.

Introduction Thijs Gillioen

!2

summarise the first two Action Research Cycles (hereafter referred to as “AR Cycle”),

research into which was presented in the author’s bachelor paper, before

commencing the discussion on the third AR Cycle, which is the current paper’s focus.

The third AR Cycle will be examined in order to assess the efficacy of the peer

mentoring programme, and to make suggestions for improvements to the

programme.

Introduction Thijs Gillioen

!3

II Research Situation: Methodology & Context

1 Methodology: Action Research

In order for this paper to make suggestions for such programme improvements, it

employs the AR methodology first discussed by Lewin (1946), and since then well-

established particularly in organisational and educational research, and in applied

linguistics (Davies & Elder 2008: 476). As an interventionist methodology, used to

“assist the ‘actor’ in improving and/or refining his or her actions” (Sagor 2000: 3), it

“inevitably lead[s] to program improvements” (Sagor 2000: 8) and “bring[s] about

improvements in classroom practices” (Richards & Schmidt 2013: 8).

Action Research involves going through a succession of subsequent reflective

“Cycles” of the given situation it aims to evaluate. These cycles are represented in

Figure 01. Since AR actively aims to improve a real-world situation, engaging with

this situation is the first step of any AR Cycle (marked as (1) in Figure 01). For this

paper, this “setting” is the writing component of the previously-mentioned ETTV

Research Situation Thijs Gillioen

Figure 01. Action Research (Lewin 1946)

(1)

(2)

(3)

(4)

(5)

(7)

(6)

!4

classes, which is offered to students wishing to obtain a bachelor’s degree in English

at UGent.

Within the situation, Action Researchers define a specific “issue”, based upon

“initial observations [and] existing data” ((2) in Figure 01). Within the context of this

research, the “observation” is that, based on existing data from two previous AR

cycles, students reacted positively to mentoring sessions, and there is the subjective

notion that mentors can help guide students towards becoming better writers. The

issue defined here is that this positive perception is based on limited data, and there

is no data that objectively quantifies either text improvement or writer development

for students in the current setting.

The next step in the AR Cycle is to plan an action ((3) in Figure 01) to respond

to the issue defined in step (2). The action planned here is to evaluate the

programme’s efficacy from both a text- and reflection-based perspective.

Subsequently, the action is carried out and reflected upon ((4) and (5), respectively).

After these steps are taken, the findings are reported (6), which is where this thesis is

situated within the current AR cycle. Although the scope of this dissertation ends at

this step (6), the improvements suggested will be tested when the research(er)

reconnects with the real-world situation (7), which will simultaneously and

subsequently serve as the basis for the next AR Cycle.

Given this interconnectedness of these AR Cycles, it is neither feasible nor

desirable to present one AR Cycle as an independent entity. As such, before detailing

the current paper’s research questions, the next sections will summarise the research

and findings of the first two AR Cycles (which were investigated as part of this

author’s BA research paper) to orient the reader to the background of this

dissertation. Accordingly, to reflect the stages of AR and to ensure a logical


!5

presentation of arguments, the relevant literature will be presented in each section,

rather than dedicating one full section to all relevant literature in its entirety.

2 Context: Literature Review & Research Setting

2.1 Literature Overview: Teaching Writing Skills

The ability for students to write “clear, well-structured texts of complex

subjects” (Council of Europe 2001: 61) is considered one of the core skills for students

to acquire over the course of their education, both during secondary school (where

this emphasis is most prominent in the primarily theoretical ASO strand), and during

their education at college or university. In Flanders, the official final attainment levels

for secondary education require students to be capable of taking into account “the

most important conventions of written language” (Flemish Government 2016), but it

is not specified what these conventions are.

The largest secondary school network in Flanders, Katholiek Onderwijs

Vlaanderen , narrows this down somewhat by requiring student-written texts to have 2

a “clear text structure”, although this is noted to be beyond the aims of the basic

teaching plan (VVKSO 2014: 20). It does not, however, define what a clear text

structure is: it only indicates it does not require students to be able to “combine

complex elements into a coherent text” (VVKSO 2014: 19-21). The other major

secondary school network, the public net GO!, does not specify beyond the

government’s requirements of students adhering to “the most important conventions

of language” (GO! 2014: 48). Therefore, in both settings, it is left up to individual

secondary school teachers to define what constitutes a well-written text.

Previously known as “Vlaams Verbond van het Katholiek Secundair Onderwijs” (VVKSO); 2

cited as such in this dissertation since the current official curriculum plans predate the name change.


!6

When students enter higher education, research has indicated this scattered

writing education results in, at best, a widely diverse range of writing proficiencies

(De Wachter & Heeren 2012: 55), and, at worst, “serious problems when trying to

apply structure to texts” (Berckmoes & Rombouts 2009: 6 [translation]). In short,

texts written by these Flemish students force the reader to “distill the meaning from

the text, or ask the sender for extra information” (Berckmoes & Rombouts 2009: 18

[translation]), making it “very hard for the reader to follow” (Berckmoes & Rombouts

2009: 17 [translation]). Based on this, students in this education system thus, at

most, only reach the first stage in the “macro-stages in the cognitive development of

writing skill”, defined by Kellogg (2008: 4), “Knowledge-Telling”, wherein they

simply “tell what [they] know” (Kellogg 2008: 3), focussing on their “thoughts[,] and

not how the text itself reads” (Kellogg 2008: 6).

Therefore, it is left to institutes of higher education to teach more complex

cognitive principles: text structuring; Higher Order Cognitions (abbreviated HOCs

(Bloom et al. 1956)); or writing for a reader, which Kellogg considers a part of his

third (and final) macro-stage, that of “Knowledge-Crafting” (2008: 4). However, in

order for an aspiring writer (here, a student) to reach this stage of “Knowledge-

Crafting”, Kellogg argues a span “more than two decades” (2008: 2) is required,

which is not realistic within the setting of a student trying to attain a university

degree.

Although Kellogg’s demand for time is impossible, his other demands are

arguably at least partially feasible, but not met, in this specific context. He argues any

attempt to improve one’s writing skills can only be achieved through “repeated

opportunities to write and through timely and relevant feedback” (Kellogg &

Raulerson III 2007: 237 [emphasis added]). Kellogg’s demand for feedback is


!7

supported by Hattie, who, in his meta-analysis of learning methods, noted feedback

to be one of the most effective tools for learning (2009: 162). In addition to feedback,

Kellogg argues the most successful model for learning writing skills is one wherein a

student can engage in “cognitive apprenticeship” (2008: 19), which involves a student

“learning by observing, rather than doing” (2008: 19), in which case the best results

can be attained by “observing readers who responded to one’s own written text plus

receiving additional written feedback” (Kellogg 2008: 19). This ‘ideal’ model of

writing instruction will be contrasted with the current system of writing instruction at

UGent in the following section.

2.2 Writing Instruction for English Majors at UGent

At Ghent University, Bachelor-level students of English have to attend three

compulsory (and successive) proficiency courses (ETTV), which include, among other

proficiencies, learning and improving writing skills. The first of these, ETTVI, enrols

between 250 and 300 students, and teaches these students about writing by gradually

presenting them the “building blocks of reader-friendly writing” (Haas 2016: 25) over

the course of a number of lectures. In between these lectures, students get to

experiment with these cognitively challenging HOCs in a number of smaller

seminars. As part of their final assessment for the course, students are given at least

three mandatory writing assignments, thus, arguably, at least partially fulfilling the

needs for repeated opportunities to write argued for by Kellogg - whose argument for


!8

weekly opportunities to write and receive feedback is, again, not practically feasible

within the (European ) university setting. 3

However, due to the realities of the current academic climate, the university

cannot dedicate enough staff members to meet Kellogg’s demand of relevant, and

particularly timely, feedback. Whereas Kellogg’s research indicates feedback is best

delivered within minutes (2008: 18), it can often take one or more weeks for students

of ETTV classes to receive feedback on their written assignments, at which point they

may already have begun writing for the next assignment. In the case of the papers

analysed for this dissertation, there was a two-month gap between the due date and

availability of feedback. Although this two-month period did include exams and the

Christmas break, this does not change the fact that this much time went by before

feedback was provided - although no further essay assignments were to be made at

this point, students did have to partake in essay exams, thus timely feedback would

still have been relevant.

Additionally, the lack of sufficient staff members renders engaging into any

form of cognitive apprenticeship, as defined by Kellogg, impossible. Given the 250 to

300 students enrolled in the ETTVI course, engaging in any form of apprenticeship

would require about ten full-time teachers - which is currently not a viable option.

Research has noted significant differences between the “American model” and the “Central 3

European University model” (Harbord 2003: 3) for writing education and assignments. These differences have led to different approaches in writing instruction: whereas the current Belgian setting allows for about 15 hours of in-class writing instruction for English majors, American universities spend at least 100 hours of in-class writing instruction for non-English majors, thus arguably coming much closer to achieving Kellogg’s demand for repeated opportunities to write.


!9

2.3 Literature Overview: Automated Essay Scoring: A Solution?

The issues of understaffing for the teaching of writing are not limited to the setting of

this dissertation, but are common throughout the world: Shermis calls into question

“the feasibility of recruiting a sufficient number of qualified human graders to

provide final scores [for writing assignments]” (2014: 54). Likewise, Vista et al.

mention “significant [financial and time-related] costs to mark, becoming exorbitant

for courses with large numbers of students” (2015: 1).

One often-cited solution to this problem is automated essay scoring (hereafter

referred to as “AES”), which hopes to significantly reduce both the time between

presenting an essay and getting feedback, and the pressure on teachers of writing. As

such, significant funding and research has been devoted to establishing the validity of

AES. However, the applicability of AES for ESL (English as a Second Language)

students is still subject of evaluation (Warschauer & Grimes 2008; Weigle 2013). In

addition, it may not be technically possible for AES in its current state to deal with

complex HOCs such as argumentation, as noted by Deane (2013: 15). Finally, current

research into AES systems shows “inter-human rater performance was significantly

better” (Shermis 2014: 55) than human-computer performance when marking

writing assignments. This human-computer reliability is even “lower in classroom

contexts” (Warschauer & Grimes 2008: 24). These realities currently prevent AES

from becoming standard practice for the purpose of marking writing assignments.

However, even if AES could be employed to reduce or eliminate the teachers’

time spent on marking essays, it does not resolve the lack of opportunities for

cognitive apprenticeship. There is only modest evidence that using AES or other

forms of Automated Writing Evaluation (AWE) has a positive influence on the quality

of the texts produced, or that the use of AWE can result in “more general


!10

improvements in writing proficiency” (Stevenson & Phakiti 2014: 63). Additionally,

Salomon noted these automated systems “are not designed to upgrade students’

intelligent engagement” (1993: 180). This limitation was observed again in recent

research that claimed usage of AWE systems resulted in “none of the broadly iterative

process through which writers hone their content, sharpen their organization, and

thus learn to transition from writer-based to reader-based prose” (Warschauer &

Grimes 2008: 33). For these reasons, neither AES nor AWE can currently provide a

resolution to the lack of opportunities to engage in cognitive apprenticeship. Because

of this, cognitive apprenticeship is still highly time-intensive when teaching large

groups, and thus impossible to achieve, even if the teacher would no longer have to

deal with marking essays.

2.4 Writing Mentoring Programme at UGent: Inception and Grounding

Because Automated Essay Scoring is not currently a viable solution to the problem of

time-intensive marking, or the lacking opportunities for cognitive apprenticeship, a

different solution is needed to combat these issues. Therefore, during the academic

year 2013 - 2014 a peer mentoring programme was established at the linguistics

section of the English Studies Group at UGent. This programme aims to improve

students’ writing by providing students of ETTV courses with more opportunities to

receive individual support on their writing assignments, in effect making them “feel

like they have had individual attention with their writing problems” (personal

correspondence, Haas: 15/09/13). To do so, the programme enrols, as “near-

peers” (Murphy 1998), a number of students (on a voluntary basis) who have

successfully completed the ETTV courses. Through doing so, the programme

administrator (who is responsible for teaching writing) hopes to at least partially


!11

fulfil Kellogg’s requirement of repeated opportunities for feedback, in addition to

providing at least some opportunities for cognitive near-peer apprenticeship. This

programme has gradually taken the form of an “Instrumental programme” (Karcher

et al. 2006: 714), which they describe as a mentoring set-up wherein “the mentor’s

role is to provide guidance and advice as the student conceptualises, researches, and

completes an assigned project” (Karcher et al. 2006: 718), which, within the context

of the ETTV courses and their assignments, is the programme’s focus.

Although concerns have been expressed at the idea of students taking a more

active (mentor) role in their fellow students’ education (as writers), research has

consistently supported the efficacy of peer mentoring (programmes), noting peer

feedback can be at least as effective as feedback provided by teachers (for example,

Topping & Ehly 1998; Gensemer 2000; Hattie 2009; Gielen et al. 2010).

Furthermore, peer mentoring has several added benefits for students, such as

reducing their writing anxiety (Martinez et al. 2011: 358), “improving (…)

understanding of concepts” (Cho & MacArthur 2010: 328) and “developing an

awareness of their writing” (Hutchings 2006: 259). On top of this, Cho & MacArthur

argue that peers are often in a better position to explain difficult concepts than

experts, “who use knowledge that novice students cannot refer to” (2010: 329).

Additionally, they indicate students’ close proximity to each other allows for them to

“detect problems from their own perspective”, thus “generating solutions to the

problems” (Cho & MacArthur 2010: 329). Some research has even posed that

“without peers’ assistance in both ideas and language, it is very difficult, if not

impossible, for [students] to grapple with these two issues [reading their text from an

outsider’s perspective and thinking of revision strategies] single-handedly” (Min

2006: 135). To further allay concerns related to mentors’ suitability to provide writing


!12

support, students wishing to become mentors in the programme receive between

fifteen and twenty hours of additional specific training in both advanced writing and

mentoring skills. Therefore, one can argue that mentors can provide meaningful

support to near-peers during their writing process.

This mentoring programme, as described above, is in many of its aspects

similar to a writing centre. Writing centres have been attached to most American

universities for several decades (e.g. DuBois et al. 2002; Karcher et al. 2006), with

Girgensohn suggesting as many as 90% of all American universities have such a

centre (2012: 127). In these universities, research has indicated the “one-to-one

writing tutorials provided by a dedicated university writing support department or

Centre” (Gopee & Deane 2013: 1629) are one of the number one forms of institutional

support requested and appreciated by students learning how to write, even in non-

language majors (Gopee & Deane 2013: 1629). Europe (and Belgium, particularly), by

contrast, has seen a relatively slow development of writing centres, to such an extent

that “they remain so few and are often institutionally invisible , that there is not even 4

a number that could be cited”, even though there is a “recognis[able] growth in

writing center work in Europe” (Girgensohn 2012: 127). The popularity of writing

centres in the US is explained by their well-established effectiveness: Hoon, for one,

argues students’ writing apprehension was reduced, in addition to noting several

“positive affects to writing” (2009: 47). Hutchings provides similar arguments, and

notes these effects may be stronger the more incorporated the writing centre is within

This institutional invisibility is evidenced in the current setting: while UGent has arguably 4

had the first Belgian writing centre(s) with the peer mentoring programme (since 2013) and the Taalonthaal initiative (since 2014), the limited visibility of these programmes has made it difficult for them to stand out in the academic landscape. This is evidenced by KU Leuven being recognised as having the first Belgian writing centre with their ILT-Schrijfcentrum initiative (since 2015).


!13

“the department […] and into disciplinary practices” (2006: 260). In this sense, the

writing mentoring programme fills a gap that is only recently, and slowly, being filled.

2.5 AR Cycle 1: Setting up a Peer Mentor Programme

It is within this setting that a peer writing mentoring programme was established

during the academic year 2013 - 2014, which is defined here as AR Cycle 1. This cycle

can be considered a “pilot year” for the programme: having only theoretical

knowledge gathered from literature, conference attendance and writing centre visits,

the programme coordinator, with the help of a few students, had to make choices

organically, based upon perceived needs of the mentors-to-be and the programme as

a whole. Although data were collected and analysed, reporting was, at this stage, only

done informally.

At the end of AR Cycle 1, the programme was in a functional state, and had

started to engage with students, both in one-on-one mentoring sessions and through

mentor attendance in ETTV seminars. It was at this stage the author of this

dissertation, in cooperation with the programme coordinator, decided that a

thorough and critical review of the programme’s workings was in order. This AR-

based review argued that significant improvements could be made to the programme

by improving its online components, and through these making the somewhat foreign

concept (for Belgian students) of mentoring easier to access for both mentors and

students (i.e. potential mentees). Having done this, the programme reconnected to

the real-world situation (step (1) in Figure 01), thus advancing into AR Cycle 2.


!14

2.6 AR Cycle 2: Lowering Barriers for Engaging with Mentoring

Making mentoring easier to access, which was the focus of research conducted during

cycle 2, required the perspective of both mentors and students. The first perspective,

that of mentors, was crucial if the programme was to be successful: it was paramount

that mentors could be retained over the course of multiple years (thus allowing them

to gather mentoring knowledge and skills, and transfer these to new generations of

mentors). Since all mentors are volunteers, ensuring they could focus on doing

mentoring work, rather than being distracted by administrative work (such as dealing

with countless mentee e-mails) or other distractions, was imperative. This

perspective is supported by Dawson who argued “technology […] resources and tools”

are essential elements when establishing a mentoring programme (2014: 140).

One online component that was reviewed was the group’s communications

platform, which is a dedicated Facebook group. The communications in this group

were analysed and placed into categories based on the framework of English &

Duncan-Howell (2008: 598). These data were used to formulate suggestions to

improve the efficacy of this group (such as clearly marking the topic of each message

in a header), keeping in mind the relevancy of posts that were less immediately

mentoring-related for the purposes of community building, as demonstrated by

Brown (2001). These suggestions, thus, made it easier for mentors to quickly find

relevant info, which lessened mentors’ barriers to staying committed to the

programme.

The second perspective, that of the students who were the programme’s

potential mentees, was no less important. For most Belgian students, as mentioned

before, the concept of mentoring is still entirely alien, due to the specific Belgian

circumstances outlined above. Thus, making it easy and straightforward for students


!15

to engage with a mentor is imperative. Through online questionnaires and an analysis

of students’ reflective writing pertaining to writing mentoring, students indicated

what barriers to mentoring they perceived. A significant barrier for many students

was the e-mailing back and forth with a mentor to establish where the mentoring

session would take place (since no dedicated room was available at this time). The

fact that mentors were, to most students, strangers, did not help this process. As a

result, student engagement with the programme was relatively low.

This situation improved when the request the programme receive a dedicated

room was granted. This change has seen significant increases in students’

engagement with the programme (for example, resulting in eighteen students

scheduling a voluntary mentoring session in November 2015, compared to two in

November 2014 (representing, percentually speaking, a 900% increase)). In addition,

a dedicated room has allowed mentoring sessions to be made mandatory for the final

ETTV writing assignment. Requiring that students see a mentor would otherwise be

impossible due to the logistical issues with arranging for locations for over 200

mentoring sessions.

At the end of AR Cycle 2, the programme had seen some substantial

improvements in its workings, which would increase student engagement with peer

mentoring during the next academic year, in which research was conducted as part of

the third AR cycle, which will be discussed below.


!16

III AR Cycle 3: Evaluating the Efficacy of the Writing Mentor

Programme

Having summarised the research conducted during the first two AR Cycles of the

programme, the following section will transition to the analysis of the third (and

current) AR Cycle, which is this paper’s focus. The focus of this paper was made

possible thanks to the programme receiving a dedicated room, the necessity of which

was argued for in the research in AR cycle 2. Because of this room, mentoring

sessions could be made mandatory. This allowed for enough data to be gathered to

commit to a comparative analysis of the pre- and post-mentoring texts written by

students, which will be used to establish if (and, if so, how) their text had changed or

improved. Before commencing the discussion on this analysis, for the reader’s

convenience, the relevant literature will be presented first, to orient the reader

towards the state-of-the art concerning the evaluation of writing mentoring

programmes and writing centres.

1 Literature Overview: Evaluating Writing Mentor Programmes

As mentioned above, writing mentoring programmes, and by extent writing centres,

have been a staple of US education for several decades, which is now also slowly

finding its way into European universities. These programmes have several decades

of experience evaluating their practices and responding to changing contexts. Such

evaluations, traditionally, have been carried out from a reflective viewpoint, mainly

taking into consideration the writer’s development process, rather than looking at the

improvement of a given text (for example, Harrington et al. 2007; Archer 2008

(partially); Bromley et al. 2015). This process is analysed through analyses of the

AR Cycle 3 - Overview Thijs Gillioen

!17

writers’ (and, occasionally, the mentors’) reflections upon their own development

and, alternately, their (perceived) self-efficacy (for example, Schmidt & Alexander

2012).

This form of evaluation reflects the working practice of most writing

mentoring programmes (including the mentoring programme at UGent): the

“minimalist” non-directive approach. This approach centres on the idea that

“students write to learn, not to make perfect papers” (Brooks 1991: 3 [emphasis

added]). As such, the student is made the “primary agent”, who takes “full

responsibility” (Brooks 1991: 2) for the paper. The mentor, through this approach, is

not relegated to the position of being an editor, nor can they be held responsible for

making the text better. The mentor only serves as a ‘guide’ who helps the student

“focus[…] on his own writing” (Brooks 1991: 2). This is done by asking the student

questions about their own writing, thus making students reflect on their writing

choices. For example, in a paper that does not properly link components of its

argument together, a directive way to solve this problem would simply be to say “this

is not linked, here is how to solve it”. However, as Brooks argues, this does not bring

a student “any closer to his own paper than he was when he walked in” (1991: 2). The

non-directive alternative is to ask the student “how are these concepts linked?”, thus

provoking the student to reflect not only on the content of their essay, but also their

writing. For mentors of the UGent writing mentoring programme, who are still

students themselves, this also offers the additional benefit of shielding mentors from

leading students in the ‘wrong’ direction: the mentor only asks questions that help

the students develop as a writer, and does not directly engage with the construction

or the content of the text.


!18

However, despite the non-directive approach having been a staple of writing

centres for several decades, research has questioned if an exclusively non-directive

mentoring relationship is the optimal way to ensure a student’s development as a

writer. Some research, such as that by Kullman, has called for flexibility in using

directive or non-directive mentoring, in accordance with “contextual

variables” (1998: 482). Similarly, Harbord argued that the emphasis on non-directive

mentoring is the symptom of “accommodation to traditional teaching, and fails to

take into account other views on teaching” (2003: 1), further specifying its

“shortcomings make it inappropriate for a European context” (Harbord 2003: 2).

This European context, Harbord argues, is different from the American context in

that European assignments tend to require longer and more academic essays (2003:

3), the goal of which is in fact to make a perfect paper. Further countering Brooks’

argument, Harbord gives the example of a student’s thesis: for many students, this is

the most elaborate (academic) writing they will ever undertake, thus making Brooks’

“write to learn” argument highly debatable (Harbord 2003: 3).

Harbord’s argument against an exclusively non-directive approach is

supported by more recent research conducted by Crasborn et al. which indicated the

most effective mentoring relationship is one that is “flexible in using the four

mentoring roles in mentoring dialogues” (2011: 328). These roles are defined as the

“Initiator”, the “Encourager”, the “Imperator” and the “Advisor” (Crasborn et al.

2011; 322); the former two use non-directive skills, whereas the latter two use

directive skills. They add “that it is important to strive for a successful match between

a mentor’s […] supervisory approach and a student’s […] learning needs during the

mentoring process” (Crasborn et al. 2011: 330), again demonstrating there is “no

single approach to mentoring that will work in the same way for every student […] in


!19

every context” (Crasborn et al. 2011: 330). In this sense, Kullman’s, Harbord’s, and

Crasborn et al’s arguments against non-directive mentoring serve as an indication of

why the traditional method of evaluating writing mentoring programmes exclusively

through reflection on writer development , is increasingly considered inadequate. 5

This sense of self-reflection being an insufficient indicator for programme

efficacy is shared by a number of other researchers. As noted by Pleasant, the field of

research into writing mentoring programmes and writing centres is “edging toward

becoming the kind of discipline that does true social science research” (2015: 11),

which implies one cannot keep “believ[ing] that writing centers do good work simply

by relying on testimonials and anecdotes” (2015: 10). Among his specific advices for

researchers is to “pick an observable, measurable outcome” (Pleasant 2015: 10).

Likewise, Archer notes the traditional approach gathers information on “students’

perceptions of the consultation, rather than on their actual writing”, adding “that a

students’ perception of improvement may not necessarily translate into demonstrably

improved writing” (2011: 133). Furthermore, she argued “student responses were

often thin and did not allow for in-depth data analysis” (Archer 2011: 133). Although

Archer still examines both students’ and consultants’ (i.e. mentors’) comments, she

also takes into account other factors such as the actual writing and the grades

obtained. Finally, an additional argument for going beyond the traditional evaluation

method is noted by Cho & MacArthur, who pose that “many questions about the

nature of peer feedback and its effects on writing remain unanswered” (2010: 329).

This dissertation does not aim to reconcile the differences outlined above

regarding how to accurately assess the efficacy of writing mentoring programmes, nor

Since the mentoring programme aims to develop students with different backgrounds and 5

(writing) skill-sets into (reasonably) good writers, this dissertation uses “writer development” (Badley (2008), quoted in Haas (2009)) in lieu of the more standard (in this field) “writing development”.


!20

does it elect to argue which side has more merit: in this author’s view, both

approaches, reflection and empirical, bring insights to the table that cannot be easily

brushed aside. As such, to assess the efficacy of the UGent writing mentoring

programme, this paper will evaluate the programme through both perspectives.

2 Evaluating the Current Programme: Two Perspectives

To evaluate from the traditional perspective of mentee reflection and perception, in

addition to the called-for perspective of a text-based analysis, the current research

poses the following two approaches through with the (workings of the) mentoring

programme will be evaluated, and through which suggestions for programme

improvements will be made.

Firstly, keeping with Pleasant’s call for an “observable, measurable outcome”

that goes beyond “testimonials” (2015: 10), this dissertation will commit to a text

analysis of the pre-mentoring version and the post-mentoring version of fifteen

different student essays. Both versions were analysed and rated using Haas’

framework for “Understanding Reader Engagement” (2016). From this, this paper

hopes to establish if, and (if so) how, students’ texts are improved as a result of

attending a session with a peer mentor. Determining the areas that have been most

improved can provide interesting data, but especially areas that have seen average to

little improvement will be of interest for the purposes of evaluating and improving

the mentoring programme’s effectiveness.

Secondly, keeping with the more traditional method of evaluating writing

mentoring programmes, students’ reflective writing was analysed to determine how

students perceive the writing mentoring programme. Although textual change may

have been noted in students’ post-mentoring essays, the process of becoming a writer


!21

is predominantly a cognitive one: therefore, establishing how students have

developed as writers as a result of the mentoring process is vital to the efficacy of the

programme.

Having given this overview of the aims of the current research, this paper

turns to the first question outlined above, which seeks to confirm the hypothesis that

student writing is indeed improved by writing mentoring, and, if so, aims to discover

in which areas mentoring offers improvement. These data points will be used to

provide recommendations to further improve the programme and its practices.


!22

3 Perspective 1: Analysis of Pre- and Post-Mentoring Texts

3.1 Data Collection: Student Papers

The papers collected as data for a text analysis of pre-and post-mentoring texts were

written as the third and final assignment for the ETTVI course, which is compulsory

for all students wishing to attain a BA in English at UGent. Students had previously

received over around thirteen and a half hours of class time instruction on writing,

and were given the following assignment:

[…] to write a short, engaging non-fiction essay. Your essay does not

need to be academic in topic, tone or style, but you must have an

explicit focus that is narrow enough to be appropriate for an essay of

500 words (+/-10%).

(ETTVI: Writer Development Assignment 3 (Appendix 1, p. 111) ) 6

This final assignment was further narrowed down by specifying the students’

task was to make “the reader want to start reading [their] essay” (S3 assignment,

Appendix 1, p. 111) and to “make sure the reader wants to keep reading your essay all

the way to the end” (S3 assignment, Appendix 1, p.111). These requirements

correspond with the “Initial Engagement” and “Continuing Engagement” sections of

Haas’ framework, which will be discussed below, in section 3.2.1.

One of the mandatory components of this assignment was for the students to

attend a mentoring session with one of the twenty writing mentors. They were

required to send in a draft of their assignment before coming to the session, so the

mentor could prepare. This draft is used as the “pre-mentoring” text for the purposes

of this dissertation. Based on the guidance given by the mentor, students could then

choose to adapt their text before they handed it in. The texts submitted to the teacher

Hereafter referred to as “S3 assignment, Appendix 1”.6

AR Cycle 3 - Student Essays Thijs Gillioen

!23

serve as the post-mentoring data point. Since no classes or other writing activities

were organised between the mandatory mentoring sessions and the deadline of the

writing assignment, this paper argues it is fair to assume the mentoring session was

responsible for the majority of changes made between the pre-mentoring text and the

final, post-mentoring essay.

In total, 175 students submitted an essay for the assignment. Out of this total,

fifteen essays were selected for analysis, resulting in a corpus of thirty texts (each

essay having both a pre-mentoring and post-mentoring version). These fifteen texts

were chosen based on the fact all authors of these texts had been to a mentoring

session with the same writing mentor (who is the author of this dissertation). This

was done to ensure potential differences in personal approach and mentoring style

between mentors can be eliminated as a cause for adjustments made to texts after the

mentoring session. All authors granted written permission for their texts to be

analysed for the purposes of academic research. This permission was reinforced via

an e-mail reaffirming they were comfortable with their texts being used for the

purposes of this specific dissertation.

3.2 Text Analysis Framework: “Reader Engagement” (Haas 2016)

3.2.1 Overview of Engagement Framework

If an analysis of students’ writing is to be done objectively and consistently, a

framework is needed that not only offers the tools to do so, but that is applicable to

the students’ specific writing instruction and setting. For these reasons, this paper

will use the “Framework for Teaching and Assessing Writing” developed by Haas

(2016). This framework has been developed and has been evolving over the course of

a seven year period, during five of which the developer has been a teacher of writing


!24

at UGent. There, the framework has been used to teach and assess students’ writing

in the ETTV courses. As such, it has a history of practical application in an academic

setting. Using this framework to rate the pre- and post-mentoring versions of student

texts, this paper seeks to establish whether or not the programme is effective at

improving students’ writing. Before going into this analysis, the following section will

offer a brief overview of Haas’ framework.

In her framework, Haas starts her assessment of student writing from the

perspective of “Reader Engagement” (2016: 25). Reader engagement is broken down

into two major components: “Initial Engagement” (i.e. what allows for a reader want

“to start reading a text” (Haas 2016: 32)), which deals with elements like formatting,

title and presentation (Haas 2016: 34-36), and “Continuing Engagement” (i.e. what

ensures “it [is] easy for a reader to keep reading, or at least not [causing] a reader to

want to stop” (Haas 2016: 37)), which argues elements like coherence and cohesion

are essential. Focus (i.e. what is the underlying ‘red thread’ throughout the text that

connects everything for the reader) is additionally identified as a major contributor to

continuing reader engagement (Haas 2016: 52).

Throughout the framework, Haas uses the distinction first defined by Bloom et

al. (1956) between Higher Order Cognitions (hereafter referred to as “HOCs”) and

Lower Order Cognitions (hereafter referred to as “LOCs”). Higher Order Cognitions

are issues appearing in (academic) writing (such as global text structure) that are

cognitively hard(er) to resolve: these HOCs are, as a result, the focus of the writing

course, since they are fundamental to writing a good text. LOCs, on the other hand,

are issues such as faulty formatting, or spelling and/or grammar mistakes (Bloom et

al. 1956; Ferris 2016: 224-225). Since LOCs are cognitively easy (or, at least easier

when compared to HOCs) to deal with, it is expected that students get them right


!25

without extensive instruction (again, as opposed to HOCs, which do require extensive

instruction). Together, HOCs and LOCs form the “building blocks of good

writing” (Haas 2016: 25). An alphabetical list of these building blocks is provided in

Figure 02, below. An overview of these issues, categorised into Initial and Continuing

Engagement, is provided below that in Table 01. A full list of definitions is beyond the

scope of this paper, but is provided in Haas’ framework (2016: 32-58).


!26


Figure 02. Alphabetical List of “Building Blocks of Engaging Text” (Haas 2016: 26)

!27


Table 01. HOCs and LOCs in Reader-Friendly Writing (Haas 2016: 27)

LOCs HOCsGeneral

(components of initial, continuing and advanced engagement)

Initial Engagement Get the reader to want

to start reading

Or at least cultivate the good will of the

reader

Publishing Presentation Formatting Deadlines

Title

Mindset Writer is Responsible Reader is not obligated to read

Care About your reader About your writing

Responsibilities Engage you reader

Adjust To writing Situation Know the conventions Adhere to conventions

Continuing Engagement

Ensure the reader wants to keep reading

Or at least make it easy for the reader to keep reading (avoid making the reader

want to stop reading)

Grammar Spelling Punctuation Typing Formatting

Story Complete Linked Balanced

Proportion Promises kept

Self-Contained

Flow Coherence

Order Given/New No Gaps

Information Logic

Cohesion Linking Transitions Signalling

Relevance Concise Precision Consistent Construction

Sentences Paragraphs Entire text

Focus Narrow Precise Consistent

Advanced Engagement

Capture the reader

Something extra done for visual appeal or for otherwise making the reading effortless for a reader

Advanced ideas Advanced language use Advanced style

!28

3.2.2 Developing a Rating Scale Based on the Framework

Haas’ framework has been used (and refined) over the course of several years to teach

students how to write engaging texts (with the long-term view of transitioning into

academic writing), and has been used reliably, without major problems, by four staff

members. However, it did not have an explicit rating scale that was useable and

accessible to raters who were unfamiliar with the framework. As such, a first step in

this research was to devise a consistent and easily applicable rating scale. Because

designing a rating scale is an involved process, the designers decided to use this

opportunity to create a scale that would have applicability not only for this

dissertation (where it will be used to analyse 30 essays), but also for the purposes of

marking large-scale assignments (for example, the 250+ essays turned in for each

assignment of the ETTVI course). Rating scale development is an iterative process

(Banerjee et al. 2015), and, as such, a number of pilot tests were conducted to

construct and refine the rating scale used here.

The pilot tests were conducted with three raters: Dr. Sarah Haas, who

developed the framework; the author of this dissertation, who would be using the

framework to mark students’ essays; and an undergraduate student who will be

employing the same framework to analyse NPO newspapers for their own Bachelor

research. For the first pilot test, the three raters started with a randomly selected

student essay that was not part of this dissertation’s corpus, but that was made under

the guidelines of the same assignment.


!29

3.2.2.1 Pilot Test I: Devising an Initial Rating Scale

The starting point of the rating scale was provided by the marking rubric for writing

assignment 3, which is based on Haas’ framework and defines a well-written text as a

text that has no HOC problems and no LOC issues whatsoever (Appendix 2, p. 117).

As such, a text that has no LOC or HOC problems was given a passing 10 out of 20

mark. This sets a high yet achievable mark for students learning writing. Since the

framework begins from a reader engagement standpoint, only a text that achieves this

minimum level of writing can be assessed additional marks for elements of

“Advanced Engagement” (Haas 2016: 27; cf. Table 01), such as a nice style. The

reason for withholding advanced engagement marks until the minimum

requirements are met is that texts with LOC or HOC problems risk disengaging a

reader before such content-related boons can be enjoyed. Because of this, texts that

do not first thoroughly attend to LOCs and HOCs do not merit credit for elements of

advanced engagement.

Essays that do not manage to keep a reader engaged throughout the entirety of

the text will therefore be given a mark lower than 10. The raters, after discussing the

pilot essay, agreed upon the following marking system. Each HOC issue in a given

text will cost a tenth of a mark (-0.1), whilst LOC issue is penalised by two tenths of a

mark (-0.2). Penalising LOCs harsher than HOCs is explained by the nature of LOCs:

since LOC issues are mostly related to formatting mistakes, which are not cognitively

hard to fix and are usually easy to pinpoint (thus indicating “the writer does not

particularly care about the reader” (Haas 2016: 34) if not properly attended to), any

LOC issue in a final text should be penalised harder than a HOC problem, which is

cognitively harder to pinpoint and fix.


!30

3.2.2.2 Pilot Test II: Refining the Rating Scale

Based upon this initial rating system, a second round of pilot testing was conducted

by the three raters. In this round, four student essays, which were also made as part

of the same assignment (but are not in this dissertation’s corpus) were marked. As a

result of this, the raters agreed some further changes had to be made in order to

ensure the grades reflected their engagement with the essay correctly. Changes that

were made for these reasons include the introduction of L-LOCs (Lower-Lower Order

Cognitions), which will be discussed below, and the decision to penalise each instance

of a specific HOC (instead of only penalising a specific HOC once per text,

irrespective of its number of appearances), which will also be discussed in what

follows.

LOCs vs. L-LOCs: Temporarily De-Emphasising Spelling and Grammar

A point of contention when using this framework for analysing student texts was its

approach to issues related to spelling and/or grammar. Based on years of writing

(analysis) expertise, the framework developer does not believe the number of spelling

and/or grammar mistakes in a text, or alternately perfect spelling and/or grammar,

to be a sufficient metric to define ‘a good text’. In addition, the researcher wants to

move towards a focus on textual issues, thus temporarily de-emphasising spelling and

grammar in favour of HOCs, rather than simply focusing on grammar and spelling, as

students are used to from secondary school. This de-emphasis works to the benefit of

feedback givers as well: de-emphasising spelling and/or grammar issues in feedback

allows for more time and attention spent discussing Higher Order Cognition

problems. Furthermore, marking 250+ student essays limits the feasibility of

marking every single one of these mistakes successfully, when aiming to keep the


!31

time between the due date of the essay and the moment of feedback as low as

possible. As such, the framework developer argued a certain number of grammar

and/or spelling mistakes should be given a pass, as long as they “would not annoy a

reader or impede understanding of the text”.

The other raters argued for a system that keeps the importance of the dangers

of emphasising spelling and/or grammar in mind, but that does also hold students

accountable for these lower-order mistakes and allows for a more objective metric

than “annoyance” for the reader. Solely focussing on reader annoyance allows for

subjective metrics to be implemented (and thus for variation in text scores) Indeed,

there was some disagreement amongst the raters as to how many, and what kind of

such mistakes caused annoyance. On top of this, the raters felt spelling and grammar

did still require some emphasis, given the context: the students assessed by the

framework are L2 learners of English and, by the end of their tenure at UGent, are

expected to attain a C1 level of English, which does not allow for these mistakes

(Council of Europe 2001).

A compromise between these positions was reached by taking spelling and/or

grammar issues into account as a new category, labelled “Lower-Lower Order

Cognitions” or L-LOCs (De Soete 2016; Haas 2016) . This categorisation expressed 7

the relatively low (textual) importance and low cognitive burden of these issues, but

simultaneously allows for them to be included in the rating scale and to be given the

proper amount of emphasis. Distinguishing L-LOCs from the regular Lower Order

Cognitions (LOCs - which still have a low cognitive burden) allowed for them to be

To reflect that this term is the result of a discussion between the three raters (as opposed to 7

being one individual’s terminology), this dissertation cites both Haas (2016) and De Soete (2016) for this term: it is used in each of the raters’ papers.


!32

marked differently, and for the emphasis of ‘regular’ LOCs to remain on textual issues

rather than language issues.

Finally, to adhere to the researcher’s concern of marking 250+ essays for

language issues and to objectify what level of language issues can result in the reader

being disengaged, a set of “buffers” was decided upon that allows for a rater to both

de-emphasise these issues, but also penalise excessive amounts of such language

issues. The first buffer was set at one L-LOC issue per one hundred words: this

reflected the point at which the raters agreed the L-LOC issues started to pose a

problem for their engagement with the text. For the purposes of the 500-word essays

analysed as part of this research, this implies these essays were not penalised for the

first five L-LOC issues encountered during the analysis. After the first five L-LOC

issues, essays lost one and a half mark (-1.5). The second buffer was set between five

and ten L-LOC problems: the raters agreed that essays with this number of L-LOC

issues started to threaten reader engagement. Any essay that broke this second

buffer, i.e. in which more than ten L-LOC problems were encountered, were

penalised by three full marks (-3): this was agreed upon by the raters as being

reflective of the fact that such essays cause complete disengagement for most readers.

Thus, these buffers were agreed upon by the raters as representative of the “levels of

annoyance” that would be created for a reader when reading essays with these levels

of L-LOC issues. Although this system of buffers still reflects some personal

preference (i.e. other readers may disagree more than five L-LOCs cause

disengagement), this system allowed for a consistent rating scale to be used for the

current research; potential future research could focus on a large-scale investigation

of the effect of L-LOC frequency on reader disengagement.


!33

Marking Each Instance of a HOC

A second point of discussion that arose during the second pilot test was how multiple

instances of a specific HOC should be approached. A certain text might, for example,

have several instances where “new” information is treated as “given” (an issue

defined as a “Given/New” HOC problem (Haas 2014; Haas 2016: 43)). The point of

discussion here pertains once more to marking: does one penalise an essay for each

instance of Given/New, or only once for each different HOC? For example, if an essay

has three Given/New problems, each appearing in a different section, does one mark

this problem once, or every time it occurs?

Like the discussion above, this point relates mostly to practicality: in longer

essays (or when facing large volumes of shorter essays), one can question if it is

feasible to start counting every single HOC to score an essay. On top of that, when

multiple instances of the same HOC appear in a single essay, one could argue each of

these instances has the same ‘cognitive’ issue underlying it. As such, one would be

penalising the same issue multiple times.

Despite these arguments, the raters agreed to mark each instance of each

HOC. Their reasons were as follows: firstly, although it is indeed arguable that

multiple instances of a given HOC problem (such as Given/New) can have the same

underlying cognitive issue, this issue does manifest itself (textually) in different ways,

resulting in variations on HOC issues that are significant enough for each to merit its

own mark. Secondly, since Haas’ framework considers ease of reading as its starting

point, it would be disingenuous not to mark each HOC problem separately, since each

HOC problem does fundamentally degrade the reading experience for the reader.

Finally, Haas’ framework does not allow for raters to internally ‘grade’ a single HOC

category (i.e., one cannot mark a certain Given/New issue as being a ‘worse’ HOC


!34

than another Given/New problem). As such, a rater cannot determine which instance

of a HOC in a given text should be marked as ‘the most important’ or as being ‘the

biggest’. For these reasons, for the purposes of assessing disengagement in students’

texts, each HOC problem in students’ essays has been marked and pointed out as

such. A text that has three Given/New issues, for example, would lose 0.3 marks.

Marking Each HOC Equally

Linked to this discussion is the question whether or not each HOC should be

considered equal when marking essays. In its current form, the rating scale assigns

each HOC problem an ‘equal weight’: in other words, each HOC issue is penalised by

the same number (one tenth of a mark).

However, one might argue that this is an oversimplification of the

intricateness of writing an (academic) essay. Consider, for example, an essay that

lacks a sufficiently narrow Focus. In such an essay, it is easy for a reader to lose track

of the focus at multiple points throughout the text. Contrast this with an essay that

has a properly narrow focus, but at one point in the text lacks a clear link to this

Focus. To most readers, the second text would be more engaging than the first text.

However, the rating scale, in its current form, does not reflect this: both essays would

lose one tenth of a mark, since the HOC problems in these hypothetical essays, whilst

different (and thus having different implications), are graded as being equal.

The raters developing the rating scale spent some time discussing this. For the

sake of simplicity, and once again considering the implications of counting multiple

differently weighted issues in large volumes of essays, they decided against using

different weights for different HOC issues at this point, although they agreed to


!35

review this issue after the essays in the corpus of this dissertation had been marked,

potentially changing it for the next academic year (and the next AR Cycle).

Marking Essays That Do not Meet “General Responsibilities”

A final issue that was noted was related to essays that do not meet the general

responsibilities outlined in the assignment brief for Scribing 3 (available in Appendix

1). One of these responsibilities, for example, is to write an essay that has “an explicit

focus that is narrow enough to be appropriate for an essay of 500 words” (S3

Assignment, Appendix 1, p. 111). An essay that is, for example, not focussed around a

single point, but rather tells an anecdote or a story in a chronological fashion, does

not meet these general requirements. This problem is comparable to students writing

a good response an an essay exam to some question, that is, however, not answering

the question posed by the exam.

An issue when marking essays that do not meet the general requirements rose

when applying the arguments which have argued in favour of analysing each HOC

problem equally (cf. above). An essay that, for example, does not meet the

requirements by being chronological, but is otherwise free of any HOC issues, would

score a 9.9 in the current rating scale (losing 0.1 for its lack of a Focus). The same

essay, that was in fact focussed, might score less due to having several Link to Focus

issues, for example. However, the raters felt that this rating solution was not

accurately representing reader engagement: despite the issues the second essay

would have, it might be more engaging due to it having an explicit Focus, which the

first essay lacked. Additionally, it would simply not be fair to grant the first essay a

higher mark: if there is no Focus, there can be no “Off-Focus” or “Link to Focus

Missing” issues, thus reducing the complexity of this essay compared to the second


!36

one. On top of this, the raters felt students who did not fulfil the minimum

instructions for an assignment, thus not practising what they were supposed to be

practising, should be penalised more strictly for not living up to their writer’s

responsibilities.

As such, a new rule was introduced into the rating scheme to reflect these

concerns. Essays that do not meet the assignment requirements (see Appendix 2, p.

117) for any given assignment would not be marked starting at 10 (out of 20), but

rather at 8.5. This score was settled upon both as a realistic representation of the

annoyance caused for a reader by a text that does not follow all instructions, and as a

suitable reflection of the lack of proper practise. In addition, this prevents students

from getting additional marks for good style or developing good ideas, which

underscores the point made in Haas’ framework about good writing being essential to

developing a good text.

3.3 Analysing Student Essays with the Rating Scale

Having successfully developed and agreed upon a rating scale corresponding with

Haas’ framework, the student essays gathered as data for this dissertation were

analysed for L-LOC, LOC and HOC issues. Two steps were taken to help ensure

objectivity and to avoid personal preferences and potential issues of bias. Firstly, on

each essay, the name of the student was crossed out. Even though the identities of the

authors were known to two of the raters, a sufficient amount of time had passed for

this to arguably no longer present any bias issues. Secondly, to further avoid any

personal bias, and to eliminate any remaining element of personal preference when

marking essays, each essay, in both its pre-mentoring and post-mentoring form, was

marked by the same three raters who developed the rating scale discussed above.


!37

Although, as noted by Saliu, “it is often argued that qualitative assessment is

subjective, imprecise and biased” (2005: 271), the varying backgrounds of the raters,

their differing levels of prior marking experience and their familiarity with the

framework and its marking system ensure a sufficiently diverse team of raters to

achieve a balance of viewpoints (Park 2015).

Because the current research focuses on finding changes in text quality

between pre- and post-mentoring essays, rather than objectifying the reliability of the

rating scale, the inter-rater reliability test conducted between the three raters was

concerned with establishing if each rater identified the same L-LOC, LOC and HOC

issues in each text. Therefore, the raters each independently analysed the text for

places of disengagement, and how these could be identified using Haas’ framework.

After this analysis, the raters compared notes and found they had, in nearly all cases,

identified the same issues, thus demonstrating a high level of inter-rater reliability (Li

Gwet 2014). Following this, the rating scale was used to mutually agree upon a mark

that was representative of text quality. Since each rater had identified the same

(number of) issues, agreeing upon a final score for each essay did not present any

significant discussions. This system allowed for a maximum focus on text issues

during the rating process. The result of this process, the analysed essays, are available

for consultation in Appendix 3. The results of this analysis are presented in the next

section.

3.4 Overview of Results

Table 02 provides an overview of the results of the data analysis of the student essays,

by listing the number of L-LOC, LOC and HOC problems encountered in each pre-

and post-mentoring essay, along with the mark agreed upon for each essay by the


!38

raters. Table 03 then elaborates on HOC issues specifically, detailing which and how

many HOC problems were encountered in each essay. A more extensive discussion of

the data provided in both tables is provided in the next section.


!39


Table 02. Overview of Marks, L-LOCs/LOCs/HOCs in Pre- and Post-Mentoring Essays

(*) Indicates an essay did not meet the general requirements of the assignment (for example, by having a chronology-based text rather than a focus-based text - cf. Appendix 2). These essays could receive a maximum score of 8.5. (**) Scores higher than 10 are to be attributed to style, development of ideas and other forms of ‘advanced engagement’ (cf. Appendix 2 and Table 01). Since mentors focus on issues of structure, scores higher than 10 are not reflective of the efficacy of mentoring. They have, therefore, been represented as "10(+)". (***) These averages were rounded up or down to the full decimal, to reflect the fact that “half a(n) L-LOC/LOC/HOC” does not exist in Haas’ framework.

!40


Note: the category “Complex Issues” was not included in the marking scheme, represented above in Table 02. This will be explained below; here, this explains why Table 02 had a total of 102 (pre) and 57 (post) HOC issues, versus Table 03’s 110 (pre) and 63 (post) HOC issues.

Table 03. Overview of HOCs in Each Essay, per HOC Category

!41

3.5 Discussion of Results

3.5.1 General Remarks

This analysis confirms the initial hypothesis brought forth by this paper (that

mentoring can guide students towards improving their written products), and the

existing literature on writing mentoring (which does not usually assess text changes,

but does indicate mentoring can help students): all analysed student essays displayed

improvements in their final version. On average, a student essay saw an improvement

of 27.29%, which is far enough above the statistic threshold of 5% to be considered

statistically relevant. Individual improvement rates vary from a low of 1.03% (for an

essay that already achieved a high score in its draft form) to a high of 81.48% (for an

essay that displayed a high number of both L-LOC and HOC problems). On top of

this, the essays that did not meet the minimum requirements in their pre-mentoring

form managed to amend this after the mentoring session(with essay 11 being the one

exception), an improvement in text quality that is arguably larger than indicated by

the numbers present. Finally, it is interesting to note that essays which do not have

any HOC problems (such as essay 15) manage to maintain this in their post-

mentoring form. Although seemingly evident, this indicates mentors are equally

capable of spotting a good essay (and thus withholding any non-directive support

that might hamper the essay) as they are of spotting issues in essays. Having given

this general overview, the next paragraphs will present a general overview of each

marking category and their relevant numbers.

Lower Order Cognition issues, in particular, are much less present in post-

mentoring essays. Most of these essays are completely devoid of LOC problems, while

those that are not are limited to a single LOC issue. For the former group, this means

a decrease of LOC problems by 100%; for the latter, it means an improvement of


!42

66.67%. This high improvement rate is explainable by the nature of LOCs: these

Lower Order Cognitions, as defined above, do not require significant work to find or

fix. Therefore, it is relatively ‘easy’ for a mentor to guide their students towards fixing

these issues. As such, it is relatively unsurprising that this category sees the highest

improvement rates.

The category of Lower-Lower Order Cognition problems sees the second-

highest improvement rate, at 42.86%. Mentors, as guides along the path of a writer’s

development, do not serve as copywriters, editors or spelling checkers (Harbord

2003): they generally only point out that ‘language issues are present’, without

further specification. This explains why these spelling- and grammar issues continue

to appear in students’ final texts. Furthermore, as students rewrite (sections of) their

essays based on their session with the mentor, new issues may arise even if old ones

were fixed. This is the case, in, for example, essay 02, which has one additional L-

LOC problem in the final version of the essay: this L-LOC problem appears in a

section that was rewritten after the student’s session with a mentor.

Finally, the category of HOCs sees a rate of improvement that is equal to that

of the L-LOC category, with an average of 42.86% less HOCs per essay. This

demonstrates two points: firstly, how these Higher Order Cognitions are, cognitively,

the hardest ones to fix, even with the guiding help of a mentor. Therefore, the fact

that HOCs remain somewhat frequent in most of these essays is to be explained by

their cognitively hard nature, which underlines the importance of Kellogg’s demand

for “repeated opportunities for practise and feedback” (Kellogg & Raulerson III 2007:

237 [emphasis added]): it would be too simplistic to think any mentor could help a

writer resolve all issues in a cognitive writing process during one short, 30-minute

session. Research in this field has indicated as much: Harbord, for example, notes


!43

such papers “are not written in a single sitting” (2003: 4), further adding that “the

mentor’s role is not to edit, but to […] help the student master conventions and

structure” (Harbord 2003: 4). Similarly, Irvin noted three tutorial sessions (in the

context of this paper, mentoring sessions) to be a “significant threshold” (2014: 2),

whereas these students were only given one. This, combined with Kellogg’s claim that

the “development of writing skills […] requires decades of learning” (2008: 20),

makes the improvement rate of 27.29% rather significant.

One other important note regarding the improvement rate of HOC issues is

that this ‘apparent’ number of HOCs that are retained in the post-mentoring version

of students’ essays is actually not entirely representative of the effects of a mentoring

session. Firstly, mentors only ‘guide’ students, non-directively: it is up to the students

to follow the mentor’s guidance and thus to actually eliminate these issues in their

essays. As such, one has to be wary of implying a too direct, causal link between the

mentoring session and the amount of HOCs fixed. Secondly, the ‘apparent’ number of

HOC issues that were solved is actually lower than the ‘actual’ number when one

takes in to consideration the effect of rewriting (sections of) an essay, as noted above

when discussing L-LOC issues. A student might rewrite one or more sentences or

sections of their essay as a result of the mentoring session, thus solving one problem,

but inadvertently creating a new HOC issue in the process. For example, essay twelve

had thirteen HOC problems in its pre-mentoring form; Table 02 indicates it had four

remaining issues in its post-mentoring form. However, in reality, none of the initial

thirteen issues remained: the four ‘apparently retained’ HOC issues found in the post-

mentoring essays were all results of the rewriting process. This is discussed in more

detail in the next section, which deals with a number of specific data trends.


!44

3.5.2 Specific Areas of Discussion: Data Trends

3.5.2.1 ‘Apparent’ Number of HOCs Retained vs. ‘Actual’ Number of HOCs retained

As noted above, there is a difference between the ‘apparent’ number of HOC

problems that remain in the text after a mentoring session, and the ‘actual’ number.

When looking at Tables 02 and 03, one might assume that, in essay 3, for example,

which had twelve HOC problems before the mentoring session, and five HOC

problems in the post-mentoring text, five HOC issues were either not spotted by the

mentor, or not solved by the student after the mentoring session. However, this does

not take into account the reality of the writing process: to fix one issue, a writer often

has to rewrite large sections of a their text, or shift around existing parts of a draft

into new positions. During this process, whilst previous HOC issues might have been

fixed, the writer may have inadvertently created new issues. As such, the ‘actual’

number of retained HOC issues is lower than what the previous two tables indicate:

these tables assume that no such rewriting process took place. To demonstrate the

difference between the ‘apparent’ change (which includes HOC problems that were

created after the mentoring session) and the ‘actual’ change (which does not include

these post-mentoring HOC problems), Table 04 provides an overview of the HOC

issues in each essay (in both pre- and post-mentoring state). HOC issues that did not

change at all (textually) in between the different versions of an essay were underlined

and italicised. These are the ‘actual’ retained HOC issues: the issues that were present

in the pre-mentoring text, and remained in the post-mentoring text.


!45


Overview of “Higher Order Cognition” Issues in Student Essays: HOC Issues that remain, unchanged, represented in italics and underlined.

Essay Pre-Mentoring HOC Issue(s) (with paragraph n°)

Post-Mentoring HOC Issue(s) (with paragraph n°)

Number of Unchanged

HOC Problems

‘Actual’ HOCs retained/solved (in %)

(vs. ‘Apparent’ %)

1 Conciseness Problem (P1) Relevance Problem (P1) Focus too Wide (P1) Conciseness Problem (P2) Relevancy Problem (P2) Link to Focus Missing (P2) Construction Problem (P2/3) Link to Focus Missing (P3) Link to Focus Missing (P5) Transition Problem (P5/6) Link to Focus Missing (P6) Balance Problem (P7)

Gap in Information (P1) Focus too Wide (P1) Link to Focus Missing (P3(*)) Link to Focus Missing (P4) Link to Focus Missing (P5)

(*) Corresponds to P4 in pre-mentoring Essay.

216.67% / 83.33%

(41.67% / 58.33%)

2 Balance (P1) Conciseness (P1) Broken Promise (P1) Double Focus (P1)

Balance (P1) Conciseness (P1) Double Focus (P1) 3

75% / 25%

(75% / 25%)

3 Focus too Wide (P1) Link to Focus Missing (P2) Link to Focus Missing (P3) Shift in Focus (P3) Shift in Focus (P3) Shift in Focus (P4) Conciseness Problem (P4) Off-Focus (P4) Shift in Focus (P4) Balance Problem (P5) Self-Containment Problem (P5)

Focus too Wide (P1) Shift in Focus (P2) Link to Focus Missing (P2) Shift in Focus (P5) Consistency Issue (P5)

1 8.32% / 91.67%

(45.46% / 54.54%)

4 Implicit Focus (P1 - GR (*)) Transition Problem (P3/4) Shift in Focus (P5) Relevancy Issue (P6) (*)General Responsibility Issue

Shift in Focus (P4) Relevancy Issue (P5)

250% / 50%

(50% / 50%)

5 Order Problem (P1) Precision Problem (P2) Precision Problem (P2) Order Problem (P5)

Conciseness Problem (P2) Balance (P1-3) Shift in Focus (P4) Order Problem (P5)

125% / 75%

(100% / -)

6 Focus too Wide (P1) Relevancy Problem (P4) Consistency Problem (P4)

Gap in Logic (P2) Link to Focus Missing (P2) Order Problem (P4)

0- / 100%

(100% / -)

7 Self-Containment Problem (P1) Broken Promise (P1) Focus too Wide (P1) Consistency Issue (P2) Link to Focus Missing (P2) Order Problem (P2) Gap in Information (P2) Gap in Information (P3) Link to Focus Missing (P3) Gap in Information (P3) Relevancy Problem (P4) Transition Problem (P4) Shift in Focus (P4) Balance Problem (P5)

Focus too Wide (P1) Gap in Information (P2) Relevancy Problem (P3) Link to Focus Missing (P3) Gap in Information (P3) Transition Problem (P4) Relevancy Problem (P4) Gap in Information (P5) Balance (P6)

8 57.14% / 42.86%

(64.29% / 35.71%)

Table 04. ‘Apparent’ Retained HOC Issues vs. ‘Actual’ Retained HOC Issues

!46


Essay Pre-Mentoring HOC Issue(s) (with paragraph n°)

Post-Mentoring HOC Issue(s) (with paragraph n°)

Number of Unchanged

HOC Problems

‘Actual’ HOCs retained/solved (in %)

(vs. ‘Apparent’ %)

8 Broken Promise (P1) Focus too Wide (P1) Coherence Problem (P3) Consistency Problem (P4) Balance (P5)

Broken Promise (P1) Focus too Wide (P1) Consistency Problem (P6) 1

20% / 80%

(60% / 40%)

9 Focus too Wide (P1) Shift in Focus (P3) Order Problem (P4)

Relevancy Problem (P3)0

- / 100%

(33.33% / 66.67%)

10 Order (P1) Conciseness Problem (P1) Balance (P1) Consistency Problem (P3) Relevancy Problem (P3) Gap in Information (P5) Relevancy Problem (P6)

Conciseness Problem (P1) Consistency Problem (P2 (*)) Relevancy Problem (P3)

(*) Corresponds to P3 in pre-mentoring Essay.

114. 86% / 85.14%

(42.86% / 57.14%)

11 Focus too Wide (P1) Relevancy Problem (P2) Shift in Focus (P3) Link to Focus Missing (P3) Shift in Focus (P4) Off-Focus (P5) Shift in Focus (P5) Link to Focus Missing (P5) Shift in Focus (P6)

Link to Focus Missing (P2) Shift in Focus (P3) Link to Focus Missing (P3) Conciseness Problem (P3) Conciseness Problem (P4) Link to Focus Missing (P5) Conciseness Problem (P5) Partly Chronological (P1-5(*))

(*) General Responsibility Issue

3 33.33% / 66.67%

(88.89% / 11.11%)

12 Consistency Problem (P1) Gap in Information (P2) Given / New Problem (P2) Gap in Information (P2) Gap in Information (P2) Order Problem (P2) Gap in Information (P2) Gap in Information (P3) Link to Focus Missing (P3) Link to Focus Missing (P3) Balance (P1-4) Order Problem (P4) Conciseness Problem (P4)

Given / New Problem (P1) Consistency Problem (P1) Given / New Problem (P2) Relevancy Problem (P2)

0- / 100%

(30.77% / 69.23%)

13 Broken Promise (Title) Broken Promise (P1) Gap in Information (P1) Gap in Information (P4) Relevancy Problem (P5)

Broken Promise (Title) Broken Promise (P1) Gap in Information (P4) Relevancy (P5)

480% / 20%

(80% / 20%)

14 Broken Promise (Title) Focus Unclear (P1) Balance (P1-4) Focus too Wide (P2) Shift in Focus (P3) Consistency Problem (P3) Shift in Focus (P3) Shift in Focus (P4) Consistency (P4)

Relevancy Issue (P2) Link to Focus Missing (P2) Shift in Focus (P5)

111. 11% / 88.89%

(33.33% / 66.67%)

15 - - - -

AVG 26. 65% / 67.24%

Table 04. ‘Apparent’ Retained HOC Issues vs. ‘Actual’ Retained HOC Issues (second part)

!47

Table 04 demonstrates the difference between ‘apparent’ HOCs retained and

‘actual’ HOCs retained, as outlined above. By isolating the HOC issues that remain

unchanged in between the pre- and post-mentoring version of an essay, it becomes

clear that the ‘apparent’ HOC improvement rate of 42.86%, as demonstrated in Table

02, is too pessimistic. The post-mentoring versions of the student essays analysed

here actually saw a decrease in HOC issues by 67.24%, or roughly two thirds, when

compared to their pre-mentoring state. This improvement rate then decreases once

new HOC issues are created during the rewriting process.

Of particular note, also, is the high individual improvement rate demonstrated

by the majority of these essays. Three students (essays six, nine and twelve) were able

to eliminate all HOC issues in their essay, after the session with their mentor. Five

more essays only retained a single HOC issue following the mentoring session,

displaying improvement rates of 80% and beyond. This clearly reinforces the

conclusion outlined above, that mentoring is in fact an effective way to help students

produce better texts. Furthermore, most post-mentoring essays also display a much

smaller number of (or, in some cases, no) new HOC problems when compared to

their pre-mentoring version, even when said essays underwent major rewrites

following the mentoring session. This could indicate the mentoring session not only

had a short-term impact (on the text itself), but also, possibly, on the student’s

long(er)-term development as a writer, thus helping them avoid making (some) HOC

mistakes in their (future) writing.

On the other hand, three essays (essays two, seven and thirteen) maintained

most of their original HOC issues, even if some of the original HOC issues were

solved. There are a number of possible explanations for this. Firstly, in the case of

essay two, the HOC issues present are relatively minor. Because the essay, in general,


!48

manages to maintain a relatively high level (both in terms of language and writing),

the raters were able to delve deeper into the text than was the case in most other

essays. This team of three raters may have been able to spot issues that the mentor

(in the time-constrained setting of a single mentoring session) did not spot.

Alternately, it is possible the mentor did spot these issues, but the student did not

believe them to be significant enough to warrant fixing: as part of their writing

education, students are introduced to the idea that it is the writer who makes the final

decision on whether or not to take up the advice given by feedback.

Secondly, in the case of essay seven and thirteen, the prevalence of other issues

may offer an explanation as to why several HOC issues remained unchanged. Both

essays contain high numbers of both L-LOC issues and LOC problems: essay seven

exceeds the second L-LOC “buffer” (defined in section 3.2.2.2, p. 30), indicating a

severe level of reader disengagement, in addition to a high number of LOC problems.

Similarly, essay thirteen passes the first L-LOC “buffer”, thus also indicating an

advanced level of reader disengagement, in addition to containing an average amount

of LOC problems. The combination of these elements, i.e. a high amount of both L-

LOCs and LOCs, results in a high level of disengagement for a reader. Because of this,

such essays force a mentor to spend a significant amount of time (of the mentoring

session) working with the student to discuss these issues, thus losing time that could

have been spent on the discussion of the present HOCs. Furthermore, the high level

of disengagement makes it hard for mentors to focus on such Higher Order Cognition

issues, when an essay is already filled with Lower Order Cognition problems.

Because of these reasons, it is not unfair to pose that the authors of essays

seven and thirteen could have benefitted significantly from an additional mentoring

session after having rewritten their draft. Both essays have significantly reduced L-


!49

LOC and LOC issues in their post-mentoring form: this would have allowed for both

the mentor and the student to focus on the HOC issues that were, at this stage, still

present in the essay. This is reinforced by Irvin’s argument about three mentoring

sessions being a “significant threshold” (2014: 2). The same is true for all other

essays: the remaining (and new) HOC issues could, potentially, have been solved by

an additional mentoring session.

However, it is interesting to take note of essay eight in this context. This essay

features an even higher number of L-LOC problems than the previous essays, in

addition to an average amount of LOC problems, thus, theoretically, causing an

equally high, if not higher, level of reader disengagement. Despite this, essay 8 only

retained a single HOC issue in its post-mentoring form, a decrease of 80% compared

to its pre-mentoring form. Therefore, one can question what sets this essay apart

from the essays discussed above that allowed the mentor (and the student) to

maintain their engagement to a point where HOC problems could, in fact, be

discussed. It is possible subjective elements such as personal interest in the topic, or

appreciation of the writing style (for example, the use of humour) play a role in

dampening the disengagement caused by L-LOC problems and LOC issues.

Alternately, it may be the case that, once a certain high number of L-LOC issues is

reached, these spelling and grammar issues actually become less relevant for a

reader/mentor, who may assume the writer simply has an insufficient command of

the language, rather than attributing it to sloppiness on the writer’s part. This would

allow for the mentor to set aside these issues temporarily, re-enabling them to look

for higher-level cognition issues. This specific topic warrants further study, which on

its own could be the focus of an entirely separate research paper. For now, this

dissertation will offer the hypotheses outlined above as an explanation for this


!50

phenomenon, and move on to other data trends observed during the analysis of the

corpus.

3.5.2.2 Essays with ‘Complex Issues’

A second trend that emerged during the data analysis was the appearance of issues

that could not be easily labelled with Haas’ framework. The difficulty of categorising

these problems lies in the fact that they are not explainable by singularly pointing to

one HOC problem defined in the framework. Instead, they often appear to be

combinations of several ‘regular’ HOC problems that manifest in unison, thus

creating an issue that is familiar, but not entirely the same as those originally defined

by the framework. These issues were not initially covered by the framework: as such,

the three raters labelled them “complex issues” (De Soete 2016; Haas 2016) . An 8

overview of such issues appearing in this dissertation’s corpus is listed in Table 05,

below.

As Table 05 demonstrates, these complex issues are generally caused by

Higher Order Cognition problems, and lead to other HOC issues. To illustrate this

effect, an excerpt from essay 07 is provided below. This specific excerpt is the first

To reflect that the category ‘complex issues’ is the result of a discussion between the three 8

raters (as opposed to being one individual’s terminology), this dissertation cites both Haas (2016) and De Soete (2016) for this term, which is used in each of the raters’ essays.


Table 05. Overview of ‘Complex Issues’ in Each Essay, per Essay

!51

half of the second paragraph of the essay, and demonstrates the complex issue that is

created by an underlying “Gap in Information” (Haas 2016: 45), which leads to a

“Consistency” problem (Haas 2016: 51). The relevant section has been underlined for

clarity; any L-LOC issues were italicised.

First of all, I eat my cookies and cakes and pies by hand. With a biscuit

as greasy as that, you won’t be able to leave the room without a stain on

your shirt. Above that you will leave a trail of grease on the floor,

because it keeps dripping of that fancy dessert.

(Appendix 3, Essay 07, p. 157 [emphasis added])

In this specific case, there is a Gap in Information between the ‘eating by hand’ in line

one, and the causing of ‘a stain on your shirt’. Arguably, the Gap in Information is not

incomprehensibly large: a reader can imagine this author wanted to indicate greasy

cookies leave one’s hands greasy, which usually results in stained shirts because

people did not wash their hands after eating the greasy cookie and they might then

touch their shirt, because of which the grease of the cookie ends up making stains on

one’s shirt. However, part of a well-written text, according to Haas’ framework, is that

the writer has “fill[ed] in all the necessary information or logical connections in a

text” (2016: 45). Additionally, such seemingly small issues can present serious

consequences if appearing in academic writing, underlining the importance of paying

attention to even the smallest HOC issues at an early stage. As such, since the

connection between ‘eating the cookie by hand’ and ‘a stain on a shirt’ is not made in

the essay itself, this section qualifies as a Gap in Information.

It is insufficient, however, to label this specific problem a Gap in Information

and to leave it as such. Because of the gap, the author has also (unconsciously)

created a Consistency problem in their own text. The essay’s focus is “disadvantages


!52

of greasy desserts” (Appendix 3, Essay 07, p. 157). The Gap in Information results in

an inconsistent text because it creates a shift: is the disadvantage the greasy hands, or

the stained shirt? Again, it is easy for the reader to assume both of these are

disadvantages, but this is another example of something that should not have been

left to the reader’s interpretation in the first place. As such, this specific issue can be

labelled as a Consistency Problem (since ‘stain on the hands’ is inconsistent with

‘stain on the shirt’), but it has a Gap in Information at its base.

The nature of these complex issues, is, of course, tied to the inherently

complex nature of language itself and (academic) writing in particular. At the same

time, it demonstrates the difficulty of developing a straightforward rating scale that

can successfully delineate every single issue into a neat category. On top of this, it can

be useful to distinguish these complex issues from ‘regular’ HOC problems, for both

mentors and students. Given the high retention level of these complex issues, it

appears further instruction for mentors on these specific issues is warranted.

Similarly, making students aware of the fact that HOC issues can have an impact on

each other could increase their own critical reading when reviewing their writing.

3.5.2.3 Specific HOC Trends

Finally, this section will discuss a number of trends related to specific HOC

categories. Therefore, Table 06 offers an overview of the frequency of each HOC

category in both pre- and post-mentoring essays. Keeping in mind the point

discussed in section 3.5.2.1, about ‘apparent’ vs. ‘actual’ retained HOCs, this table

makes the same distinction for post-mentoring essays.


!53

The HOC categories related to “Focus” are clearly the most frequent source of

textual problems, in both pre- and post mentoring texts. On its own, this is not

entirely surprising: “Focus” is defined by Haas as being one of the most difficult

“building blocks” to master (2016: 55), and this specific assignment was the first time


Table 06. Number of HOC Issues, per HOC CategoryNumber of HOCs, per HOC Category, Pre-Mentoring vs. Post-Mentoring

HOC Category # Pre-Mentoring # Post-Mentoring (retained/new)

Delta retained (# and %)

Focus too Wide 8 4 / 0 -4 (-50%)

Focus Unclear 1 0 / 0 -1 (-100%)

Double Focus 1 1 / 0 -0 (-)

Focus Shift 14 3 / 3 -11 (-78.57%)

Off-Focus 2 1 / 0 -1 (-50%)

Link to Focus Missing 12 4 / 6 -8 (-66.67%)

Relevancy Problem 9 5 / 3 -4 (-44.44%)

Order Problem 7 1 /1 -6 (-85.71%)

Conciseness Problem 6 1 / 2 -5 (-83.33%)

Coherence Problem 1 0 / 0 -1 (-100%)

Consistency Problem 6 2 / 2 -4 (-66.67%)

Balance Problem 8 2 / 1 -6 (-75%)

Given / New Problem 1 2 / 0 +1 (+100%)

Transition Problem 3 1 / 0 -2 (-66.67%)

Construction Problem 1 0 / 0 -1 (-100%)

Gap in Information 13 4 / 1 -9 (-69.23%)

Gap in Logic 0 0 / 1 -

Broken Promise 6 2 / 1 -4 (-66.67%)

Self-Containment Problem 1 0 / 0 -1 (-100%)

Precision Problem 2 0 / 0 -2 (-100%)

Complex Issues 8 6 / 1 -2 (-25%)

!54

students had to grapple with it; thus, some levels of problems can be attributed to the

nature of this specific HOC and the circumstances of the assignment.

Despite this, however, the “Focus too Wide” problem in specific is one of the

most persistent issues, even in post-mentoring texts, seeing the second-lowest rate of

improvement following a mentoring session. This can indicate two things: either

mentors are (too) conservative in their mentoring approach to essays whose focus is

too wide, or mentors do mention this issue, but students do not (sufficiently) narrow

their focus following their conversation with the mentor. Either way, it is clear this is

an area where improvements to the programme’s conduct can be made, for example

by increasing mentors’ training in this specific area.

Similarly, the categories “Link to Focus Missing” and “Focus Shift” are very

frequent in pre-mentoring essays, and continue to cause reader disengagement in

post-mentoring essays. Unlike the “Focus too Wide” trend, though, these issues do

see relatively high improvement rates following the mentoring session (66.67% and

78.57%, respectively), indicating that some of the missing links were corrected.

However, this amelioration is impacted negatively by the creation of new problems in

these categories during the rewriting process. This, as noted above, underlines the

importance of the call for repeated conversations with a mentor. In addition, one can

also question if this is a proper place for a more flexible mentoring relationship

between the mentor and the student, as argued for by Harbord (2003) and Crasborn

et al. (2011): by non-directively pointing to and resolving the problem, then taking a

more directive approach (for example, by providing general examples on how this

issue could be eliminated), the creation of these new HOC issues might be prevented,

whilst also still primarily putting the focus on the students’ development as a writer,

rather than the improvement of one specific essay.


!55

In the non-focus related (or, given the interconnectedness of these categories,

less-closely-to-focus-related) categories, “Relevancy” and “Gap in Information” are

the most frequently-appearing problems. This serves to demonstrate how

interconnected some of these categories are: it is often ‘because of’ a Gap in

Information, that a section becomes less relevant to the essay’s focus. Both categories

have relatively high improvement rates, though, indicating mentors are able to deal

with these issues when they occur. However, new relevancy issues continue to occur

in the post-mentoring essay, suggesting this is one area mentors may need to adopt a

more directive attitude, or assume the role of a more directive peer tutor (as opposed

to a non-directive peer mentor (Topping & Ehly 1998)), in order to ensure students

can prevent these problems from appearing in (sections of) their newly (re)written

text.

Finally, one must also look at the categories that have seen the highest rates of

improvement. Following the mentoring session, issues related to an “Unclear Focus”,

“Order”, “Coherence”, “Balance”, “Construction”, “Self-Containment” and “Precision”

were entirely or mostly resolved. This can be explained, partly, by the fact that these

are areas in which mentors have been extensively trained, and partly by the nature of

these issues: they are subjectively noted to be some of the “easier” HOC problems in

the sense that they are, cognitively, somewhat easier to distinguish than issues that

are, for example, related to Focus. Nevertheless, the high improvement rates in these

categories are an example of the efficacy of writing mentoring, demonstrating that

mentors are capable of guiding students to resolve a large number of any given text’s

issues.


!56

3.6 Recommendations for Programme Improvements

Having discussed the results of the analysis of students’ pre- and post-mentoring

texts, the following section aims to suggest how these data can be used to improve the

mentoring programme’s practices, and its efficacy. Several of these suggestions have

been touched upon in the discussion of the data analysis; they will be expanded upon

here.

The possibility of students attending several mentoring sessions for one

specific assignment has already been mentioned during the discussion of the data

results, and is backed up by research conducted by Harbord (2003) and Irvin (2014),

and further reinforced by Kellogg’s demand for repeated opportunities for feedback

(Kellogg & Raulerson III 2007). From a subjective, experience-based standpoint, the

author can attest that working with the same student, during various (writing) stages

of their assignment, has proven to have a significant effect on both the students’

development as a writer, and especially on their written product. In the programme’s

current setting, mentoring sessions are slotted to have a maximum duration of half

an hour, a time period that is relatively narrow if the mentor is faced with an essay

that has a high amount of HOC problems (such as essays seven, eight and eleven).

Mentors are currently free to request for students to return for a follow-up session,

but this is rare in practice, partly due to time constraints on both students and on

mentors (who, in the current setting, are all contributing to the programme on a

volunteer basis), and partly due to the unfamiliarity of Belgian students with the

concept of peer mentoring, which is still proving to be somewhat of a barrier. Despite

this, the data discussed above clearly show mentors are capable of guiding students

towards significant textual improvements, an effect that can only increase when

multiple sessions can take place. Therefore, although this recommendation does not


!57

go as far as to suggest attending multiple mentoring sessions be made mandatory in

the current setting, strongly encouraging mentors to ask students to return for (a)

follow-up session(s) - or, ideally, recommending them to reach “the three [session]

threshold” (Irvin 2014: 5) - is one path to increasing the efficacy of the programme in

its current form.

The second recommendation that can be made for programme improvements

relates to mentor instruction. There are a number of specific areas where further or

more intensive instruction for mentors could increase the programme’s efficacy. Of

the retained HOC problems in the essays analysed above, 44.44% relate to focus

(Focus too Wide, Link to Focus Missing, Double Focus, Focus Shift). Focus too Wide,

in particular, is an issue retained in 50% of the post-mentoring essays. Similarly,

complex issues are an area of low improvement in essays, with 75% of the identified

issues remaining in the post-mentoring essay. This can be explained partly by the

inherent complexity of these issues, and partly by the fact that these issues were only

identified as part of the inter-rater reliability process of this paper. As new additions

to Haas’ continually-evolving framework, mentors will need to receive additional

training in order to ensure they can identify and remedy such issues in future

students’ texts.

One way to familiarise mentors with HOC problems and practise identifying

these issues is by involving mentors in the marking process. This could have two

advantages for those involved in the programme: firstly, the teacher would receive

valuable (and, arguably, experienced) assistance in providing feedback on the large

number of essays (exceeding 200+ essays per assignment) that have to be marked for

the ETTVI course, allowing them more time to engage with students (and their

writing) personally.


!58

Although unusual in the current (Belgian) setting, the inter-rater reliability

test at the onset of this chapter has demonstrated Haas’ framework might be applied

by non-experts with relatively little difficulty; furthermore, research has shown peer

assessment can be as effective as staff assessment (e.g. Zundert et al 2010; Iraji et al.

2016), especially when working with well-defined criteria or a specific rubric (in casu,

Haas’ framework) (Falchikov & Goldfinch 2000; Panadero et al. 2013; Jones &

Wheadon 2015). On top of this, mentors are already familiar with the framework they

would be using, having learned it in the ETTV classes and used it as mentors.

Familiarity with a framework enhances the validity of using it as a tool for assessment

(Falchikov & Goldfinch 2000: 315).

Secondly, being involved in marking can help not only the teacher, but

mentors as well. Their involvement could lead to a deeper understanding of HOC

issues “through meta-processes such as reflecting on and justifying what they have

done” (Liu & Carless 2006: 289). As such, it could provide considerable amounts of

real-life practise for mentors, helping them identify issues (faster). Aside from short-

term practice, research has also indicated peer assessors can gain other valuable

skills, such as problem-solving skills (Çevik 2015: 256), and can “[increase] student

learning and achievement” (Vista et al. 2015: 13), all of which could prove

advantageous as long-term improvements for mentors.

Thus, to address the issue of retained problems in the text, this paper offers

the recommendation that mentors be given specific instruction on the issues that are

retained most often, and that they be involved in the marking process. This offers

both immediate advantages for the teacher and mentors alike, additionally resulting

in long-term benefits such as mentors gaining valuable skills that can aid them in

their goal of guiding students to become better writers.


!59

4 Perspective 2: Analysis of Student Reflections

Having analysed and discussed the textual improvements to students’ writing that

can be attributed to a mentoring session, and having made suggestions for improving

the programme’s efficacy further, the current section turns to the second perspective

on the research question outlined at the beginning of this dissertation.

Ultimately the goal of writing mentoring is not to improve texts, but rather to

assist students during their writing process and during their development as writers.

Therefore, an evaluation of programme efficacy can not solely be done from an

analysis of written products, but must also include the perspective of students as

writers-in-development, and, more specifically, the role and effect of writing

mentoring on this process. Getting the student perspective on this is important since

the writer development course does not solely focus on producing good text: it

focusses on turning an audience of students who have various writing backgrounds

into (reasonably) good (academic) writers. As such, understanding how students

experience the mentor process is an important component in establishing its efficacy,

and how said process can be improved in the future.

4.1 Data Collection: Student Reflective Writing

The reflections that will be used to study students’ view on writing mentoring were

written as part of their third writing assignment. In addition to being required to

write a formal essay, students were also asked to submit 1,000 words of reflective

writing. The assignment for this was to “write at least 1,000 words of Reflective

Scribbling. Reflecting on your writing decisions and on [if/how] giving/receiving

feedback helps you as a writer and helps other writers” (S3 assignment, Appendix 1,

AR Cycle 3 - Student Reflections Thijs Gillioen

!60

p. 112). This “reflective scribbling” was analysed for content that is relevant to

mentoring.

As with the students’ writing assignment, written permission to use their

reflective writing for academic research was obtained from all students prior to the

analysis of the texts for the purposes of this dissertation. To avoid any privacy

concerns, names and pronouns referring to either mentors or students were removed

and replaced with neutral references and pronouns; in addition, only the pieces of

reflective writing that are directly quoted from are included in the appendices to this

dissertation. To further ensure the privacy of those specific writers (and to ensure

maximum relevancy of the appendices), only the immediate context of the quote is

provided (usually the full paragraph from which a quote has been extracted). The

student comments presented in this dissertation were not otherwise altered unless

explicitly indicated. Before turning to the results of this analysis, the following section

will briefly discuss the analysis methodology that was employed to find out how

students perceived the programme.

4.2 Methodology: Content Analysis

A total of 162 pieces of reflective writing were submitted by students, and were

subsequently studied for the purposes of this dissertation. This reflective writing was

categorised in a number of ‘rounds’, based on lexical analysis. This categorisation

process is represented schematically in Table 07.


!61

In a first selection round, texts that offered some comment related to writing

mentoring and/or writer development were separated from those that did not; this

latter group was not studied further. Comments that were considered related to

writing mentoring and/or writer development were chosen based on keywords like -

and related to the concepts of - “(writing) mentor”, “mentoring session”, “writer

development”, “writing career” and “change in writing style”. This selection process

reduced the total number of relevant reflective essays to 119, with a total relevant

comment count of 229.


Reflection Categorisation Process

Original data set

Step 1: Categorisation

based on relevancy

Step 2: subcategorisation

Step 3: subdivision Step 4: further subdivision

where possible (final categories)

162 pieces of Student Reflective Writing

119 ‘relevant ‘ essays;

229 ‘relevant’ comments

“Helpful” category (213 comments)

“Practical Reasons” “Practical Reasons”

“Psychological Reasons”

“Confidence and/or Motivation Boost”

“Extra Explanation Offered and/or Helps to Understand Theory”

“Appreciation of Mentors(‘) (Style)”

"Appreciation of Approachability”

“Appreciation of Non-Directive Approach”

“Student Advice on Mentoring (Programme)”

“Student Advice on Mentoring (Programme)”

“Attending Multiple Mentoring Sessions”

“Attending Multiple Mentoring Sessions”

“Did not Feel Helped” “Did not Feel Helped”

“Writer Development”

category (16 comments)


(no further subcategorisation)


43 ‘non-relevant’ essays (not studied further)

- - -

Table 07. Overview of Reflective Writing Categorisation Process

!62

During step 2 (cf. Table 07), these data were then further analysed for

comments related to the writing mentoring programme, and students’ perceptions of

its effect on their development as a writer. From this analysis, these comments were

divided into two major subcategories, again based on lexical content. The first

subcategory contained 213 comments that were related to the short-term results of

the mentoring session, and was labelled “Helpful”: these students indicated the

mentoring session helped them in one or more ways (for example, “The session with

my mentor also gave me the right directions to find the real focus” (Appendix 4,

sample 1, p. 214) ). The second subcategory contained 16 comments that were related 9

to the long(er)-term effects of the mentoring session, and was labeled “Writer

Development”. An example of such a comment is: “After the mentoring session I am

now convinced that if I had to write another focus-based story, I would find a good

focus all by myself!” (A4S2, p. 215).

In the following step (3), the “Helpful” subcategory was further subdivided,

based on lexical indicators, into narrower categories that each reflect a central reason

why students perceived the mentoring session to be useful. This ultimately resulted

(step 4) in the following eight categories: “Practical Reasons”, “Confidence and/or

Motivation Boost”, “Extra Explanation Offered and/or Helps to Understand Theory”,

Appreciation of Approachability”, “Appreciation of Non-Directive Approach”,

“Student Advice on Mentoring (Programme)”,“Attending Multiple Mentoring

Sessions” and “Did not Feel Helped”. Each of these categories was identified on the

basis of lexical items, and are defined as follows.

The first category, “Practical Reasons”, was based on a trend that was

immediately observable: students finding short-term practical benefits (such as an

Hereafter, references to comments from the reflective scribbling will be formatted as 9

“(Appendix 4, sample [x], p. [x])”, which will be abbreviated as “(A4Sx, p. x)”.


!63

improved text, or getting specific pieces of advice or having received help to narrow

down one’s focus) of attending a mentoring session. Therefore, this category is

comprised of comments that relate to the immediate, practical effects of writing

mentoring on students’ texts. This was based on lexical items like, for example, “the

mentor I had was very helpful and gave a lot of advice to help me to finish and

perfect my scribing at home.” (A4S3, p. 215 [emphasis added]).

A second category that quickly became evident during the analysis in step 3

was “Psychological Reasons”. This category was defined by lexical items such as word

groups like “increased confidence” or “motivation boost” and by the use of emotive

verbs such as “comforting” and “reinforcing”. This category comprises the

psychological and/or emotional benefits to attending a mentoring session. One of

these benefits indicated by a substantial number of students, for example, is finding a

renewed motivation to write following the mentoring session. Similarly, many

comments also indicated students felt a confidence boost thanks to positive

reinforcement from the mentors on their current written product. An example of such

a comment is the following: “When I got home afterwards, I felt a lot more motivated

to start again than I did when I left my apartment in the morning.” (A4S4, p. 215).

Upon further analysis during step 4, this “Psychological Reasons” category was

narrowed down into two separate, final, categories: “Confidence and/or Motivation

Boost” and “Extra Explanation Offered and/or Helps to Understand Theory”. The

first of these, “Confidence and/or Motivation Boost”, retains those comments from

the “Psychological Reasons” category defined by the words groups and emotive verbs

defined above. The second category, “Extra Explanation Offered and/or Helps to

Understand Theory” was defined by lexical items such as “increased understanding”

and “explaining [a given HOC]”. This category also relates to the psychological (and


!64

cognitive) effects of a peer mentoring session, but, more specifically, is comprised of

comments from students indicating they felt more motivated not (primarily) by the

support, but because the mentor was successfully able to explain some concepts from

Haas’ theoretical framework that were previously not well-understood by the

student(s). One example of such a comment is the following: “Also, he once again

explained to me the difference between a topic and a focus and I think I’m finally on

the way of figuring out what a focus really is – or at least I think I discovered

something that day.” (A4S5, p. 216 [emphasis added]).

A fourth category defined during the analysis in step 3 was “Appreciation of

Mentors(‘) (Style). This category was mainly defined by emotive verbs, combined

with adjectives or word groups with adjectives, indicating appreciation. Examples of

such indicators are “appreciate”, “liked the relaxing attitude” and “relaxed and pretty

fun”. This category consists of comments by students indicating an important

element of their willingness to listen and talk to a mentor was the more casual,

‘relaxed’ setting of the mentoring session (as opposed to a regular class).

Following further analysis in step 4, this category too was refined into two

smaller groups. The first of these is “Appreciation of Approachability”. Comments in

this group specifically mention mentors’ status as peers (since mentors are all

students themselves) as an important element in establishing mentor-to-student

relatability and approachability, thus allowing students to feel less uncomfortable

when taking their text to, essentially, a stranger. An example of such a comment is the

following: “They understand our difficulties - regardless how petty - better in some

way, because, not that long ago, they were sitting exactly where we are sitting

now” (A4S6, p. 216).


!65

The second of the “Appreciation” subcategories defined in step 4 is

“Appreciation of Non-Directive Approach”. Lexical indicators for this category were

mentions of the non-directive mentoring style maintained by mentors, which could

be either explicit (by mentioning “non-directive”), or implicit (indicated by inferring

that the mentor did not do the work for them, instead merely asking questions that

lead to solutions for textual issues; for example, “thinking for myself”, “only asked

questions” and “helped but did not tell me what to do”). One student, for example,

wrote that “the fun thing was that he didn’t just tell me how I could do improve my

text and make everything link, he made me do it myself. He asked very direct

questions and my answers created the solution” (A4S7, p. 217).

This appreciation for the mentors’ approach is also expressed in a sixth

category, “Student Advice on Mentoring (Programme)”. This category was identified

in step 3 by the prompt “One piece of advice I would give to a first-year student next

year is …”, which is one of the prompts students were given as part of their Scribing 3

assignment (Appendix 1, p. 114) for use in their reflective writing. A group of students

responded to this question by saying that they would advise future students to go to a

mentoring session for any written assignment. The following piece of advice is one

example of this:

One piece of advice I would give to a first-year student next year is to go

to a mentoring session because it really helps. Even if you are too shy to

go just try it one time. The mentors give you really good feedback and

they are willing to help you with the mistakes or flaws in your text. It

can only help you and improve your assignment to go see a mentor.

(A4S3, p. 215)


!66

Similarly, many students either advised students to go see a mentor before it is made

mandatory to do so, or expressed a wish they had done so themselves: “If I had to

give someone advice for next year I would tell them to go see a mentor. It helped me a

lot and I regret I didn’t do it for the other assignments” (A4S8, p. 217). Finally, a

number of students also advised the course coordinator to make seeing a mentor

compulsory for earlier assignments, instead of only making it compulsory for the final

writing assignment of the course: “It might be a good idea to make this obligatory for

the first two essays too” (A4S9, p. 218). All of these advice-related comments were

gathered in the “Student Advice on Mentoring (Programme)” category.

The seventh category, “Attending Multiple Mentoring Sessions”, was identified

by students’ explicit mention of the idea of going to several mentoring sessions for a

given assignment, with a small number within this group explicitly mentioning

actually having done so. The following example is from one of those students:

I decided it was time to call in the help of a writing mentor (thank God

they exist). I went to our appointment […] In that moment it stroke to

me I still had no clue at all what focus meant. So luckily my writing

mentor helped me a lot trying to figure out what this whole focus thing

was all about. […] I decided to make another appointment with my

writing mentor and was hoping this time I would have it right. Luckily I

did and I was quite proud of myself I had mastered the focus-skill.

(A4S10, p. 218)

Finally, the eighth category, “Did not Feel Helped”, is made up of comments by

students who left the mentoring session not feeling very well-helped. Comments in

this category contain lexical items referring to disappointment or “having higher

expectations”, as is reflected in the following comment: “Don’t get me wrong, my


!67

mentor was the nicest [mentor] in the world, but I expected to get more specific

feedback” (A4S11, p. 218-219 [emphasis added]). The varying reasons for students

not feeling helped will be discussed in detail in the discussion section below.

4.3 Overview of Results

Once the categories detailed in section 4.2 had been established and refined based on

a full reading of all comments, all 229 reflective comments were analysed once more

and added to the relevant category. Table 08, below, offers an overview of the

frequency of each category, along with the total number of comments per category.

These findings will be discussed in detail in section 4.4.


Table 08. Overview of Categorised Reflective Writing

Reflection category # of comments % of total

“Helpful” categories 213 93.01%

Practical Reasons 77 33.62%

Confidence and/or Motivation Boost 33 14.41%

Extra Explanation Offered and/or Helps to Understand Theory 26 11.35%

Appreciation of Non-Directive Approach 23 10.04%

Student Advice on Mentoring (Programme) 19 8.30%

Attending Multiple Mentoring Sessions 15 6.55%

Appreciation of Approachability 13 5.68%

Did not Feel Helped 7 3.06%

Writer Development category 16 6.99%

Total 229 100%

!68

4.4 Discussion of Results

4.4.1 General Discussion

From Table 08, it is clear that a large majority of students find the peer mentoring

programme useful and/or important to their writing process in one way or another,

with 93.01% of all relevant comments indicating their mentoring session was helpful

primarily in the short term. For many students (33.62%), “Practical Reasons” are the

primary positive effect of a mentoring session. Seeing that practicality is the number

one concern among students (rather than long-term effects of mentoring, such as

assisting one’s development as a writer) is not particularly surprising, given that

students’ primary concern would be to pass the course. In this sense, this data

reinforces Harbord’s assessment that evaluation of mentoring programmes also has

to consider the practical effects of writing mentoring (2003). It also demonstrates

that a large subset of students perceive the mentoring programme to be beneficial to

their writing, and thus, efficacious, at least at guiding students towards improving

their writing for this particular assignment.

Aside from the short-term practical effects of a mentoring session, a significant

number of students also noted short-term psychological effects of a mentoring

session. 14.41% explicitly indicated they felt more confident after the mentoring

session, in addition to another 11.35% who asserted they had a better understanding

of either the task at hand or the underlying theoretical framework following the

mentoring session. Similarly, mentors’ approachability and non-directive approach

were also mentioned (5.68% and 10.04%, respectively) as important elements

contributing to making a student feel a mentoring session is a worthwhile part of

their writing process. Combined, these psychological elements comprise 41.48% of

students’ comments, thus eclipsing the “Practical Reasons” group. As such,


!69

psychological effects of mentoring sessions on students should not be underestimated

as contributors to the programme’s efficacy.

On top of the short-term effects of mentoring sessions on students themselves,

a group of students also indicated they would advise future students of English to

attend a mentoring session for (any of) their writing assignments, even if it was not

mandatory for them to do so. Although it can be argued that most comments in the

categories above can be read as implicit recommendations of mentors to future

students, 8.30% of students’ comments explicitly mention this advice. Although this

number, on its own, is not extremely high, it is significant considering that many

first-year students are overwhelmed with the amount of work they have to do - which

has been established in literature (e.g. Chraif 2015; Ruiz-Gallardo et al. 2016) and in

the reflective comments: “the obligated mentoring session we had to attend was –

despite the loss of time that we need sooooooo bad – actually not a bad

thing.” (A4S12, p. 219 [emphasis added]). Thus, these already busy students

recommending taking up an additional, non-obligatory activity, can be seen as an

indicator of at least some level of student satisfaction and of programme efficacy.

The group “Attending Multiple Mentoring Sessions” received 6.55% of the

total comments gathered during the analysis. The fact that some students voluntarily

attended multiple mentoring sessions, or would recommend seeing a mentor

multiple times for a given assignment, again serves as an indicator that these

students perceive the mentoring programme as efficient in its ability to guide

students in their writing process.

However, a small subgroup (3.06%) of students indicated their experience

during the mentoring session was not positive, or, at least, that the mentoring session

did not help them as much as they had expected. Although this group is, statistically


!70

speaking, too small to be considered relevant, this dissertation will discuss this group

below, in section 4.4.2.8, in order to understand why these students, in these specific

cases, did not feel helped. Given that the large majority of students indicated having a

positive experience during their mentoring session, it is possible that elements

extrinsic to the mentoring session itself (such as faulty expectations or a

misunderstanding of the workings of the mentoring process) are tied to the

dissatisfaction expressed by these students.

Finally, sixteen comments were gathered that relate to the long-term effects of

the mentoring session, representing 6.99% of the gathered data. The fact that any

comments related to long-term writer development are present, is another indication

the programme has at least partly succeeded in moving beyond the ‘here-and-now’ of

assisting students’ writing, instead having had an impact on students’ long-term

development as writers. These comments will be discussed in detail in section 4.4.2.9.

4.4.2 Discussion of Reflections per Category

Having given an overview of the results of the categorisation process and the

frequency with which each category appears, the following section will now look at

each of the categories defined above and attempt to establish what, in each category,

is a contributor to the programme’s efficacy, or students’ perceptions of its efficacy.

4.4.2.1 Practical Reasons

As noted above, this “Practical Reasons” category consists of student comments

indicating they perceived the mentoring session to be useful because it helped them,

in the short term, to refine and finish their essay. One example of such a comment is

the following: “the session with my mentor gave me the right directions to find the


!71

real focus [of my text]” (A4S1, p. 214). Students’ appreciation of such support is not

surprising, on its own, since this support will (indirectly) help them gain a higher

grade.

It is interesting to note, however, that students found practical benefits to

mentoring sessions in different stages of their writing process. Some students

indicated they had difficulty finding a focus for their assignment, but were able to

find a “specific focus on [their] topic during the mentoring session” (A4S13, p. 219).

Furthermore, students who came in with a limited draft found their session with a

mentor useful because it helped them develop their ideas, as is indicated by: “I have

to admit, [the mentor] was a great help in figuring out all of this. What [the mentor]

said, the ideas [the mentor] had on my text were in line with what I thought. Only I

couldn’t really put it down on paper, whereas [the mentor] did” (A4S14, p. 220).

Likewise, another student found “the mentor [they] had was very helpful and gave a

lot of advice to help [them] to finish and perfect [their] scribing at home” (A4S15, p.

220). For these students, the mentoring programme’s efficacy lies in its ability to

guide students’ thoughts as they process their ideas into text.

For students who had an advanced or complete draft, support from mentors

came in the form of help in refining their current texts. This refinement could pertain

to the structure, which “mentor[s] could really [help them] get right” (A4S16, p. 220),

or to the focus of the students’ texts. Pertaining to focus, some students indicated

they were helped in “keeping [their] focus more narrow” (A4S17, p. 220), in making it

“explicitly stated” (A4S18, p. 221), or even in finding a new focus that “suited the text

much better than the one [they] initially had” (A4S19, p. 221). Aside from focus-

related support, students also noted mentors could help them find issues they “saw,

but could not define” (A4S20, p. 221) or issues students just “really hadn’t seen


!72

before” (A4S21, p. 222). Students posed this form of support is made possible thanks

to the mentors’ ability to “deconstruct the parts of [students’] own writing” (A4S22, p.

222): thus, part of the programme’s efficacy lies in mentors offering students the

opportunity for their drafts to be read by readers who have been trained to offer them

support.

It is somewhat surprising to find students this receptive to near-peer support:

teachers often note students either simply do not care about feedback, or “do not act

on [it]” (Hepplestone et al. 2011: 118). Peer support, specifically, has often been

found to evoke “mixed, and often negative reactions” (Best et al. 2015: 332), in

addition to being “a process that is laden with strong negative emotions” (Best et al.

2015: 344) for students. Best et al. have argued the peer support process is often

negative for students, not because of the process itself, but because of group

dynamics, wherein a dedicated student might be ‘matched’ for support with a less-

dedicated student, thus resulting in an “unfair” situation (2015: 345). It can be

argued, however, that this group dynamics issue is rendered void in the current

programme. This is the case partly because all mentors are volunteers (thus, are

dedicated by definition), and partly because they have all received identical training

and (as such) have near-identical expertise. One student summed it up as follows:

I was afraid for my mentoring session […]. I was just thinking about the

fact that it would be a waste of time and that I'd come outside even

more puzzled. My worries were unauthorized. I had an appointment

with [a mentor], who was very talkative. I got rid of my stress

immediately. [The mentor] was very nice and when I told [the mentor]

my name, [they] said: 'Once upon a time, isn't it?' I was so so positively


!73

surprised with that answer, because it seemed to me that [the mentor]

was very well informed and put a lot of effort in [their] part of the deal.

(A4S23, p. 222)

As well as being dedicated, students’ perception of mentors as ‘experts’ is

widespread: they indicated that “writing mentors clearly have a lot more

experience” (A4S24, p. 223), “aren’t hesitant and have more experience” (A4S25, p.

223) and “clearly know what they’re talking about” (A4S26, p. 223), which made at

least one student note they took “the advice/feedback from a mentor way more

serious[…] than from a peer” (A4S27, p. 224). An additional issue for engaging with

support on written assignments, noted by Best et al., are “vague comments[, which]

students find confusing and discouraging” (2015: 343). Because of the one-to-one

nature of a mentoring session, anything that is not immediately clear to a student can

be cleared up ‘on the spot’ by the mentor, thus again eliminating the issue. One

student phrased it as follows: “I also had some questions about my focus, if it was

near enough, and the structure of my text. It made thin[g]s clear for me and

afterwards, I knew exactly how I wanted to change my text” (A4S28, p. 224). As such,

mentors’ position as near-peers, but also relative ‘experts’, combined with the

possibility for students to ask them questions during at stages of the writing process,

are the main contributors to the programme’s efficacy in this category.


!74

4.4.2.2 Confidence and/or Motivation Boost

In addition to finding practical benefits to a mentoring session, many students

indicated the mentoring session had a positive psychological effect on them. One of

these effects was an increase in confidence, or a boost in motivation. Again, students

indicated they could find psychological support in the mentoring programme

throughout different stages of the writing process.

For students who had not yet completed their draft, a mentoring session was,

quite often, a powerful motivator to keep writing. One student noted that “when

[they] got home afterwards, [they] felt a lot more motivated to start again than [they]

did when [they] left their apartment” (A4S4, p. 215). Similar reflections noted

mentors could help their peers get “[their] enthusiasm back” (A4S29, p. 225), feel

“much more secure about [their] work” (A4S30, p. 225), “relieved” (A4S31, p. 225),

“reinvigorated” (A4S22, p. 222) and “confident […] because [their] writing mentors

had made clear that [they] [were] doing well” (A4S32, p. 226). Finally, one student

felt like they were given “a bit of control back”, which gave them “a bit of reason to

keep writing” (A4S33, p. 226).

Students who had been able to finalise a draft were similarly encouraged.

Although sessions with mentors often result in extra work for students, students

indicated they were “very positive about [their] writing again” (A4S20, p. 221), in

addition to feeling “sure and a bit more confident” (A4S34, p. 226). Even students

who did not have a lot of changes to make following their mentoring sessions noted

feeling “a bit more confident” (A4S32, p. 226) and “incredibly proud […] like I’m

getting the hang of it” (A4S35, p. 227).

The source of the programme’s efficacy in this field can be found in mentors’

positions as ‘students-who-have-been-there’, and through this, their ability to reduce


!75

student anxiety. Research supports the perception that peer mentors can be

significant in reducing such anxiety (Rodger & Tremblay 2003: 13), which is

corroborated by students in the current setting indicating mentors are not just

mentors, “but also […] still students. […] They understand our difficulties - regardless

how petty - better in some way, because, not that long ago, they were sitting exactly

where we are sitting now” (A4S6, p. 216). One student even indicated mentors “make

the transition from high school to university easier” (A4S36, p. 227), which again

underscores the point made above with regards to decreasing student anxiety levels.

As such, although the effects of this category on students may not be immediately

visible, they are important to student wellbeing, ultimately resulting in increased

retention rates and increased academic gains (Campbell & Campbell 1997; Salinitri

2005), thus indicating long-term programme efficacy.

4.4.2.3 Extra Explanation Offered and/or Helps to Understand Theory

Another psychological area where the mentoring programme appears to have

achieved some efficacy is in offering a limited form of peer tutoring. During

mentoring sessions, it became clear some students were still struggling to grasp some

elements of Haas’ framework for writing, which they had been taught over the course

of the first semester. The mandatory mentoring session offered these students

another opportunity to receive additional explanation, thus indirectly resulting in

textual improvements, writer development, and psychological wellbeing (by feeling

more secure in understanding the theory). One example of such a comment is the

following: “in the session they explained the concept of focus really good, so I finally

got it” (A4S37, p. 227).


!76

Mentors’ individual and personal styles are an important contributor to

programme efficacy in this area. Although all mentors are capable of explaining Haas’

framework thanks to having been given extensive training with it, they are able to

bring in their own perspectives on the theoretical framework, thus offering new ways

of explaining it to students. One student noted a metaphor used by a mentor: “a little

scheme” of her own that “was veeeeeery helpful” (A4S38, p. 228), whereas another

student was helped by “the example of the dog” (A4S39, p. 228). In addition,

mentors’ personal approach made one student feel like they had “finally learned

something” (A4S40, p. 228). In this sense, these comments from students support

the assertion of Cho & MacArthur (discussed above) that noted students’ close

proximity to each other allows for them to “detect problems from their own

perspective”, thus “generating solutions to the problems” (2010: 329). On top of this,

in their own meta-study of mentoring research, Orland-Barak & Hasin noted “the

ability to talk about teaching in ways that connect between theory, practice, and the

particular context of the mentee” to be a characteristic of good mentoring (2010:

429), thus underlining the importance of this category for programme efficacy.

4.4.2.4 Appreciation of Non-Directive Approach

Another short-term trend indicated as positive by many comments (10.04%) was

mentors’ non-directive approach. Instead of offering immediate ways to ‘fix’ issues

spotted in student texts, mentors guide their mentees towards solving the problem on

their own. One student summarised it as follows:

Even though I thought it was hard to answer [the mentor’s] questions

about my text, I did like the way [the mentor] did it. [They] let me think

about my own text, which is very hard, but refreshing when you see


!77

where you went wrong. [The mentor] didn't just say what [they] did and

didn't like, but [they] let me think and search myself, which I didn't

expect, but liked very much. This way, I have the feeling I did learn

something and above all, I understand [the mentor’s] remarks, because

eventually I saw them myself.

(A4S41, p. 229)

This appreciation may, at first glance, appear contradictory. Students’ priority

is generally to pass their assignments, with learning becoming a secondary goal.

However, in this specific context, several comments indicated the non-directive

approach was appreciated because students were given the “chance to think for

[them]self” (A4S42, p. 229), instead of explicitly being told what to do. Similar

comments noted they appreciated that “it was up to [them] to decide what was going

to be [their] focus and how [their] text would look like” (A4S43, p. 229) and that by

“the questions that were asked to [the student], [they] found out [for themself] what

was wrong with what [they] had written and that was actually the nicest thing”.

(A4S44, p. 230). Or, as one student put it, “it’s way more educational when [they] are

set on a path to discover for [them]self” (A4S45, p. 230). Therefore, the programme’s

main form of efficacy for these students is that it empowers them, allowing to resolve

issues themselves, rather than giving them a straightforward solution. This is

supported by research conducted by Zepke & Leach, who note “enabling students to

work autonomously” is an important factor in student engagement (2010: 170).


!78

4.4.2.5 Student Advice on Mentoring (Programme)

In their reflective scribbling, students could also give advice to their successors

regarding the writing course. A number of students elected to give advice pertaining

to the mentoring programme. From this, two trends emerged: firstly, students

advised their future peers to go see a mentor (sooner); secondly, students advised the

course teacher to make mentoring sessions mandatory for earlier assignments.

The first recurring idea was that students gave the advice to see a mentor, or to

go see a mentor before the last assignment. Some students expressed this by saying

they “wish[ed] [they] had used them sooner” (A4S46, p. 231), or “sincerely regret not

having seen a writing mentor” (A4S47, p. 231) for earlier writing assignments. Others

explicitly “advise[d] next year’s students to use [mentors] from the start” (A4S46, p.

231), since “asking for mentor help is not a shame or a nerdy thing to do, neither is it

a waste of your time” (A4S48, p. 232), but it “really helps[,] even if you are too

shy” (A4S3, p. 215), and, after all, “[the mentors] don’t bite” (A4S49, p. 232).

Aside from pointing out the usefulness of a mentoring session to future

students, some of the current mentees also advised the teacher of writing to make

mentoring compulsory for earlier writing assignments, as is demonstrated by a

certain student commenting that “it might be a good idea to make this [mentoring]

obligatory for the first two essays too” (A4S50, p. 232). This advice is, in nearly all

cases, motivated by students’ initial surprise to the usefulness of a session: “if I knew

that a mentoring session would be like this, I would have done it for my other essays

too” (A4S51, p. 232). Despite the fact that students were informed multiple times as

to how a mentoring session works, several students noted its usefulness only became

clear when having an actual conversation with a mentor.


!79

The “Student Advice on Mentoring” category does not provide any explicit

indications of the efficacy of the mentoring programme. However, by inversion, the

fact that a significant number of students advise their future peers to make use of its

services, and the fact that they propose to have more mandatory sessions (and thus,

more work), indicates these students perceive the programme to have a high efficacy

at what it does. These students’ recommendations can be seen as significant

suggestions for programme improvement, which will be discussed further below.

4.4.2.6 Attending Multiple Mentoring Sessions

One of the recommendations outlined in section 3.6, based on the analysis of student

texts, and existing literature, was to encourage mentors to guide their students over

multiple mentoring sessions. A number of students referred to this in their reflective

writing: some by indicating they actually did attend multiple sessions, and some by

indirectly indicating how a second session could have been useful to them.

A small number of students voluntarily attended more than one mentoring

session. These students indicated the second session was useful as well, partly

because they received confirmation of the improvement over their first draft, and

partly because mentors could guide them in resolving any remaining (or new) issues:

I have seen my mentor twice because I wrote a completely different text

after my first mentoring session and it was way easier to write that

second draft, I even enjoyed writing it. I did also feel a lot more

confident about my second draft. My second mentoring session went

good as well. My text had improved a lot and there were only a couple of

linking problems. Those problems were a bit harder to fix because I


!80

didn’t want to change too much. I do think I managed to link them

without writing something else entirely.

(A4S52, p. 233)

Another comment that evidenced the effect of confirmation and further help is the

following:

I met up with the [second] writing mentor who was actually a really

kind person. [The mentor] gave me some helpful advice as well as

several compliments. It has kind of cheered me up and gave me some

more motivation.

(A4S53, p. 233)

In one case, a student attended a mentoring session where the mentor could

“point out nothing that would make [the student’s] text better”. Although, in such a

case, the practical usefulness of a second session is quite low, the student still

experienced some benefit from the session, psychologically, since they “couldn’t be

happier […] and […] so incredibly proud” (A4S54, p. 233). This underscores the

potential, both practically and psychologically, of encouraging students to attend

multiple mentoring sessions. The fact that these students voluntarily returned for a

second mentoring session can be seen as an indicator that these students perceived

the programme to be efficient in its goal of supporting them during the writing

process.

In a number of other cases, comments indicated how a second mentoring

session could have been helpful to certain students. One comment revealed a student

was still unsure about their writing, being afraid that “after rewriting, […] [they]

made the same mistakes again and failed again” (A4S66, p. 239). Given the data

analysed in section 3.5.2.1 (p. 44), indicating some students do retain HOC problems


!81

(or create new ones), it is not unfair to argue this student could have benefitted

significantly from a second mentoring session. This student felt they had to “rewrite

[their] whole text”(A4S66, p. 239): if mentors are aware of such cases, urging such

students to return for a second session could prove beneficial to the student.

Although instances like these do not directly support programme efficacy, they do

support the suggestion made above (section 3.6) with regards to multiple mentoring

sessions.

4.4.2.7 Appreciation of Approachability

A final contributor to short-term programme efficacy was the mentors’

approachability, which was noted in 5.68% of the reflective comments. Although

approachability is not an immediate contributor to programme efficacy, it allows for

students to feel at ease when conversing with a mentor (who is ultimately still a

stranger), thus making the support process easier. In addition, an approachable

attitude makes it more likely that students will voluntarily return for another session

afterwards. As such, this category represents an indirect but important long-term

contributor to programme efficacy and sustainability.

A significant element in maintaining an approachable atmosphere was

mentors’ status as peers. To students, this enables mentors to “understand our

difficulties - regardless how petty - better in some way, because, not that long ago,

they were sitting exactly where we are sitting now” (A4S6, p. 216). The fact that

mentors ‘have been there’ allows them to tap into a shared pool of experiences that

decreases the uneasiness of the situation, as indicated by this comment: “we spent

[some time] talking about Latin, secondary school, study choices and other

interesting stuff. I loved that we could do that (…) I simply think it was amazing that


!82

we could just chat like that.” (A4S55, p. 234). This also made the sessions a “lot less

scary than […] expected” (A4S56, p. 235). Mentors’ position as peers also created a

sense that sessions were “relaxed and actually pretty fun” (A4S57, p. 235).

Another element that contributed to approachability was mentors’

preparedness for the session. As noted by Best et al., students often feel negatively

about feedback from unprepared peers (2015: 345), but students do assert peer

feedback can be valuable when the peer is prepared. The following comment indicates

a student’s positive surprise to the peer mentor belonging to the latter category:

I was just thinking about the fact that it would be a waste of me and that

I'd come outside even more puzzled. My worries were unauthorized. I

had an appointment with [a mentor], who was very talkative. I got rid of

my stress immediately. [The mentor] was very nice and when I told

[them] my name, [they] said: 'Once upon a me, isn't it?' I was so so

positively surprised with that answer, because it seemed to me that [the

mentor] was very well informed and put a lot of effort in [their] part of

the deal.

(A4S23, p. 222)

A final contributor to mentors’ approachability was the mentoring room where

sessions took place. Students noted that the “cozy atmosphere” (A4S58, p. 235) made

it easier for them to feel at ease when speaking with a mentor, which in term allowed

for students to focus on developing their writing skills.

These elements contribute towards an “interpersonal relationship” (Orland-

Barak & Hasin 2010: 429) being created between mentors and students, which they

note as another important characteristic of good mentoring. This relationship allows

for mentors to be approachable sources of support, rather than another


!83

institutionalised (and thus, distant) form of help students would be less eager to

access.

4.4.2.8 Students Who Did not Feel Helped

Having discussed the students who experienced the mentoring session in a positive

way, the current section now turns to the minority of students (3.06%) who did not

feel helped by the mentoring session, or whose expectations were not entirely met.

Understanding why these students did not perceive the mentoring session to be

helpful or successful can lead to interesting data points for further programme

improvement.

An explanation for these students’ disappointment may lie in the fact that it

appears these students expected the mentor to be directive, or to provide explicit

instruction on how these students’ texts could be ‘fixed’. One example of a comment

indicating as much is the following:

[…] Don’t get me wrong, my mentor was the nicest [mentor] in the

world, but I expected to get more specific feedback. […] Then [the

mentor] asked me what my focus/topic was. I had no idea myself so I

couldn’t quite answer [their] question. […] The concept of ‘beetles’ was

too wide and I should narrow it down. I didn’t agree with that because,

in my opinion, I had already narrowed it down. […] I asked [the

mentor] to give me an example of a good focus for my text. For some

reason [they] refused to give me one... I was pretty disappointed once I

left the mentor room...

(A4S59, p. 236)


!84

Thus, in this comment, the student’s disappointment stems from the mentor’s refusal

to provide an adequate focus for the student’s text. Since mentors are explicitly

instructed to maintain a non-directive posture, the mentor was simply following

programme guidelines. However, this resulted in a less-than ideal experience for this

student, who was hoping to have their text ‘fixed’. A similar situation is presented in

the following comment:

[The mentor] said what my peers said, but I already told [them] I know

what’s wrong. [They] said there was too much new given information,

but how I could avoid this hasn’t [they] mentioned. When I left the

mentoring room, I was so confused: my peers said to me that the stuff

I’m working with for my text is great, but I have to work on it. [The

mentor] said the same things, but I had the feeling [they] didn’t like the

topic or focus I had in mind.

(A4S60, p. 236)

Again, the issue here appears to be that the student expected the mentor to be

directive, i.e. to tell them how they could fix their specific issue (here: a problematic

“Given/New” HOC problem).

On the other hand, the comment above also indicates the student did not

entirely comply with the assignment details: students were instructed to re-write

their text based on their peers’ feedback, before seeing a writing mentor (S3

Assignment, Appendix 1, p. 112). A different student also noted that “everything [the

mentor] had to say, my peer had already written down” (A4S61, p. 237), which again

could have been avoided had the student complied with the assignment details.

Despite this, dismissing these students’ issues entirely would be to dismiss

these students as a whole; since the programme aims to reach all students,


!85

considerations should be made to ensure that these students, too, can be assisted

during their writing process. The strict application of the non-directive approach has

resulted in a (small) number of students leaving the mentoring session dissatisfied;

this underlines the argument of Harbord (2003) and Crasborn et al. (2011) for a

flexible approach based on each student’s needs.

4.4.2.9 Writer Development

In addition to the (relatively) short-term benefits to mentoring discussed in the

sections above, a small group (6.99%) of comments signal a long(er)-term advantage

to attending a mentoring session. Since the comments analysed for the purposes of

this research were collected after a single mentoring session, one has to be careful

about drawing long-term conclusions of its effects. Despite this, a few students

explicitly noted the mentoring session aided them in their development as writers. In

what follows, two groups of these comments will be discussed; they are representative

of the other comments in this category.

The first group contains comments related to students’ ability to give their

peers support during their writing process. One student said “[they] already feel like

[they] developed some kind of skill that [they] didn’t have before [they] went to

university” (A4S62, p. 237). Another student similarly commented:

[A fellow student] had already gone to see a writing mentor, so [they]

[were] able to tell me things about my text that the writing mentor had

also said about [their] text, which was of course very helpful, because

[they] already knew what [they] had to improve and [the fellow student]

saw the kind of same defaults in my text. [That student] told me that I

should state my focus more explicitly in every paragraph. But [they]


!86

also told me that my text already had a pretty clear focus and that it was

structured well, so that was nice to hear. Those three things are also

kind of exactly what the writing mentor ([…]) told me about my text.

(A4S63, p. 238)

Given that this student indicates their peer was able to find several issues in their text

due to having had a mentoring session, it can be argued that the mentoring session

not only helped this peer improve her own writing, but also her ability to help others

during their writing process. This indicates a possible long(er)-term developmental

effect of the mentoring session.

The second group contains comments related to students’ ability to write.

These comments indicate that, following the mentoring session, students felt like they

would be able to write better texts for any future assignment. An example of such a

comment is: “After the mentoring session I am now convinced that if I had to write

another focus-based story, I would find a good focus all by myself!” (A4S2, p. 215).

These comments provide some evidence that the programme was not only able

to help students in the short term, but also in the long term, despite the fact that the

data analysed here only pertain to one specific assignment. This might serve as

another indirect indicator of (long-term) programme efficacy.


!87

4.5 Recommendations for Programme Improvements

Having elaborated upon each category of students’ reflective comments, this section

aims to suggest how these data can be used to improve the mentoring programme’s

practices, and its efficacy. Although only a small subset (3.06%) of students posited

not feeling helped by the mentoring session at all, there are a number of ways in

which the programme could improve to both reduce this number, and further assist

students who did feel helped during their writing process.

Firstly, the programme’s strict adherence to the non-directive approach to

mentoring may be in need of reassessment. Although this dissertation has not

specifically focussed upon this element, it became clear at several points during the

discussion above that this approach limited the programme’s ability to aid some

students in an optimal manner. Although many students declared that they

appreciated the non-directive approach, if the programme aims to reach all students,

the reality that this mentoring style does not work for some mentees has to be

considered and adapted to. This situation in the programme’s particular context is

similar to claims made in the literature discussed above (e.g. Kullman 1998; Harbord

2003); likewise, Crasborn et al. argued that “mentoring is a multifaceted

phenomenon and that no straightforward description or guidelines for success can be

given” (2011: 328).

Such adaptations have to account for the two points of view that inform the

programme. The first of these perspectives is that of the mentors: for them, the non-

directive approach offers the advantage of avoiding regressing into editors, in

addition to shielding them from making content- or analysis-related comments,

which could pose the possible problem of the mentor being wrong, or the assignment

instructor not appreciating such an intervention. However, in addition to mentors,


!88

students are the second group to which the programme is beholden: for them, the

non-directive approach enables them to reflect on their own work, thus enhancing

their opportunity for learning. Despite this, for students, passing the course is often a

primary concern, to which learning benefits (created by the non-directive approach)

have become secondary. Facing this divide, if the programme is to increase its

efficacy, both sides have to be accounted for.

Since the majority of students did indicate feeling helped, and since there are

some indications of writer development following a mentoring session, abandoning

the non-directive approach entirely would be too drastic. However, since some

students do need more directive support, one possibility would be to allow ‘senior’

mentors (i.e. mentors who have two or more years of mentoring experience) to go

beyond non-directive support and give students directive assistance. Although this

does limit mentors’ ‘shielding’ (discussed above), containing this directive advice to

an essay’s structure only (thus not commenting upon content), combined with senior

mentors’ experience, should limit their ‘risk’. For less experienced mentors, this

offers a double benefit: by referring students in need of directive support to senior

team members, ‘junior’ mentors can continue practising non-directive skills with the

majority of students for whom the non-directive approach does yield positive results.

Additionally, this removes any doubt or ‘uneasiness’ (junior) mentors may experience

when a student does not respond positively to the non-directive approach. By

opening up options for directive support, the programme could reach a larger

audience, and extend its usefulness to the current audience, thus improving its

efficacy.

The second and third recommendations pertain to introducing the mentoring

programme (and its practices) to students sooner in the semester. As discussed


!89

above, several students regretted not seeing a mentor earlier, thus advising their

future peers to approach a writing mentor at a less advanced stage of the semester.

Several students students noted their initial apprehension to see a mentor stemmed

from being “shy” (A4S3, p. 215) or finding the idea of a mentoring session

“scary” (A4S3, p. 215; A4S15, p. 220; A4S56, p. 235). Other students indicated they

“didn’t quite know what to expect of it” (A4S64, p. 238), and that the session was

“nothing like [they] expected” (A4S57, p. 235). Clearly, students’ apprehension is, in

part, caused by their unfamiliarity with mentoring (as a concept, and as a practical

instrument). Therefore, the following two recommendations aim to provide

suggestions to improve this situation.

A first way to make students more familiar with the concept of mentoring, and

the writing mentors themselves, is to implement a full mentoring session in one of

the writing classes. The familiar environment of the classroom, coupled with the

presence of a teacher as an ‘official’, could significantly reduce students’ anxiety when

talking to (at that point) a ‘stranger’ about their writing. Furthermore, by introducing

mentoring in class, students would get a clear view on what can (and cannot) be

expected from a mentoring session, thus clearing up the ‘unease’ indicated by the

students quoted above. Given the mostly positive post-mentoring experiences

discussed above, it is not unfair to pose that, once students are more familiar with

mentoring, they would be more inclined to visit a mentor, either for follow-up

mentoring sessions or for other assignments.

A second way to introduce students to mentoring is by making mentoring

mandatory earlier on in the semester, either by introducing it in-class, as suggested

above, or by making attendance of a mentoring session compulsory for earlier writing

assignments. This suggestion was spurred by comments from students:


!90

Something I would change about the writing class is a mandatory

writing mentor session for the first of second assignment already.

Because to me, it was really useful and I wish I had done it for my

previous assignments as well.

(A4S65, p. 239)

Mandating mentoring earlier in the semester serves a double purpose: firstly, it

would allow students to ‘grasp’ the concept of mentoring early on, thus making it

more accessible for future assignments. Secondly, it would allow mentors to guide

students from an early stage in the semester (rather than merely at the end of it), thus

increasing the potential for mentors to (through cognitive apprenticeship) have an

impact on students’ development as writers. The first suggestion could lead to short-

term improvements in programme efficacy; the second could lead to (increased)

long-term efficacy.


!91

IV Limitations, Conclusion, and Future Research

Having analysed and discussed the efficacy of the UGent peer writing mentoring

programme from two perspectives, and having made suggestions for programme

improvements based on these analyses, the current section now turns to discuss the

limitations of the current research. Following that, a conclusion of this research is

presented, before suggesting some avenues for future inquiry.

1 Limitations of the Current Research

Throughout the current AR cycle, the aim has been to conduct a thorough and

founded evaluation of the programme’s practices, and to use this evaluation to

propose grounded suggestions for improvements. However, as with any research, this

project was constrained by time and scope; therefore, it faced a number of

limitations.

A first limitation is that the essays analysed as part of the pre- and post text

analysis were written by students who had seen the same mentor (who is the author

of this dissertation). In the context of this research, this was required to avoid bias,

and because of practical limitations: it was impossible, within the short timeframe, to

account for personal differences in mentoring style and mentors’ differing personal

relations to students influencing the mentoring process. However, if due recognition

is given to these potential differences, considering the fact that all mentors are

volunteers (and thus more or less equally dedicated) and the fact that all mentors

have received the same training, this paper has argued these results can be

reasonably extrapolated to the programme as a whole. However, a more thorough

Conclusion Thijs Gillioen

!92

evaluation could include texts mentored by different (and, ideally, all) mentors

participating in the programme.

A second limitation is the possibility of researcher bias in evaluating the texts.

Even though attempts were made, by conducting inter-rater reliability tests, to

mitigate possible influences, the likelihood of some bias remains. Each of the three

raters was, to a certain extent, personally invested in the programme’s success: the

author and the undergraduate student have been writing mentors for several years;

the framework developer, who was the third rater, is also the programme’s

administrator. Although, while rating, each rater independently found the same

issues with every text, one cannot entirely eliminate the possibility of some common

bias. Completely eliminating any bias was not practically possible, and may, by

nature, not be possible at all. To avoid bias, raters from outside the programme would

have to be employed. Thus, one would have to ask either the other teachers involved

in the ETTV course (who may not have time to do so, and, given their familiarity with

the framework and the mentoring programme, can arguably also be considered

biased), or external raters (who would have to be taught the framework, and would

have to be paid for their time). As such, although some bias could not be completely

avoided, measures were taken to mitigate this bias.

Thirdly, the texts and reflection used as data for this analysis were made as

part of a single assignment, because of which this evaluation of the programme’s

efficacy was based on a limited point in time: the final weeks of the first semester of

the academic year 2015 - 2016. Thus, objective claims can only be made about this

period. More, and ideally longitudinal, research is required to more fully assess the

programme’s efficacy and the effects of suggestions such as those proposed above

upon said efficacy.


!93

Finally, this paper has focussed uniquely on the student perspective on

programme efficacy (by analysing their written product, and their reflections). The

points of view of other ‘stakeholders’ in the programme, such as the other mentors,

the teachers responsible for teaching and assessing writing, and the English Studies

Group as a whole, could not be included. Future investigations into these perspectives

could reveal further areas where programme efficacy could be improved.

2 Conclusion of the Current Research

During the academic year 2013 - 2014, a peer mentoring programme was established

in the English Studies Group at UGent to assist the teaching of writing by offering

students additional opportunities for support on their written product, as well as at

least some opportunities for cognitive apprenticeship, both of which were defined as

essential by Kellogg (2008) in his research on writer development.

The current Action Research-based applied linguistics paper aimed to conduct

(within the limitations outlined above) a thorough evaluation of the programme’s

efficacy. Based on literature on the evaluation of similar programmes and writing

centres, this evaluation was conducted from two perspectives. The first perspective

was that of a text analysis, comparing pre-mentoring texts to post-mentoring texts.

The second perspective looked at programme efficacy as perceived by students,

through their reflective writing.

To conduct an analysis of pre- and post-mentoring texts, a reliable rating scale,

based on Haas’ framework (2016), had to be developed first. To do so, the author and

two other raters conducted a series of pilot tests and developed such a scale: it was

designed to be applicable to both the number of essays studied in this dissertation,

and large(r) volumes of essays, such as essays submitted for (other) courses. Based on


!94

these pilot tests and post-test discussions, a number of points, such as how to deal

with spelling and/or grammar mistakes (later called L-LOCs), and how to assess

essays that did not meet the assignment’s minimum requirements, were discussed

and ultimately agreed upon.

The rating scale was then applied by the three raters to the corpus of essays

collected. Each rater independently read each essay to identify places of

disengagement, which were then analysed using Haas’ framework. After this, a

general mark was agreed upon, based on the rating scale. This process revealed that

mentoring has a positive effect on the written product of students, with an average

improvement of 27.29%. L-LOC problems in post-mentoring essays are reduced by

42.86%, whereas the frequency of LOC issues was found to decrease by 66.67%. HOC

problems are 42.86% less frequent. This decrease rises to 67.24% if one discounts the

new HOC issues created by students while rewriting their essays after the mentoring

session. In 20% of the cases, all HOC problems of the original draft were resolved;

53.33% of the essays studied show a HOC resolution rate of equal to or higher than

75% following the mentoring session. These numbers indicate the programme

achieves a relatively high efficacy rate, considering the limited one-off timespan of

thirty minutes for a mentoring session.

The analysis also revealed a number of specific areas where the programme

can (further) increase its efficacy. Whereas some HOC problems, such as “Precision”

problems or “Self-Containment” issues have a 100% improvement rate (thus,

generally speaking, solving any such issues that were present in the pre-mentoring

text), other HOC problems showed lower improvement rates. Examples of these are

“Focus too Wide” (50% improvement), “Relevancy” problem (44.44%) and “Complex

Issues” (25%).


!95

To advance the programme’s efficacy in general, and specifically in the areas

described above, a number of suggestions were made that could lead to programme

improvements. The first of these was to encourage mentors to engage with their

student mentees multiple times by asking them to return for follow-up mentoring

sessions. This recommendation is supported by research noting that three mentoring

sessions is an important threshold (Irvin 2014). Doing so could allow for further

guidance of the students, and could give way to resolving the HOC issues created by

students following the mentoring session.

The second recommendation then pertained to mentor instruction. It was

argued that additional instruction, specifically in the areas that saw little HOC

improvements, could lead to additional programme improvements. To do so, the

research suggested employing mentors in the marking process for ETTVI

assignments. This would allow mentors to practise the identification of HOCs and

thus gain a deeper understanding of those cognitive issues, in addition to facilitating

a number of long(er)-term benefits such as increased problem-solving skills. Aside

from that, such involvement could reduce the work load on the course teachers, a

benefit which is backed by research confirming the validity and usefulness of peer

assessment (e.g. Iraji et al. 2016).

The second perspective from which programme efficacy was evaluated was

that of the students. Their written reflections were analysed for elements that indicate

where they perceive the programme to be efficacious, and where further

improvements to efficacy can be made. This analysis of 175 pieces of reflective writing

yielded 229 relevant comments, which were then categorised in eight categories. Only

a small subset of students indicated they did not feel helped; the other seven

categories each pertained to a different reason why students found their mentoring


!96

session to be useful. Aside from practical reasons, such as helping students find a

focus, students indicated the mentoring sessions was helpful to them because they

were (re)motivated to write, because the mentor could explain some concepts they

had not yet fully grasped or because they felt empowered by mentors’ non-directive

approach. Other students offered indirect indicators of programme efficacy by

voluntarily attending multiple mentoring sessions, or by giving their future peers the

advice to attend a mentoring session. In addition to these short-term effects, some

signs of long(er)-term writer development were identified.

Based on these data, a number of additional recommendations for programme

improvements were suggested. The first of these posed that the strict adherence to

the non-directive approach may need to be abandoned in favour of an approach that

mixes the non-directive and directive approaches (as suggested by, for example,

Crasborn et al. (2011)). By, for example, allowing ‘experienced’ mentors to go beyond

non-directive support, more students could be reached, and more students would feel

‘helped’ by their mentor. The second recommendation was to incorporate a

mentoring session into the regular writing classes. This could have the effect of

introducing students to the concept and workings of a mentoring session in a (more)

familiar environment, thus alleviating the anxiety felt by many students when first

approaching a writing mentor (who is, ultimately, a stranger); this could, ideally,

result in students feeling more inclined to schedule a session with a mentor of their

own accord. Finally, the third recommendation suggested mandating attendance of a

mentoring session earlier on in the semester. This would, again, allow for mentors to

be introduced to students earlier, in addition to opening up opportunities for long-

term guidance of students by mentors.


!97

Based on the data analysed, it is argued that, by implementing the suggestions

detailed in this paper, fundamental improvements to both short-term and long-term

programme efficacy can be made, as such increasing the programme’s ability to guide

students during their evolution as writers. Through doing so, this project aspires to

have provided a meaningful contribution to the evolution of the UGent writing

mentoring programme, which, although no longer entirely new, is still growing and

learning how it can optimally meet students’ (and mentors’) needs. It also hopes to

have made a valuable contribution to the field of research concerning peer and

writing mentoring, a field that is still open to much potential research.

3 Suggestions for Future Research

Having summarised the current research, this final section will suggest a number of

avenues for future research in the field of peer mentoring. The UGent writing

mentoring programme, like the ETTV courses within which it largely functions,

constantly evolves and adapts to meet both the changing circumstances as well as the

student audiences it tries to reach. Therefore, it would be improvident to assume that

the programme can reach its end goal merely by implementing the suggestions made

above. More research, more AR cycles, will be needed to re-evaluate and reassess the

programme’s efficacy, and make proposals for further improvements. The

suggestions for such research, below, are not intended to be exhaustive: they are

merely indications of where future research projects might lead.

During the literature overview, this paper summarised the ongoing discussion

between proponents of the non-directive approach, and proponents of a ‘mixed

approach’ (that incorporates both non-directive and directive methodologies). This

discussion has been referred to at several points, and could be an interesting avenue


!98

for future research. Such research could examine audio recordings of mentoring

sessions, and establish if both directive and non-directive mentoring are happening,

and, if so, when and why mentors decide to transition from one approach to the

other. It would be interesting, then, to see which approach is appreciated more by

students, and if a combined approach results in more text improvements. Data for

this point have already been collected, but analysis could not be pursued due to

limitations of time and scope. However, future research projects could pick up this

thread and provide further investigation, thus adding to the ongoing debate in this

field, and potentially uncovering further ways to improve the programme’s efficacy.

A second option unexplored in the current setting is, as noted above, the

perspective of other stakeholders in the mentoring programme. Analysing the points

of view of mentors could reveal ways for further programme optimisations. Do

mentors believe themselves to be efficacious? Do mentors feel like the extra training,

mentioned above, is warranted? And do mentors have other suggestions that could

facilitate the programme’s workings? All of these aspects could be the subject of

further inquiry. Additionally, other stakeholders such as teachers who use essays for

assessment, or the English Studies Group as a whole, could offer additional

perspectives on the programme. These “outsider perspectives” could reveal valuable

information or suggestions that “insiders” may not have thought of; likewise, their

opinions on the programme may help integrate the writing mentoring programme

into more courses, thus again allowing mentors to continue guiding students beyond

the confines of the ETTV courses.

Thirdly, future examination could investigate the applicability of the

mentoring programme’s style and methodology to other languages offered in its

current setting. In other words, this investigation could assess the cross-language


!99

applicability of mentoring, examining if writing mentors can be of help to students

studying languages other than English, and, if so, what adaptations would have to be

made to conform to other languages’ writing standards or needs. Such research would

not require additional mentors, since most members of the current group are already

enrolled in another language (in addition to English). As an added benefit, this could

help integrate the programme in non-English language courses.

Finally, this paper will advocate the need for so-called “slow research” (Lillis

2010). The current research has conducted an evaluation of the programme’s efficacy

at one point in time. For the programme to evolve, conducting a thorough

investigation that includes the perspectives listed above, a longitudinal study

spanning multiple AR cycles would be ideal.

Such a longitudinal timeframe opens up a number of possibilities for research

that are, otherwise, not possible. The first pertains to writer development, which is,

by nature, a slow process that spans decades (Kellogg 2008: 2). Therefore, while this

paper has found limited evidence of writer development, any claims about students’

development as writers can only be truly substantiated by extensive data gathered

over the course of at least several writing assignments, if not multiple years.

Secondly, such long(er)-term inquiries enable the Action Researcher to conduct

repeated reflective cycles. This would allow for more fine-grained analysis and

suggestions than can be achieved within one AR cycle. Thirdly, if other stakeholders

would find it beneficial to expand the programme to languages beyond English,

investigations into its progress and efficacy would require multiple AR cycles. Finally,

it is not unimportant to note that the opportunity to conduct such longitudinal

research, within the Belgian context, is unique to UGent: there are currently no other

institutes (of higher education) that make use of peer mentoring (and peer tutoring)


!100

to the extent that the current mentoring programme does. This offers Ghent

University a unique position, on the one hand, to conduct research that can

potentially fill (a) gap(s) within this field; on the other hand, to maintain its forefront

position in Belgium in this regard by advancing the programme through further

research.

For these reasons, the current paper argues that longitudinal research is

optimal for the continuing improvement and refinement of the peer writing

mentoring programme. Regardless of whether or not such longitudinal research is

possible, the programme will continue to require investigation and adaptation, in

order for it, and its mentors, to continue succeeding in their goal to guide students in

their development as writers.


!101

V References

Archer, Arlene. 2008. Investigating the effect of writing centre interventions on

student writing. Reprinted in Arlene Archer & Rose Richards (eds.), 2011.

Changing Spaces. Writing Centres and Access to Higher Education, 131-146.

Stellenbosch: AFRICAN SUN MeDIA.

Banerjee, Jayanti, Xun Yan, Mark Chapman & Heather Elliott. 2015. Keeping up with

the times: Revising and refreshing a rating scale. Assessing Writing 26. 5-19.

Berckmoes, Dirk & Hilde Rombouts. 2009. Intern rapport verkennend onderzoek

naar knelpunten taalvaardigheid in het hoger onderwijs. Antwerpen:

Universiteit Antwerpen/Linguapolis.

Best, Karen, Laura Jones-Katz, Bailey Smolarek, Marie Stolzenburg & Derek

Williamson. 2015. Listening to our students: An exploratory practice study of

ESL writing students’ views of feedback. TESOL Journal 6(2). 332-357.

Bloom, B.S. (ed.), M.D. Engelhart,, E.J. Furst, W.H. Hill & D.R. Krathwohl. 1956.

Taxonomy of educational objectives, handbook I: The cognitive domain.

New York: David McKay Co Inc.

Bromley, Pamela, Eliana Schonberg & Kara Northway. 2015. Student perceptions of

intellectual engagement in the writing center: Cognitive challenge, tutor

involvement, and productive sessions. Writing Lab Newsletter 39(7-8). 1-6.

Brooks, Jeff. 1991. Minimalist tutoring: Making the student do all the work.

Writing Lab Newsletter 15(6). 1-4.

Brown, Ruth E. 2001. The process of community-building in distance learning

classes. The Journal of Asynchronous Learning Networks 5(2). 18-35.

References Thijs Gillioen

!102

Campbell, Toni A & David E. Campbell. 1997. Faculty/student mentor program:

Effects on academic performance and retention. Research in Higher

Education 38(6). 727-742.

Çevik, Yasemin Demiraslan. 2015. Assessor or assessee? Investigating the differential

effects of online peer assessment roles in the development of students’

problem-solving skills. Computers in Human Behavior 52. 250-258.

Cho, Kwangsu & Charles MacArthur. 2010. Student revision with peer and expert

reviewing. Learning and Instruction 20(4). 328-338.

Chraif, Michaela. 2015. Correlative study between academic satisfaction, workload

and level of academic stress at 3rd grade students at psychology. Procedia -

Social and Behavioral Sciences 203. 419-424.

Council of Europe. 2001. Common European Framework of reference for

languages: Learning, teaching, assessment. Cambridge University Press /

Council of Europe.

Crasborn, Frank, Paul Hennissen, Niels Brouwer, Fred Korthagen & Theo Bergen.

2011. Exploring a two-dimensional model of mentor teacher roles in

mentoring dialogues. Teaching and Teacher Education 27(2). 320-331.

Davies, Alan & Catherine Elder. 2008. Handbook of applied linguistics. New York:

John Wiley & Sons.

Dawson, Philip. 2014. Beyond a definition: Toward a framework for designing and

specifying mentoring models. Educational Researcher 43(3). 137-145.

Deane, Paul. 2013. On the relation between automated essay scoring and modern

views of the writing construct. Assessing Writing 18(1). 7-24.


!103

De Soete, Alexander. 2016. Analyzing Reader Engagement in NPO Newsletters: Is

Haas’ Reader Engagement Framework Applicable to Dutch NPO

Newsletters?. Ghent: Ghent University BA paper.

De Wachter, Lieve & Jordi Heeren. 2012. TaalVaardig aan de STart. Gerichte

ondersteuning van academische taalvaardigheid aan de KU Leuven.

Proceedings of the 2012 “Van Schools tot Scriptie” Colloquium. 55-68.

Domínguez, César, Arturo Jaime, Ana Sánchez, José Miguel Blanco & Jónathan

Heras. 2016. A comparative analysis of the consistency and difference among

online self-, peer-, external- and instructor-assessments: The competitive

effect. Computers in Human Behavior 60. 112-120.

DuBois, David L., Bruce E. Holloway, Jeffrey C. Valentine & Harris Cooper. 2002.

Effectiveness of mentoring programs for youth: A meta-analytic review.

American Journal of Community Psychology 30(2). 157-197.

English, Rebecca & Jennifer Duncan-Howell. 2008. Facebook© goes to college:

using social networking tools to support students undertaking teaching

practicum. MERLOT Journal of Online Learning and Teaching 4(4). 596-601.

Ferris, Dana R. 2016. Promoting grammar and language development in the

writing class. Why, what, how and when. In Eli Hinkel (ed.), Teaching

english grammar to speakers of other languages, 222-246. New York:

Routledge.

Flemish Government. 2016. Secundair onderwijs - Derde graad ASO - Moderne

vreemde talen Frans-Engels - Eindtermen. http://www.ond.vlaanderen.be/

curriculum/secundair-onderwijs/derde-graad/aso/vakgebonden/moderne-

vreemde-talen-frans-engels/eindtermen.htm. (1 May, 2016.)


http://www.ond.vlaanderen.be/

!104

Gielen, Sarah, Elien Peeters, Filip Dochy, Patrick Onghena & Katrien Struyven. 2010.

Improving the effectiveness of peer feedback for learning. Learning and

Instruction 20(4). 304-315.

Girgensohn, Katrin. 2012. Mutual growing: How student experience can shape

writing centers. Journal of Academic Writing 2(1). 127-137.

GO! Onderwijs van de Vlaamse Gemeenschap. 2014. Leerplan ASO - 3e graad -

Basisvorming AV Engels. http://pro.g-o.be/blog/documents/2014-014.pdf. (1

May, 2016).

Gopee, Neil & Mary Deane. 2013. Strategies for successful academic writing —

Institutional and non-institutional support for students. Nurse Education

Today 33(12). 1624-1631.

Haas, Sarah. 2009. Writers’ groups for MA ESOL students: collaboratively

constructing a model of the writing process. English Language Teacher

Education and Development 12. 23-30.

Haas, Sarah. September 15, 2013. “Peer Mentoring for first year students”. Personal

e-mail correspondence with author and other potential mentors.

Haas, Sarah. 2014. ETTVI lecture slides [PowerPoint slides]. Retrieved from

http://minerva.ugent.be/main/course_home/course_home.php?

cidReq=A00286802014. (3 March, 2016).

Haas, Sarah. 2016. Understanding reader engagement: A framework for teaching

and assessing writing. [manuscript].

Harbord, John. 2003. Minimalist tutoring - An exportable model? The Writing Lab

Newsletter 28(4). 1-5.


http://pro.g-o.be/blog/documents/2014-014.pdf

http://minerva.ugent.be/main/course_home/course_home.php?

!105

Harrington, Kathy, Peter O’Neill & Savita Bakhshi. 2007. Writing Mentors and the

writing centre: producing integrated disciplinary writers. Investigations into

university teaching and learning 4(2). 26-32.

Hattie, John. 2009. Visible Learning. A synthesis of over 800 meta-analyses

relating to achievement. Milton Park, Oxon: Routledge.

Hepplestone, Stuart, Graham Holden, Brian Irwin, Helen J. Parkin & Louise Thorpe.

Using technology to encourage student engagement with feedback: a literature

review. Research in Learning Technology 19(2). 117-127.

Hoon, Tan Bee. 2009. Assessing the efficacy of writing centres: A review of selected

evaluation studies. Pertanika Journal of Social Sciences & Humanities 17(2).

47-54.

Hutchings, Cathy. 2006. Reaching students: lessons from a writing centre. Higher

Education and Development 25(3). 247-261.

Iraji, Hamid Reza, Mostafa Janebi Enayat & Mahyar Momeni. 2016. The effects of

self- and peer-assessment on Iranian EFL learners' argumentative writing

performance. Theory and Practice in Language Studies 6(4). 716-722.

Irvin, L. Lennie. 2014. What a difference three tutoring sessions make: Early

reports of efficacy from a young writing center. The Writing Lab Newsletter

39(1-2). 1-5.

Jones, Ian & Chris Wheadon. 2015. Peer assessment using comparative and absolute

judgement. Studies in Educational Evaluation 47. 93-101.

Karcher, Michael J., Gabriel P. Kuperminc, Sharon G. Portwood, Cynthia L. Sipe &

Andrea S. Taylor. 2006. Mentoring programs: A framework to inform program

development, research, and evaluation. Journal of Community Psychology

34(6). 709-725.


!106

Kellogg, Ronald T. & Bascom A. Raulerson III. 2007. Improving the writing skills of

students. Psychonomic bulletin & review 14(2). 237-242.

Kellogg, Ronald T. 2008. Training writing skills: A cognitive developmental

perspective. Journal of Writing Research 1(1). 1-26.

Kullman, John. 1998. Mentoring and the development of reflective practice: concepts

and context. System 26(4). 471-484.

Lewin, Kurt. 1946. Action research and minority problems. Journal of Social Issues

2(4). 34-46.

Li Gwet, Kilem. 2012. Handbook of inter-rater reliability. Third edition. The

definitive guide to measuring the extent of agreement among raters.

Gaithersburg, MD: Advanced Analytics, LLC.

Lillis, Theresa. 2010. The value of local research for sustaining writing development

in higher education: the case of ‘academic literacies’. WDHE [Writer

Development in Higher Education] Conference June 2010. Retrieved from

http://www.writenow.ac.uk/news-events/wdhe-conference-2010/conference-

presentations/monday-28-june-2010/#Keynote1. (10 May, 2016).

Liu, Ngar-Fun & David Carless. 2006. Peer feedback: the learning element of peer

assessment. Teaching in Higher Education 11(3). 279-290.

Martinez, Christy Teranishi, Ned Kock & Jeffrey Cass. 2011. Pain and pleasure in

short essay writing: Factors predicting university students’ writing anxiety and

writing self-efficacy. Journal of Adolescent & Adult Literacy 54(5). 351-360.

Min, Hui-Tzu. 2006. The effects of trained peer review on EFL students’

revision types and writing quality. Journal of Second Language Writing 15(2).

118-141.


http://www.writenow.ac.uk/news-events/wdhe-conference-2010/conference-

!107

Murphy, Tim. 1998. Motivating with near peer role models. The Japan Association

for Language Teaching: On JALT’ 97: Trends & Transitions. 201-206.

Orland-Barak, Lily & Ronit Hasin. 2010. Exemplary mentors’ perspectives towards

mentoring across mentoring contexts: Lessons from collective case studies.

Teaching and Teacher Education 26(3). 427-437.

Panadero, Ernesto, Margarida Romero & Jan-Willem Strijbos. 2013. The impact of a

rubric and friendship on peer assessment: Effects on construct validity,

performance, and perceptions of fairness and comfort. Studies in Educational

Evaluation 39(4). 195-203.

Park, Soo-Kyung. 2015. The interplay of task, rating scale, and rater background in

the assessment of Korean EFL students’ writing. English Teaching 70(2).

55-82.

Pleasant, Scott. 2015. It’s not just beans anymore; It’s our bread and butter. Writing

Lab Newsletter 39(9-10). 7-10.

Reason, Peter & Hillary Bradbury (eds.). 2001. Handbook of action research.

London: Sage Publications.

Richards, Jack C. & Richard W. Schmidt. 2013. Longman dictionary of language

teaching and applied linguistics. London: Routledge.

Rodger, Susan & Paul F. Tremblay. 2003. The effects of a peer mentoring program

on academic success among first year university students. The Canadian

Journal of Higher Education 33(3). 1-18.

Ruiz-Gallardo, José Reyes, José L. González-Geraldo & Santiago Castaño. 2016. What

are our students doing? Workload, time allocation and time management in

PBL instruction. A case study in Science Education. Teaching and Teacher

Education 53. 51-62.


!108

Sagor, Richard. 2000. Guiding school improvement with Action Research.

Alexandria: Association for Supervision and Curriculum Development.

Salinitri, Geri. 2005. The effects of formal mentoring on the retention rates for

first-year, low achieving students. Canadian Journal of Education 28(4).

853-873.

Saliu, Sokol. 2005. Constrained subjective assessment of student learning. Journal

of Science Education and Technology 14(3). 271-284.

Salomon, Gavriel. 1993. On the nature of pedagogic computer tools: The case of the

writing partner. In Susanne P. Lajoie & Sharon J. Derry (eds.), Computers as

cognitive tools. 179-196. Hillsdale, NJ: Lawrence Erlbaum Associates.

Schmidt, Katherine M. & Joel E. Alexander. 2012. The empirical development of an

instrument to measure writerly self-efficacy in writing centers. The Journal

of Writing Assessment 5(1). pp. n.a.

Shermis, Mark D. 2014. State-of-the-art automated essay scoring: Competition,

results, and future directions from a United States demonstration. Assessed

Writing 20. 53-76.

Stevenson, Marie & Aek Phakiti. 2014. The effects of computer-generated feedback on

the quality of writing. Assessing Writing 19. 51-65.

Topping, Keith & Stewart Ehly. 1998. Peer-assisted learning. New York: Routledge.

Vista, Alvin, Esther Care & Patrick Griffin. 2015. A new approach towards marking

large-scale complex assignments: Developing a distributed marking system

that uses an automatically scaffolding and rubric-targeted interface for guided

peer-review. Assessing Writing 24. 1-15.

Vlaams Verbond van het Katholiek Secundair Onderwijs (VVKSO). 2014. Engels -

Derde graad ASO - studierichtingen met component moderne talen: leerplan


!109

secundair onderwijs. http://ond.vvkso-ict.com/leerplannen/doc/

Engels-2014-003.pdf. (1 May, 2016).

Warschauer, Mark & Douglas Grimes. 2008. Automated writing assessment in the

classroom. Pedagogies: An International Journal 3. 22-36.

Weigle, Sara Cushing. 2013. English language learners and automated scoring of

essays: Critical considerations. Assessing Writing 18(1). 85-99.

Zepke, Nick & Linda Leach. 2010. Student engagement: Ten proposals for action.

Active Learning in Higher Education 11(3). 167-177.

Zundert, Marjo van, Dominique Sluijsmans & Jeroen van Merriënboer. 2010.

Effective peer assessment processes: Research findings and future directions.

Learning and Interaction 20(4). 270-279.


http://ond.vvkso-ict.com/leerplannen/doc/

!110

VI Appendices

Overview

Appendix 1 ETTVI Scribing 3 Assignment pp. 111 - 116

Appendix 2 Overview of General Responsibilities pp. 117

Appendix 3 Analysed Student Essays pp. 118 - 213

Appendix 4 Samples from Reflective Writing pp. 214 - 239

Appendices: Overview Thijs Gillioen

Because the appendices to this paper contain samples of student writing and student

reflective writing, they were not made available for open access, so as to avoid any privacy

concerns.

The appendices can be requested for consultation at [email protected].

mailto:[email protected]

writing mentors guiding student...

Documents