Academic SkillsSteve Campitelli Dr Jane Page

Academic Skills M Teach Early Childhood

Anatomy of a successful

partnership

http://www.ptid.com.au/assets/images/projects/UniMelb%20GSE/UniMelb_GradSchEdu_04.jpg

Academic Skills

Part 1: collaboration

Part 2: research project

Presentation

Academic Skills

http://www.ptid.com.au/assets/images/projects/UniMelb%20GSE/UniMelb_GradSchEdu_04.jpghttp://education.unimelb.edu.au/__data/assets/image/0005/1144292/early-childhood-web.jpg

Master of Teaching, Early Childhood (EC)

MGSE Globally ranked

Graduate Program

Clinical model

Interventionist teachers

Theory / practice nexus

Local and international cohort

Sts from range of backgrounds

Academic Skills

(*Stevenson & Kokkinn, 2012)

Academic Skills (AS)UoM professional group – 11 staff

Working in academic area

Context*: Faculty/Discipline based

Individuals: 1-to-1 ‘itutes’, 4 X 30-min appts / semester, sts self-book, can choose adviser – focus shift from provider to user

Groups: workshops / presentations - open & targetted

Construct*: “teaching of academic literacies”

Academic independence – not long term dependence

Skills – not proofreading

Ability to engage with academic program

Collaborate with academics

Academic Skills

Enabling independence through skills, not content

Academic Skills

Support for students - IndividualFollow up individual sessions at AS for any sts Also sts flagged as requiring assistance at diagnostic and during program

Nature of the collaboration

AS-EC collaboration well-established & multi-faceted

Initial student contactAS workshop – Transition to PGDiagnostic writing assessment

Advice to AcademicsFeedback on assignment briefs: focus clarity, accessibility

Support for students - GroupMeet academics, discuss assignmentTargeted (tailored) workshops in Sem 1 Support specific assignmentsTails off in Sem 2 - independence

Academic SkillsNature of the collaboration

Attendance at EC staff meetings

AS attend EC meetings

Keep abreast of program

Have input into program issues

Advice to Academics

Join EC Committees

E.g. assessment, international sts needs

Developed assessment grading documents

Academic SkillsKey elements of collaboration

Starts early – AS support from day 1

Continues through semester in a linked way

Multi-faceted initiative – dichotomous

Students & teaching staff

Group & individual level

In class & out of class

Academic SkillsKey elements of collaboration

Tailored for target group

Consistent communication between AS & EC

Academic (staff) buy-in

AS knowledge of what EC is doing and vice-

versa

Academic SkillsWhy does it work? Graduate School

Strong knowledge of each other’s work and focus

Shared common vision and commitment to interdisciplinary partnerships

Valuing of expertise of AS and MGSE

Meaningful to both academic and AS staff members – collaborative learning

Integrated throughout the program

Relates and flows into student support processes

Academic Skills

Interventionist orientation – positivist model

Sts know AS – relationship established

AS is viewed as ‘part of the EC team’ – workshops, meetings, committees – not external or ‘add-on’

Good service fit: EC values AS work (not always a given!)

Complementary: EC content + AS skills

EC highly values student welfare, committed to engagement

Understanding of nature of service – not just ‘remedial’

Champions in Graduate School set tone

Why does it work? Academic Skills

Academic SkillsKey Learnings

Locate / cultivate a champion

Champion needs to be proactive, a ‘doer’, as do you!

Encourage a ‘can do’ or a ‘do what you say you will’ attitude (within reason)

Source good service fit – AS fits the interventionist model of MGSE

Academic SkillsKey Learnings

Value of the service related to provision, but also to the attitude of the other party – champion is essential in this

Identify key stakeholders – get on board

Communicate and deliver

Multi-faceted, timely, linked approach is successful

Led us to thinking about evaluating impact …

Academic SkillsResearch: “So … what difference does it make?”

Project

Measure impact of individual advising

What differences does academic advising make?

Focus on writing – most common assistance

EC a rich and willing research area

Academic SkillsLiterature

Tendency to focus on justification rather than evaluation Wider context of economic rationalisation on 1-1 service provision

(Stevenson & Kokkinn, 2009; Chanock, 2007)Evaluation of 1-1 is difficult & lags behind program evaluation; often not

open or scrutinised (Chanock, 2002)

Kasper (1997): content-based instruction courses for ESL students Song (2006): impacts of content-based instruction courses on results

and pass rates Woodward-Kron (2009): how writing improves with disciplinary

knowledgeBaik & Grieg (2009): impact of a first-year adjunct ESL program

Storch & Tapper (2009): impact of an EAP course on writing of postgrad students

Academic SkillsLiterature

Clerehan (1997) & Chanock (2000): dialogic learning

Chanock (2007): how 1-1 informs classes, rel’n to group teaching (also

1996 conf.)

Berry et al. (2013): evaluation process of individual consultations

Chanock (2002): variables that make 1-1 eval difficult; intangibles of

the 1-1 context & wide range of influences on student success;

cautions reliance on marks

McLean & Webb (2002): assumptions our work impacts positively,

variables influencing outcomes are beyond our control

Academic SkillsResearch

Intention - aimsTo quantify the effect of Academic Skills (AS) intervention with

a selected group of 1st year Master of Teaching Early

Childhood (EC) Teacher Candidates (TCs)

Research questionWhat measurable effect does AS intervention have on the

performance outcomes in academic writing tasks completed

by first year Master of Teaching (EC) TCs in the first semester

of their academic program?

Internal question:

What if we weren’t here?

https://zombiesruineverything.files.wordpress.com/2013/09/pro6.jpg

Academic SkillsResearch

How?

Students produce an essay response to an assessment

requirement in their first semester

2000w / 50%

Examine non-assisted & assisted versions of a piece of

writing for differences in result

Tried for a ‘closed loop’

Academic SkillsResearch - process

Workshop on essay

task

Sts work on DraftPaper #

Sts submit Draft #

Draft blind

marked

1st AS itute* on Draftpaper

Sts work on Draftpaper

2nd AS itute on

Draft

Submit Finalpaper

Final paper

marked

2 papers compareditute: indiv. 30 minall cohort can access

# unassisted

Academic Skills

AS Workshop 5/3 – all cohort

All cohort submit Final paper 19/42 X AS appts from 27/ 3 to 17/4, Participants work on Final paper

Participants work on draft papers, unassisted

Submit draft 27/5

Workshop to 1st draft: 22 days1st to Final draft: 22 days

Academic SkillsAnalysis

Quantitative Analysis

Difference in marks btw draft and final

Post-assistance questionnaire to participants

Qualitative Analysis

Comments on papers from markers

Comments on papers from Adviser

Reflections from markers and Adviser

Academic SkillsParticipants

Participants: 12 students9 int / 3 Eng L1 (looking for 40! …)

First semester, first year Masters

Issues around timing re placement: early in program

Academic Markers: 2Pre-marking meeting: papers blind marked, same markers

Familiarity with assessment

Rubric / criteria developed internally

Academic SkillsMarking mechanismConceptual

understanding50%

Structure & organisation

30%

Evidence of research

20%

Analytic criteria- generic

Raw scores out of 10 then re-weighted

Bands correspond to UoM grading bands: H1, H2 etc.

Academic SkillsResults of marking from draft to final

All marks went up, none went down!

1 grade pos average shift - ave Pass grade to ave H3 grade*

(*cohort final average H3)

11 of 12 sts improved 1-2 grades; 1 sts’ grade stayed same

(improved 3%)

14.3% average positive mark shift

Shift varied from 3% to most dramatic shift of 30% (2 sts)

4 fails in draft 1 to 0 fails at final

6 students with word count issues to 1 (penalties apply)

H1 80+H2A 75-79H2B 70-74H3 65-69P 50-64F <50

Academic SkillsResults of marking from draft to final

Gain areas (out of 10):

1.0 ave gain in Structure / organisation

0.95 ave gain in Research / referencing

0.8 ave mark gain in Conceptual understanding

Shifts: Draft:

1 X H2B

2 X H3

5 X Pass

4 X Fail

Final:

4 X H2B

5 X H3

3 X Pass

0 X Fail

So:

Managed to flip profileH1 80+H2A 75-79H2B 70-74H3 65-69P 50-64F <50

Academic SkillsSurvey of participants

Survey results: Likert scale N=10 – all agreed that

Workshop increased confidence

Itutes helped me better understand task; structure & org; express myself;

grammar; referencing

Open question results

How itutes most helped you:

Linking & flow / Structure organisation of ideas 5

Grammar / Language help / expression - word choice 4

Helped me get it in on time / pace myself 3

Referencing 3

Academic SkillsSurvey of participants

Open question results

Effective features of itutes :

Instructions on cohesion / linking 4

Structure and organisation of essay 3

Citation and referencing 3

“Really good for my mental health”!

Least effective features

Nothing 3

Only 30 mins long / Felt rushed 2

Closed question results

Get other help?

No 5

Yes 5 (classmates, partner)

Used AS before?:

No 9 Yes 1

Would use again?

Yes 10

Academic SkillsResults – what did markers identify as issues?

Expression / language – awkward, unclear, grammar, word choice, sentence length, lack of signposting, run-on sentences or overly complex confusing sentences , ‘shopping list’ expression, inappropriate (overly informal)

Organisation – paragraphing (at times, severe), distinct sectioning and separation of information

Inappropriate presentation of info – tables bullet lists

Referencing / citation issues – use of first name, not in list, conventions, missing citations, overuse of long quotes, ref too old, not what authors said

Clarity of argument – point being made, argument hard to follow

Representative of common student issues (Berry et al., 2012)

Academic SkillsResults – what did markers identify as strengths?

Most had very clear intro and conclusion sections

Clear expression

Good support, link to theory / literature

Good examples, clear definitions

Strong points – well expressed

Logical structure and flow

Main learnings

The markers and I agreed on the weak and strong points

BUT … I did not identify the places for extra theoretical support or particular academic sources as much as the academics did

AS focus on expression, structure, links, cohesion, citation – though there is flow-on effect from the mechanical to the conceptual

Academic SkillsLimitations

Sts knew 1st draft had to be ready to be marked, but knew

assistance was going to happen, so may not have ‘tried’

100% to submit a ‘finished’ first draft

On placement, time - rushed

Submitted first version earlier than everyone else

Markers subconsciously looking for improvement

No access to markers’ draft feedback

Small group, one paper, single discipline

Not longitudinal

Academic SkillsMarkers’ reflections

Disconnect between workshop and what sts are doing

Made me think about how I can connect to AS service

Made me analyse what a mark is, what it means, what an essay

looks like, my feedback

Made me change my assessment structure

Value of clear marking rubric – shared understandings

Emotional landscape of writing – making sure the feedback and

marks are right

Training in feedback – marking private, not scrutinised

Reasonable for ppl to want to know what we do (Chanock, 2002)

Academic SkillsEffects of advising on writing?

Advising on the right things

Most effect in expression/language, structure, organisation,

linking, citation, word count, getting paper submitted

Limited content influence … but it is also there

We do make a positive difference

Importantly…

we don’t muck things up! – great points stay as great points!

Academic SkillsAdviser’s reflections

Disconnect between stated workshop advice and what they are doing – timing? Early v JIT

Greater awareness of support being required at key content points -

boundaries between the mechanical aspects of writing and

content (Berry et al., 2013)

Some students feel the itute context is rushed

Itutes providing the windows into issues (Chanock, 2007)

Value of being on the ‘same page’ as academics in terms of task,

expectations, structure, marking system – holistic

Academic SkillsConclusions

Make a positive difference – this study: a grade’s worth

Areas where we have expertise – structure, org, expression, citations – but that this has positive flow-on effect in content

Adjustments to cultural requirements of writing required –– the ‘craft of academic writing’ (O’Mahony et al., 2013)

Linked, multifaceted, holistic approach works – esp. when there is shared understanding of the task

Timing of workshop intervention – early to JIT

Approach to marking: emotional investment, feedback

Approach to assignment structure: 1-1 effect on classes (Chanock, 2007)

One-to-one assistance works

Academic SkillsFuture directions

More research – follow marks, does advice apply further?*

*crude result: all sts rec. higher marks for subsequent essay in other subj. suggesting advice being applied out of 1-1 context

Longitudinal, cross-disciplinary study …

Academic SkillsSo …

… what would happen if we weren’t here?

Students wouldn’t perhaps do as well as they could, but …

Would perhaps not be able engage as completely and

rigorously with the program as they could

We are able to enhance engagement with the program:

enabling sts to optimise the academic inputs they receive

with the skills to engage with them

Academic SkillsQuestions

Academic SkillsBibliography

Baik, C. & Grieg, J. (2009). Improving the outcomes of undergraduate ESL students: the case for discipline-based academic skills programs. Higher Education Research & Development. 28(4), 401-416.

Berry et al. (2013) Individual consultations: towards a 360-degree evaluation process. Journal of Academic Language & Learning. 6(3). 16-35.

Chanock, K. (2000) ‘You get me to explain myself more better’: Supporting diversity through dialogic learning. In R. James, J. Milton & R.Gabb (Eds). Research and Development in Higher Education volume 22: Cornerstones of higher education (pp. 53-67). Melbourne: HERDSA.

Chanock, K. (2002). Problems and possibilities in evaluating one-to-one language and academic skills teaching. In J. Webb & P. McLean, P. (Eds.). Academic Skills Advising: Evaluation for program improvement and accountability (pp. 199-221). Melbourne: VLLN.

Chanock, K. (2007). Valuing individual consultations as input into other modes of teaching. Journal of Academic Language and Learning, 1(1), 1-9.

Clerehan, R. (1997). How does dialogic learning work? In K. Chanock, V. Burley, & S. Davies (Eds.). What do we learn from teaching one-to-one that informs our work with larger numbers? Proceedings of the conference held at La Trobe University November 18-19, 1996 (pp. 69-81). Melbourne: Language and Academic Skills Units of La Trobe University.

Kasper, L. (1997). The impact of content-based instructional programs on the academic progress of ESL students. English for specific purposes. 16(4), 309-320.

O’Mahony, B., Verezub, E., Dalrymple, J,. & Bertone, S. (2013). An evaluation of students’ writing support intervention. Journal of International Education in Business. 6(1), 22-34.

Song, B. (2006). Content-based ESL instruction: long-term effects and outcomes. English for specific purposes. 25(2006), 420-437.Stevenson, M. & Kokkinn, B (2009). Evaluating one-to-one sessions of academic language and learning. Journal of Academic Language &

Learning. 3(2). 36-50.Storch, N. & Tapper, J. (2009). The impact of an EAP course on postgraduate writing. Journal of English for Academic Purposes. 8(2009),

207-223. Webb, J. & P. McLean, P. (2002) (Eds.). Academic Skills Advising: Evaluation for program improvement and accountability. Melbourne:

VLLN.Woodward-Kron, R. (2009). ‘‘This means that.’’: a linguistic perspective of writing and learning in a discipline. Journal of English for

Academic Purposes. 8(2009), 165-179.