SOUTHAMPTON SOLENT UNIVERSITY Solent Teaching and Learning Community

Dialogue

2

January 2012

2

SOLENT TEACHING AND LEARNING COMMUNITY

Dialogue

SOLENT PEDAGOGIC RESEARCH NETWORK TEAM

Sara Briscoe • FBSE Donna Peberdy • FCIS

Chris Patterson • MARTEC

Number 2 • January 2012

Contents

Editorial………………………………………………………………………………………………………………………………4 Engaging Blended Learning Students: An evolving approach to engaging students through the VLE Steve Hogg and Andrew Doig…..…………………………………………………………………………………………5 Assessment: exploring a new discourse Dr Elizabeth Mytton………………….………………………………………………………………………………………11 Learning strategies for student engagement and achievement Carina Buckley…………………………………..………………………………………………………………………………21 Report for Dialogue on Professor Graham Gibbs’ seminar; How to change assessment of degree programmes to improve student learning. Deborah Wright…………………………….……………………………………………………………………………………28 Reading Lists and myReferences: Integrating RefWorks and myCourse Hannah Young…………………………………….………………………………………………………………………………31 Editorial Guidelines……………………………………………………………………………………………………………39

3

Editorial

Welcome to this second edition of Dialogue the journal of the Solent Teaching & Learning Community (STLC) at

Southampton Solent University. Dialogue seeks to disseminate pedagogic research and promote the exchange of best

practice in teaching and learning. The papers in this edition are timely and clearly focus on the student learning

experience from a number of differing perspectives.

Steve Hogg and Andrew Doig highlight the importance of the VLE in supporting student learning and reflect on recent

developments a Solent University. The blended and distance learner are characterised and the authors discuss the role

of the Flexible Delivery Development and Support Team (FDDST) in developing the Solent Online Learning Standard and

working with academics to support courses.

In an insightful paper Elizabeth Mytton examines contemporary assessment approaches in the context of legal education.

This paper discusses how the development of the new LLB (Hons) was informed by the University steer on designing

curricula to develop relevant skills and provide real-world learning opportunities.

Carina Buckley explores the use of one to one student skills tutorials. Based on her observational experience and a

follow up survey she presents a model that highlights appropriate learning strategies in relation to both the student’

confidence level and the purpose of the student’ visit. This paper is useful in strengthening our understanding of how

we can better support student skills development.

Deborah Wright offers a useful report on Professor Graham Gibb’s seminar held at Oxford Brookes Business School. This

report offers some practical suggestions on how to engage the student more effectively in the assessment process.

Finally, Hannah Young reports on the Library pilot project investigating the feasibility of using RefWorks bibliographic

software to generate reading lists and make them available to students on myCourse. Level four and five student’

experiences with the system are reported together with an evaluation of the software for wider use.

From working with students face-to-face and one-to-one, to supporting distance students, from curriculum to

assessment, we hope you will find the variety of perspectives and approaches to teaching and learning useful and

informative.

Thank you to all of these authors for sharing their work. We hope you will consider submitting your research for inclusion

in further editions. Finally we would like to thank Jenny Watson for helping to collate this second edition.

EDITORIAL BOARD

Sara Briscoe, FBSE

Donna Peberdy, FCIS

Chris Patterson, MARTEC

4

D I A L O G U E

Engaging Blended Learning Students: An evolving approach to engaging students through the VLE

Steve Hogg, Head of Learning Technologies LIS Andrew Doig, Flexible Courses Instructional Developer, LIS

Abstract Students taking part-time, distance or blended learning units who are also in employment face high commitment demands of work, life and family in addition to their study. They do not have time to face the additional challenge of making sense of difficult to access learning materials. These students are also often highly discerning, and will talk with their feet – failing to complete units that don’t engage them. At Southampton Solent University (SSU), we recognise the need to make online or blended units accessible and supportive. In order to engage students on these, we have developed a set of institutional standards for online course development that aim to make materials intuitive, easy access, clearly introduced and well-signposted. The standards also identify levels of support and collaboration in order for students to feel both engaged by and to gain maximum benefit from the learning processes. At the same time, we have established a Flexible Delivery Development and Support Team (FDDST) which collaborates with academic staff in course planning, writing and delivery. This team works with tutors to achieve the standards while aligning learning outcomes and assessment with online and blended learning activity. This paper explores the stages that led to establishing our institutional standards for online course development and the scope of activities for the FDDST. We shall explore ways in which the activities of the team have been responsive and adaptive to student experiences, and illustrate some impacts of these developments on both actual and anticipated student engagement, achievement and retention. This is an edited version of the paper that was presented in draft form as an interactive workshop at the ALDHE 2011 conference in Belfast, and has since been submitted for publication in the Journal of Learning Development in Higher Education in 2012. Keywords: online, distance, blended learning, engagement, students in employment, virtual learning environment

The Discerning Adult Learner As an institution, Southampton Solent University (SSU), like many other providers in the sector, is increasing its provision of courses aimed at part-time, mature, often professional learners delivered through blended and distance learning. The online aspects of this blended learning are enabled through the use of the University’s Moodle-based Virtual Learning Environment (VLE). However, blended learning courses are not entirely new to the University, and through a survey of the academic staff delivering existing blended learning courses, some recurring characteristics of this new constituency of student could be identified.

5

D I A L O G U E

These students:

• are in work • are in a variety of industries • have reached a ‘peak’ level in their profession and so require further qualification to

move up or on • have a very specific first degree • often have young families • have limited available time • have not been in formal education for some time (Doig 2011)

When presenting on their experience of running a blended learning Masters degree at SSU, Patrick and Newell (2009) shared an impactful quotation from one of their students: “one point about myCourse is, that’s effectively our university, that’s our lives”. The VLE is central to the learning experience of the blended and distance learner, and they will demand that the VLE is put to effective use to enable their learning. The demands of taking on study while working, and very often taking care of a family, mean that these students will become impatient with learning provision of variable quality. The adult learner is more discerning and expects a quality learning experience. Further, as universities strive to find new and more efficient income streams and to attract new learner constituencies, it is likely that the balance between blended and distance learning will move more towards distance. This is already occurring at SSU, where we have seen demand for entirely distance provision increase against blended because of the inability of some students to take on the commitment of face-to-face hours. It is imperative then to keep in mind that the distance student can easily feel isolated. Creating a sense of community and creating a supportive environment is essential. However, it is very difficult to make e-learning engaging and a true learning experience. How do you create a learning experience within the VLE that is every bit as good as attending a great lecture or seminar session? How is it possible to create an experience where the student feels afterwards: ‘I really learned something there’?

Building a concept of high-quality course content In 2009, SSU was awarded funding from the Higher Education Funding Council for England (HEFCE) and created a Strategic Development Fund to run across three years. Under this funding and the Strategic Development Programme (SDP) that resulted from it, SSU’s e-Development Centre successfully proposed the creation of a Flexible Delivery Development and Support Team (FDDST). The objective of this team is to bring together learning technologists, instructional and media developers along with academic staff to work on the development of high-quality course content for online blended and distance learning courses (and build on the good work to date carried out in providing technology-enhanced learning and blended learning within the University). One of the first activities of the FDDST was an informal survey of the e-learning landscape within, and beyond, the Higher Education (HE) sector. The survey was advantaged by the move towards open education resources among institutions such as the Open University (OU), the University of Nottingham, MIT and Stanford, as well as the wealth of other examples of e-learning provision openly available, such as the BBC Academy. This survey was unstructured, as in it did not depend on fixed criteria of what defines high-quality but rather was based on the received knowledge of our internal team of e-learning experts drawing comparisons and conclusions on what would be achievable through our available human and technical resources. A variety of impactful features were identified from various providers. For instance:

• The OU makes excellent use of the Moodle book, providing extended narrative delivery of content and enabling a tutor presence, while also embedding various kinds of

6

D I A L O G U E

learning activities aimed at student engagement within the materials • Stanford University has gone to great lengths to provide a series of video lectures that

appear in all of their Open Educational Resources, thus giving a strong sense of university identity, as well as personalising the expertise held within the institution

• Nearly all providers used a consistent presentation of content within the courses or units that they deliver; various features such as learning or communication tools as well as navigation features are presented consistently within the same frames of the web page

• Commercial providers such as lynda.com and Adobe TV present very clear introductions to each set of learning materials through high-quality video; highly engaging as well as clarifying the purpose of the resources.

Arriving at a recipe for success - the Solent Online Learning Standard The results of this survey, combined with reflection on the many years of collected experience of working in e-learning design and delivery have been used by the FDDST to arrive at a recipe for successful online course delivery. This recipe is a collation of standards and strategies that we believe improve the learning experience for students participating in online blended or distance learning courses. We have chosen to call this collation the ‘Solent Online Learning Standard’ (SOL Standard). This term has been applied usefully at an institutional level to suggest a set of standards or principles that all course developers and facilitators can aim to achieve. These standards being incorporated by SSU’s Academic Services in to policy placed in the Academic Handbook relating to a ‘Framework for online learning’. It is understood that new components to the SOL Standard will be recognised as new courses are delivered and feedback received, as well as that, not all components will be applicable to all teaching contexts. It is our hope that a washback effect will occur, where learning and understanding gained in the application of the Solent Online Learning Standard with distance and blended courses can be applied more broadly across the rest of the university’s, mainly full time face-to-face courses. We are assisted in this by the work conducted by Almpanis et al (2010) in developing their Blended and Flexible Learning Framework, which identifies four different designs for blended learning that include varying levels of engagement with learning materials and peers, moving from minimum requirements at design 1 through to mixed individual and group study in design 4. The four designs of the Blended and Flexible Learning Framework should help us to provide design templates aimed at allowed tutors to more easily apply different levels of learner engagement through their online course materials. The elements of the SOL Standard are illustrated below:

7

D I A L O G U E

Figure 1: Southampton Solent University ‘Solent Online Learning Standard’ for Online and Blended Learning

A consistent approach In the past, as is common with support of technology-enhanced learning within a large institution, the learning technologist works in a capacity of enabling academic staff members to deliver their VLE content independent of continuous support. Learning technologist

Figure 2: Model of support from learning technologists

Academic staff

Enable

Deliver

Students

8

D I A L O G U E

This model has several advantages, but most especially it means that the expert knowledge held by the learning technologist can be shared effectively with a large number of academic staff, who are then able to deliver to an even larger number of students, hopefully creating great impact within the institution. One drawback, however, is that the enabled academics may go on to deliver with varying degrees of success as the relationship does not necessarily include continued support. However, with the FDDST, the model works slightly differently and is very much about the members of the team working collaboratively and continuously with the academics who develop and deliver the technology-enhanced learning experiences via the VLE.

Collaborate

FDDST

Academic staff

Deliver

Students

Figure 3: Model of support from the Flexible Delivery Development and Support Team

The huge advantage here is that the academic staff, individually or more usually as a course team, are continuously enabled and supported in order to deliver their VLE content. This means that the output in terms of online course content is maintained at a much more consistent standard, and the provision of online learning is enabled by a much more consistent approach to delivery. One challenge to this model which must be considered is that this level of support is resource hungry in terms of the time spent by the members of the support team. At the moment, this is maintained through funding available from the Strategic Development Programme. It is important for the FDDST, during the duration of the available funding, to establish a costing model that weighs the effectiveness of this support against the efficiency of the provision, and hopefully establishes a strategic demand for its longevity.

Conclusion In its outset, the FDDST set about trying to clarify what SSU means by ‘high-quality online learning’. In the long run, with the ongoing development and application of the SOL Standard, what has been provided is a means of expressing the experience and knowledge of the team in relation to online distance and blended learning teaching and learning. This of course, is enhanced by the great deal of work that has and continues to go on in the academic faculties in developing courses delivered in this way. Further, with the activities of the team, we have also been able to develop a new way of engaging with academics to try to ensure that this experience and knowledge is put to good effect. The future potential of these activities is the continuingly improving provision of more flexibly delivered courses, and potentially a greater competitive edge in a rapidly changing higher education market place.

9

D I A L O G U E

References ALMPANIS, T., S. PATRICK, R. MCLELLAN, C. DINSMORE, A. FAUSTINO, & W. BASUKI, 2010. Proposing a Framework for Blended and Flexible Course Design. In: Kinshuk et al., eds. Cognition and Exploratory Learning in Digital Age: 7th International Conference, CELDA2010, Timisoara, Romania, October 15 - 17, 2010. Proceedings: IADIS International Association for the Development of the Information Society. Conference Proceedings, pp. 263 – 267 DOIG, A., 2011. Online Usability for Students in Employment. 5th International, Education and Development Conference INTED 2011: Valencia, Spain 8th March 2011 Proceedings: INTED 2011, pp 805-810 PATRICK, S. & J. NEWELL, 2009. Transition into or back into HE by Flexible Learning. [conference presentation] Solent Event, September 2009. Southampton Solent University

Contact

Steve Hogg

Head of Learning Technologies, LIS 02380319860 steve.hogg@solent.ac.uk

Andrew Doig

Flexible Courses Instructional Developer, LIS 02380319187 andrew.doig@solent.ac.uk

10

D I A L O G U E

Assessment: exploring a new discourse Prof. Elizabeth Mytton Law Subject Group, FBSE Introduction This paper combines a theoretical framework with reference to how contemporary assessment approaches might operate with practical experience gained from recent events within the Faculty of Business, Sport, and Enterprise (FBSE). The focus of the paper is assessment and the spheres of influence which impact on the thinking and implementation of new assessment techniques. It provides arguments for supporting the proposition that the Faculty is adopting learning, teaching, and assessment practice at the cutting edge of Higher Education. This claim is made in the light of the new funding methodology and the current indications of what Higher Education is for in the new order. The paper is based on the experience of recent developments in Legal Education and the extent to which such indications can provide common areas across disciplines. The emphasis is on current change in Education rather than Law per se. Having considered the theoretical framework and prevailing external factors, the paper identifies current initiatives being undertaken in the Faculty to meet the strategic developments of the University.

In terms of the theoretical framework, the paper revisits themes identified in the author's previous work (Mytton, 2003) in relation to spheres of influence on Legal Education and reconsiders this with reference to recent experience in working with assessment strategy and its implementation. It also takes into account the extent to which revalidation events enable teams to review their professional practice and opportunities to develop a new discourse. The theoretical framework considers first principles such as ‘Why assess?’ (Brown et al, 1994). This is considered against a background of understanding the institutional context from theoretical perspectives on ‘perfomativity’ (Lyotard, 1984) and ‘supercomplexity’ (Barnett, 2000).

A profound shift in the educative experience is manifest from the origins of the human experience of learning derived from the Aristotlean notion of the educative to that which is now digitally driven. The implications are captured by Boud (Boud et al, 2008):

New technology makes access possible to a vast range of digital sources. The environment makes some activities possible and constrains others but it does not change the fundamental processes of human learning.

In addition to striking a balance between the advantages of digital sources and human learning, tutors are also tasked with accommodating external factors related to assessment such as:

The National Student Survey Assessment as a key driver in terms of student satisfaction and retention The integrity and validity of assessed work The cost of verification and impact of Turnitin Compliance with the Equality Act and reasonable adjustments Enabling students to engage with the assessment process Tutors’ responses to student performance in shaping teaching

11

D I A L O G U E

Whilst tutors come to terms with designing relevant and contemporary assessment strategies, the external environment also needs to be taken into account not least in terms of the government’s stance on Higher Education. The impact of the new funding mechanism is of significant interest to Heads of Department. Those Heads with devolved budgets need an entrepreneurial flair to make things work to best effect. Financial acumen is necessary to deal with budgets, targets, attracting income, and allocating available funds to best effect.

The Market The key theme here is competition. Private providers are more able to control their business, they have less complex structures and decision-making is far more straightforward. Such providers have greater opportunity to control costs than University law schools. A key concern affecting law schools is where there is cross-subsidy; an issue which does not arise in the private sector. It is becoming clear that private providers have the ear of employers and indeed the government. They are more able to provide 2 year degrees designed for qualifying lawyers who have committed to the profession. To date there are less than 10 Universities providing 2 year ‘fast-track’ degrees which may suit career changers and mature students far better than school leavers, who have a different set of life and learning skills to develop before committing to an expensive professional course of training. Private tutors are not required to have the same qualifications as university law academics who hold academic and professional qualifications. The growth of multiple discipline practices is also having an impact of the shape of Legal Education and the training of professionals. There is a significant paralegal population of well-qualified lawyers who are unable to secure a pupilage or a training contract and are working for a very modest income.

Professional bodies Currently, there are discussions among the professional bodies relating to ‘the moral question’ about the oversupply of Law graduates. It may be acceptable on leaving University to have a significant number of Law graduates who are able to seek a range of a career options. However, the question arises as to those students who graduate from the vocational stage. Suggestions focus upon a possible cap on numbers and aptitude testing which is currently being piloted by the Bar. It is recognised there are perils associated with aptitude testing and that very able students may not get through and some who get through may not make it. There are also issues in relation to diversity and access. Law schools are continually under scrutiny in terms of what should be taught. Very often it is felt City firms are driving the agenda yet they tend to recruit from a relatively narrow sector. Post 92 universities often design courses with employability and business skills built in whereas the traditional universities tend to focus on academic areas which reflect the strengths of the discipline and intellectual endeavour. It is becoming clearer that Law graduates are expected to have a sense of business awareness and prepare for careers in the event they cannot or do not wish to proceed with a professional Law qualification or a career in Law. Indeed, typically only 40% of Law graduates proceed to practice. It is suggested that universities make it easier for students to transfer across courses. We certainly need to take the model of alternative business structures into account, advise students well, and provide a menu from which students are able to make appropriate choices. Currently, a major review ‘Legal Education 2020’ is underway. This is being led by the Law Society, the Bar, and the Institute of Legal Executives. The outcomes will be known in due course, however, there are aspects of current Legal Education and alignment with the professional world which are becoming apparent. Legal Education requires students to gain business awareness and the ability to engage with the world of business. However, there are different views across the sector about what a University education is for. Some Universities will claim to be sites of pure academic endeavour and will resist signing up to the world of preparing students for employability; such knowledge and skills are not for the purists. It may

12

D I A L O G U E

well be the case that qualifications from a traditional University import a brand and that may well work for them. That said, in the light of the new funding methodology, employability opportunities now count in a way they never did before. This is where SSU is ahead in setting a scene in which students can embark on a course of study which has already been designed with students’ futures in mind. Hearts and minds are already focussed on employer engagement and enabling students to study courses which will enable them to do well.

The Solent experience: a comparative comment Recent events such as the Faculty away-day and the Faculty conference clearly demonstrate our approaches to teaching, learning, and assessment are contemporary, relevant, and real. Whilst this is the prevailing discourse at SSU it is clear that there remains a more traditional view in the academy. It is for this reason that SSU is poised to provide a learning experience which students will be seeking in increasing numbers given the changes in the market economy and funding. Clearly, there are universities which will claim to be sites of pure academic endeavour and will resist signing up to the world of preparing students for employability; such knowledge and skills are not for the purists. (Bradney, 2003) refers to the liberal University law school and asks ‘what it should be doing in terms of its teaching, research and administration’. He asserts ‘the liberal law school responds to the inalienable curiosity that is at the heart of human nature.’ Further, he claims that the liberal curriculum ‘should not attempt to determine what students will become or take out’ whilst acknowledging the proposition is in conflict with intended learning outcomes (ILOs). Perhaps Bradney’s preference for a liberal education cannot accommodate the idea of what University education means to students who are keen to pursue their own career path regarding University as a means to an end rather than an end in itself. It may well be that the traditional University imports a brand and that may well work for them and their graduates. That said, in the light of the new funding methodology, employability opportunities now count in a way they never did before.

Assessment: a theoretical framework Barnett et al. (2008) suggests there are six qualities which help to make out the student’s ‘educational being’: courage, bravery, determination, persistence, integrity and sincerity. Whilst acknowledging this position it would be interesting to know the extent to which such qualities are considered when constructing an assessment strategy. Do educators have in mind the qualities and dispositions to ensure the ‘authentic educational being’? We have a learning environment which could not even have been imagined a decade or two ago.

13

D I A L O G U E

If we can only make sense of the new order, we can play it to our advantage. With one eye on what is required, a little imagination, and the courage to push the boundaries, the deeply held traditions of what legal scholarship stands for can co-exist with new possibilities.

Performativity and values The starting point must be that legal scholarship within the context of university Law teaching is framed by measurement since only that which can be measured can be valued according to Lyotard (1984). The underlying flaw which threatens the essence of the educational endeavour is that moral values and the moral good cannot be measured. Teaching Law is a microcosm of the wider socio-political environment in which competing claims abound. The central question is the extent to which external spheres of influence determine the essence of the University and furthermore whether it is legitimate that they should do so. If the purpose of the University today is to produce resourceful people equipped to enter and participate in the world, it creates a contested discourse about what is valued. This has a degree of resonance with Lyotard for whom ‘performativity’ captures that which might be described as an underlying epistemological shift within the academy. Lyotard deconstructs different knowledges recognising that what is of use or what can be measured is what is valued. Given that universities are essentially corporate bodies it would appear that a corporate body is only capable of valuing that which can be measured. Performativity raises questions about the University value system. The first question to ask is whether an institution can have a value system. Barnett (2000) recognises the University may have its place within an accepted value background, e.g. ‘the pursuit of knowledge is a good thing’ or ‘social justice is worth striving for’. Further, he observes that in an era that has difficulty with large ideas there is also some difficulty in sustaining such a value position. Barnett, in exploring the underlying challenges, draws upon architectural metaphor by suggesting that ‘the building will shake but it will not fall down’. The essential difficulty for him seems to be that ‘the University in the contemporary era is unsure of its value basis; and that is to put the matter charitably’. It can be argued that it is not only the building which is

14

D I A L O G U E

shaking but educators too are required to make sense of a rapidly changing environment. We are constantly making judgements about what counts and what has to be considered when engaging with educative endeavours. Contemporary education may be far removed from the Aristotelian Greek state but, for some, those values hold good and a way has to be found for them to subsist within a world which Barnett refers to as ‘supercomplexity’ (Barnett, 2000). Erben (1999) traces the use of the word educative to mean two things; as a synonym for educational, and a word associated with ethics and values where educative experience and educative knowledge carry a wider compass that that characterised by the term educational. He refers to the Aristotelian idea that ‘in both formal and informal environments the educative relates to a conception of personal identity originating in moral choices…from infancy to feel joy and to feel grief at the right things: true education is precisely this’. He refers to the Aristotelian proposition that moral good is identifiable with social good.

Yet Law educators as academics themselves are the determinants of values and what it means for them to be part of the community of Law educators. Deeply held traditional understanding is a feature of the legal academic community for example that which has been espoused by Birks (1998). That said, different ideological and epistemological perspectives are at play. Barnett argues the influx of both internal and external ideologies have impacted on the University to the point that it is problematic to sustain the idea of the University solely as a site of reason.

Many terms of reference used in assessment are deeply embedded and it is sometimes difficult to invite others ‘to do assessment differently’. Why should they? Whose interests are being served? Educators make judgements and the recipients are judged. We need to appreciate the impact of what is being undertaken. It is relatively straightforward to draft the documents required for institutional purposes yet reflexivity is profoundly significant in terms of the reality of the student and indeed the tutor’s experience.

Assessment: a critical discourse A starting point is to ask ‘What assessment is and what it can do?’ This is explored by Boud and Falchikov (2008). They acknowledge that ‘assessment affects people’s lives’ and that: ‘Assessment is a value-laden activity surrounded by debate about academic standards, preparing students for employment, measuring quality and providing incentives’. Furthermore, they assert that ‘Examination systems are resistant to change as they unreflexively embody many socio-political assumptions about what education is for’. There are contradictions in terms of what we are required to do and what is worthwhile. Boud et al. argue ‘It is only through establishing a counter-discourse to the one that currently dominates higher education that some of the fundamental problems created by current assessment assumptions and practices can be addressed’.

His study surveys a range of institutions to discover their statements on how they frame assessment concluding, as one might have anticipated, that there is ‘an assessment bureaucracy’. He suggests the time has come to contest this dominant discourse and reframe assessment to emphasise the more important underlying purpose of ‘informing judgement’ and suggests three key features. Firstly he argues the need to connect assessment and learning and that it is necessary to look at the consequences. This can be achieved by asking the question, ‘Do assessment acts actively promote development of students’ capacity to make judgements about their own work and its relevance to future learning’. Secondly, he proposes assessments should ‘foster reflexivity and self –regulation’. This goes to the root of building confidence and self-image of oneself as an active learner as opposed to the learner being solely directed by others. Thirdly, he espouses the case for having a new focus on the variety of contexts in which learning occurs such as real work settings. It may well be the case that all three of these features can be found in law schools and there is scope for further study in this regard.

15

D I A L O G U E

Why Assess? If we are to reflect on the idea of ‘Doing Assessment Differently’ it is worth asking why assess at all? This question is explored by (Brown, Rust and Gibbs, 1994) who suggest seven basic reasons for assessing: Motivation Creating learning opportunities Feedback to students such as strengths and weaknesses Feedback to staff to indicate how well the message is getting across Judge performance Quality Assurance (internal) Quality Assurance (external) When devising an assessment strategy it would be interesting to know how law schools approach such a task (Bone, 1999). Sanders found in his exercise for Heads of Law that there is no consistency in terms of whether there is an assessment policy across law schools. It would be interesting to know who determines whichever policy exists in law schools where such policy exists. The significance of this relates to how law academics and law educators prioritise assessment practice and the terms of reference they use. In a Business School, for example, if the assessment policy is drafted by a non-lawyer, what are the mechanisms for ensuring compliance with Qualifying Law Degree (QLD) rules and the Quality Assurance Agency (QAA) subject benchmark for law?

Assessing Law differently: the rules Given the context of competing spheres of influence upon legal scholarship this paper focuses particularly on the possibilities of assessing differently whilst balancing regulation and in pursuit of innovation. First things first; we cannot rely entirely on claims to unfettered academic freedom – there are limits. It is necessary to know the limits and operate freely to the extent to which freedom is possible. In terms of regulation the spheres of influence upon a QLD operate externally and internally. In terms of the latter the key determinant is the Joint Academic Stage Board Handbook. This is read in the context of individual law schools who in turn determine how to deliver the programmes of study in accordance with the rules. Professor Andrew Sanders has produced a set of findings in relation to ‘Assessment and regulation of assessment on QLDs’ having undertaken a questionnaire sent to Heads of UK Law Schools to which 38 responded (Warwick, 2009). His findings are more extensive than is possible to report here but the following observations can be made for the purpose of simple illustration:

Half of law schools have a policy on traditional closed book exams and other forms of assessment

Not all law schools felt constrained by the move away from examinations but some felt exams contribute to upholding standards

The majority did not feel constrained intellectually regarding what is appropriate for a QLD

The balance between exams and other forms of assessment varies considerably Law schools are differently located in terms of their approaches to Legal Education. In broad terms, the epistemological character of a law school will be largely dependent upon whether it is essentially a site of academic endeavour in its own right or whether it has an emphasis upon the professions and vocational training. Within a typical law school with a Qualifying Law Degree (QLD), there will be the following reference points: External Professional networks

16

D I A L O G U E

Joint Statement Law Benchmark statement National Qualifications Framework (NQF) Quality Assurance Agency (QAA) Higher Education Funding Council (HEFCE) Solicitors Regulatory Authority (SRA) Bar Standards Board (BSB) Joint Academic Stage Board (JASB) Research Excellence Framework (REF) National Student Survey (NSS) League Tables Statutory requirements e.g. the Equality Act 2006 Internal University strategic plan Academic citizenship Team meetings Subject expertise VLE Subject teams Equality & Diversity Quality Assurance Unit guides Assessment regulations Research Student Satisfaction Surveys

Innovation at Southampton Solent University: developing a new discourse for assessment Southampton Solent University is strongly committed to the student experience and prevailing ethos is to enable students from all backgrounds to have the opportunity to benefit from a University course of study. This ethos impacts on all aspects of the learning, teaching and assessment experience. The Business School focuses on the theme of ‘Solentness’ designed to enable students to become well-equipped to align their studies to employer enhancement and personal effectiveness. There is considerable emphasis on supporting the student and enabling them to reach their potential. This context impacts significantly on the approach to assessment. Recently, the Law provision has been comprehensively reviewed which provides opportunities for reviewing traditional approaches to assessment. It was clear the design and implementation of innovative and stimulating assessment requires a robust infrastructure. It is particularly important that technical staff and academic staff feel able to communicate effectively at every stage in order to develop a contemporary pedagogic discourse where the following is recognised and becomes part of the quest for doing assessment differently:

Knowledge exchange is the essence of intellectual discovery and learning New vocabulary and new ways of thinking about access to knowledge are critical The dynamic interface between inspiration and implementation Different ways of thinking about what is possible in problem-solving Excitement about the generation of new ideas, new technology, and vocabulary New modes of communication between academics, students and technologists

to produce new ideas in the design of a dynamic learning environment.

17

D I A L O G U E

FBSE away-day and conference Student presentations at the away-day indicated that courses at SSU are on the right track but there is still a way to go in ‘making it real’. This was also borne out at the FBSE conference in the session on ‘Assessment: Exploring a new discourse within an institutional framework’. From those two events the following key points emerged. Students need more ‘live’ opportunities for learning; techniques which are very easy to incorporate to reflect business practice to enable students to recognise what they will actually encounter. The time has come for a move away from entirely academic exercises for undergraduates such as essay writing which may no longer be the most effective away to enable students to acquire the skills required for business. Some examples were identified as:

Live client briefs Practical exercises Presentations Report writing Business mentors Co-teaching and learning with business professionals and student.

In addition to the above, a further list of desirable knowledge, skills, and attributes were obtained from external key stakeholders in Law.

Clarity of thought and expression Applied knowledge Business practice Client relationships Client management Client perspective

Project management Tax Accounting Good flair for business Writing and Drafting Presenting Risk analysis

Key Changes to the Law Degree The above factors were taken into account when designing the revalidated LLB (Hons). A single new course was designed to reflect the aspects of Law and practice most relevant to students, academics, practitioners and business. From the outset, Law was taken to be integral to the business community. Given that around 40% of all qualified Law graduates go on to practice Law it is recognised the course needs to reflect both Law and Business practice. A unit has been introduced at level 5 ‘Lawyers Working with Business’ to enable students to learn the realities of the world of work. A skills spine was integrated through the course. At Level 4, new units were introduced to provide opportunities for learning subject knowledge whilst the ability to gain a range of skills. Resources were obtained to set up a Mooting Room. Mooting had been piloted and, as a result, it was found to be a particularly effective method to incorporate staff, students and practitioners in the teaching and learning process. A unit named ‘Aspects of Law and Practice’ (ALP) was also introduced to promote self-development and reflective practice. There are four parts to this unit reflecting the key themes through levels 5 and 6. Mahara (ePortfolio system) is a key opportunity for students to make informed decisions about their chosen

18

D I A L O G U E

paths. ALP comprises parts which optimise existing strengths in the Law Subject Group: Criminology, Social Law, and Commercial Law. These themes provide students with the chance to test out their preferences before they commit to the rest of the course. Their reflective practice and personal development is supported through Mahara. In addition, it was recognised that traditional part-time courses do not provide the required mode of learning for professionals, new learners and career changers. A new idea was to introduce a form of ‘Flexible Learning’. This provides non-standard participants with flexibility according to their needs supported by a Flexible Learning Tutor. Through the revalidation process there many lessons learned illustrated by the following examples:

Play to our strengths Generate buy-in Excite, inspire, lead Understand the chaos of change Pilot new innovative methods of assessment Listen to the student voice Students are the future – appreciate their world Interface with professionals Apply current best practice in everything we do

It became clear a focus needed to be found to bring these elements together so as to ensure we maintain momentum and play to our strengths. The Southampton City Law Network is being established to bring the Law and Business community together. This will promote placements and ‘live’ learning opportunities for students, as well as continuing professional development (CPD) and professional training for employers. A particularly innovative arrangement is where a Law firm supports students in ‘live’ activities and awards prizes. The firm gains publicity and is able to provide professional opportunities for students.

Concluding remarks Taking these experiences into account it is fair to say that SSU LLB (Hons) is very well placed to meet the objectives articulated through current University strategic aspirations through the Strategic Development Programme and the ‘six strands’ which set out how the student experience can be aligned with the SSU learning experience. Certainly, the revalidation of the Law provision generated many opportunities for review of our practice and how to align student learning with current expectations from students, practitioners and key stakeholders. The impact of the funding methodology will impact on us all. The good news is that students choosing to embark on a learning and career path at SSU will be starting from a very strong position taught and supported by staff who understand the huge significance of ‘making it real’.

References BARNETT, R., cited in Boud, D. and N. Falchikov, 2008. Rethinking Assessment in Higher Education. Oxford: Routledge.

BARNETT, R., 2003 Beyond all Reason: Living with Ideology in the University. Buckingham: SRHE/OUP

BARNETT, R., 2000. Realizing the University. Buckingham: OUP

BIRKS, P.,1998. The academic and the practitioner. Legal Studies 18 (4) 397-414

19

D I A L O G U E

BONE, A., 1999. Ensuring Successful Assessment (NCLE) (Warwick). Committee of Heads of Law Schools, Warwick, (2009).

BOUD, D. & N. FALCHIKOV, 2008. Rethinking Assessment in Higher Education. Abingdon: Routledge BRADLEY, A., 2003. Conversations, Choices and Chances Oxford: Hart Publishing BROWN, S., C. RUST & G. GIBBS, 1994. Strategies for Diversifying Assessment in Higher Education. Oxford: Oxford Centre for Staff Development.

ERBEN, M., 1999. The biographic and the educative: A question of values, in Scott, D. (ed.) Values and Educational Research, London: Institute of Education, University of London, pp 77-92.

LYOTARD, J. F., 1984. The Postmodern Condition: A Report on Knowledge. Manchester: MUP.

MYTTON, E., 2003 Lived experiences of law teachers, in The Law Teacher: International Journal of Legal Education, 37(1), 36-54.

MYTTON, E. & J. HANSON, 2002. W(h)ither the Academic Law Teacher. In: D.S. Prenton, ed The University of Crisis. NY: Rodopi..

MYTTON, E., 2000. Quality: an epistemological enquiry in West Indian Law Journal, Special edition pp 78-90.

Contact

Prof. Elizabeth Mytton

Senior lecturer in law, FBSE 02380319071 elizabeth.mytton@solent.ac.uk

20

D I A L O G U E

Learning strategies for student engagement and achievement

Carina Buckley Learning Skills Tutor, LIS

Abstract: This paper presents a research-based model for helping students identify opportunities for taking control of and responsibility for their own learning, developed through one-to-one skills tutorials with students across a range of subjects and year groups. Although interventions must ultimately accommodate the needs of the individual, they can be broadly characterised so that certain strategies will be more effective than others according to confidence level and purpose of visit. For students low in confidence coming to check their work is correct, having a practical checklist to follow connects to their concerns for task diligence. For more confident students, rehearsal and discussion are central to their self-improvement. Where the purpose of the tutorial is more questioning, low confidence students would benefit from goal-setting and planning, to structure progress and break down tasks, whereas more confident students are likely to appreciate learning for its own sake and so guidance on placing each task in context will be useful for them. By helping students take control of their learning, their motivation and engagement are likely to increase, giving them a more positive learning experience and potentially improving their academic outcomes.

Introduction One-to-one tutorials with a learning skills tutor can be seen as a way to bridge the gap between subject competence and academic insecurity, by guiding students through the structural barriers that might prevent them from expressing their subject knowledge. Learning cannot be divorced from the context in which it takes place (Prosser and Trigwell, 1999); knowledge and the skills needed to communicate it well are similarly entwined (Cottrell, 2001). For students to come to a learning skills tutorial implies an appreciation of this relationship and a desire to improve it. Moreover, as these tutorials are entirely voluntary, attendance is in itself a measure of engagement, motivation and at least some level of students’ awareness of themselves and their abilities. This paper presents a research-based model for helping students identify opportunities for taking control of and responsibility for their own learning, developed through one-to-one tutorials with students across a range of subjects and year groups. When students recognise that their learning is within their control, their learning effectiveness is likely to increase, giving them a more positive learning experience and potentially improving their academic outcomes.

Learning skills at Solent The learning skills tutor is part of the Library and Information Service, existing independently of the Faculties. Tutorials are therefore accessible to all students throughout the university,

21

D I A L O G U E

either by dropping in or by emailing to arrange an appointment. Students can also be referred to the tutor by a librarian, lecturer or other support service. Between 1st September and 17th December 2010, 115 individuals had attended a total of 223 appointments, with most falling towards the end of the term and the approaching assignment deadlines. The majority of students were first and third years, and there was no single course or faculty that made significantly more use of the service than any other; students were drawn from all corners of the university. In terms of student queries, essay writing was by far the most common topic, but this masks an underlying complexity that became the foundation of the research informing this paper. Whereas some individuals were concerned with understanding the requirements of the question and how best to structure their answer, others were more worried about checking their grammar and language. Some students therefore queried the underlying architecture of the essay, while for others it was sufficient to have their verb choice, vocabulary and punctuation checked. A similar pattern was seen in the nature of referencing queries, with a division between those questioning citing and referencing as a concept and those unsure of where to place the comma or how to reference an email. The split between the mechanical, performance-oriented approach and the more structural, learning-oriented approach gave rise to the first dichotomy of the model the purpose of the tutorial. In turn, the purpose is mediated by the level of confidence the student has in their own ability and capacity to succeed (high vs. low), creating four possible broad categories for each student to fall into. It is unlikely that these tendencies (which will vary from assignment to assignment) are restricted to academic skills but instead also underpin each student’s approach to their subject area.

A learning strategies model The model was developed over three months’ observation of student activity at tutorials. It became clear early on that the types of queries and the student’s confidence in themselves and their work resulted in the need for different types of learning and feedback strategies. Although interventions must ultimately accommodate the needs of the individual, they can be broadly characterised such that certain strategies will be more effective than others according to confidence level and purpose of visit. Consequently, each of the four categories identified above will have its own learning strategy, best suited to purpose and confidence level. These observations were followed up and corroborated by a formal questionnaire sent to 15 students who have attended tutorials several times, in order to establish their confidence, motivations and goals in more detail. The questionnaire asked each respondent to rate their level of confidence on a number of issues, such as producing work to the required standard, writing in an appropriate academic style, understanding the topics discussed in lectures and seminars, and remaining motivated until the end of their degree. It then asked them to choose one answer from a range of possibilities to indicate the main reason they came to learning skills tutorials; what they considered to be the most useful outcome of a tutorial; and the most likely reason for doing well in an assignment (Table 1):

The main reason for coming to learning skills tutorials

To get the grammar or referencing checked To make sure I’ve answered the question properly To get help with structuring my work well To make sure I understand what I’ve got to do For help with academic practices like writing in the 3rd person, etc.

22

D I A L O G U E

The most useful outcome of a learning skills tutorial

To have any grammar mistakes corrected To talk through the question and what I have to do to get a good mark To get some ideas on how to go about completing the task To learn more about academic skills so I get it right next time

Reason for success in an assignment

My own ability Putting in the effort Luck (e.g. lecturer marking it was in a good mood) I got help (from a tutor, friend, etc) It wasn’t too difficult

Table 1

Establishing the goals of a tutorial and attributing success Although confidence can and does vary with context and activity, along a wide continuum (Sander and Sanders, 2006), the students questioned here can be categorised as being relatively more or less confident in terms of their academic ability. When this was mapped against what they felt was the most common reason for coming to a tutorial, there were some overlaps evident in their reported preferred outcomes. From this information, it was possible to build up a model of the learning strategy that would be most effective for each kind of learner (Table 2). ‘Checking’ is considered to be focussed on areas such as grammar, spelling, referencing and writing in the third person, while ‘questioning’ covers the underlying elements of structure, understanding the brief and building a relevant argument:

CONFIDENCE

Low High

Checking Practical tasks

Checklist

Rehearsal

Discussion PURPOSE

Questioning Planning

Goal-setting/monitoring

(Task in context)

(Activity)

Table 2

Student-valued learning strategy according to confidence and purpose of tutorial Those students who were low in confidence who came to have their work checked appreciated using checklists and strict, narrow guidelines for fulfilling requirements, such as referencing examples. In this way they could take control of their learning by following small, set tasks, gaining a feeling of competence as they gradually worked down the list. For questioning students lacking in confidence, a similar strategy can be employed but on a more structural level. Agreeing tasks and activities for the student to complete – either in that session or by the following one – engendered a similar feeling of control and progress, but looked a little deeper below the surface to the underlying architecture of the assignment. Tasks set might include developing an essay plan, drafting an introduction or preparing aims

23

D I A L O G U E

and objectives. The purpose is to help the student take control by goal-setting and planning the different components of the assignment. Where confidence is greater, the learning strategy changes accordingly as the concordant increase in self-efficacy allows for improved performance. Self-efficacy describes an individual’s own perception of how well they will be able to deal with a particular real or potential situation (Bandura, 1982). Checking for these students becomes more a matter of rehearsal, of trying out tasks, taking more of a risk and discussing the requirements in more detail. These students often query the meaning of particular words in essay titles, for example, but come with their own definitions to test. In theory, students high in confidence who come with a questioning stance would be likely to value taking this further and looking at each task in the context of their course or learning as a whole, making links between each piece of work and the skills involved. As yet, no student comfortably falls within this category although some certainly come close.

Student motivations and goals It is the reasons behind these choices and preferences that can provide a gateway to student motivations and greater understanding. To be constructive and useful, a teaching session must help the student to make progress within the context of the individual’s expectations, prior experiences and perceptions of the task demands. It is often feared that by encouraging ‘surface’ learning processes, the student will continue to rely on them, but the best approach chosen is the one that best matches the student’s motivational state and experience of the particular teaching environment (Biggs, 1994). So although checking may appear to be a surface strategy concerned only with outward appearances and thus should be developed towards a more questioning stance, for the student low in confidence that would not necessarily be the most useful direction for improvement. Gains in confidence is often cited as a principle benefit of being a student (Morgan and Holly, 1994) and it may be more appropriate for that student to retain a more mechanical approach, but become more confident in their work and their ability by focusing on the task. In this way, the uncertain student has the opportunity to become comfortable with the discourse of learning and take a more active role in their own knowledge construction, since to grasp the meaning behind the discourse is to create new meaning and take control of learning (Marton, 1975). Insight into the processes that drive these preferences can show the best tactic to take with different students. The model is expanded below to underline the students’ approach and overall motivation for learning (Table 3).

CONFIDENCE

Low High

Checking

1. Viewpoint: temporary exertion

Task diligence

2. Viewpoint: aptitude

Self-improvement

PURPOSE

Questioning

3. Viewpoint: chance

Making progress

4. Viewpoint: expectancy

Task appreciation

Table 3

Student approach and motivation

24

D I A L O G U E

For students low in confidence coming to check that their work is correct, having a checklist to follow and focussing on corrections and improvements that are immediately achievable connects to their concerns for task diligence (sector 1). Achievement goals of this nature do not rely on prior experience, ability or confidence for success but do nevertheless allow for student engagement with the task (Covington, 2000). In a sense this rewards the student for investing in their learning but is not a stable, long term strategy. The expectancy of success is a key motivator for action but depends on whether the cause for success can be considered to be stable or not. When a positive outcome can be attributed to a stable cause, such as aptitude, then the student is more likely to believe that the next attempt at a similar task will be equally successful. However, a cause such as effort is more unstable, and so the outcome cannot be predicted: it could be just the same as before, or it could be different (Weiner, 1985). The aim, therefore, is to move the low confident, checking student, towards a viewpoint of aptitude with the goal of self-improvement, represented in sector 2 of the model. The use of discussion and rehearsal acknowledges a student’s higher level of confidence and adds challenge, but remains contained and controlled. As the emphasis is on the student’s own aptitude, the causes of success are considered to be more stable and therefore assured, and so aspiration will continue to rise (Weiner, 1985). This creates a positive feedback loop, as a student confident enough to anticipate success – and, moreover, to attribute that success to an internal cause (their own ability) – will experience enhanced self-esteem when the result is indeed successful, and thus raise their confidence further. For these students, the difference lies in their expectations. Low confidence students display outcome expectations, whereby fulfilling certain behaviours, such as checking their work against referencing guidelines, will lead to a certain outcome. Students with more confidence concentrate instead on their own ability to produce the behaviours that will lead to a satisfactory outcome (Eccles and Wigfield, 2002). For students to be able to transfer their learning from one context to another, they need to be able to recognise commonalities in the underlying structure of their work (Cottrell, 2001). In terms of learning skills tutorials, those students who start with more of a questioning mode, seeking to understand that structure and its meaning, these students are focussed on making progress overall, rather than just in the particular task at hand. Although their confidence may be low (sector 3), they are becoming active agents in their own learning, setting goals and planning ways to improve (Dickinson, 1995). The focus on mastering the task at hand is a way of increasing their competence, but the planning aspect helps to remove the task from the immediate context and allow for its connection to other, similar ones. The goals set are generally manageable and achievable, and directed towards learning overall. By shifting the focus from the task to the person in their learning context, the student is better able to break out of what might be a constrained pattern of behaviour to one which is more responsive to the situation. Instead of restricting their activities to those which conform to an idea of a good student, i.e. one who can cite and reference correctly, or construct a grammatically correct sentence, they are more aware of their academic goals generally and their own capacity for dealing with any challenges that may arise as a result (Ridley, 1991). Chance is considered to be these students’ dominating viewpoint because their low confidence can inhibit their sense of self-efficacy. Where self-efficacy is lacking, people can behave inefficiently regardless of how well they might know what it is they have to do, as success is predicated on people making optimal use of their abilities. Questioning and clarifying task requirements and regularly appraising performance and progress can help to strengthen self-efficacy by demystifying academic skills and practices, as well as by providing a means of coping with task demands (Bandura, 1982). In addition, setting realistic, achievable goals, planning ahead and beginning to take responsibility can all contribute to helping the student take control of their learning (Dickinson, 1995), since the achievement of goals and subgoals marks out the distance travelled and the progress made, thus confirming increases in self-efficacy (Bandura, 1982). The aim for students who fall into this category would be to help them develop and enhance their confidence, in order for them to internalise the locus of control and feel responsible for their successes thus moving into sector 4 of the grid. Ultimately, students will see the task in

25

D I A L O G U E

context, understand how it relates to their learning generally and approach it with an expectancy of success through confidence in their own ability (Eccles and Wigfield, 2002). As yet, none of the students who answered the research questionnaire appeared to fall into this category. Of the 15 respondents, ten gave answers that suggested they belonged in sector 2. Most of these were third years, so it would be interesting to expand the research to establish whether there is a change over time and with experience. These students are motivated to succeed and want to do more to understand. Over time, and with continued practice, a deeper appreciation of how their learning connects and relates will be available to them. An alternative pathway into sector 4 is open to the three respondents placed in sector 3. For these students, time, practice and encouragement to reflect on their achievements will translate their more luck-based viewpoint into one of expectancy. The final two students both identified themselves as belonging to sector 1, and their development could take either of the two pathways possible; which one depends on student attributes, prior experiences, motivations and self-awareness. Neither should be considered as less useful or desirable than the other.

Conclusion The benefits of this model to those involved in student learning are twofold. First of all, it can help structure the content of tutorials towards what would be most immediately beneficial to the student, answering their needs for control vs. reassurance; task performance vs. self-improvement. However, more than that it can help map out a route towards learner autonomy and outcome expectancy. By understanding how students approach their tasks in terms of their motivations, confidence and sense of control, it is possible to understand the reasons behind their behaviour and suggest more targeted ways of improving it. It is often not simply a question of ability but the interlinking of self-awareness, self-efficacy and confidence to create an attitude to learning that may be at odds with that desired by tutors. However, this model identifies a number of strategies that can be employed to enhance and emphasise student engagement and motivation, giving them a more positive learning experience and potentially improving their academic outcomes.

References BANDURA, A. 1982. Self-efficacy mechanism in human agency. American Psychologist 37(2),

122-147 BIGGS, J. 1994. Student learning research and theory: where do we currently stand? In Gibbs,

G. (ed.) Improving student learning: theory and practice. Oxford: The Oxford Centre for Staff Development, pp. 1-19

COTTRELL, S. 2001. Teaching study skills and supporting learning. Basingstoke: Palgrave Macmillan

COVINGTON, MV. 2000 Goal theory, motivation, and school achievement: an integrative review. Annual Review of Psychology 51, 171-200

DICKINSON, L. 1995. Autonomy and motivation: a literature review. System 23(2), 165-174 ECCLES, JS. and A. WIGFIELD, 2002. Motivational beliefs, values and goals. Annual Review of

Psychology 53, 109-32 MARTON, F. 1975. What does it take to learn? In Entwistle, N. and D. Hounsell, (eds.) How

students learn. Lancaster: Institute for Research and Development in Post-Compulsory Education, pp. 125-138

MORGAN, A. and L. HOLLY, 1994. Adult change and development: the interactions of learning with people’s lives. In Gibbs, G. (ed.) Improving student learning: theory and practice. Oxford: The Oxford Centre for Staff Development, pp. 127-135

PROSSER, M. and K. TRIGWELL, 1999. Understanding learning and teaching: the experience in higher education. Buckingham: The Society for Research into Higher Education and Open University Press

RIDLEY, DS. 1991. Reflective self-awareness: a basic motivational process. Journal of Experimental Education 60, 31-48

26

D I A L O G U E

SANDER, P. and L. SANDERS, 2006. Understanding academic confidence. Psychology Teaching Review 12(1), 29-42

WEINER, B. 1985. An attributional theory of achievement, motivation and emotion. Psychological Review 92(4), 548-73

Contact Dr Carina Buckley

Learning Skills Tutor, LIS 023 8031 9336 carina.buckley@solent.ac.uk

27

D I A L O G U E

Report for Dialogue on Professor Graham Gibbs’ seminar; How to change assessment of degree programmes to improve student learning.

Deborah Wright Quality Enhancement Officer, AS The seminar was held at Oxford Brookes Business School in March this year; Graham Gibbs presented his views on some practical ways to go about changing the assessment pattern at programme level, and the impact this can have on improving student learning. His findings draw on ‘theory and empirical evidence [which comes] in part from a national project that currently involves a dozen universities changing assessment systems...’ (Oxford Brookes University: ASKe seminar outline) From his research over many years Gibbs has gained a perspective on students’ approaches to their studies. He has found that students in the UK do not do enough work (we have some of the shortest degree programmes in the world); do not make use of feedback; and do not understand goals and standards. He used the term ‘selective negligence’ to describe how students make choices about how to apportion their study efforts (Snyder 1971), and highlighted a study by Miller and Parlett (1974) which looked at ‘the examination game’ and found that students, very early on in their programmes of study, identify and excise the bits they don’t need to do. Gibbs offered some practical suggestions on how to design assessment tasks which require students to be active participants in the assessment process. This is to move the focus from assessment done to students to assessment done with and by students. These were presented under four headings:

1. Changing the perceived demand changes the way students approach the task On a teacher training course a compare and contrast essay task asked for a

comparison of two philosophies (out of several presented on the course) of approaches to classroom management. The replacement task required students to watch a previously unseen video of a teacher in the classroom; they then had to present an argument for possible strategies the teacher could use supported by the relevant philosophies. As the students did not know what challenges would be presented in the video, they needed to study a wider range of philosophies, as well as demonstrating how these could be applied to choices of classroom management. Deep rather than surface learning was thereby acquired and evidenced, and was of greater value to the students both in terms of knowledge and practice.

Get the students to self check. Tutors had found themselves repeatedly writing

the same corrective comments over and over again, with no real improvement. One programme gave students a checklist frontsheet to attach to their work

28

D I A L O G U E

asking them to ‘tick all the things you’ve done wrong’. This resulted in perfect lab reports.

Sampling coursework. For example on a programme with 25 pieces of work

across the year, with 4 overall Learning Outcomes for the programme, students are told that their work will be sampled at random for one LO per sample. Not knowing which piece of work will be sampled, nor which LO will be the focus of that sample, students ensure that they address all the LOs in all their tasks.

2. A mark is a substitute for thinking about quality

Formative assessment at Oxford encourages students to focus on what they

don’t know or understand. Oxford has a high volume of formative and low volume of summative assessment: students every year ask for marks for their formative tasks – and every year Oxford declines, as they maintain that it is unreasonable to mark students on task they are still learning how to do.

3. Diversity may be a red herring

Offering many different types of assessment results in students not taking feedback on board as they know they will never do another similar type of assessment.

4. LOs don’t help students understand what they need to do

Contrary to the belief that greater explicitness of learning outcomes and criteria will naturally result in students understanding what it is they have to do, and to what standard, Gibbs (2010) and Rust (2002) have found that students become much clearer about requirements and expectations if they have opportunities to work with exemplars.

Gibbs suggested trying ‘Traffic light’ peer assessment – indentifying ‘good’ and ‘needs fixing’ (now in use in primary schools).

Summary In summary, the key to improving student learning lies in the assessment design sequence. It works when it captures

‐ sufficient student time and effort: time on task ‐ high quality learning which matters to the student: engagement and deep

approach ‐ a focus on learning rather than marks: feedback ‐ learning to self-supervise: feed forward

To conclude, Gibbs offered some examples of what doesn’t work and what does. Strategy

What doesn’t work What does

Clear assignment briefs Greater and greater detail results in student cunning and selectiveness

Discussing and marking exemplars; critique each other’s work.

Assessment tasks Too many summative tasks More formative tasks, with prompt and targetted feedback; cumulative tasks which lead to a final summative task.

Feedback: speed Four weeks is too slow ‘Quick and dirty’ feedback within 2 weeks maximum

Feedback: making use of Volume of written feedback does not count

Make development a priority: e.g. a tear off slip for the

29

D I A L O G U E

feedback sheet to be attached to the next assignment will remind student and marker of focus for improvement

Diversity Too much variety means students are uncertain of expectations, and have little opportunity to practice and improve.

Link across units: students need to see the common principles of the types.

References GIBBS,G., 2010 Using assessment to support student learning www.leedsmet.ac.uk/publications MILLER, C.M.I. & M. PARLETT, 1974 Up to the Mark: a study of the examination game. Guildford: Society for Research into Higher Education. In G. GIBBS and C. SIMPSON (2004) Conditions Under Which Assessment Supports Students’ Learning. Learning and Teaching in Higher Education, Issue 1: 3–31 RUST, C., 2002 The impact of assessment on student learning. Active Learning in Higher Education 3(2) 145-158 SYNDER, B.R. 1971 The Hidden Curriculum. New York; Alfred A. Knopf

Contact Deborah Wright Quality Enhancement Officer (Teaching and Learning), AS 02380319560 deborah.wright@solent.ac.uk

30

D I A L O G U E

Reading Lists and myReferences: Integrating RefWorks and myCourse

Hannah Young Information Librarian – Law & Human Sciences, LIS This article was originally published in the May 2011 issue of Multimedia Information and Technology (MMIT) 37(2) pp. 26-29 and is reproduced here with the kind permission of the editor, with minor amendments. The project has since moved a long way forward with further developments, pilots and feedback leading to many changes and enhancements to the original software. A subsequent article will be published in Dialogue to bring the story fully up to date. In the meantime, for more information about project developments, please contact Hannah Young.

Introduction In April 2010, the Library set up a reading list project group with the aim of discovering whether there were any alternative ways of providing lists to students. The Library currently uses LearnBuild to produce and manage reading lists. Whilst this software provides a good basis for reading lists, it does not meet all of the Library’s needs (such as the ability to easily export references) and academics are not currently involved in editing their own lists due to the complexities of the system. The current process for creating lists is as follows: members of Library staff upload and update lists as they are sent through by academics; students access these from a Reading List link within their units on myCourse [Figure 1]. The software is used to produce lists in a range of formats, either as weekly lists or general recommended reading and can include direct links to journal articles and digitised chapters where required. The system also allows the addition of notes to resources and automatically links to the Library Catalogue for book details and availability.

Figure 1. Example unit on myCourse with Reading List link at the top Following a review of alternative reading list systems by the group, it was decided to investigate and pilot TELSTAR. The TELSTAR project was developed at the Open University (OU) and uses RefWorks bibliographic software to generate reading lists and make them available to students in the Moodle Virtual Learning Environment (VLE) directly, rather than via a separate link. Since Moodle has already been successfully adopted as the University’s VLE, and as existing subscribers to RefWorks, this seemed a real possibility. Additional

31

D I A L O G U E

features that were attractive include: using the SFX link-resolver to automatically link to ejournal articles thus reducing the need to manually create durable links; having a cut-down version of RefWorks software available in the VLE where students can save references and generate bibliographies and in-text citations [myReferences – see Figures 2-3]; and the ability to export reading list references automatically into a myReferences area. It was felt that having these facilities within myCourse might encourage students to use bibliographic software to help with their referencing and inspire them to move onto RefWorks for more advanced use as they progressed in their course. Due to the potential of the system, it proved possible to obtain some Strategic Development Funding (SDP) to set up a pilot project to investigate further.

Creating the system During summer 2010 and after liaison and support from the OU, the TELSTAR code was loaded onto myCourse. The live system comprises of two elements – the Resource Page module (which allows for the creation and display of a reading list) and the myReferences area (which is the cut-down version of RefWorks). Several months were spent testing the code and making localised changes until December 2010. One highly time-consuming element was customising the two Solent referencing styles (American Psychological Association 5th ed [APA] and Harvard – British Standard) so that they matched University guidelines and could be used by students during a pilot. It was also necessary to change the layout/order of the myReferences fields that are displayed for each resource type to ensure that all the required basic fields are available on the first screen and notes added where appropriate (thus ensuring that students do not need to ‘Display all fields’ to create a correct reference and know how to enter multiple authors) [see Figure 4].

Figure 2. Student view of myReferences area

32

D I A L O G U E

Figure 3A. Student view of creating a bibliography and selecting a customised style

Figure 3B. Student view of in-text citations and example bibliography generated in APA Solent style

33

D I A L O G U E

Figure 4. Student view of Create a Reference option – different fields selected for ‘Book Whole’ or ‘Journal Article’ type and local customisation of note for Authors Local customisations enabled staff to be satisfied that the system can reference materials correctly in the University styles and has the advantage that these styles can then be implemented in RefWorks so all students can use them for their referencing. However, the more local customisations that are added to the system, the harder it is to revert to the original code for subsequent upgrades, thus producing more work in the longer term. This is a key consideration for the project and how it can be taken forward, especially with forthcoming upgrades to RefWorks 2.0 and Moodle 2.0. Other local customisations included adding two key features to help with the creation of reading lists: a DOI link-up with CrossRef which enables staff to enter an article DOI and automatically retrieve referencing details; and a similar link-up with the Library Catalogue for books [See Figure 4 – Get data options]. All journal articles automatically link to the SFX link-resolver but this can be overridden. The override option is critical: digitised article links that are not available via the link-resolver are added here and any problematic links can also be replaced. Books automatically link to records in the Library Catalogue displaying the location and Dewey number. The system also allows the creation of shared accounts, meaning that in the future, academics and librarians could both enter new referencing details and update reading lists collaboratively. Within the Resource Page module, a reading list can be generated using an RSS feed from RefWorks – the librarian creates a shared account in myReferences and adds reading list references using the link-up features above, manually entering, or importing from RefWorks. Folders can be created for each week/topic area/resource type etc depending on requirements. These folders are then ‘published’ and an RSS feed is generated. The RSS feed is then added to a myCourse unit via the Resource Page module and references are displayed in Harvard or APA style. Each reference links to SFX, the Library Catalogue or an alternative URL [Figures 5-7].

34

D I A L O G U E

Figure 5. Example Reading List generated from RefWorks RSS in APA style

Figure 6. Textbook records link to the Catalogue showing location details

35

D I A L O G U E

Figure 7. Articles automatically link to SFX menu where available online or can link to digitised articles or URLs where required

Using RSS has great advantages since references can be updated in myReferences and changes are automatically reflected on the live list for students. The potential for providing reading lists by RSS is very encouraging, but it must be noted that obstacles have been encountered. For example, depending on the format of lists, it may be necessary to have many separate RSS feeds as individual list items cannot be moved around or headings inserted between them. Notes can be added to individual resources, which is a useful feature, but there is not quite the flexibility that some reading lists require. An additional problem is the need to add book records several times into each RSS feed – this enables specific page numbers to be added for weekly readings, but means that multiple records for the same item exist in the myReferences shared account. It is now likely that the Resource Page module will be discontinued by the OU, but work on RSS has still proved useful – it may be possible to develop an in-house system instead which uses RefWorks RSS feeds to display reading lists in a more flexible manner and with different styling.

36

D I A L O G U E

Piloting the system In Semester 1 and 2, 2010-11, the system was piloted with Level 4 and Level 5 Psychology students (using APA style) and Level 4 Criminology students (using Harvard). An export file of each reading list was created – this allows students to click on a link in their myCourse unit and have all their reading list references imported automatically into their own myReferences area, ready to use in a bibliography. A workshop demonstration and hands-on exercise using this import, and adding references into their myReferences area, was provided for the Level 4 students. Level 5 students received a demonstration in a lecture. All students were asked to try out the system and to give feedback. At this point, whilst there are obvious advantages to using RefWorks within Moodle, it should be noted that the myReferences area does have limitations. As a cut-down version of RefWorks, students have to cut and paste reference details into their Word documents rather than automatically downloading them into Word format. Also, there is no automatic import facility in myReferences from library subscription databases, so students either need to manually enter their reference details or push journal article references into RefWorks, and then access these from within myReferences.

Feedback It proved difficult to obtain feedback from the Level 5 students – many were still using the old ‘Reading List’ link in myCourse as this had been their access route to core readings in Level 4. A few students answered a questionnaire, indicating that the reading list was helpful, but no students attended pre-arranged focus groups or gave substantial information on their use of the system. From the questionnaire analysis, six students stated that they had not used myReferences because it was too complicated; two because they did not know what it offered; and two because they preferred to do their own referencing. This indicates that myReferences may not be intuitive to use and that training sessions with students might be required, as are currently offered for RefWorks. Indeed, one student commented that a more in-depth introduction to the system would have been helpful. The Level 4 students who received a workshop gave useful feedback – there were some concerns about the system including the need to improve some labelling on fields in myReferences (i.e. ‘title’ to ‘article title’ and ‘periodical’ to ‘journal title’; and an author example demonstrating the need to put the surname first) along with comments on the display and placement of certain features. Overall though, feedback has been positive and students seem to like the idea of having their reading list references automatically available, and the ability to add new references to their personal area. An additional issue highlighted in the workshops is that if students create a RefWorks account separately with a non-University email, this results in students having two accounts, so the link-up between myReferences and RefWorks does not exist. It will therefore be necessary to ensure that all RefWorks accounts are created with University details to prevent this occurring.

The future myCourse statistics, along with feedback from a questionnaire and three interviews held with the Level 4 students are currently being analysed to gain more information about the use of myReferences. This will enable a decision to be taken as to whether to make the area available to more students and progress with further customisations/updates as suggested from feedback during the workshops. Usability testing with additional students has also been proposed. RSS feeds will continue to be investigated as a potential way of providing reading lists to students within the VLE.

37

D I A L O G U E

Conclusion The project’s overriding aim was to discover whether there were new and alternative ways of providing reading lists, and more specifically, the relevance and suitability of myCourse/Refworks integration. RefWorks RSS does appear to be a possible solution but further testing and the creation of an in-house system would be required to take this forward. It will therefore be useful to ascertain whether any other Universities are using/considering RSS for reading lists as part of this development work. Whilst this has been a worthwhile project, setting up and customising such a system has been very time consuming. It appears that there is simply not an easy and cost-effective out-of-the box solution to providing reading lists since there are so many different requirements and issues to address. The use of myReferences has demonstrated how students struggle with referencing in the first year, and wish to be given more help – but a true evaluation of the Level 4’s use of this area is still required in order to decide whether further work is justified and whether it should be made available as an alternative/addition to RefWorks in the next academic year. For further information about the project and to view example reading lists, please see: http://mycourse.solent.ac.uk/refworksproject

Acknowledgements This paper was written with contributions from Kathryn Ballard (Information Librarian) and Roger Emery (Learning Systems Development Manager)

Contact Hannah Young Information Librarian (Law and Human Sciences) LIS 02380319687 hannah.young@solent.ac.uk

38

D I A L O G U E

39

Editorial Guidelines

Background Dialogue is the internal Solent Learning Community Journal. The Solent Learning Community

(SLC) was formed as a result of a TQEF project 2007-2009 which identified how a Community

of Practice approach could be used to support pedagogic research and share good pedagogic

practice across SSU.

Dialogue is an academic journal aimed at both practitioners and policy makers. It is intended

that it will be published internally twice a year.

Submission Articles need to be submitted electronically to Jenny Watson.

Solent Pedagogic Research Network project team have jointly edited this first edition: Sara

Briscoe (FBSE), Donna Peberdy FCIS and Chris Patterson MARTEC

Instructions for Authors Contributions to Dialogue will normally fall into one of the following categories. However,

contributions of different lengths will be considered by the Editorial Board:

· Articles: 2000 - 3000 words (accompanied by an abstract.)

· Brief reports (up to 1000 words) on, for example, innovative practice, conference events,

etc

· Book Reviews

Format

Articles should be typed, single spaced and have only one title.

The title will appear in bold.

D I A L O G U E

We encourage the use of subheadings (which will also appear in bold) to divide the

article.

All artwork (material that is not textual) such as tables, figures, diagrams, charts,

graphs, illustrations should be in black and white or shades of grey. Once your article

has been accepted we will ask you to provide a hard copy of text with diagrams or

other visuals to help with accurate setting at the design stage.

Use Trebuchet 11 MS Font

Footnotes should be avoided.

Please contact a member of the editorial group for a word document template.

Writing Style

Papers should be written in an easily accessible style, suitable for an audience of academics,

policy makers and practitioners.

All papers should use UK English spelling, grammar and punctuation.

Referencing Use the Harvard Referencing System. Further guidance can be obtained via http://portal-

live.solent.ac.uk/library/leaflets/resources/US06.pdf

About you Underneath your title we need your name and designation. At the end of your piece we will

use:

Your name

Faculty and role

Contact details

D I A L O G U E