Department of Computing, Engineering & and Technology

University of Sunderland

Will embedded ePortfolio-based supervision lead to greater student engagement

in a capstone project?

Dr Siobhan Devlin & Dr Gary Unthank

Module Context• Final year UG project – capstone module - worth 40 credits (1/3 of

year). So, over time: – Students have had the pervasive initial pedagogy that frames and

prefigures professional preparation – This is one of the pervasive capstone apprenticeships (real world

experience) – It is also a culminating sequenced and balanced portfolio (the sum of the

parts of the software engineering process – problem analysis & specification, client interaction, literature review, product build (design, build, test, evaluate) and dissertation write up)

• They are computing students – Computer Science, Computing, ICT, Computer Forensics, Business

Computing, Network Computing, Games Software Development – Some have done a placement (additional capstone apprenticeship)

Supervision Process• Students have individual supervisor but it is a timetabled interaction

in the project room – At any one time there’ll be several supervisors sitting with one

student each for a 20 minute interaction

• Supervisor is not the project sponsor – The student is not doing a project for the supervisor e.g. in their

area of academic research – As we are an institution with a big focus on employability we

ensure students interact with both a client and a supervisor as each entail different skills development

• Supervisor is often the personal tutor/programme leader – Embedded PDP

Previously…• Progress and advice recorded on carbonated paper

– A copy for the student and a copy for the supervision file kept in the project room

• Easy to do but: – Difficulty deciphering hand writing – Students lose their copies – Students could only demonstrate what they brought in paper

files or on own electronic device – Students chose to engage or not. Missed sessions meant no

record/no progress check/no advice given – Students could hide lack of progress/bury their heads in the

sand – Supervision “after the fact”; not active involvement

Enter Mahara…• We had previous experience of using it with first year students

(these same students now in final year) – With less structure – Less focus – At that time we thought the students would take the tool

and run with it (they’re permanently online, and they’re computing students!) – but they didn’t

• So, some trepidation in both tutors and students – “First year I knew a lot less about the way to structure and use the

system properly, …” (student) – “I know originally I did not like the thought of using it or adding blog posts

…” (student)

But…it was a new and improved version of Mahara

• Better interface • Better functionality • Which both staff and students could see

• And the module leader had done an online course on ePortfolios…

• So we were hopeful

Initial hopes…• We wanted to be better able to track and encourage

engagement

• For engagement, read “professional disposition”

• In Shulman’s (2005) Signature Pedagogy terminology “Professional education is not education for understanding alone; it is preparation for accomplished and responsible practice in the service of others”.

Signature Pedagogy of Software Engineering?

SE is: systematic, controlled, cost-effective, scientific approach to the development, operation, and maintenance of SW, using appropriate tools and techniques, according to the constraints you are working within

SP of it must be: problem-based/ problematised, systematic, controlled, cost-effective, scientific, authentic, hands on (the instruction & critique that Shulman describes in an engineering lab), interactive & reactive, evolving, emergent, team based – including peer review, constrained (including time),

What did we do?

Students & staff given the above template & training

Then, pretty much work as usual but with the eP being used as the supervision vehicle

What have we found so far?

• 7 weeks in

• 1st assessment in week 6 (unfortunately not yet marked)

• But several supervision sessions in we can start to draw some tentative conclusions

Student feedback

Strong agreement• The eportfolio is a good way to keep my supervisor in the loop/

informed about what’s going on in the project • I am able to share evidence of what I’ve done with my supervisor

more easily than if we weren’t using an eportfolio • The eportfolio helps a supervision to run smoothly as everything is

stored in one place • The eportfolio enables me to see my achievements build up bit by bit

on the project

• The eportfolio makes me think about what I’m doing/have done before I go to a supervision session

• The eportfolio helps me to track my own progress • The eportfolio helps me to reflect on my progress

Some agreement• I am able to share evidence of what I’ve done with my

client/sponsor more easily than if we weren’t using an eportfolio (not many at that stage yet)

• Seeing my achievement build up on the eportfolio helps motivate me on my project

• The eportfolio means I can’t hide lack of progress from my supervisor (or myself)

• I am uploading a variety of media-rich artefacts to my eportfolio (text, images, audio, video etc) (not much opportunity yet)

• I am only uploading text documents (not much opportunity yet)

No agreement • The eportfolio has a negative effect on the supervision conversation • I find I am able to do bits and pieces ‘on the go’ on the eportfolio

because of its portability (Staff think this is happening with ‘weaker students’ but they aren’t necessarily the ones who have responded to this q’naire yet)

• I only really put stuff there before I go to a supervision • Using the eportfolio is a pain • I feel that the eportfolio helps supervisor communication outside of

the supervision sessions • It’s only used in the supervision session • I feel like using the eportfolio is helping with my professional

development – What do their agreed points add up to if not professional behaviour?

“Personally I think that the e-Portfolio is allowing us to monitor our progress much better than last year’s

paper based systems.”

“It’s nice to be able to fill in the record before supervision and then not having to remember to bring all the paper based records ...

Getting feedback on your supervisions is much easier than last year and you can’t lose the copy of paper!”

“If you keep your blog up to date with what you are doing then the supervision blog is very easy to do

and takes next to no time.”

“ePortfolio is a good way of keeping track of what I have done and submitted, I can look

back on previous progress and compare with what I need to achieve.”

“Supervision sessions are very structured in what is checked

…then the feedback given is always at hand so improving upon what I have done is easier for me.

It also looks to be a good way of ensuring regular progress …and that deliverables are given to support the claims.”

• “I am using the eportfolio to keep a weekly blog of what I am doing on my project and also to keep a blog on what I am doing on another module as well. I think it is a good way to keep all of my work together as well.”

Staff Feedback

Q: are the students doing anything differently?

• No, not really although some students are more active in creating documents that they can upload, e.g., client meeting notes.

• It is helping them reflect on their own progress better. If a student hasn’t done enough in previous weeks, it is difficult for them to ‘hide’ from this. It has also eliminated the need for students to bring work in paper or digital format to the sessions, which streamlines the process for us.

• There may be a slight improvement in the timeliness of when students do things

• The better students add far more detail to review prior to coming to the supervision meeting

• Those engaging are creating blog entries and appear to be in better control

• It’s getting some of them to think more – they’re certainly keeping up with learning logs etc.

“There may be a slight improvement in the timeliness of when students do

things.”

Has the supervisor role changed?

• No • Not in terms of what I say but perhaps in how I give examples/feedback • I don’t feel my role has changed, but I do feel the portfolio enables students to

fulfil their role more appropriately and to take greater control of the project experience from the onset.

• I feel a little like I have a better chance at influencing & encouraging as I can see more at a glance and can point out what they have. I feel a little more ‘involved’ whereas writing comments on paper feels ‘after the fact’

• Yes – now with the ‘good’ students I review their progress (which has a lot more detail). For weaker students the role remains the same.

• I feel like I can concentrate more on talking to the student and then write a shorter summary comment on the ePortfolio rather than filling in the three boxes on the old forms.

• I am also finding that I’m expecting more work to be provided for me to review in the session whereas in the past they often didn’t have this with them.

Issue where supervisor wants to be project manager rather than scrum master, or assessor rather than

facilitator?

• It’s great for accessing and tracking the docs – its just typing in the feedback. I want to give the students quality feedback and I end up wasting time correct my silly typing errors etc.

• The students occasionally forget to create a supervision record before coming to their supervision slot. This means I have to ask them to log in and create a page (I want to keep records sequential.) I have also had one case where a student restructured the links in her portfolio and has ‘lost’ / hidden my supervision feedback for the previous weeks. She has agreed to resolve this by our next session.

• supervisor need to supervise efficiently vs student choice & flexibility

• Questions/discussion?

% staff agreement100%

70%

67%

80%

90%

• I can track my students’ progress easily on the eportfolio system • I can track my students’ progress better on the eportfolio than I

could on paper • I’m not able to compare but I feel the eportfolio allows me to

track progress better than a paper based system would • I see evidence of student deliverables more readily than when

students had to remember to bring additional things to the supervision meeting

• My students are not able to hide a lack of progress now that we use the eportfolio system

% staff agreement40%

30%

20%

10%

30%

• My students seem more engaged with their projects than usual

• It seems as though the students are engaging whenever/wherever they can since the supervision ‘tool’ became a flexible online tool.

• The evidence of progress that I can see is media rich/is more media-rich than previously

• The student reflections are meaningful/seem more meaningful than previously

• The students seem to be developing/presenting more professional dispositions