SmartEvidenceA Mahara Development ProposalShane Nuessler and Misty Kirbyshane.nuessler@canberra.edu.au and misty.kirby@canberra.edu.auUniversity of Canberra

BackgroundAnecdotal evidence that

● Students needed just-in-time interventions (instructions, guidance on what evidence is, etc) when building their portfolios for assessment.

● Students found Mahara difficult to use unless they had step by step instructions.● Various disciplines have adopted a template approach to provide interventions and scaffolding,

however students found Mahara unintuitive and did not engage unless required. ● A “how to use it” training approach took time away from the most important task of students

learning the standards, what appropriate evidence consists of, how to collect evidence, confidentiality and privacy, ethics, etc.

● Excessive time taken to assess portfolios due to:○ multiple ways students present evidence, regardless of the template○ having to scan the portfolios visually linking artefacts on screen to particular standards and

competencies○ spending time assessing portfolios with missing or inadequate rationale for the presence of

the evidence.

Preliminary Investigation Findings

Themes from initial conversation between UC and Catalyst● Concerns around quality of evidence being submitted● Teachers need to locate evidence efficiently● Need multiple ways to sort/search/display evidence● Need to re-communicate interventions through system● Want students to be able to self assess● Need to support/encourage reflective process● Need simple way for students to display and share

evidence/rationale/self assessment etc.● UC has multiple disciplines with competency/standards● Take into account use of additional strategies

Themes from literature● Reliability and validity of ePortfolios ● Reflective component of portfolio practice is critical to deep

learning● Purpose needs to be clearly defined● Assessment criteria (and use) needs to be clearly defined ● Need multiple engagement/intervention strategies

Preliminary Investigation FindingsFindings from focus groups and interviews with teachers and students

Students● Had trouble visually connecting groups of evidence together on a page● Had need for technical training/support● Needed guidance around what constitutes good evidence (on screen)● Need for more personalisation● Students don’t realise they need to be making connections from evidence to multiple standards● Some want a standard template approach/ others prefer creating their own

Teachers● Workflows vary from teacher to teacher● Would like different ways to access and display evidence e.g. grid view of the standards.● Clicks/complexity was an issue for everyone● Need to provide clear purpose and direction to students● Students generally good at describing the evidence but not deriving meaning/connections esp to multiple standards.● Fear that standardisation would lessen student creativity

Proposed SolutionUnderpinning concepts:1. Usefulness:

○ Develop student critical thinking through provision of relevant evidence interventions that are built into the interface, displayed on demand and when students attempt to provide evidence against a standard.

○ Mahara provides mechanism to store multiple standards frameworks and related interventions (making tacit expectations and requirements explicit).

○ Staff can efficiently assess student performance against any given standard. 2. Usability:

○ Students are able to choose the standard(s) that apply to their context, limiting student exposure to only those standards relevant to them.

○ Mahara makes it easy for students to link evidence to standards - providing an intuitive and simple means so students can focus on the primary task.

○ Staff can choose how Mahara displays evidence provided by students - e.g. on a student organised portfolio, or a drill down through matrix of students, or through a matrix of standards, via reports that highlight lack of evidence against certain standards, etc, etc.

Mockups - A

dmin Settings

Mockups - Student Settings

Mockups - Providing evidence

Mockups - D

isplaying Evidence