midterm report - indiana universitysandbox.informatics.iupui.edu/~presrini/portfolio/... · 5 | p a...
TRANSCRIPT
1 | P a g e
Midterm Report
David Brainer-Banker | Christina Dunbar | Steve Goforth | Preethi Srinivas
Fall 2012
Mike Wilson
I543 Usability and Evaluative Methods
2 | P a g e
TABLE OF CONTENTS
EXECUTIVE SUMMARY .......................................................................................................... 4
I. PRESENTATION OF APPLICATION .................................................................................. 6
II. INSPECTION METHODS USED ....................................................................................... 11
A. COGNITIVE WALKTHROUGH ................................................................................................... 11
1. Tasks and Scenarios .......................................................................................................... 11
2. Severity rating system ........................................................................................................ 16 B. HEURISTIC EVALUATION ........................................................................................................ 16
1. Features inspected ............................................................................................................. 16
III. USABILITY INSPECTION RESULTS ............................................................................. 17
A. COGNITIVE WALKTHROUGH ................................................................................................... 17 B. HEURISTIC EVALUATION ....................................................................................................... 31
APPENDIX .................................................................................................................................. 34
CONCEPTUAL MODEL ................................................................................................................. 34 USER PROFILES & CHARACTERISTICS ......................................................................................... 36
SCENARIOS ................................................................................................................................. 42
REFERENCES ............................................................................................................................ 45
3 | P a g e
TABLE OF FIGURES
FIGURE 1. CANVAS INSTRUCTURE .................................................................................................................................. 6
FIGURE 2. CANVAS INSTRUCTURE SITEMAP .................................................................................................................. 7
FIGURE 3. CANVAS HOME PAGE ...................................................................................................................................... 8
FIGURE 4. DISCUSSIONS FEATURE .................................................................................................................................. 8
FIGURE 5. SETTINGS PAGE .............................................................................................................................................. 9
FIGURE 6. GRADE BOOK FEATURE ................................................................................................................................ 10
FIGURE 7. MESSAGES AND INBOX ................................................................................................................................. 10
FIGURE 8. CALENDAR FEATURE .................................................................................................................................... 11
FIGURE 9. CUMBERSOME AND NON-INTUITIVE STRATEGIES FOR CONNECTING TO THIRD PARTY SYSTEMS ................... 17
FIGURE 10. USAGE OF TECHNICAL TERMS AND POSITIONING OF CONNECTION OPTIONS AT MULTIPLE LOCATIONS ....... 18
FIGURE 11. DIFFICULTIES IN IDENTIFYING AND ENABLING FACEBOOK NOTIFCATIONS .......................................... 20
FIGURE 12. NO SUPPORT FOR UPLOADING DOCUMENTS FROM MULTIPLE GOOGLE DOCS ACCOUNT .............................. 22
FIGURE 13. DIFFICULTIES IN IDENTIFYING TECHNIQUE FOR SYNCING CANVAS CALENDAR WITH GOOGLE CALENDAR . 23
FIGURE 14. GOOGLE DOCS ALLOWED TO BE DOWNLOADED IN HTML FORMAT ........................................................... 24
FIGURE 15. NO CLEAR DISTINCTION BETWEEN POSTING TO A DISCUSSION OR ADDING A SIDE NOTE ............................. 25
FIGURE 16. ISSUES WITH COLLAPSING OR EXPANDING POSTS ....................................................................................... 26
FIGURE 17. INBOX FEATURE NOT FOLLOWING USUAL EMAIL CONVENTIONS ................................................................. 27
FIGURE 18. JUMBLING OF ASSIGNMENTS AND MESSAGES IN INBOX .............................................................................. 28
FIGURE 19. HIDDEN HOVER FEATURES IN GRADE BOOK ............................................................................................... 30
4 | P a g e
Executive summary
An expert review was used to evaluate the Canvas by Instructure learning management system
(LMS). The review consisted of two inspection methods--a cognitive walkthrough and a
heuristic evaluation. The review included evaluating the interface for the cloud-based LMS
service and its core features including: Gradebook, Messages, Discussions, Settings, as well as
its Web services (Google Docs, Google Calendar, SMS, social media and RSS). The cognitive
walkthrough comprised eight scenarios and ten tasks to analyze the user’s workflow when
interacting with each of the noted core features of the LMS. The heuristic evaluation referenced
Nielsen’s heuristics to identify key usability issues within the interface design.
The expert review revealed the following key findings for Canvas:
Fails to follow standard conventions
o Inbox (Conversations) does not follow normal email conventions. There are
ambiguous and mismatched labels (Inbox, Conversations, and Discussion
Replies) o Inbox does not use standard features for the expert user are missing (e.g. selecting
all read and unread messages). o Messages and assignment notifications do not have distinct labels within
Conversations although they serve two different functions. o In the discussion area the difference between the side note and the general posting
features is not apparent.
o The Settings feature does have consistent and accurate labeling or categorization
for the social media and web services.
o Social media icons cannot be easily identified or found. Their location is not
consistent with the grouping in the Settings are.
o Calendar does not allow for different viewing options. Only a monthly viewing
option is available.
Limited integration of social media and web services
o No obvious social media or web services capabilities until you enter Settings.
o The Google Docs web services feature does not allow for standard formats when
downloading, only HTML format.
o Facebook limited to only push notifications to timeline or inbox.
Navigation and workflow is unintuitive, inconsistent and cumbersome
o Hidden hover states in the Gradebook cells for inputting grades and comments
impede the instructor workflow instead of enhancing it with shortcut options and
flexibility of use. o The discussion area is missing visual cues for the collapsing feature.
5 | P a g e
The key recommendations for Canvas are as follows:
Follow standard conventions
o Provide visual cues for shortcuts in the Gradebook without having to hover over a
cell.
o Use one consistent label for the Inbox (Conversations) and follow standard email
conventions.
o Allow for standard expert user shortcuts to select all messages that have been read
or all that are unread in Conversations.
o Provide clear differentiation between messages and assignment notifications.
o Allow for standard calendar viewing options.
Make navigation and workflow more intuitive, consistent and user-friendly
o Provide visual cues to inform of additional postings when discussions have been
collapsed.
o Provide distinct language to show the difference between auto-clearing text for a
general posting and a side comment.
o Make the labels more transparent for general posting versus side comments.
o Place related information in relevant proximity within Settings.
o Provide a more appropriate and accurate label for social media and web services
besides “Other Services”. Possibly two distinct categories.
Improve social media and web services integration
o Make social media obvious from homepage of LMS
o Allow users to download standard formats for Google Docs such as .doc, .docx,
and .pdf.
o Allow for more collaborative uses of Facebook and Google Docs inside the LMS
6 | P a g e
I. Presentation of application
Canvas Instructure is a learning management system (LMS) that is built using Ruby on Rails
web application framework. As a partner with Amazon Web Services, Instructure provides
additional robust services by integrating with several other online services such as Facebook,
Google Docs, LinkedIn, Twitter etc. Leveraging other modern technologies such as HTML5,
Instructure helps educators and learners collaborate more efficiently.
Figure 1. Canvas Instructure
A video illustrating some of the capabilities of Canvas Instructure was created by our team and
can be found here http://www.youtube.com/watch?v=RM4PC_Ixi_A
The following site map gives an overview of the architecture of Canvas.
7 | P a g e
Figure 2. Canvas Instructure Sitemap
8 | P a g e
Sample screenshots of some of the features of Canvas are as follows.
Figure 3. Canvas home page
Figure 4. Discussions feature
9 | P a g e
Figure 5. Settings page
10 | P a g e
Figure 6. Grade book feature
Figure 7. Messages and inbox
11 | P a g e
Figure 8. Calendar feature
II. Inspection methods used
A. Cognitive walkthrough
1. Tasks and Scenarios
A total of 8 scenarios were created for the two user profiles, student and instructor. The goals
and its constituent tasks were identified based on the scenarios. A total of 10 tasks were
identified for the two user profiles.
12 | P a g e
The user tasks were then evaluated through cognitive walkthroughs that answered the following
4 questions [1]:
Will the user try to accomplish the task?
Will the user notice that the correct action is available?
Will the user associate the correct action with the effect to be achieved?
If the correct action is performed, will the user see that progress is being made toward
solution of the task?
Priya, Graduate student Key Tasks: Submitting to a Collaboration and Checking Messages Priya is from Northern India. She came to IUPUI right after completing her bachelor’s degree in
visual communication, and now lives with two roommates in Indianapolis. Her interests include
art and photography, and she will be a pursuing master’s program in visual communication
design at Herron School of Art and Design at IUPUI. She likes to communicate and socialize
with people and gain additional knowledge through the process. She believes in teamwork and
tries to stay connected and on-track with people during projects using tools such as Google
Calendar. She often feels connected with the nature and loves visiting new places, camping,
trekking, and taking photographs or videos. Being trained in visual communication, she is quite
proficient in using computer programs such as Microsoft Word, Adobe suite of products such as
Photoshop, Lightroom, etc. She owns a digital slr camera, a Macbook Pro in addition to iPhone
that she always carries around with her to stay connected with her friends through Facebook,
Skype and emails. She is new to IUPUI and has not yet had the opportunity to try OnCourse. Priya has been assigned to a team project that focuses on capturing nature and wildlife. Her team
consists of four members including her. Each member chose a region of focus to gather pictures
from and contribute to a final report before they started. Priya chose to report on the Tiger
reserves of India and is hence visiting India for two weeks. During her stay in India, her role with
respect to the project will be to capture as many interesting pictures or videos related to tigers,
write her portion of the final report and post possible interesting discussion points for the entire
User profile Tasks
Student
Instructor
• Changing preferences in Settings to receive notifications from Facebook
• Checking To Do List and Submitting to assignment with Google Doc
• Submitting a post to Discussion Forum
• Checking Grades
• Syncing Calendar to Google Calendar
• Reviewing student work turned in for assignment due
• Using SpeedGrader to grade assignments and give feedback. Mute assignments turned in.
• Revising a quiz
•Checking inbox and responding to message
• Sending an Announcement
13 | P a g e
class or the teacher to comment on. Priya logs into her canvas account. Since Priya is registered
for only one course for the semester, she can see only one course listed under the courses menu
item. She proceeds to find out if there are any new messages for her from her teammates or from
the class lecturer. She also checks at the online discussion forum to see if other students have
shared anything interesting. She makes a quick check of the calendar to ensure no new item has
been added to it. She then gets to the course related to her project, loads the common Google
document that has been setup for the project report and starts reporting her experience for the
morning. Once she is done writing her report, she saves the document, does a check of her inbox
again for new messages and logs off. After she is done with writing the report, Priya ensures she
backs up the photographs she captured for the morning, uploads one best photo from that batch,
and writes a catchy caption for her classmates and teacher to see and discuss about. She then
heads out to get hot tea and breakfast.
John, Senior Key Tasks: Submitting an Assignment and Checking Grades (Login, Check To Do List, Submit
assignment with Google Docs, Review grades) John is excited to finish his history degree at IUPUI. He hopes to get a job working in
communications at a marketing firm when he graduates. He generally spends most of his day in class
or doing homework. When he isn’t doing school related activities then he is spending time with his
friends or playing rugby. It is Thursday afternoon and John just got out of a history class. He knows he has a rugby game this
weekend out of town and will not have much time to do homework, so he decides to get started on
his assignment due for his Monday class. He logs into his Canvas account on his iPhone. First thing
he checks is the To Do and Coming Up lists. He sees on the To Do list that a paper on humanism is
due for Monday. On the Coming Up list he sees a paper on the agrarian world will be due next week. He walks to his reserved study room in the library. Once there, he logs into Canvas on his Macbook.
He clicks on the link for the humanism paper in the To Do List. He reviews the assignment
instructions, and then opens up another web browser to begin work on the paper in Google Docs.
When he is done he returns to Canvas to submit his Google Doc to the assignment. After submitting
the paper he decides to see how he is doing overall in his courses. He clicks on Grades in the main
menu and sees that he has a B+ average in 2 of his classes and an A average in the other two. He then
logs out of Canvas and heads off to grab something to eat.
Justin, Junior Key Task: Reviewing Syllabi and Syncing Calendar to Google Calendar Justin is a pre-med student studying biology. He received a scholarship to do research when he
applied to the university. He did such a good job on his research the last two summers that his
scholarship director named him an assistant researcher. He enjoys doing research and hopes to enter
into a joint MD/PhD program when he gets to medical school. He also participates as a member in a
few health related clubs on campus. Justin plans to take the MCAT this year and knows he will need
to stay on top of assignments and work ahead if possible. He logs into his Canvas account a week
before the semester begins to check his course syllabi and calendars. He plans to sync his Canvas
Calendar with his Google Calendar. He reviews the syllabi for his chemistry, biology, and physics
courses. He notices that the fifth week of the semester there is a conflict with his lab for chemistry
since he plans to present a research poster at a conference. He decides to send his lab instructor a
message letting her know that he will not be in lab on that date. After reviewing his calendar and
syllabi he clicks on Assignments from the main menu. He immediately looks at the To Do list and
begins working on the assignment that is due first.
14 | P a g e
Stephanie, Sophomore
Key Task: Using Facebook to communicate through Canvas and checking the to-do list. Stephanie is from a small town and came to IUPUI right after graduation. She lives with two
roommates in Indianapolis. She goes to school part-time, since she has not yet decided on a
major. She works long hours to keep her school debt down and as a result she has less time for a
social life than she had originally hoped for. She is often checks Facebook on her breaks at work
and between classes. She Stephanie received a Windows 7 laptop for a graduation gift. It is the
first computer she has ever owned. While at work Stephanie receives a Facebook notification
that her post from last week’s discussion was commented on by her Instructor. She reads the post
and decides to respond back to instructor. Once off work, she heads to her favorite place to
study, the local coffee shop where she never has to worry about finding a seat. She logs into
Canvas and checks her To Do list to see what homework to complete first. She notices that she
has to read two chapters from her sociology book and post a discussion by tomorrow. She pulls
out her book and begins reading. When she is done reading she submits a post to this week’s
discussion.
Zach, Associate Professor
Key Tasks: Reviewing Submitted Assignments and Grading Zach is a professor at Kelly School of Business at IUPUI. Aged 35, he is married and has two
sons. Zach is a hard-working, enthusiastic professor who focuses on entrepreneurship and is
interested in guiding students launch start-up companies by coordinating with financiers.
Considering he has to connect and contact with several people (clients), he is always on the
move. Although he has a desktop at his office space, he owns a tablet with android operating
system and carries an iPhone with him wherever he travels to stay in touch with clients while at
the same time completing his work as a professor (such as assigning homework to his students,
grading their work etc.) during free time. Zach is waiting to meet with one of the potential clients at Starbucks café. He had to drive 20
miles to get here and it is 5 pm in the evening. He is tired, but has to complete grading his
students’ assignments before that night. He had promised spending time playing baseball with
his sons and hence, wants to be done with work soon. He sees a message from his client being
half hour late and decides to start grading while he waits. Hot coffee helps Zach feel fresh and
more ready to start grading.
Raquel, Adjunct Professor
Key Tasks: Reviewing Student Assignments and Grading (Login, Check Assignments, Grades,
Announcements) Raquel works full-time as a physical therapist and has been an adjunct professor at IUPUI for the
past 4 years. She teaches an evening course in kinesiology in the school of health and
rehabilitation sciences. She enjoys teaching the course because she likes sharing her knowledge
and staying on top of the latest news in her field. She typically sets up her course and makes all
materials available online two weeks before the semester begins. She devotes her Sundays to
grading student work and providing feedback. It is Sunday and Raquel needs to grade her
students’ reflection papers. She typically has the students complete reflection papers each week
applying what they have learned. She logs into Canvas and clicks on the Assignments from the
main menu. Once on the Assignments homepage for her course, she reviews the Upcoming
15 | P a g e
Assignments and the Undated Assignments. Next, she clicks on the assignment for the reflection
paper in the Upcoming Assignments list. It takes her to the reflection paper assignment, so she
can review the assignment instructions before grading papers. After reviewing the instructions
she is ready to grade and clicks on Grades in the left menu bar. She then goes down the roster
and grades each student’s paper leaving feedback. Once finished grading she decides to send out
an announcement with an interesting video she found on kinesiology movement analysis, which
she thinks will be a great addition to the lecture on Monday. All done with grading she decides is
ready to meet up with her fiancé for a concert.
Don, Professor
Key Tasks: Creating, revising and checking student quizzes and responding to student questions. Don is a professor of economics and editor of a well-known journal. When he isn’t staying on
top of the latest economic trends and journals, then he is advising the economics club. He took
an interest in the economics club when a few students asked him to come and speak about
various governmental, economic, and social issues. He told the students that he would advise
them only if they could find a co-advisor to handle logistics of the club, since he preferred only
to lead discussions. Don typically follows a strict schedule with his microeconomics course. He
has been teaching the course for five years and has revised the syllabus each semester to include
some of the latest journal articles. The only changes he will make are on the quizzes which he
releases each week. He logs into Canvas to check his inbox each evening. He knows that there
are typically students who struggle with the course content due to not studying enough. He finds
three messages from students concerned about their grades in his inbox and responds to them.
Then, he clicks on the microeconomics course in the Courses menu. He wants to see how many
students have already completed the quiz due tomorrow. He notices that most students have
finished the quiz. He looks to see which questions posed a challenge. He makes a note to discuss
the questions that students had difficulty with in lecture tomorrow. Finally, he revises makes
revisions to the quiz for next week.
Sonya, Assistant Professor Key Tasks: Using the Canva Speed Grader to grade and mute assignments and responding to student
messages. Sonya works long hours at the office and at home. She is wants to become a tenured professor in
the school of engineering, so she puts in the extra hours to help her students and do research. She
is the director for the engineering simulation lab and has four students that work for her. She also
teaches a 300 level engineering course on Tuesday and Thursday afternoons. When she is not in
the lab doing research or in her office grading or helping students, then she is spending time with
her family. Her 2 teenage sons often ask her to attend their sports games and her husband often
asks her counsel on house related matters. Sonya does not have much time to spend on grading
and lives by the motto, “work smarter, not harder”. She logs into Canvas and uses the Speed
Grader to grade assignments daily. She does not want to spend a whole Sunday grading, but
prefers to grade as she gets student work. The typical weekly assignment requires students to
answer a word problem, so that she knows they are grasping the lecture material. The students
must show the calculations for how they came up with an answer. She grades the six assignments
that were turned in and provides feedback to the students, but chooses to Mute Assignment. She
does not want a student seeing their grade until she has graded all returned assignments. Next,
she checks her inbox to see if there are any messages. She finds a message from a student who
16 | P a g e
wants to set up an appointment. She responds to the student with her availability. After grading
and checking messages, she heads to her son’s soccer game.
2. Severity rating system
Throughout our evaluation process, three severity levels were chosen to represent the priority of
the issues found in Canvas; Low, Medium, High.
Low (Minimal) - Minimal impact on the user’s ability to complete task, low priority. Does not
impede the user’s ability to complete task.
Medium (Cautionary) - Moderate impact on user’s ability to complete task. Somewhat
impedes the user’s ability to complete task.
High (Critical) - High impact on the user’s ability to complete task. User cannot complete a
task. Impedes on the user’s ability to complete task.
B. Heuristic evaluation
1. Features inspected
The overall design and work-flow were evaluated for the following features. More details on the
individual features can be found below.
Discussions: Discussions provides the user instructor and students to have a streamlined, forum-
like, conversation. During the evaluation, the features inspected include posting, notifications,
dates displayed and the unread and assignments options.
Settings: Settings allows the user the opportunity to modify personal account settings and sync
3rd party accounts such as Facebook, Twitter and Google Docs. During the evaluation, the
features inspected include add and remove email addresses, phone numbers, social media
accounts and syncing Google Docs. Gradebook: Gradebook gives the instructor the opportunity to add, modify and leave feedback
on student grades. During the evaluation, the features inspected include Speed Grader, Mute
Assignment and the hidden (hover-over) drop-downs.
Conversations (Messages): Conversations allows the users to communicate with one another
via course, group or by individual user. During the evaluation, the features inspected include
creating and responding to messages (including file attachments) and notifications.
Calendar: Calendar provides the user with the option to keep track of their upcoming
assignments, quizzes and tests on the Canva’s interface or sync it with the personal Google
Calendar. During the evaluation, the features inspected include the ability to add to the calendar
and syncing with Google Calendar.
17 | P a g e
III. Usability inspection results
A. Cognitive walkthrough A total of eleven problems were identified based on the 10 tasks. They are listed below.
1. Connecting to third party systems is a cumbersome and not intuitive.
Figure 9. Cumbersome and non-intuitive strategies for connecting to third party systems
a. There is no clear indication that it is possible to connect to Facebook.
Description: You must browse to your user settings screen before the option to connect to
Facebook is even presented to you. Once there the option is still in a difficult to find location.
Task: 1- Changing preferences in Settings to receive notifications from Facebook.
Severity: Low
Category: Notification
18 | P a g e
Implication: Users will be unaware of the options available to them and will lose any benefit that
the features could have provided.
Recommendation: Move the option to a location where other notification-related information is
displayed and/or promote the option to the user in prominent locations such as the dashboard.
b. There is no value proposition as to how connecting to Facebook will benefit you.
Description: It is not particularly clear what connecting to Facebook will do and why you would
want to do it.
Task: 1- Changing preferences in Settings to receive notifications from Facebook.
Severity: Low
Category: Notification
Implication: Users will be unable to make appropriate choices about which features to use and
will lose any benefit that the features could have provided. Users may closely monitor what apps
post to and scrape from their timeline and be concerned about allowing the app to connect.
Recommendation: It would be helpful to have the intended purpose and value of connecting to
Facebook made clear to the user. At the very least this information should be included as help
text with the options.
Figure 10. Usage of technical terms and positioning of connection options at multiple locations
19 | P a g e
c. The option to enable Facebook notifications is difficult to find and identify.
Description: The user must enter their user “Settings” screen to look for the option. Once there,
the Facebook icon is displayed but it is quite far down and would be below the fold for some
people. It is not grouped in an area which indicates that it has anything to do with notification,
but rather is under a section entitled “Web Services” which may not mean much to someone
without a background in technology. There is a section to the right which does include
notification options (and this is, in fact, where an indication of Facebook notification is displayed
once the process is complete) but the “Add Contact Method” option available in that section only
applies to text messages.
Task: 1- Changing preferences in Settings to receive notifications from Facebook.
Severity: Medium
Category: Notification
Implication: Without outside instruction, all but the most determined, curious or savvy users will
be unlikely to discover how to enable Facebook integration.
Recommendation: Improve clarity of the options available by renaming the “Web Services”
section and including descriptive text to explain what connecting to each available service will
entail and what will be provided in return. Alter the “Add Contact Method” button so that it
offers options for all available contact methods.
20 | P a g e
Figure 11. Difficulties in identifying and enabling Facebook notifcations
d. The notifications screen within settings does not contain any obvious links to or mention
of the process of adding new notification methods.
Description: Users must ignore the “Notifications” menu option on the left hand side if they
want to add a new notification method. Following that option will take them to a screen where
they can choose which notifications are sent to which service, but the addition of new services
happens only on the general settings page.
Task: 1- Changing preferences in Settings to receive notifications from Facebook.
Severity: Medium
Category: Notification
Implication: Users might become confused by the lack of clarity and unintuitive organization of
the screens and sections.
Recommendation: Move the “Other Services” (social networking) options to the dedicated
notifications page for consistent grouping. A section “advertising” the availability of these
options could remain on the general settings page, and the “Add Contact Method” button should
continue to link to all available contact methods.
21 | P a g e
e. It is not possible to use files from multiple Google Docs accounts.
Description: If the user must use a document from a separate Google Docs account, they will
need to logout, then log back in with the other account.
Task: 2- Checking To Do List and Submitting to assignment with Google Doc
Severity: Medium
Category: Assignment Submission
Implication: Users will not be able to pull documents from two separate Google Doc’s accounts.
The user will be forced to log out of one account, then log into the other.
Recommendation: Allow users to add multiple Google Doc’s accounts within Canvas. This will
let users have access to different documents from separate Google Doc’s without needing to log
out and into another account.
22 | P a g e
Figure 12. No support for uploading documents from multiple Google docs account
f. It is difficult to figure out how to add the Canvas calendar to your Google calendar.
Description: For the novice user, it is going to be difficult to figure out how-to subscribe to the
Canvas Calendar in Google Calendar. Unless the user has subscribed to a Google Calendar
before, they will not know to click on small “Calendar Feed” on the Calendar page, copy the
URL then paste it into their Google Calendar.
Task: 5 – Syncing with Google calendar
Severity: High
Category: Calendar
Implication: Users won’t know they have the opportunity to sync their calendar with Canvas.
23 | P a g e
Recommendation: Change the nomenclature from saying Calendar Feed, to “Sync Canvas with
Google Calendar” and provide a link that sends the user directly to the Google Calendar page,
asking for permission to sync.
Figure 13. Difficulties in identifying technique for syncing Canvas calendar with Google calendar
g. Viewing or downloading assignments submitted in Google Docs format.
Description: If an instructor chooses to view or downloads a student’s assignment that has been
submitted in Google Docs format, the document’s file format will be .HTML. HTML is not a
format that most users use to download documents. If a user tries to download a document and
they are prompted to save an HTML file, they will think that the server is down.
Task: 1 - Reviewing student work turned in for assignment due
Severity: Medium
Category: Gradebook
Implication: Inability to add annotation (for critique student’s work), overall confusion over file
type.
24 | P a g e
Recommendation: The file needs to be downloaded in a widely recognized format; for example,
.doc, .docx or .pdf. This will allow instructors to annotate and leave comments for students in
the document. By changing the file format, it will also eliminate all confusion on whether or not
they are really downloading a student’s submitted work.
Figure 14. Google docs allowed to be downloaded in HTML format
2. Familiar standards and conventions are not followed.
a. It can be difficult for students tell apart the fields related to posting to a discussion and
those for adding a “side note.”
Description: This issue is particularly noticeable for long discussions. The fields used to add
discussion posts and those used to add side notes are very similar in appearance and neither is
very well labelled - the only distinctions between them are indentation, background color and the
“ghost” text displayed in the field before entering content. For long discussions the screen is
littered with these text boxes, and once the user has scrolled all the way to the bottom the option
to add a side note to the very last post may appear more logical than scrolling all the way to add
a new post at the top. In addition, the text box for adding a new post is labelled with “Write a
reply...” and based on common conventions people may think of a “reply” as being something
more similar to what is called a “side note” here.
25 | P a g e
Task: 3- Submitting a post to Discussion Forum
Severity: High
Category: Discussion
Implication: Grading assignments may be more difficult for the instructor if the students become
confused and accidentally add side notes instead of proper replies. It will also become confusing
to follow the discussion in this case.
Recommendation: Increase the emphasis of the “Write a reply...” text box, improve labeling for
both types of text box, and consider updating the ghost text to something more indicative of the
task users are trying to perform (adding a post to the discussion).
Figure 15. No clear distinction between posting to a discussion or adding a side note
b. Clicking on a post in the forum collapses all of the options and side notes under it
without providing any indication that the collapsed content exists after it is hidden.
Description: While the ability to collapse items in a threaded discussion is a common feature,
users need some indication that there is content hidden from their view after collapsing items so
that they will not inadvertently miss important details.
Task: 3 - Submitting a post to Discussion Forum
26 | P a g e
Severity: Medium
Category: Discussion
Implication: Users may miss important information that was posted in the side notes without
realizing it.
Recommendation: Add something that indicates that additional items are available when a post
is collapsed. As a nice touch it might also be nice to indicate when something can be collapsed in
the first place.
Figure 16. Issues with collapsing or expanding posts
c. Inbox feature does not follow the usual email conventions.
Description: Email clients typically perform a range of standard tasks - composing, reading,
replying to, and deleting messages - and a set of conventions has been adopted by nearly all of
them which enable users to perform these tasks using largely similar actions. Canvas deviates
from these conventions for no obvious reason, making the inbox much more difficult to deal with
than would be expected. When replying to a message you click on a message on the left and a
27 | P a g e
conversation-style view is brought up on the right showing all of your messages from the sender
of the message you originally clicked on. A message composition dialog box also appears on the
right above the conversation view with a text box labeled “Message”. Additionally, to delete a
message you must click on a small gear icon in the top left corner of the message and choose the
option from a drop down menu. The organization of the Inbox is not made particularly clear
either, seeming to collapse messages into groups by “participants” (all senders/recipients) of the
messages.
Task: 4 - Checking the inbox and responding to messages
Severity: High
Category: Communication
Implication: The average user has a deep familiarity with email clients and may become
frustrated with the lack of consistency when using Canvas. It is possible that they will choose to
use regular email instead of the tool in canvas.
Recommendation: Modify Canvas to behave more like standard email clients. Also adopt the
conventional terminology - when composing a reply the text box should be labeled “reply”
rather than “message.”
Figure 17. Inbox feature not following usual email conventions
28 | P a g e
d. Messages and assignment submissions are mixed together in the Inbox and the
differences are not clearly delineated.
Description: The Canvas Inbox jumbles many different types of messages together into a single
view and does provide clear visual cues as to which type of message a user is currently viewing.
Assignments are formatted slightly differently in the right-hand conversation view, with the
assignment name, link and score displayed at the top, but this formatting could certainly be
clearer. In addition, there is no differentiation at all on the left hand side.
Task: 4 - Checking the inbox and responding to messages
Severity: Medium
Category: Communication, Assignments
Implication: Users, particularly instructors, may lose track of messages, especially around a time
when many assignments are due at once.
Recommendation: Add a label or icon which more clearly differentiates between assignments
and messages, and also include that label or icon in the summary list view on the left.
Figure 18. Jumbling of assignments and messages in inbox
29 | P a g e
3. Key information is hidden from view.
a. It is not immediately obvious that important features and information is hidden in
unmarked hovers.
Description: Important features within Canvas Gradebook (Assignment Details, SpeedGrader,
Message Students Who..., Set Default Grade, Curve Grades, Download Submissions, Mute
Assignment) are being hidden by drop downs that only appear after you hover over the
assignment header then click the downward arrow button. Another option that is hidden by
hover-overs is the ability to comment on a student’s assignment. To find the button, the user
must hover over the assignment then click the tiny chat bubble that appears in the upper right
hand corner of the assignment.
Task: 1 – Reviewing student work turned in for assignment due
Severity: High
Category: Gradebook
Implication: If this is the user’s first time using gradebook, they will have extreme issues not
knowing how to use the interface. Though this learnability issue could be resolved after using
gradebook a few times, it is still needs work. The UI should be intuitive and easy to use for first-
time users.
Recommendation: Make the drop down and comment options visible without the need of
hovering. Similarly, the comment feature needs to be visible at all times.
30 | P a g e
Figure 19. Hidden hover features in Grade book
31 | P a g e
B. Heuristic Evaluation
A heuristic evaluation inspected 5 areas: Discussions, Messages, Calendar, Grades, and Settings.
DISCUSSIONS FEATURE Severity Heuristic Description Recommendation
High Visibility of system status
• Collapsed postings not obvious. System status is difficult to comprehend. • No clear distinction between “writing a reply” and “adding a side comment”. • The dates are ambiguous as there is no label to signify what the date represents (e.g. due date or start date) • Numbering system listed next to the discussion titles is ambiguous. Discussions without posts are not consistent with numbering system.
• Improve visual cues for collapsed postings • Follow conventional discussion standards and have only option to “write a reply” • Provide labels for dates in discussion • Consistent numbering system for discussions that have postings and those that do not (i.e. 0 = zero postings)
High User control and freedom
• Confusing hierarchy of labels for buttons that are in close proximity to each other: “Unread” and “Assignments”. • No clear indications as to which posts are unread within a discussion. • Limited visible search terms in search field. Only permits searching by title, body, or author in unread discussions. No option available for other search terms or for searching discussions that have been read.
• Consistent hierarchy in labels that are in close proximity (e.g. “Unread” and “Read”) Remove Assignments button label. • Use standard labels and visual cues for unread and read discussion postings. • Broader search options
High Consistency and standards
• “Unread” and “Assignments” are not consistent in hierarchy with one another. Acceptable nomenclature would be “Read” and “Unread”. “Unread” is a state of being, not a to-do item like assignments. • Inconsistency across LMS: Discussions and Inbox (Conversations) represent unread messages in an inconsistent manner. Discussions represents unread messages with a bold font. Conversations represent unread messages with a blue dot next to the conversation.
• Consistent hierarchy in labels that are in close proximity (e.g. “Unread” and “Read”) Remove Assignments button label. • Make sure that unread and read messages are consistent across the LMS
Medium Recognition rather than recall
• Ambiguous dates next to discussion topics on discussion homepage.
• Provide labels for dates
32 | P a g e
SETTINGS FEATURE Severity Heuristic Description Recommendation
High Consistency and standards
• Social networking icons are divided into two categories but there is not description of the differences. • Google Docs, Skype, Facebook, Linkedin, Twitter, Delicious, Diigo do not belong in the same category. These services have separate uses and it is not clear how each application is used within Canvas.
• Facebook, Linkedin and Twitter could be under a “Social Media” category • Google Docs, Skype and Diigo could be under “Online Collaboration Tools” category • Delicious could be under “Aggregator”
High Recognition rather than recall
• Terms “Web Services” and “Approved Integrations” may not be recognized by novice users. • The button “New Access Token”may also be difficult for a novice user to understand.
• Use more transparent terminology
Low Help and documentation
• None help and documentation easily accessible or visible
• Provide quick help menu or more obvious functions for features in Settings
GRADEBOOK FEATURE Severity Heuristic Description Recommendation
High Visibility of system status
• The page looks static, like a read-only document (i.e. features to comment on a grade, options to mute an assignment, opening speed grader are not visible etc.).
• Provide visual cues for hidden gradebook features without needing to hover over it.
High Recognition rather than recall
• There are several hidden information in terms of hover-overs
• Provide visual cues for the types of possible interactions for buttons at-a-glance and without having to hover over them.
High Consistency and standards
• Confusing labels: the mute assignment icon is also used with a slash for “instructor is working on grades” icon.
• Distinguish labels for different functions. Follow typical LMS icons for grading progress.
Low Help and documentation
• For the lack of visible controls on the screen, there should be some sort of quick help guide like in Calendar
• Provide quick help menu options
33 | P a g e
MESSAGES (CONVERSATIONS) FEATURE Severity Heuristic Description Recommendation
High Consistency and standards
• The “reply text box” looks too similar to the new message text box. • The right hand side of the Conversations page is not intuitive. Composing a new message can also be reply to a previous message that is added to the discussion. • No apparent difference between Inbox, Conversations, and Discussion Replies. Yet all three labels are used. • The messages is not similar to other email applications in functionality (i.e. reply is separate from same level functions like forward and delete)
• Use consistent labels • Use standard email conventions
High Recognition rather than Recall
• The number in a grey box next to each message is ambiguous. (i.e. does is represent new messages or unread messages?
• Follow standard email conventions or provide a label
Medium User control and freedom
• Users have to click on each message to mark read or unread. There isn’t a way to select all messages “read” or all messages “unread”.
• Use standard email conventions
Low Help and documentation
• The tutorial doesn’t provide information on Discussion Replies. The tutorial is not extensive enough for the various features in Conversations. • Some parts of Canvas have quick help documentation while others force user to go to Canvas guides.
• Improve quick help menus and/or documentation • Need consistent help documentation across the LMS
34 | P a g e
CALENDAR FEATURE Severity Heuristic Description Recommendation
High Consistency and standards
• The only visible option for the user is to see a monthly view of the main calendar. • The link to add calendar feed has low visibility (on the bottom right of the calendar) • Undated Events label is visible, but Undated Assignments label is not shown.
• Follow standard conventions for calendar viewing options • Improve visibility of calendar feed • Include a visible label for Undated Assignments
High User control and freedom
• The calendar does not appear to allow for viewing by day or by week. • There is no visible way to close the popup for the undated events without refreshing the browser.
• Allow for standard calendar views • Add a close button to popup for undated events
Medium Visibility of system status
• The pastel color coding scheme for the calendars is hard to distinguish on the neutral gray background. May cause the user confusion. • Refresh icon appears static and does not look clickable.
• Easier color scheme for quick glances • A refresh icon that looks clickable
Medium Recognition rather than recall
• There is no obvious view by day function. Only option for user is scrolling a bar within a cell (when there may be several items).
• Allow for standard calendar views and at-a-glance viewing
Low Help and documentation
• The “Create a new assignment” help tip has a hyperlink that is not active in the documentation.
• Correct the hyperlink
APPENDIX
Conceptual model The conceptual model below gives an overview of the core interactions that occur between the
key users of Canvas.
35 | P a g e
Overview
Enlarged views – Admin and Teacher
36 | P a g e
Enlarged view – Student
User profiles & characteristics Characteristics of the student and instructor key users are identified below.
Student profiles
John, 21, Senior
Background: John is a full-time student studying history. He is single and lives with 3
roommates. John works at a pizzeria a few hours every week.
He is excited to finish his history degree at IUPUI. He hopes to get a job working in
communications at a marketing firm when he graduates. He generally spends most of his day in
class or doing homework. When he isn’t doing school related activities then he is spending time
with his friends or playing rugby. John owns a smartphone and laptop.
Motivations:
Playing rugby
Spending time with friends
Knowing the latest tech news
Graduating and getting a job
37 | P a g e
Frustrations:
Not having clear instructions for assignments
Roommates staying up late and making noise
Being bored, wasting time
Out-of-date information online
Goals:
Quickly check assignments, due dates, and details
Communicate with classmates easily
Get immediate feedback or questions answered by instructor
Glance able learning objectives and outcomes (what I will learn)
Justin, 22, Junior
Background: Justin is a pre-med student studying biology. He received a scholarship to do
research when he applied to the university. He did such a good job on his research the last two
summers that his scholarship director named him an assistant researcher. He enjoys doing
research and hopes to enter into a joint MD/PhD program when he gets to medical school. He
also participates as a member in a few health related clubs on campus.
Justin is a full-time student, has a girlfriend and lives off-campus with a roommate. He owns
iPhone, iPad and a Macbook Pro.
Motivations:
Socializing in school club activities
Club Sports
Hanging with friends
Keeping above a 3.8 GPA
Getting accepted into graduate school
Frustrations:
Not finding time to see girlfriend
Not being in-charge of group research
Roommate not picking up after himself.
Not knowing when assignments have been posted
Goals:
Knowing when assignments are posted to the online course
Being able to contact classmates easier through the online course.
38 | P a g e
Meeting deadlines by knowing when changes have been made on the online course.
Stephanie, 19, Sophomore
Background: Stephanie is from a small town and came to IUPUI right after graduation. She lives
with two roommates in Indianapolis. She goes to school part-time, since she has not yet decided
on a major. She works long hours to keep her school debt down and as a result she has less time
for a social life than she had originally hoped for. She is often checks facebook on her breaks at
work and between classes. She Stephanie received a Windows 7 laptop for a graduation gift. It is
the first computer she has ever owned.
Motivations:
Finding the right major
Managing the cost of school with minimal input from her family
Social time with her roommates
Frustrations:
Feeling overwhelmed with coursework, group projects and job
Lack of personal attention from instructors when stuck
Uncertainty when so many of friends seem to know where they are headed
Goals:
Getting assignments done quickly without wasting precious time.
Finding ways to do homework on downtime at job
Feedback from instructors, TAs or even classmates when she needs it the most.
Priya, 24, Graduate student
Background: Priya is from Northern India. She came to IUPUI right after completing bachelor’s
degree in visual communication, and now lives with two roommates in Indianapolis. Her
interests include art and photography, and she will be pursuing masters program in visual
communication design at Herron School of Art and Design at IUPUI. She likes to communicate
and socialize with people and gain additional knowledge through the process. She believes in
teamwork and tries to stay connected and on-track with people during projects using tools such
as Google Calendar. She often feels connected with the nature and loves visiting new places,
camping, trekking, and taking photographs or videos.
Being trained in visual communication, she is quite proficient in using computer programs such
as Microsoft Word, Adobe suite of products such as Photoshop, Lightroom, etc. She owns a
digital slr camera, a Macbook Pro in addition to iPhone that she always carries around with her
39 | P a g e
to stay connected with her friends through Facebook, Skype and emails. She is new to IUPUI
and has not yet had the opportunity to try Oncourse.
Motivations:
Travelling to new places and meeting new people
Photography and filming
Working on interesting group projects
Graduating and working for a well-established commercial firm
Frustrations:
Difference in teaching pattern from that experienced in India
Lack of clear instructions on assignments
Lack of knowledge on when assignments or grades or online class discussions are posted
Lack of communication and co-ordination between team members while working on
group projects
Goals:
Understanding and coping with education pattern at IUPUI
Having clear up-to-date information about assignments or grades or online discussions
Being able to contact other team members and staying on-track with respect to a group
project
Being able to write reports as a team with each team member contributing to the project
Finding internships
Instructor profiles
Raquel, 33, Adjunct Professor
Background: Raquel works full-time as a physical therapist and has been an adjunct professor at
IUPUI for the past 4 years. She teaches an evening course in kinesiology in the school of health
and rehabilitation sciences. She enjoys teaching the course because she likes sharing her
knowledge and staying on top of the latest news in her field. She typically sets up her course and
makes all materials available online two weeks before the semester begins. She devotes her
Sundays to grading student work and providing feedback.
Motivations:
Meeting the needs of her physical therapy patients
Helping her students get the core concepts
Staying on top of work and teaching duties
Spending time with fiancé
40 | P a g e
Frustrations:
Spending a lot of time grading
Traffic congestion to campus
Students that make excuses
Not being able to find related resources online
Goals:
Make sure students know what’s required
Easy access to students work that has been submitted
Quickly return grades to students with feedback
Provide novel ways for students to learn
Don, 52, Professor
Background: Don is a professor of economics and editor of a well-known journal. When he isn’t
staying on top of the latest economic trends and journals, then he advising the economics club.
He took an interest in the economics club when a few students asked him to come and speak
about various governmental, economic, and social issues. He told the students that he would
advise them only if they could find a co-advisor to handle logistics of the club, since he preferred
only to lead discussions.
Motivations:
Highly involved with the Economics club as an advisor
Writing more articles
Going home and relaxing after teaching
Tending to his herb-garden
Collecting vintage baseball cards
Frustrations:
Student excuses
Grading student work online
Staying up to date with three separate email accounts.
Goals:
90% Pass-rate in class.
Communicating to students easier.
Knowing if students have been active in checking new announcements on the course site.
Finding new ways to make science relative and interesting to students
Having a successful semester.
41 | P a g e
Seeing his students graduate.
Sonya, 42, Assistant Professor
Background: Sonya works long hours at the office and at home. She is wants to become a
tenured professor in the school of engineering, so she puts in the extra hours to help her students
and do research. She is the director for the engineering simulation lab and has 4 students that
work for her. She also teaches a 300 level engineering course on Tuesday and Thursday
afternoons. When she is not in the lab doing research or in her office grading or helping students,
then she is spending time with her family. Her 2 teenage sons often ask her to attend their sports
games and her husband often asks her counsel on house related matters.
Motivations:
Gaining tenure
Making sure her students have real-world experiences
Spending time with family
Doing groundbreaking research in her field
Frustrations:
Managing heavy workload
Coordinating service learning part of her course
Not having enough time with family
Goals:
Set up field experiences as quickly as possible
Finding ways to do homework on downtime at job
Spend minimal time grading
Using class time for collaboration and teamwork
Being recognized by colleagues
Zach, 35, Associate Professor
Background: Zach is a professor at Kelly School of Business at IUPUI. Aged 35, he is married
and has two sons. Zach is a hard-working, enthusiastic professor who focuses on
entrepreneurship and is interested in guiding students launch start-up companies by coordinating
with financiers. Considering he has to connect and contact with several people (clients), he is
always on the move. Although he has a desktop at his office space, he owns a tablet with android
operating system and carries an iPhone with him wherever he travels to stay in touch with clients
while at the same time completing his work as a professor (such as assigning homework to his
students, grading their work etc.) during free time.
42 | P a g e
Motivations:
Always interested in talking to people, establishing contacts as clients for his students to
launch potential ideas as start-up companies
Making sure his students gain practical experience by assigning them to real-time
projects and grading the same
Advising and guiding his students
Spending quality time with his family
Frustrations:
Multi-tasking and managing time schedules with meeting clients and classes
Lack of some student’s interest and slacking student performance
Grading students’ work
Lack of ability to stay up-to-date on every student’s performance with ease
Students’ not aware of recently posted assignment to be completed unless notified
separately through extra emails or messages
Unable to spend quality time with his sons
Goals:
Sync personal calendar with calendars for all the courses handled to stay up-to-date and
up-to-time while teaching classes or attending meetings with clients
Finding ways to clearly notify students of any new assignments with ease
Grade students’ work and report the grades and feedback comments with ease
Seeing potential student’s ideas to be launched as start-up companies
Scenarios
Student
It is Thursday afternoon and John just got out of a history class. He knows he has a rugby
game this weekend out of town and will not have much time to do homework, so he
decides to get started on his assignment due for his Monday class. He logs into his
Canvas account on his iPhone*. First thing he checks is the To Do and Coming Up lists*.
He sees on the To Do list that a paper on humanism is due for Monday. On the Coming
Up list he sees a paper on the agrarian world will be due next week.
He walks to his reserved study room in the library. Once there, he logs into Canvas on his
Macbook*. He clicks on the link for the humanism paper in the To Do List*. He reviews
the assignment instructions, and then opens up another web browser to begin work on the
paper in Google Docs. When he is done he returns to Canvas to submit his Google Doc to
43 | P a g e
the assignment*. After submitting the paper he decides to see how he is doing overall in
his courses. He clicks on Grades* in the main menu and sees that he has a B+ average in
2 of his classes and an A average in the other two. He then logs out of Canvas* and heads
off to grab something to eat.
Justin plans to take the MCAT this year and knows he will need to stay on top of
assignments and work ahead if possible. He logs into his Canvas account* a week before
the semester begins to check his course syllabi and calendars. He plans to sync his
Canvas Calendar with his Google Calendar*. He reviews the syllabi* for his chemistry,
biology, and physics courses. He notices that the fifth week of the semester there is a
conflict with his lab for chemistry since he plans to present a research poster at a
conference. He decides to send his lab instructor a message* letting her know that he will
not be in lab on that date. After reviewing his calendar and syllabi he clicks on
Assignments from the main menu. He immediately looks at the To Do list and begins
working on the assignment that is due first*.
While at work Stephanie receives a Facebook notification that her post from last week’s
discussion was commented on by her Instructor. She reads the post* and decides to
respond back to instructor*. Once off work, she heads to her favorite place to study, the
local coffee shop where she never has to worry about finding a seat. She logs into
Canvas* and checks her To Do list* to see what homework to complete first. She notices
that she has to read two chapters from her sociology book and post a discussion by
tomorrow. She pulls out her book and begins reading. When she is done reading she
submits a post to this week’s discussion*. Her friend calls her because she needs a ride to
the grocery store.
The sun rises on early Sunday morning as Priya sits waiting to catch a glimpse of a tiger
or two at the enchanting preserve of Royal Bengal Tigers. The initial rays of sunlight
shining through dew drops on the leaves and the surrounding rivers of Ganges,
Brahmaputra and Megna creating a delta with lush vegetation is indeed splendid and
Priya cannot help but click photographs of the scene. Priya manages to see a streak of
tigers involved in the ‘wild’ and captures several photographs. She then returns back to
her room near the tourism office and manages to connect her laptop to the Internet. Priya
has been assigned to a team project that focuses on capturing nature and wildlife. Her
team consists of four members including her. Each member chose a region of focus to
gather pictures from and contribute to final report before they started. Priya chose to
report on the Tiger reserves of India and is hence visiting India for two weeks. During her
stay in India, her role with respect to the project will be to capture as many interesting
pictures or videos related to tigers, write her portion of the final report and post possible
interesting discussion points for the entire class or the teacher to comment on.
Priya logs into her canvas account*. Since Priya is registered for only one course for the
semester, she can see only one course listed under the courses menu item. She proceeds
to find out if there are any new messages for her from her teammates or from the class
44 | P a g e
lecturer*. She also checks at the online discussion forum* to see if other students have
shared anything interesting. She makes a quick check of the calendar* to ensure no new
item has been added to it. She then gets to the course related to her project, loads the
common Google document* that has been setup for the project report and starts
reporting* her experience for the morning. Once she is done writing her report, she saves
the document*, does a check of her inbox again for new messages and logs off. After she
is done with writing the report, Priya ensures she backs up the photographs she captured
for the morning, uploads one best photo from that batch*, and writes a catchy caption for
her classmates and teacher to see and discuss about. She then heads out to get hot tea and
breakfast.
Instructor
It is Sunday and Raquel needs to grade her students’ reflection papers. She typically has
the students complete reflection papers each week applying what they have learned. She
logs into Canvas* and clicks on the Assignments from the main menu*. Once on the
Assignments homepage for her course, she reviews the Upcoming Assignments and the
Undated Assignments*. Next, she clicks on the assignment* for the reflection paper in
the Upcoming Assignments list. It takes her to the reflection paper assignment, so she
can review the assignment instructions before grading papers. After reviewing the
instructions she is ready to grade and clicks on Grades in the left menu bar*. She then
goes down the roster and grades each student’s paper leaving feedback*. Once finished
grading she decides to send out an announcement* with an interesting video she found on
kinesiology movement analysis, which she thinks will be a great addition to the lecture
on Monday. All done with grading she decides is ready to meet up with her fiancé for a
concert.
Don typically follows a strict schedule with his microeconomics course. He has been
teaching the course for five years and has revised the syllabus each semester to include
some of the latest journal articles. The only changes he will make are on the quizzes
which he releases each week. He logs into Canvas to check his inbox each evening*. He
knows that there are typically students who struggle with the course content due to not
studying enough. He finds three messages from students concerned about their grades in
his inbox and responds to them. Then, he clicks on the microeconomics course in the
Courses menu*. He wants to see how many students have already completed the quiz due
tomorrow. He notices that most students have finished the quiz. He looks to see which
questions posed a challenge. He makes a note to discuss the questions that students had
difficulty with in lecture tomorrow. Finally, he revises makes revisions to the quiz for
next week*.
Sonya does not have much time to spend on grading and lives by the motto, “work
smarter, not harder”. She logs into Canvas and uses the Speed Grader to grade
assignments daily*. She does not want to spend a whole Sunday grading, but prefers to
45 | P a g e
grade as she gets student work. The typical weekly assignment requires students to
answer a word problem, so that she knows they are grasping the lecture material. The
students must show the calculations for how they came up with an answer. She grades the
six assignments that were turned in and provides feedback to the students, but chooses to
Mute Assignment*. She does not want a student seeing their grade until she has graded
all returned assignments. Next, she checks her inbox* to see if there are any messages.
She finds a message from a student who wants to set up an appointment. She responds to
the student with her availability*. After grading and checking messages, she heads to her
son’s soccer game.
Zach logs into his canvas account*, checks his inbox* for new messages and finds none.
He proceeds to super grader. He notices that all his students had turned in the report* that
was due before the deadline and this makes him feel happy. He views the first student’s
report. He refers to the rubric* he had initially designed for that assignment and quickly
selects the points* for each such rubric item based off the report. In addition to grading,
Zach adds comments* to parts he thinks needs improvement and parts he thinks are really
good. Towards the end, Zach is very happy with the report and leaves an audio message*
congratulating the student for the work on the report. In a similar manner, Zach grades
other reports and is done grading all the student reports by the time his client arrives. He
is very pleased with the quality time he had spent while waiting to meet with his client
and is looking forward to spending the rest of the evening following this meeting with his
sons.
References [1]- http://www.usabilityfirst.com/usability-methods/cognitive-walkthroughs/