table of contents - idetportfolio.com€¦ · web viewtable of contents. part 12. team members2....

HCI: Final Project 2 May 2011ED PS 6440 Laura Adams, Rose Defa, LaVar Edwards

Table of Contents

PART 1.....................................................................................................................................2Team Members............................................................................................................................................................................ 2Statement of Problem/Challenge........................................................................................................................................2Review of Related Products/Systems................................................................................................................................2Summary of User Interviews.................................................................................................................................................7Summary of Participant Profiles..........................................................................................................................................9Design Priorities....................................................................................................................................................................... 10Design Issues..............................................................................................................................................................................11Persona......................................................................................................................................................................................... 12Task Statements....................................................................................................................................................................... 14

PART 2...................................................................................................................................15Internal Testing: Design Revisions...................................................................................................................................15

Interviews.................................................................................................................................................................................... 15Mockups....................................................................................................................................................................................... 15Cognitive Walkthrough......................................................................................................................................................... 25Heuristic Analysis..................................................................................................................................................................... 27

User Testing................................................................................................................................................................................31Design............................................................................................................................................................................................ 31Experimental Protocol........................................................................................................................................................... 31Tasks.............................................................................................................................................................................................. 32Participants................................................................................................................................................................................ 32Summary of Results/Overview of Findings................................................................................................................... 33

PART 3...................................................................................................................................33Recommendations/Revisions from User Testing......................................................................................................33

High Priority Design Issue.................................................................................................................................................... 34Moderate Priority Design Issue.......................................................................................................................................... 34

Final Design................................................................................................................................................................................ 34What is the Final Design and How Will it Meet the Needs of Members?..........................................................34Meeting the Needs of Members.......................................................................................................................................... 35

Future Issues.............................................................................................................................................................................. 35End of Alpha Release.............................................................................................................................................................. 35End of Beta Release................................................................................................................................................................. 35Hosting and Plans for Growth............................................................................................................................................ 36Seeking Volunteers.................................................................................................................................................................. 36Conference Presentations..................................................................................................................................................... 36

Pg 1 of 36


PART 1

Team Members

"Laura Adams" <[email protected]>"Rose Defa" <[email protected]>"LaVar Edwards" <[email protected]>

Statement of Problem/ChallengeAn increasing number of schools and districts are securing funds and technology grants that will provide every student with a mobile handheld device to use in the classroom. Administrators and teachers are overwhelmed at the extensive selection of applications and find that the available search tools for these apps are not focused enough to identify high-quality, free apps that support the educator’s instructional goals.

Our goal is to design a rating and review system for educators to identify apps based on their effectiveness as an educational tool and to foster a community for educators to share their knowledge and experience with other teachers.

Review of Related Products/SystemsWhen comparing our project website with other related websites, it’s evident that we are defining a niche in the App Search space. This is because the functionality needed by educators to quickly find free, high-quality apps without advertisements is not currently being addressed. Education Week recently reported finding useful apps for classroom use is difficult. Several sites have created lists of recommendations, but APPS4EDU.org is the first comprehensive search, rating, and recommendation website available.

As of Jan 11, 2011, Apple’s iTunes App Store has sold 10 billion Apps; many were at no cost. Currently as of May 2011, 31,749 educational apps are “active” and available. However, the Apple Search Engine does not allow searching by subject, grade level, rating, cost, or genre (education). iTunes functionality does not allow the user to copy and paste text or save a list of apps for further review, even when interesting apps are identified within iTunes.

Pg 2 of 36


The iSchool Initiative is typical of Educational App recommendation sites; it supplies lists of recommended apps. However, no search mechanism, you can’t make your own recommendations, or review existing apps.

Pg 3 of 36


This is not to say we did not have any inspiration for designing our project. Amazon.com GoodReads.com, Yelp.com, CitySearch.com, and eBay.com where all mentioned during our interviews as websites with excellent search, filter, rating, review, and/or recommendation functionality.

Pg 4 of 36


Pg 5 of 36


Pg 6 of 36


Summary of User InterviewsThe average age of our interviewees is between 42 and 64 years with 17 years of teaching experience. All interviewees hold at least a master’s degree and most have specialized in some type of educational technology. Positions include specialists in media, instructional, and school technology, as well as an Assistant Superintendent. When asked to rate themselves in technical proficiency on a scale of 1-5, 1= novice, most rated themselves a “4”. They all use mobile devices, primarily iPad or iPod Touch, and apps daily.

All of the interviewees work in the same school district. Within that district, wireless Internet is available in all secondary schools when using school district equipment; personal wireless devices are not allowed. Wireless Internet access should be available in elementary classrooms within the next two years. Generally, our interviewees were frustrated that personal iPads could not be used in the classroom and recommended that district iPods be available to all students.

When asked about using ratings on the Web, almost all referenced apps; all indicated they used ratings and reviews. All but one listed Amazon as their most frequented site to access ratings and reviews. Other popular sites for ratings were GoodReads and

Pg 7 of 36


Yelp. Overall, they use ratings and reviews consistently for assessing apps and determining whether to trust a site based on the number of other people using it.

Interviewees described information that would be useful to know before installing an app. They indicated they would like to view a screen shot. One interviewee stated that she would like to watch a short “video demo” before installing. Interviewees rely on websites, magazine reviews, and peer recommendations to find useful apps. They may also look for new apps from developers they already know and like. Most value the ratings and reviews to evaluate apps and some had rated an app themselves.

Interviewee preference is to search for apps by subject, grade level, rating and price; however, they would like to take their searches further and have the capability to drill down to subsets of subjects or the national core curriculum. One interviewee suggested the capacity to perform a boolean search. They listed searching for apps as a major issue and expressed that many apps are not worth the time to search and download. Interviewees requested easier navigation and better keyword searching to find more targeted and specific apps. In addition, they felt there are far too many choices to sift through in order to find the right app that meets their precise need.

All interviewees preferred free apps to those they had to pay for and commented that they would like a better selection of free apps that are more appropriate to their subject. There were multiple concerns that quality free apps were becoming difficult to find. Most felt that making recommendations to the developer for improvement, fixes, and/or upgrades to their app was worth the time invested. Some of the interviewees had already made recommendations to the developer. Others had not, nor did they think it was worth the time.

In comparing the Internet to apps, about half preferred apps. They pointed out that not all students have iPads or iPods; some said it would be nice to have both. They expressed enthusiasm with the benefits of apps in the classroom, such as ease of access, motivation and ability to self-pace, but expressed concerns about students being off-task, behavior problems, losing devices, handing in assignments and printing.

Interviewees suggested that to improve the quality and experience of mobile technology in education, they would like students to be able to download (district policy prevents this). They identified some technology issues such as battery life, wireless issues, and old routers. They would like to see improvement and greater ease in wireless communication, updating and upgrading devices, and easier search features. Their greatest concern when downloading apps is compatibility and bandwidth.

Pg 8 of 36


Summary of Participant Profiles

Demographics 42 - 64 years old, female, masters level in educational technology; average of 17 years teaching experience; works in Granite School District..

Experience with technology

On a scale of 1- 5, 1 = novice, participant is a 4. Uses Apple technology such as iPod, iPad touch daily.

Rating and review process

Participant values the rating and review of apps and is experienced rating an app. They use Amazon primarily, but also GoodReads and Yelp. They use ratings and reviews consistently for assessing Apps and determine whether to trust a site based on number of other people using it.

They like to search by subject, grade level, rating and price, but want to drill down to subsets of subjects, the national core curriculum, perform a boolean search or watch a short “video demo” before installing.

They like making recommendations to developers for improvement, fixes or upgrade.

Usage of apps In comparing Internet to apps, participant prefers apps. Knowing that not all students have iPads or iPods, they want both. Participant wants more user-friendly navigation for finding specific apps and relies on third-party resources to decide whether to consider an app or they look for trusted developers.

They are enthusiastic about the benefits of apps in the classroom, such as ease of access, motivation and ability to self-pace but express concerns about students being off-task, behavior problems, losing devices, handing in assignments and printing.

Access Participant is concerned with access, compatibility, bandwidth and impact. Within the district, wireless Internet is available in all secondary schools (although Internet should be available in elementary classes within the next two years), but they are frustrated that personal iPads cannot be used in the classroom;

Pg 9 of 36


they want district iPods for all students.

They think that quality free apps are becoming difficult to find and believe that technology issues such as better battery, wireless issues, old routers, update and upgrade devices will impact application in the classroom.

They want more free apps, and apps more appropriate for subject; searching for apps is a major issue given the sheer number and difficulty finding the best app.

Design Priorities

Our design priorities and the needs of Barbara Persona go hand in hand. Is it important to Barbara? Then it is important to implement in the website. We found there were things we could not do within the scope of the website that Barbara is concerned with, such as classroom and student use, compatibility and bandwidth issues.

Website Design It should be obvious the site is an educational app search engine No external advertising

Search Functionality Support search by subject, grade level, rating, price, and “search text” Easily identify “important” characteristics on first page. Limit the need for drill

down Should offer similar search functionality to a site like Google Should offer similar filter functionality to a site like Amazon

Rate and Review Functionality Allow members to rate and review apps Allow members to rate other member’s reviews Should offer similar functionality to sites like Amazon, GoodReads, Yelp, and

eBay.

Pg 10 of 36


Recommend Allow members to create lists; lists would include “My apps”, future research,

recommendations, and custom lists Should offer similar functionality to Amazon and GoodReads

Design Issues

Website design Present information succinctly; educators do not have a lot of extra time. Site should be written to frugally use bandwidth; schools do not have a lot of

excess capacity. Site should follow “common internet standards”; links underlined, text readable,

high contrast between text and backgrounds, limit use of icons, responsive, use terminology of the target audience, and not require training (aka intuitive).

Search and Rate and Review (R&R) functionality The success of the search functionality is dependent on an adequate number of

educational apps containing rating and review information. The inherent problem with the Apple and Android sites is the lack of categorical educational data. The solution proposed relies on the creation and attribution of categorical information to existing apps. After the site is made available, a significant amount of time and effort will need to be devoted to rating and categorizing apps until enough external educators are rating and reviewing apps to sustain the effort and reach critical mass.

Existing Data A mechanism needs to be developed to sync the data between Apple and our

website.

Android or Blackberry or Microsoft? A decision will need to be made if Android and/or Blackberry and/or Microsoft

Apps will be included in the future. Database, data structure, and page layout should incorporate their may need to

be a high-level distinction to include which platform supports which App.

Pg 11 of 36


Persona

Mrs. Barbara Caldwell, age 42, lives in South Jordan, Utah with her husband and two children, ages 11 and 14. She attended Weber State College where she earned her B.S. in Biology Composite Teaching with a Minor in Earth Science Teaching. She graduated in 1996 and started teaching Science at Kearns High School. She is a popular teacher among students, because of her enthusiastic teaching style, knowledge, and sense of humor. Although Barbara has always considered herself somewhat tech-savvy, she felt there was more she could do to use technology more effectively in her teaching. Motivated to learn better methods of technology integration, she enrolled in graduate school at Utah State University where she earned her M.Ed in Instructional Technology in 2002. Barbara enjoys teaching high school science, but eventually wants to become a district technology coordinator so she can train other teachers to better utilize technology.

Barbara’s school encourages students to utilize mobile devices and access the Internet through the school’s wireless network. Although technology has its challenges such as outdated devices, district controls, and limited bandwidth, Barbara nonetheless feels empowered. She is utilizing this new technology in an attempt to teach her students how fun and interesting Biology and Earth Science can be, rather than simply a required class all students must pass for graduation. Barbara believes the mobile Apps are more effective at motivating marginal students than an outdated textbook. Occasionally, she tries a new technology product, only to find that it did not work so well in her classroom. Either it was ineffective, or it made classroom management difficult. Barbara likes to be innovative and takes pride in her progressive state-of-the-art teaching approach. Thus, taking a chance on new technologies is worth the risk of an occasional lesson flop.

Barbara utilizes the iTouch at her school and teaches her students how to access and evaluate resources on the web. She collaborates with the other faculty and likes to exchange ideas to improve her classroom instruction. She sometimes feels frustrated

Pg 12 of 36


that other teachers don’t share their ideas and strategies as much as she would like; the only thing teachers have less of than money is time.

As for Barbara’s personal technology profile, she believes that any task that can be accomplished online ought to be. Her favorite web sites for sharing information are Amazon, GoodReads, and Yelp. She prefers to visit solid, well-known sites rather than unknown sites she doesn’t trust. She thinks most sites are too cluttered and prefers a clean interface. She commented that she doesn’t like a lot of marketing noise. She doesn’t trust a site when she sees a lot of mixed marketing messages. It makes her think that the site developers are desperate and confused about what they are promoting. She spends about 2 hours per day on the Internet, not including email, attendance, grading, and other admin tasks. At school she uses a Macbook. At home she uses Windows. Barbara owns an iPad and keeps it with her at all times. Her primary goal when she purchased the iPad was email access, social networking, and web browsing when her husband or kids were using the computer. Since then, she has come to realize she prefers the touch screen to the keyboard. Barbara likes how easy it is to install and use Apps on her device and she doesn’t miss needing to wait for the computer to boot up, the virus database to be updated, or required patches to be installed.

Barbara has found many apps that she likes. Some of her personal favorites include Amazon, Facebook, and Pandora. Barbara also downloads and tries various Apps for her classroom that she has either heard of through word of mouth or look interesting in the online App store. However, she wants to be very confident in the educational value of an App before recommending it to be installed on all of her students’ devices. There is quite a bit of time and effort involved to coordinate the installation of an App on hundreds of devices. If the App costs money, this is also an expensive proposition. Barbara would use more Apps, more often, if they were free and easier to install, update, or remove from all of her students’ devices.

Barbara would like a website focused on addressing the problems of using apps in education; how to find high quality, preferably free, apps for use in the classroom. Ideally, she would like to search for all of the apps useful in a high school biology class. Even when she knows she would like to use an app to study the “Periodic Table of Elements”, the App Store search result yields 14 apps, ranging from Free to $4.99 each. To see the Rating and Review information, she needs to select each individually which is very time consuming. Even more frustrating is a search for “High School Biology” that only identifies seven apps, none of which are the Periodic Table of Elements used in every Biology class. Like most teachers, Barbara does not have extra time to waste.

Pg 13 of 36


Task Statementsa. Barbara wants to search for a free App for use in the classroom that will assist students in learning the Periodic Table of Elements. Because of the time and effort involved, she wants to research candidate apps before installing and testing. Using Chrome, Firefox, or Explorer, she browses to Apps4edu.org and searches for an interactive App that teaches and showcases the Periodic Table of Elements. She is looking specifically for an App designed for use in all high school level science courses (including biology, chemistry, physics, environmental and general science). The app must be free of charge and highly rated. She has rated Apps and relies on “a group consensus” to assist her decision making process. She first evaluates the ratings then reads several reviews to help match her educational needs for a Periodic Table of Elements. This looks like the perfect app until she views the screenshots and realizes the Periodic Table of Elements app doesn’t actually include a Periodic Table of Elements. At a minimum, the app needs to be more functional than the poster on her wall or the two pages in the textbook. Barbara pressed the back button and selects another app to review. She checks the ratings, reviews, and screenshots before realizing there is a video available. Two minutes later, the video clearly demonstrates the functionality she requires and clicks on the link to the external site so she can download the app.

b. Barbara wants to warn other educators not to waste their time evaluating a Periodic Table of Elements app that does not include a Periodic Table of Elements. She searches for and finds the app. She selects Review App and indicates she dislikes the app and gives it a 1 star rating, the lowest possible. Furthermore, she types a review stating she can’t recommend the app because it is missing the fundamental functionality, specifically the table of elements. The organization of the elements in the standardized table is one of the most important items for her students to learn. She saves her changes and believes her review will save other teachers from wasting time bothering with such an inferior app.

c. Barbara has used numerous apps and wants to use a Periodic Table of the Elements App as an example so that other educators can learn from her experience and decide whether to use similar apps in their classrooms. She decides to start with ones that she is currently using for her Biology Technology class. Using Chrome, she browses to Apps4edu.org and selects an app related to the Periodic Table to review. She analyzes the cost, grade level, subject and search narrative. She also considers the site’s rating system and protocol to rate the app. She writes an optional review that will provide constructive feedback for other potential users of the app such as ease of use, applicability to grade level, and other search functions. She inputs her rating, indicating the app name, grade level and subject, and her rating. She notes that the cost and the last update are recorded in the review.

Pg 14 of 36


PART 2

Internal Testing: Design RevisionsInterviewsOur interviews actually marked our first mental design and re-design of our system. Initially, we interviewed random educators thinking statistically that this idea of “randomness” was a more favorable method for data collection. Our first group of interviewees did not use or have access to mobile devices in their classrooms for educational use. Therefore, they were not able to provide useful information regarding their needs, let alone give us any reason to believe they would have use for our system. We conducted another round of interviews - this time targeted to our ideal user of mobile technology and apps in the classroom. As a result of these interviews, our persona came to life and we had an ideal user.

Mockups The mockup phase was the shift from the intangible to the tangible. The light bulb moment occurred during this phase. This is when it became clear our project went from 10 pages, or so, a hundred, or so. This is where search, member, rate and review, recommend, login, and a few edit screens met forgot password, privacy policy, login error, categorization edit, a bunch of admin screens, member management, etc. If it were not for the use of JavaScript and AJAX (Web 2.0 techniques), the page count would have easily exceeded 250. Knowledge of current development techniques streamlined the mockup design process.

Pg 15 of 36


Mockup cont.

App Search screen

Description: The search screen is similar in functionality to Google or Bing. Type anything and see the results of the search. It will only search for educational Apps. Filter will allow the search to be restricted based on the categorical information. A result of 0 rows found will display search suggestions; initial screen will display search suggestions.

Pg 16 of 36


App Detail screen, Rating tab

Chemical Elements Free

Ratings: 4.9 avg (61 reviews)Subject: Science (Biology, Chemistry, Earth Science, Physics)Grade: Elementary, Secondary, Post SecondaryType: ReferenceCost: FreeAds: No

| RATINGS | Description | Screenshots | Video | (tabs / links to navigate to additional data)

Ratings Detail:5 (90%) displayed as graphic instead of text4 (10%)3 (0)2 (0)1 (0)

Most liked review

91% like the following review. Do you?

Will look great on any table, periodic or otherwise, 7 Oct 2009Review by Barbara Caldwell (Teacher)

The Elements is a photographic tour de force of items from Theo Gray's personal collection of element samples. If he were to put on a museum show or do a PBS series, this would be the companion book. …

Other reviews follow …

NOTE: Thumbs up = like

Thumbs down = dislike No circle = inappropriate; review, screenshot, video, whatever would be removed until clip was reviewed..

Pg 17 of 36


Description: The App Detail screen contains the name of the App, button to write a review, expanded rating information, and all of the associated categorical information.The Ratings tab contains detailed rating breakdown, the “most liked” review, and a complete list of all reviews.

App Detail screen, Description tab



| Ratings | DESCRIPTION | Screenshots | Video |

NOTE: the App developer writes the description

This app gives you a vast amount of element information, directly from you iPhone, in a matter of seconds!

Great for students, professors, scientists, or anyone with an interest.

It is very easy to view the different Elements which are grouped by:

-Element Names -Atomic Numbers -Element Symbols -Element States

Each Element includes information about its:

-Atomic Weight -State -Period -Group -Year it was discovered -Category -Melting point

Pg 18 of 36


-Boiling point

Each Element has two buttons for quick navigation to its page on WIKIPEDIA, and WOLFRAM|ALPHA for more in-depth information.

… more …screen would scroll

Description: The App Detail screen contains the name of the app, button to write a review, expanded rating information, and all of the associated categorical information.The Description tab contains the detailed description of the app.

App Detail screen, Screenshot tab



| Ratings | Description | SCREENSHOTS | Video |

Share a screenshot button / link to upload screenshot(s). Screenshots would sort by Like

Pg 19 of 36


100% like this screenshot. Do you? 100% like this screenshot. Do you?

Additional screenshots would cause screen to scroll Description: The App Detail screen contains the name of the app, button to write a review, expanded rating information, and all of the associated categorical information.The Screenshots tab contains all of the app screenshots and a button to add new screenshots.

App Detail screen, Video tab



Pg 20 of 36


| Ratings | Description | Screenshot | VIDEO |

Share a video ← button / link to link / embed video(s). Videos would sort by Like

100% liked the following video. Do you? Title: iPadVideo by Barbara Caldwell (Teacher)

…Additional links to embedded video could be included…

Description: The App Detail screen contains the name of the app, button to write a review, expanded rating information, and all of the associated categorical information.The Video tab contains all of the app videos and a button to add new embedded video links.

Pg 21 of 36


Home

NOTE: All of the highest rated Core and Career / Technical apps appear on the home page.

“Highest” is a function of the average rating and number of votes. (e.g. An average rating of 5)

Description: The home page is the 1st page people are likely to see, however, due to increased use of search engines, this is not necessary.

The home page is intended to be a useful location appropriate for every visit. As such, no background information about APPS4EDU can be found here. Although the page currently displays Highest Rated Apps by Subject Area, this is likely to change based on member feedback.

Pg 22 of 36


Login

Description: The login panel is available on every page. A cookie is set to automatically login on future visits; the user sets this capability.

Barbara Caldwell’s Profile – Awards tab

Profession: TeacherExperience: 14 yrsDegrees: M.Ed Instructional Technology

BS Biology; Minor Earth Science Teaching Tech: iPad, Android Phone

Follow me on:

| AWARDS | Reviews |

Barbara’s Apps3Edu.org awards (click on award for more information)

Pg 23 of 36


NOTES: Information, including name, would be displayed or hidden based on privacy settings. A pseudonym could be used instead of name. Photos are optional

Description: The My Profile page provides detailed information related to a login. A login must be a verified email address, however, the profile can be a pseudonym or real name. Simple links will allow logins to share links to external sites like Facebook or Twitter. Photos can be uploaded. Educational information, degrees, experience, etc.

The Awards tab displays all of the awards given to the login. The intent is to identify the prolific reviewers, raters, etc.

Barbara Caldwell’s Profile - Review tab

Profession: TeacherExperience: 14 yrsDegrees: M.Ed Instructional Technology

BS Biology; Minor Earth Science Teaching Tech: iPad, Android Phone

Follow me on:

| Awards | REVIEWS |

Review of Chemical Elements FreeTitle: iPadPosted: 10 Jan 2011


Pg 24 of 36



Review of Chemical Elements FreeTitle: Will look great on any table, periodic or otherwisePosted: 7 Oct 2009

The Elements is a photographic tour de force of items from Theo Gray's personal …NOTE: More reviews would scroll. Reviews would be sorted reverse chronologically

Cognitive WalkthroughWe conducted our first formal cognitive walkthrough early on to identify ease of use and flaws in the hierarchy or sequencing of task design. To accomplish this, we used our initial paper prototype design. The initial task we used was to have our persona, Barbara, search for an app to use in the classroom that would assist her students in learning the Periodic Table of Elements.

We identified deficiencies in the paper prototype and worked together on a system redesign intended to prioritized the design issues and correct the serious flaws in the interface.

Summary of Design Issues and Revisions from Cognitive Walkthrough AnalysisOur greatest design issue that we needed to address was the lack of a home page. Obviously this was critical and necessary for our design functionality. This was an obvious issue and didn’t actually take a full cognitive walkthrough to discover. Another critical issue that became an immediate redesign priority was the lack of clear navigation within the system. We had no clear path to let the user know how to navigate through the system and get from one page to another. Minimally, we needed to add a uniform navigation bar on every page.

Pg 25 of 36


The “back button” issue became a greater priority than we initially thought it would. Although there was expressed concern during our group planning discussions about the system not having a designated back button, we agreed that for simplicity’s sake, the browser’s back button would be sufficient. After our initial cognitive walk-through, it became apparent that there was a deeper design issue to consider. Having (or not having) a back button was a small issue – having to use the back button as a means to navigate was a huge design issue. Our user should not have to click the back button continually to find their way back to a previous page. Our redesign needed a clearly marked exit or link to take them back to another page in one click.

Several other moderate issues that needed redesigning were some explicit directives and/or prompts for navigation and use of website features, aside from the main navigation bar. These included better directions for using the search filters, a prompt for clicking the rating stars to access the reviews, an additional short-cut back button that prompted and took the user back to their search results, and an obvious link to take Barbara to the external site for downloading. Barbara will probably also want a drop-down menu tool where she can sort her filtered search (for example, sort ratings or price from highest to lowest). These were probably considered more minor to moderate issues with experienced web users such as Barbara, however, a few minor fixes and clarifications would make the site more efficient and faster for Barbara to reach her task goal.

Our redesign priorities were addressed according to the severity of the problem with the consideration of our design team’s available time, knowledge, and resources. We focused our attention on designing better navigation paths and clear navigation buttons to get to those paths. To address this need, we added a uniform and consistent navigation bar at the top of every page and implemented better use of color coding and highlighting to indicate links within the system.

Pg 26 of 36


System Redesign after Cognitive Walkthrough

Before Cognitive Walkthrough Redesign After Cognitive Walkthrough

Heuristic Analysis Generally, we found issues with consistency, search functions and navigating back once an App has been selected. The following summarizes the issues within the heuristic categories and methods or plans to increase usability.

DialogWe found issues with navigation, saving searches and confusion about links. Although all screens use simple and natural dialog common to the user and the simplicity would appeal to users who are looking for Apps specific to education and generally know their search criteria, users may be looking for more information, such as to whether they can

Pg 27 of 36


save searches, how they navigate back and forth between screens, and detail about certain links.

We determined to add a Recently Viewed function, a Return to Home Page button and Back and Forward buttons. We will identify dialog boxes that can clarify links. A second analysis found that we need to add more information to the user, such as “Refine your Search,” and feedback if the search did not return results, etc.

LanguageThe language is comparable to similar websites and uses both educators’ language and Internet terminology. There is an issue with the icons and we realize we need to be cautious about using icons that may not have a universal meaning. We will find other representations for these voting choices.

As a result, we changed the icons to become more intuitive and provided language familiar to the users, such as, “Checking for valid email address” so the user knows the system is working.

Minimize Memory LoadInformation on pages is available to user and remains visible until user navigates away from page and selects another App to examine. However, a user may often review an App and then want to look at others before deciding to download. We will need to add a “Recently Viewed” to the search page so users can refer to apps they may have viewed previously within their session. We also may want to add a “Return to Search Results,” so they can easily refer back to the original search rather than reversing through numerous screen shots.

After making changes, our analysis revealed we still needed to increase the navigation options for users. They were confused (did not remember) how to perform the basic search in the tasks, so we added some additional search boxes to make search more user friendly. We also considered whether to add a tutorial so that search success is not dependent on a user’s sophistication with the search function.

ConsistencyNavigation bars are consistent on each page. The App “snapshot-dashboard description” is uniform and consistent at the top of each page except the Search and Home pages which need specific functionality. The button to write a review is in the same location on every page. Will need to decide whether/how to standardize the snapshot-dashboard description Search screen with other screens and we need to ensure consistent right-hand placement of navigation bar.

Pg 28 of 36


We made changes and then analyzed whether these improved the navigation and search. We simplified the right-hand navigation bar by adding clear search categories that corresponded with our user’s desired search criteria.

FeedbackFeedback is provided when user clicks on the app name at the top by taking them to the app description page, and when the user clicks on ratings stars by taking them to the ratings detail page. Also, the voter’s “Like” icons provide feedback to the user to let them know their vote has been successfully submitted. We will need to provide dialog boxes or short tutorials for users not familiar with uploading. Feedback is limited on the reviewer profile pages and profiler award pages to protect the users privacy and integrity. The users may vote on whether they like a review written by the showcased reviewer.

After our analysis, we changed the icons so they will be grayed out once the user has voted to avoid “stacking the ballot.” After the user votes, they will receive feedback to let them know their vote has been successfully submitted.

Clearly Marked ExitsExits are not explicit. User will need to rely on web familiarity and intuition to know to use the escape and back buttons. This may be a potential problem, escape and back buttons may not be intuitive for all users. We will add a Return to Home page and a Next and Previous button.

After the user testing analysis, we realized we cannot rely on web familiarity and intuition; teachers who may want to use this App may have a wide variety of experience with Web navigation so we added an additional search box. Users also still seemed confused as to how to move around the website so we added more navigation and error messages.

ShortcutsThe user will be able to navigate with one click to the various features and descriptions for the current app that she is examining. The user will not however be able to explore a different app from the original search without going back to the original search results. This is functionally comparable to a Google search and although we thought this would remain as such in the final design, we need to decide if there is a value-add if the user can search on different apps from their current app. The user can look at the reviewer’s Award page by clicking on their name from one of their reviews. This is not exactly a short cut to get to the reviewer’s information. We will examine this in our design and analyze if this can be structured to be a little more streamlined.

Pg 29 of 36


Further analysis revealed a need for more shortcuts and functionality in moving back and forth between searches and search features. Final design will need to assess search functions based on our analysis and user testing.

Good Error MessagesWe determined that we needed to add error messages in our updated design. For example, if the search does not bring up any results, we will need to incorporate some feedback messages such as “Try broadening your search to include more options.” If the search brings up too many results (over 20 results), then we will need a feedback message that says, “Try using the search filters to narrow your results.”

Our edited design resulted in creating additional error messages in the login and register pages, such as the login / register page.

Prevent ErrorsWe can avoid possible errors by making the rating stars and other clickable locations either highlighted, colored, or somehow more obvious that it is a direct link to something. Errors can be avoided by adding pop-ups to explain links, for example, when clicking on the app name, indicating that will take them to the description.

Subsequently, we changed the login and register button so they change color with correct information.

Pg 30 of 36


User TestingDesignOur testing design included two tasks representative of activities that Barbara would want to perform as a user: Search for Apps and Rate Apps. We used Adobe Captivate to capture the users “think aloud” and related mouse clicks.

Experimental ProtocolWe provided users with written protocol in an effort to create consistency in testing. The tester read the information to each candidate. After assuring the user that the study was voluntary, and that they had full control over whether to continue, information was provided about confidentiality. We then explained the Think Aloud method and guided them through a practice task. We described the website and our overall goal:

“Apps4edu.org is website established to expand the use of mobile applications (aka Apps) in education. The goals of the site are to create a location where educators can find and rate apps based on their effectiveness as an education tool and to create new educational Apps tailored to fit the needs of educators. The website functions similarly to other sites such as Amazon.com and Yelp.

Pg 31 of 36


The user was then provided with the following tasks:

Tasksa. Use Apps4edu to search for an App for use in the classroom that will assist students in learning the Periodic Table of the Elements. Using your browser of choice, go to Apps4edu.org. Search for an interactive App that teaches and showcases the Periodic Table of the Elements. Include only Apps appropriate for grades 9-12 and designed for use in all high school level science courses (including biology, chemistry, physics, environmental and general science). The App should be free of charge and contain no advertising, in accordance with School District Policy. Using the site, determine if the App will benefit your students. Save each App that meets the criteria for further investigation. b. Use Apps4edu.org to evaluate the Periodic Table of the Elements App so that other educators can learn from your experience. Login to Apps4edu.org, then search for, and select the desired App to rate and review. Update and/or provide information for one or more categorical references about the App that includes the objective categories: grade level(s), subject(s), advertising, and search text keywords. Use the site’s rating system and protocol to rate the App. In addition, write an optional review that will provide constructive feedback for other potential users of the App. Results should display, at a minimum, App name, rating, grade level, and subject.

ParticipantsOur user testing participants were Instructional Design and Technology and Nursing Informatics students, and had been previously instructed in the process of Think Aloud protocol. We attempted five user tests, two of which produced partial results due to computer problems with the classroom computer connectivity. Participants displayed a wide variety of computer expertise, which affected their ability to complete the task. For example, Participant #1 had significant difficulty with the test tasks, and indicated she did not “use these kinds of websites as much as others might.” Two participants demonstrated skills similar to Barbara and were able to intuitively apply the search activities. Participant #5 seemed to move around the site most easily and quickly solved issues. He completed Task A easily, but stumbled on Task B when he was confused by having to log in to complete a rating.

There was, however, similar feedback from each participant. The Think Aloud revealed most definitively the challenges with navigation, language and feedback. For example, Participant #2 could not find the cost even though it was clearly stated; she was looking for the word “free.” Participant #4 had difficulty with the search function, and in fact,

Pg 32 of 36


could not complete the task without guidance. Once he understood the search parameters, however, he did complete the task.

Although the test users were not exactly representative of our persona, Barbara, this testing provided critical feedback in moving toward our most optimal design and resulted in numerous design changes.

Summary of Results/Overview of FindingsHow did HCI methods inform design decisions? When three consecutive people made the same comment, we determined it was a good indicator for making a change.

We made numerous "navigation" design changes based on feedback. Since the majority of the site does not require a login, we wanted it to be easy to log in from anywhere.

We created two distinct navigation areas; three areas if you include sub-menus and combined all of the navigation into a single location AND added a "breadcrumb" to assist members in finding their location.

Many thought the term "users" was derogatory; so we chose to replace the term with “members” instead. Overall, we attempted to model our site after familiar sites identified by target users during the interviews.

PART 3

Recommendations/Revisions from User TestingThroughout the process we made design changes. There were instances when we grappled over whether certain edits would increase usability. The user testing provided the final word on edits and even framed our future edits, including those that were “show stoppers” and those that were minor improvements.

A further breakdown of these recommendations and revisions can be found in our in class power point presentation.

Pg 33 of 36


High Priority Design IssueThe initial design and the final design are drastically different. The major shift in design occurred between the first set and second set of interviews. The first set of interviews included educators with a superficial interest in using mobile devices and their apps in education. This led to a very wide range of opinions and little consensus. The second group of interviewees was educators using mobile technology in the classroom daily and had experienced the frustrations of finding and keeping track of apps. This led to a major rewrite of the Persona which led to a major redesign of the site.

These high priority issues included structural and database changes with consequences rippling through the entire application. For instance, the second group wanted specific filtering of search results, not simply a change in the search phrase. This filtering required attributes to be added to the apps, such as, subject, grade level, and rating.

Moderate Priority Design IssueFrom this point, there were many substantive changes, but not many structural, fundamental, or encompassing changes. All issues identified during testing and compatible with the persona have been addressed.

Moderate issues were almost exclusively HCI issues. Changing “0.00” to “Free”, changing “Cost” to “Price”, changing the order of rating, subject, grade level, advertising to be consistent.

Final DesignThe final design is a website to allow members a place to find, rate, and recommend apps for education, hence the name APPS4EDU.org.

What is the Final Design and How Will it Meet the Needs of Members?Our implementation of a “Final Design” includes the concept of continuous improvement. As such, the site will never be finished, literally. If a development metaphor, the site will follow a Linux style development cycle instead of a Microsoft style development cycle. In a Linux style process, pages on the site will be updated daily. Identified bugs and member suggestions will be addressed as time and resources are available. Each individual page will be modified, tested, and released to the public independently. In a Microsoft development model, the site would be designed, updated, tested, reworked, then deployed as a whole. In our model, the Search page may update 100+ times by the end of summer and the Forgot Password page and corresponding

Pg 34 of 36


email may only update twice. The effort put into the site will correspond directly to the needs of members (aka users). Our design philosophy focuses development efforts based on member needs. If we have 30 requests relating to the search page and 20 requests for the remainder of the site, the search page will receive most of the development resource allocation.

Meeting the Needs of MembersWe believe the site, as it is designed today, meets the needs of the members. However, we expect the member needs and wants to shift over time as the mobile technology becomes more integrated in the classroom. As such, our Final Design includes the concept the site will continue to grow and evolve. Ultimately, we believe the site addresses a quantifiable existing need. How the Internet community of educators will accept our project has yet to be determined.

Future Issues

Finish the SiteThe number one issue is to finish the site and allow educational community to become members of APPS4EDU.org. The design and implementation philosophy is continuous improvement; we will seek daily incremental progress and deliver members a site that is constantly evaluated and improved in the light of their efficiency, effectiveness and flexibility. The site is intended to be in Alpha until the end of May, in Beta from June thru August, and available for wide-scale use to begin in Sept.

End of Alpha ReleaseIn our vernacular, the end of the Alpha phase will conclude with all of the desired functionality implemented in the site and no known substantive errors on any page. As such, the site is a collection of pages. Each page implements a single function or is part of a functional sequence. To move from Alpha to Beta, all of the pages for the final site need to be implemented and there will be no known errors on any individual page. Page functionality will range from very easy, such as static text concerning the privacy policy, to the complex, such as the of merging a search engine with a custom database containing tens of thousands of records to create a search results page.

End of Beta ReleaseBeta will conclude with the site being used by a few hundred users who have used the site to rate, recommend, and categorize tens of thousands of Apps. During this process, many internal components will need to be optimized to withstand the demands of multi-

Pg 35 of 36


user loads. All suggestions, including HCI specific requests, will be addressed as part of the continuous improvement process.

Hosting and Plans for GrowthLaVar is currently hosting the site in a data center in Salt Lake on excess hardware and bandwidth available through his consulting business. If the site was to become very popular, we would need to identify an entity willing to support the APPS4EDU.org site, whose stated policy is no advertising and no marketing of member information. An alternative is to identify and apply for grants and/or working out a deal where LaVar can pursue a Doctorate and the institution will host, or possibly feature, the site.

Seeking VolunteersAbout 50 volunteers have already consented to help with categorizing apps and creating reviews. We will turn to social networking sites and educational groups to attract more early-adopters as the site has been more thoroughly vetted.

Conference PresentationsThe APPS4EDU.org presentation was submitted to the Oct 2011 Southern Utah Educators Conference (SUECON, http :// suecon . org /) , the Nov 2011 Colorado Technology Education Association (CTEA, http :// www . cteaonline . org /) conference, and will be submitted to the Utah Coalition for Educational Technology (UCET, http :// www . ucet . org /) conference in 2012. We will attempt to identify other opportunities to showcase this project and the accomplishments of a few University of Utah Instructional Design and Educational Technology (IDET, http :// idet . ed . utah . edu / ) students.

Pg 36 of 36

table of contents - idetportfolio.com€¦ · web viewtable of contents. part 12. team members2....

Documents