· web viewdesired outcomes. ensure that navigation is natural. check students can find a course...

Path UAT TestingName:

Student number: s1107694

Desired outcomes Ensure that navigation is natural. Check students can find a course and get some information about it. Check students can find a given degree programme and understand it. Check students can access the builder for their degree or for a generic degree. Ensure the builder is a natural thing to use and makes sense. Check students know if they’re satisfying their requirements.

IntroductionThank you for taking part in this User Testing session.

Today you will be testing some of the existing functionality - and some new features - of the Path website, which is used for course and degree programme information in the schools of Mathematics, Engineering and Physics & Astronomy.

We will give you four scenarios to complete and ask for feedback along the way. If you have anything to add that isn’t asked explicitly, please note it in the margins, at the end of this document, or anywhere else where we can find it.

ScenariosFinding a courseYou are looking for information on the mathematics course “Topology”. Try to find that information.

1. How did you get to the course information page? Did you have any trouble?Courses – School of Mathematics – Geometry & Topology – Topology full year course

No trouble.

2. Can you think of any other ways that you might get to that page?DPT – School of Mathematics – Year 4 – Topology

3. Can you take this course? How do you know that?Yes, in my final year. This is stated in the description.

Does it have any prerequisites? – Not readily available

4. What sorts of student feedback does this course have?9 reviews by students.

77% of students from last year recommend it.

Feel free to play around and look at other courses as you wish.5. Do you have any other comments about course pages on Path?The link for prerequisites should be more obvious as this is an important piece of information when choosing courses.

Topology does have prerequisites. See question 5 below.

In response to this, we changed the panel stating “This course has requirements” to have a more striking icon and draw attention to it.

Choosing coursesNavigate to the course listing for ‘Mathematics’, and try to store some of the courses you’re considering in ‘My Courses’.

1. How did you store the courses in ‘My Courses’?By clicking and dragging to the icon as stated at the top of the page.

2. Can you see any other ways to add courses to ‘My Courses’?Yes. Clicking on a particular course and then clicking “add to My Courses”.

Try adding some courses from another subject area.

3. How did you get to those courses?From the courses page; by school. Or by “recommended outside courses”.

4. Do any of the courses timetables you’ve added clash? How do you know this?I can construct my timetable but the only way to determine a clash is to physically look through it – tedious.

Feel free to play around with the course lists.5. Do you have any other comments about course lists on Path?

A few users have noted that manually determining clashes is a pain. However, this is not currently something we can fix as it is handled by T@ED.

Reviewing a courseWe are now going to ask you to write a review for a course. Navigate to the a course you have taken and write a review for it. (This is all on a test version of Path, so this review will never be made public.)

1. Was it clear what a review consisted of and the sort of thing you were expected to write?Yes. Clearly stated under “review guidance”.

2. Was it straightforward to write a review? What factors influence this?Yes. The layout is well structured making it easy to review courses.

3. Are you happy to write reviews on Path? Why?Yes. It is quick and easy. It would be useful to know if my lecturers have seen it though.

This is not something we had previously considered. However, reviews being marked as “seen” might give the wrong impression as it doesn’t necessarily mean they’ve taken it on board. Hopefully the future feature of staff feedback should allay these concerns, as staff will be able to prove that they have taken reviews on-board.

Finding a degree programmeFrom now on we’re going to be using your actual degree. Please enter the title of your degree and your current year in the boxes below:

(If your current situation proves unexciting – for example if you have no optional courses – feel free to pretend to be in a different year or on a different degree within the three schools. But make sure you tell us in these boxes.)

Degree programme YearMathematics BSc Hons Year 3

Now find the information for your degree programme on Path, and answer the questions below. Look across all four years of your degree where possible.

1. How did you get to the degree programme information page? Did you have any trouble?Degrees – BSc Hons – Year 3

2. Can you think of any other ways that you might get to that page?No.

3. How many compulsory courses does your degree have in third year?4

4. Where there are optional courses, do you understand what the requirements are on what you can take?Yes.

The hope here was that students would click the “My Degree” button, however this student was using a test account without their enrolment data so this option was not available.

5. If there is a course picker on your degree programme (with a drop down box of schools), is it clear how this works? (If there isn’t, just write “N/A”)Yes

Feel free to look around other degree programmes as you wish.6. Are there any further comments you would like to make about the degrees on Path?It seems very user-friendly.

Using the programme builderNow open the programme builder for your degree in your current academic year.

Path’s programme builder allows you to choose your subjects for the current year, as well as looking backwards and forwards at other years. For this scenario, we’ll pretend it’s the start of the term and you’re deciding which courses to take this year.

1. Are the courses you took in previous years marked as such? (This won’t be the case if you are pretending to be in a different situation)Yes.

2. In your current year, is it clear which courses are compulsory?Yes.

3. In your current year, is it clear what optional courses are available and the requirements on them?Yes.

4. In your current year, are there any other requirements you can see? If so, what are they?Yes, that I did not take the course in a previous year. The number of courses per block required is clearly stated.

5. Can you add and remove courses from your selection and, when doing so, is it clear how this affects your fulfilment of the requirements?Yes. The number of credits I am taking is clearly stated at the top.

6. If you look at future years, can you identify courses you might want to take and determine whether or not they are available?Yes. The colours make this easy.

Feel free to play with the programme builder for a while, adding and removing courses, and changing degree/year if you wish.

7. Are there any further comments you would like to make about the programme builder?

Final thoughts1. Do you think Path is a useful service?Yes.

2. Did you find Path easy to navigate?Absolutely.

3. Did you like the design of Path?Yes.

4. Do you think Path would be useful if expanded to include other schools?Yes. Definitely.

If there is still time, feel free to browse around Path some more and look at other courses and degrees.

5. Please leave any closing comments in the space below.


1. How did you get to the course information page? Did you have any trouble?Home -> Courses -> Maths -> Topology

2. Can you think of any other ways that you might get to that page?Degrees -> Maths -> BSc -> Year 4

Or build your programme -> etc.

3. Can you take this course? How do you know that?Yes. I did.

Also more helpfully it is listed in the MMath programme page.

4. What sorts of student feedback does this course have?77% recommendation

Feel free to play around and look at other courses as you wish.5. Do you have any other comments about course pages on Path?Nice and blue and friendly looking.


1. How did you store the courses in ‘My Courses’?‘Add to my courses’ button inside course info.

2. Can you see any other ways to add courses to ‘My Courses’?Drag course icon into ‘my courses’ on mathematics page (I like this it’s very satisfying!)


3. How did you get to those courses?Navigate back to courses, then click on another course.

4. Do any of the courses timetables you’ve added clash? How do you know this?Yes, opened up the timetable (link to Ted) and looked for problem – Didn’t see any sort of alert?!

Feel free to play around with the course lists.5. Do you have any other comments about course lists on Path?

As said in response-1, this is not something we have any control over right now.


1. Was it clear what a review consisted of and the sort of thing you were expected to write?Yes

2. Was it straightforward to write a review? What factors influence this?Until I clicked done, then it tried to give me the impression if had not worked at all.

3. Are you happy to write reviews on Path? Why?Yes in principle. But I am inherently lazy and rarely do such things in practice without motivation.

This was due to an error on the test database which has since been fixed.

We have considered adding some various motivators to Path, but really this is something schools will need to encourage.



Degree programme YearMMath 5


1. How did you get to the degree programme information page? Did you have any trouble?Home – degrees – maths - MMath

2. Can you think of any other ways that you might get to that page?

3. How many compulsory courses does your degree have in third year?4

4. Where there are optional courses, do you understand what the requirements are on what you can take?Underneath the compulsory ones. Yes

5. If there is a course picker on your degree programme (with a drop down box of schools), is it clear how this works? (If there isn’t, just write “N/A”)Yes

Feel free to look around other degree programmes as you wish.6. Are there any further comments you would like to make about the degrees on Path?



1. Are the courses you took in previous years marked as such? (This won’t be the case if you are pretending to be in a different situation)Yes

2. In your current year, is it clear which courses are compulsory?Yes

3. In your current year, is it clear what optional courses are available and the requirements on them?Yup

4. In your current year, are there any other requirements you can see? If so, what are they?No

5. Can you add and remove courses from your selection and, when doing so, is it clear how this affects your fulfilment of the requirements?Yes, the progress bats on the right are sensual!

6. If you look at future years, can you identify courses you might want to take and determine whether or not they are available?I have no future…


7. Are there any further comments you would like to make about the programme builder?Love the colour coordination and progress bars. Way more awesome than path 1.

We were hoping users would note the “programme hint” at the top of the page, which provides additional instructions from the school. However, this may have been missed due to the wording of the question, which the user seemed unclear about.

To clarify, user 2 is in the final year of their degree.

Final thoughts1. Do you think Path is a useful service?Yes

2. Did you find Path easy to navigate?Yes

3. Did you like the design of Path?Defo!

4. Do you think Path would be useful if expanded to include other schools?The old system was useless. So certainly.


5. Please leave any closing comments in the space below.

By “the old system”, the user meant the previous combination of DRPS/timetab/T@Ed. Notably, this “system” wasn’t an integrated process in the same way that Path operates.

User 2 had to leave the session to get a lecture, and so the second half of their testing was slightly rushed, hence the one-word answers in the last couple of sections.


1. How did you get to the course information page? Did you have any trouble?Browse courses – Mathematics – Year 4

No trouble

2. Can you think of any other ways that you might get to that page?Search topology

3. Can you take this course? How do you know that?I can’t as it is in the year 4 section and I am in year 3.

4. What sorts of student feedback does this course have?77% recommend it out of 22/60 folk enrolled

Feel free to play around and look at other courses as you wish.5. Do you have any other comments about course pages on Path?Well organised, easy to navigate, brilliant software. Very useful for choosing courses.

Due to information present:

Recommendations Assessment breakdowns Reviews Pre-requisites

This isn’t necessarily reliable, particularly due to the flexible degrees in the University. We perhaps need to draw more attention to degree programmes, which are more reliable in this regard.


1. How did you store the courses in ‘My Courses’?Add courses

2. Can you see any other ways to add courses to ‘My Courses’?Drag and drop! Fantastic! Possibly something to let people know how to drag and drop.


3. How did you get to those courses?Went back to courses and chose Physics and so dragged and dropped.

4. Do any of the courses timetables you’ve added clash? How do you know this?, Quantum Mechanics and Jewels of Algebra. From opening up the timetabling software and seeing for myself.

Feel free to play around with the course lists.5. Do you have any other comments about course lists on Path?Not at all, pretty positive. Very colourful, neat and easy to use.

On the “My Courses” page and in the help system it is noted that you can drag-and-drop courses, there is also a banner on the top of every page for the first 3 days of visiting stating this, which we consider enough guidance.


1. Was it clear what a review consisted of and the sort of thing you were expected to write?Yes, clear. Easy to navigate through Path.

2. Was it straightforward to write a review? What factors influence this?Teaching methods, tutorials, assignment difficulty, exam difficulty, how difficult course was.

3. Are you happy to write reviews on Path? Why?Easy to do. Easy to navigate the GUI.

Whilst interesting, we were more asking what factors made it straightforward or not to write a review.



Degree programme YearMathematics (BSc Hons) 4


1. How did you get to the degree programme information page? Did you have any trouble?Tab on the top, no trouble

2. Can you think of any other ways that you might get to that page?Search bar

3. How many compulsory courses does your degree have in third year?4.

4. Where there are optional courses, do you understand what the requirements are on what you can take?Clever system! Optional courses under compulsory ones. Requirements are on each page under requirements.

5. If there is a course picker on your degree programme (with a drop down box of schools), is it clear how this works? (If there isn’t, just write “N/A”)Yes this is entirely clear, chose Physics and Astronomy as I have a keen interest and it all fits well together. i.e. easy to use!

Feel free to look around other degree programmes as you wish.6. Are there any further comments you would like to make about the degrees on Path?Amazing system. I would really struggle choosing course and timetabling otherwise. This in comparison to timetab is monumentally better.



1. Are the courses you took in previous years marked as such? (This won’t be the case if you are pretending to be in a different situation)Yes, they are marked as selected green.

2. In your current year, is it clear which courses are compulsory?Yes, there are none!

3. In your current year, is it clear what optional courses are available and the requirements on them?Yes, they are under optional… so simple yet effective.

4. In your current year, are there any other requirements you can see? If so, what are they?Satisfying the credit requirements on the blocks.

5. Can you add and remove courses from your selection and, when doing so, is it clear how this affects your fulfilment of the requirements?Yes, take courses away and the bars don’t fill up. Need bar to fill up before I have satisfied requirements.

6. If you look at future years, can you identify courses you might want to take and determine whether or not they are available?Yes, the blocks on the right are very useful. Helps heaps for planning courses next year.


7. Are there any further comments you would like to make about the programme builder?Great software; clear, concise and straight to the point. I would especially like to use this when planning for my masters next year. Colours are useful, I would recommend Path to my peers.

Final thoughts1. Do you think Path is a useful service?Incredibly. Well organised. Revolutionary software in comparison to the useless software before: timetab, DRPS.

2. Did you find Path easy to navigate?Extremely.

3. Did you like the design of Path?Colourful, nice user interface, user friendly

4. Do you think Path would be useful if expanded to include other schools?Useful does not even cut it for Path. It MUST be expanded to other schools. My (economics) flatmate said to me how jealous he was of the timetabling.


5. Please leave any closing comments in the space below.I have said enough, truly remarkable system. Definitely pursue getting this to other schools as they would benefit tremendously.


Student number:





Larger text in body font and in bold under the icons would ease the manipulation/reading. Same goes for the list of degrees.

User 4 added some comments to the front of the UAT document before realising that the interior pages broke the feedback down.

Path uses standard font sizes, but adheres to user style sheets, so visitors can increase/decrease the font size as they please. However, we did increase the font size in the navigation bar as this was noticeably smaller than it should’ve been.


1. How did you get to the course information page? Did you have any trouble?The web address for programme guide should be a hyperlink – it would make the navigation easier. I got to the page by using the search function on the top right corner of the screen.

2. Can you think of any other ways that you might get to that page?Yes. Trying all the Maths degrees/years of study until I find it would be another way; however this would take much more time.

3. Can you take this course? How do you know that?I take it by clicking “Add to my courses”. I was told that I should first click “Requirements”, however I found this rather unintuitive. There should be a pop-up menu which pops up to tell the students the requirements if the student has not fulfilled the requirements.

4. What sorts of student feedback does this course have?Reviews, recommendations. In my opinion, “recommendations” should be a tool which appears on the same page as “reviews”. It would be more intuitive and more easy to find.

Feel free to play around and look at other courses as you wish.5. Do you have any other comments about course pages on Path?Generally, there should be brighter colours (of course, this is a matter of taste…) and everything that is written in block (except large bodies of text such as reviews) should be written in bold and a larger font. Overall, I think it is pretty good.

We believe by this, the user means the links to degrees on the search results page (there was some misunderstanding of what they should be doing).

This confusion was caused by the user not realising that there were course listings as well as degrees. Considering the layout of navigation and the home page, I’m not sure how to make this clearer.

The “My Courses” section is for storing courses for navigation, and doesn’t dictate which courses the student is enrolled on. We considered changing the text on My Courses to clarify this, but feel it is already clear and just went unread by this user.

We didn’t want to over-complicate the reviews panel, but agreed that recommendations should feature. As a result, a smaller pie chart of recommendations has been added under the header for each year of reviews to give a brief indication of approval that year.


1. How did you store the courses in ‘My Courses’?By clicking on the course name and then clicking on “Add to my courses”.

2. Can you see any other ways to add courses to ‘My Courses’?No.


3. How did you get to those courses?By clicking “courses” in the top right corner of the screen, and then I went to the courses I wanted.

4. Do any of the courses timetables you’ve added clash? How do you know this?None of the courses clashed. I therefore chose courses which do (on purpose) but there was no alert/message to notify me. I think it would be useful if such an alert appeared, with details about on which day of the week the clash is, at what hour etc.

Feel free to play around with the course lists.5. Do you have any other comments about course lists on Path?I think it would be better to have a more “dynamic” way of visualising the (future/proposed) timetable, in a table which shows days of the week and courses, other than just a list. This dynamic timetable should be available as soon as a course is chosen (even tentatively) by clicking “Add to my courses”.

Due to the otherwise overwhelming show of support for the colour scheme on Path, and the opinion that bold text would draw the eye to much, we decided not to follow this advice at this point, but have taken it on board for the future.

As mentioned in other responses, this is a limitation of T@Ed.

As above, this is with T@Ed, not Path.


1. Was it clear what a review consisted of and the sort of thing you were expected to write?No, it was not clear. Some guidelines (or, even better, detailed guidelines) about how to write, and about what is expected from a review, would be very useful.

2. Was it straightforward to write a review? What factors influence this?I did not find it was straightforward. Giving a brighter colour (perhaps red) to the “write a review” button, as well as making it bigger, would be very helpful in my opinion.

3. Are you happy to write reviews on Path? Why?As with most reviews, the more someone writes in a review, the easier it is to guess who wrote it, even though it is anonymous. So students may feel uncomfortable about writing reviews. As for myself, I am happy to write reviews on Path.

There are guidelines on the page used to write reviews.

In response to this, we coloured the “Give feedback” button orange on each course page of courses that the logged-in student hadn’t yet reviewed.



Degree programme YearMaster of Physics with Honours Physics 3


1. How did you get to the degree programme information page? Did you have any trouble?By clicking on Degrees -> My Degree.

Yes, the name “degree programme information page” was rather confusing to me. A simpler name would be better e.g. “My degree – Information Page”

2. Can you think of any other ways that you might get to that page?Yes, by using the list of degrees under the “Degrees” tab.

3. How many compulsory courses does your degree have in third year?6 compulsory courses

4. Where there are optional courses, do you understand what the requirements are on what you can take?Yes, I do -> by clicking the requirements tab

On the site, the words “degree programme information page” aren’t used.

5. If there is a course picker on your degree programme (with a drop down box of schools), is it clear how this works? (If there isn’t, just write “N/A”)Yes, it is clear how it works.

Feel free to look around other degree programmes as you wish.6. Are there any further comments you would like to make about the degrees on Path?Like compulsory courses appear in blue, maybe it would be good if optional courses appeared in a bright “warm” green. In this way they would be easier to spot.



1. Are the courses you took in previous years marked as such? (This won’t be the case if you are pretending to be in a different situation)Yes, they are.

2. In your current year, is it clear which courses are compulsory?Yes, it is.

3. In your current year, is it clear what optional courses are available and the requirements on them?Yes, it is clear.

Optional courses do have an orange background (as opposed to the blue of compulsories). This may be subtle, but page position and layout should also give indication of the section the user is in.

4. In your current year, are there any other requirements you can see? If so, what are they?I couldn’t find any other requirements.

5. Can you add and remove courses from your selection and, when doing so, is it clear how this affects your fulfilment of the requirements?Yes I can.

Yes it is clear, although I think

6. If you look at future years, can you identify courses you might want to take and determine whether or not they are available?No, because there was not a list of instructions to tell me that courses in blue are available. I later found out by asking that there was a list of instructions but in my opinion such a list should be on every page.


7. Are there any further comments you would like to make about the programme builder?Yes, the courses for each of the years of study take too much time to load. This may become a bigger problem when more students start using Path.

The instructions appear the first time you visit the Programme Builder and must be closed by the user. To combat premature closing, we added a link to the instructions in the menu of the Builder. There are also now videos demonstrating how to use the Builder.

This is something we are aware of, and we intentionally moved loading to be asynchronous so the user isn’t waiting on a blank screen. We have made efforts to speed up page loading, and will continue to do so.

Final thoughts1. Do you think Path is a useful service?Yes, definitely.

2. Did you find Path easy to navigate?My answer to this question would be a qualified “yes”. In some cases it was not so easy to navigate.

3. Did you like the design of Path?Yes I did, however I think more lively colours would be a way to improve it.

4. Do you think Path would be useful if expanded to include other schools?Yes, I definitely think so.


5. Please leave any closing comments in the space below.I think Path is a good initiative, especially because it helps students understand how their current choice of courses can affect their possible choices in future years.

User 4 was very keen to provide as much feedback as possible, and raised some points we hadn’t thought about. Whilst we disagreed with some design decisions, the user brought some things to our attention that have now been implemented.

With regard to using more lively colours, an optometrist said that the colours currently used were approaching unsuitable for those with “colour deficits”. As such, we are unlikely to make the colours used any more luminous.

· web viewdesired outcomes. ensure that navigation is natural. check students can find a course...

Documents