bis usability report
DESCRIPTION
TRANSCRIPT
![Page 1: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/1.jpg)
BIS website
![Page 2: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/2.jpg)
2 / 50
Executive Summary
Amberlight are a specialist user centred design consultancy, that have been commissioned by the BIS to conduct research to inform the following goal:
Evaluate proposed new design for BIS website and inform optimal information architecture
Approach:
6 two-hour workshops
7 one-to-one sessions to test interactive prototype
Top-line findingsMethodology
User sample:
27 users in workshops (6 target groups)
7 users in one to one sessions
Ways to access the content: E-mail alerts primary, then Search (Google and internal search engine), bookmarks, then navigation within the site.
Consider to add personalised links to the homepage (in "I need to"): history of pages visited, and some content selected according to the same principle as selecting content for the e-mail alerts. Also, consider to allow users to customise the home page orgainsation like on the bbc site or iGoogle.
Home page prototype look and feel: The new layout is "clear", "corporate", but tabs are less legible than on the current version. Could look more like a government site. Does not look innovative.
Users need to better understand what BIS covers, what is a "partner", and who does what.
Users would like an advanced search.
Top level navigation categories work well.
No consensus on policies classification. Different target audiences have different expectations. Two-level classification may clarify the organisation and strengthen the “Policy” part of the BIS.
It is not obvious for users that regulations can be found in policies.
Users need to identify quickly the types of content (e.g. speeches, regulation, policies) in the search results.
Specific requests by Media users: Media resources (pictures, videos, etc).
Specific request by Specialist managers: classification of the content by sectors and profession.
![Page 3: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/3.jpg)
![Page 4: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/4.jpg)
4 / 50
Research Objectives
1
2
User research took place in London in order to inform the following goals:
4
3
![Page 5: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/5.jpg)
5 / 50
Research approach
One-to-one sessions:
120 minutes each
Current use of the site
Home page content requirements
Top level navigation validation (closed card sorting)
Policies themes organisation (open card sorting)
Overview of Participants
All BIS or BERR or DIUS users on a regular basis.
6 user profiles were involved in the sessions:
• Delivery partners, stakeholders and intermediaries• Media• Policy influencers and lobbyists• BIS staff• Political stakeholders• Specialist managers
60 minutes each
Detailed evaluation of the prototype
Focus on homepage, About BIS, Speeches, Policies and publications
Workshops: Current content architecture
Desk research
![Page 6: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/6.jpg)
![Page 7: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/7.jpg)
7 / 50
The key positives
Overall
Policies
Search
• The top level navigation works well and looks easier than on the legacy sites.• Users like the new look and feel.
• ‘Ministers responsible’ in the Policies content page is well received, however the information should be available at a higher level in the website.
• Search results look clearer than on the legacy sites.
![Page 8: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/8.jpg)
8 / 50
To be improved areas
• Participants are confused about BIS mission and domains: what is BIS doing? Who is doing what?
• Many users do not understand what partners are.• ‘Policies’ does not sound like it includes regulations: users sometimes do not know
where to look for regulations.• Users need more advanced search options on the site.• Content selection for email alerts is not specific enough.
✗✗
✗✗
✗✗
✗✗
✗✗
![Page 9: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/9.jpg)
![Page 10: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/10.jpg)
10 / 50
Workshop method
Structure of the workshop: Introduction
Workshop presentation; Ice-breaker
Current website experience
What do participants do on the websites today? What do they like? Do they have difficulty using the websites? How do they access the websites? What kind of information are they usually looking for? Etc…
BIS prototype: homepage review
What information do they expect to find on the homepage? What do they think of the homepage layout?
Top level navigation review (Closed card sorting)
How do participants spontaneously organize information in the top-level navigation? Does the top-level navigation match user expectations?
Policies review (Open card sorting)
How do participants spontaneously organize policies? What are the relevant policy themes? How do they want to look for policies?
Closing discussion
Wrap-up and conclusion. Overall feedback on the BIS site.
![Page 11: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/11.jpg)
11 / 50
One to one sessions method
Individual interviews – 1 hour each
Sessions structure Opening discussion
Session presentation; Ice-breaker; Interview on current usage of BIS, BERR and DIUS websites
Prototype review (Task-based evaluation)
Participants are asked to perform tasks on a clickable prototype of BIS.
They are asked to think out loud: they explain what they are doing and tell what they are thinking at the time they are doing or thinking it.
Media panels review
What media panels are participants interested in? Do media panels bring an added value to the website.
Closing discussion
Wrap-up and conclusion. Overall feedback on the prototype.
![Page 12: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/12.jpg)
12 / 50
One to one sessions method
Task-based evaluation scenarios: Scenario 1 – Homepage discovery
“Please have a look at this page and tell me what you think.”
Scenario 2 – About BIS
“You would like to have more details about what BIS is doing. How would you do that on this website?”
Scenario 3 – Speeches
“You have heard about Lord Mandelson’s statement on Royal Mail and would like to read it. How do you do this on the website?”
Scenario 4 – Policies
“A friend of yours told you that from October 1st 2009, tips, gratuities, and cover charges can no longer be used to make up National Minimum Wage pay. You would like to find more information on that topic on this website. How would you do that?”
Scenario 5 – Publications
“You are a company owner and are interested in how to build partnerships for research and innovation between industry and universities. You have been told that you can find a guide on the topic on the website. Please show me what you would do.”
Scenario 6 – Search results
“Say you are looking for information on ‘digital Britain’: an initiative from the UK government to develop the digital and communication sectors. You have used the search tool of the website and here is the result page that you get. What do you think about this page?”
![Page 13: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/13.jpg)
![Page 14: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/14.jpg)
14 / 50
Priority scale used in this report
Symbols are displayed on pages of this report to indicate the level of priority of observation/recommendations.
The 3 priority levels are marked as follows
1.High priority: Issue that affects the overall usability of the website and the way users perceive BIS. These issues have a long term negative effect on the website.
2.Medium priority: Issues that block or slow down the user regarding specific actions, or make it difficult to find some information. They do not have a big impact on the overall usability but are annoying for users.
3.Low priority: Cosmetic issues / nice to have. These issues do not have a major impact on the usability. Taking care of these issues will fine tunes the interface to the user’s needs.
M
L
H
![Page 15: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/15.jpg)
![Page 16: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/16.jpg)
16 / 50
Content access usage patterns
The 2 reasons why users go to the BIS website:
1. Look for specific information (They know what they are looking for before they start looking for it).
Specialists and lobbyists mainly access the site for this purpose.
2. Check for updates on the site.
Media and political stakeholders mainly access the site for this purpose.
A majority of participants subscribed to e-mail alerts.
It is an easy way to keep up to date on BIS new content.
![Page 17: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/17.jpg)
17 / 50
Content access usage patterns
When they look for specific information, participants access the website by different means (Most frequent to least frequent):
1. E-mail alerts
Participants receive an email alert and click on the links they are interested in, which opens the appropriate page on the website.
2. Search by Google
Participants click a link on Google, which is generally successful.
3. Search tool on the site
On the BIS website, participants use first the search tool if they look for a specific information.
4. Bookmarks
Participants have bookmarked the pages they are most interested in.
5. Navigate on the site
Participants use the menus to look for information.
![Page 18: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/18.jpg)
18 / 50
Content access usage patterns
When they look for updates, participants access the website by different means (Most frequent to least frequent):
1. E-mail alerts
Participants receive an email alert and click on the links they are interested in, which opens the appropriate page on the website.
2. Homepage
Participants check the latest updates on the site’s homepage.
3. News
Participants look for updates in this section.
4. Bookmarks
Participants bookmarked pages with updates on subjects they are interested in for a quicker access.
![Page 19: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/19.jpg)
19 / 50
Confusion around the remit of BIS
Participants who have used BERR (or DIUS) for a long time think that the BERR (or the DIUS) site is actually the BIS site.
They do not realize that BIS is a merger of BERR and DIUS.
When they navigate the site, they do not realize when they link through to another site.
For instance: they are on BERR and click on a link that leads them to BIS interim site.
(Quote from a participant who actually uses only the BERR site)
![Page 20: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/20.jpg)
![Page 21: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/21.jpg)
21 / 50
Home page look and feel
We asked users what adjectives they would use to describe the home page prototype
![Page 22: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/22.jpg)
22 / 50
No need to change the look and feel
Participants like the colours.
There is no need to have a key associated to the three colours blue, green and purple.
![Page 23: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/23.jpg)
23 / 50
Look and feel must be consistent across time
Participants are fed up with constant changes to the sites.
They do not want the site to change every 6 months and would like all government websites to have a consistent look and feel.
Recommendation:
Whenever possible, keep the same structure, section labels, etc. on all government websites.
![Page 24: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/24.jpg)
24 / 50
Top level navigation design
The top-level navigation looks better and more legible on the current website than on the prototype.
Separations between tabs make it more legible.
The contrast is better.
Recommendation:
Keep the current tabs design.
Current web site
Prototype
M
![Page 25: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/25.jpg)
![Page 26: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/26.jpg)
26 / 50
Home Page content – Workshop Method
"What content do you expect in these areas? Fill the blanks according to what you expect.”
Individual work.
Then share results and discussion.
Results confirmed by the closed card sorting exercise.
Individual work Discussion: Sharing results on a poster
![Page 27: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/27.jpg)
27 / 50
Home Page content – 1 to 1 Method
Participants are presented with the homepage and asked to give their feeling and opinion about it.
”Please look at this page and tell me what you think about it?”
“Please explain what you are / are not interested in, and why?”
![Page 28: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/28.jpg)
28 / 50
Expected content
Overall, participants expect 2 types of content on the home page:
1.BIS scope and purpose: What Is BIS doing? Who is doing it? Who to contact according to the subject?
2.Latest news and updates: participants expect quick access to the latest updates from the homepage (News, speeches, regulations…)
![Page 29: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/29.jpg)
29 / 50
Expected content: scope of BIS
The structure and remit of BIS are unclear, even for people who have known BERR, DIUS or BIS for several years.
The notion of Partners (and how they relate to BIS) is unclear as well.
There is need to define what a Partner is.
Users require a list of the sub-departments.
Ideally, create a map with the departments, how they work together, how they are linked to Partners.
For each department : mission, concrete actions, contact person.
Participants expected to find this information in the "About BIS" section
H
![Page 30: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/30.jpg)
30 / 50
Expected content: Daily updated
Updates need to be in a prominent place on the home page. The location of the Daily updated news is fine on the the prototype.
The three tabs should be used to sort the content by Business (first tab), Innovation (second tab) and Skills (third tab).
Typical content expected: New consultation, policies updates, latest speeches by ministers ("speech of the day"), new publication.
News should be sorted by date.
Each main section of the site should also have alist of updates (e.g. New publications in the Publication landing page, Speeches in the Ministers landing page).
News alerts: Should be internal news and external news. Users would be interested in having news from other sources than BIS, if they impact their business or life (e.g. EU regulations, strikes, statistics).
L
![Page 31: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/31.jpg)
31 / 50
Expected content: Monthly updated
Key campaigns, events or "hot" policies.
E.g.: Climate change, Low carbon business, Employment matters, Recession help
Success stories.
E.g.: A company which adapt to the new economic context, SME taking benefit of the employment policies, etc.
News on a theme (e.g. Science)
Short reports
Overall, when reacting to the prototype, participants find the example content matches their expectations quite well.
![Page 32: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/32.jpg)
32 / 50
Expected content: Bottom Partners
Government sites (e.g. Direct gov, Digital Britain)
Legal sites
Top 10 partners sites
European issues.
A calendar with alerts and reminders.
E.g. consultations deadlines, legislation progress (parliament activity)
"How to" ....
E.g. How to "green" your company
Forms
Opportunities (e.g. funding)
Interactive features: online forum, Q&A, FAQ, feedback by users, blogs.
Top 10 of the most viewed content
BIS annual report
![Page 33: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/33.jpg)
33 / 50
Expected content: 'I need to'
All participants find the link ‘Sign up for email alerts’ useful.
Participants ask to be able to personalise at least a part of this section:
History of past searches / pages recently visited."Something like an automatic bookmark"
Selection of content (e.g. content related to a sector), the same way users select content for e-mail alerts.
Users would like to be able to choose and place content like on the BBC site. On the BBC site, users can select the type
of information they would like to see on a part of the homepage.
(http://www.bbc.co.uk/)
M
![Page 34: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/34.jpg)
34 / 50
• Top info box
Participants are indifferent to this info box.
They cannot think of any reason that would justify keeping the box.
Top info box and breaking news
L
![Page 35: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/35.jpg)
35 / 50
Top message box and breaking news
Breaking news
During workshops, participants said they don't read the top line on the current web site, and they would probably not read it either with the new layout…
…But half of the participants in the 1 to 1s noticed it and appreciated it!
The new design makes it clearer that this area is breaking news, and users are indeed interested in this kind of information.
The buttons (previous, next, pause) are also useful but are not visible enough.
Current web site
Prototype
M
![Page 36: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/36.jpg)
36 / 50
Top message box and breaking news
Recommendations:
Get rid of the top message box.
Breaking news:
Keep the words ‘breaking news’: it clarifies what the information is.
Give more room to the breaking news: they can be displayed on 2 or 3 lines.
Draft display suggestion for the breaking news
![Page 37: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/37.jpg)
![Page 38: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/38.jpg)
38 / 50
Workshop method – Closed card sorting
List of 28 contents (cards)
Instruction: "On which section of the site would you click in order to find this content?"
Quick exercise: 5 minutes maximum to allocate the 28 contents.
![Page 39: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/39.jpg)
39 / 50
Workshop Method - Open card sort
The 28 cards were selected as follows:
A couple of categories were picked from the current BIS interim, BERR and DIUS websites.
Category titles had to be as unambiguous and easy to understand as possible.
![Page 40: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/40.jpg)
40 / 50
1 to 1 method – Task-based evaluation
Each task implicitly required participants to use specific sections of the prototype
When participants did not click on the right menu section, the reason why they clicked on another menu section was systematically investigated:
“Why did you click here? What did you expect?”
“Why do you think this content should be here?”
Tasks were designed to assess ‘About BIS’, ‘News’, ‘Policies’ and ‘Publications’
Inputs gathered during the 1 to 1s confirmed the findings of the workshop sessions.
![Page 41: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/41.jpg)
41 / 50
Overall results
The top level navigation works well.
Tabs labels are clear.
There was a wide consensus on the sorting in most of the cases.
It is good to have the Ministers tab close to the News tab: the 2 access points are relevant for announcements and speeches.
BUT
The notion of Partners is often unclear.
‘Policies’ doesn't sound like it includes regulations. Participants wonder where the regulations are.
![Page 42: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/42.jpg)
42 / 50
No significant differences according to audiences
Example: sorting results for specialists
* Sorting results for the other groups are available in appendix
![Page 43: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/43.jpg)
43 / 50
Unambiguous menu sections
Most of the contents were almost always sorted in the same menu section.
It shows that these menu sections are clear and that participants know what kind of information they will find there.
The following menu sections are the easiest to understand:
Ministers
Consultations
Contacts
![Page 44: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/44.jpg)
44 / 50
Some cards were difficult to sort
• 'Shareholder executive' was the most difficult content to classify
About half of the participants did not understand what to expect behind this.
The other half would have put it in the Partner section.
After showing them the corresponding page on the BIS site, they all decide that the Partner section is more appropriate for this content.
Recommendation:
Place ‘Shareholder executive’ in the ‘Partners’ section.
M
![Page 45: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/45.jpg)
45 / 50
Some cards were difficult to sort
"Consumer Law Enforcement" is expected in Policies and in Publications.
It is linked to the fact that participants do not always expect to find regulations in policies.
Apprenticeships was sometimes sorted in “Partners”.
It is linked to the fact that participants do not understand what “Partners” mean.
![Page 46: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/46.jpg)
46 / 50
Duplicates (Content in more than one section)
It is fine to have duplicates.
Participants often place the cards in more than one category.
Examples that work fine:
On the Home Page: News, Opened consultations, Announcements, Press releases, etc.
Ministers in Ministers (1), Contacts (2), or sometimes News (3).
Publication Contacts primarily in Publications... and also contacts.
Speeches in Ministers(1), News (1), Home page (2).
List of ministers and MPs also expected in Contacts.
Governance in About (1) and Ministers (1).
![Page 47: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/47.jpg)
47 / 50
Duplicates
‘Companies Act’ can be in ‘Policies’ and ‘Publications’.
More generally, users need to differentiate the regulations, the policies and the intents (like speeches / consultations).
If a policy document has been published, it is fine to have it in the two sections: ‘Policies’ and ‘Consultations’.
Participants expect A ‘News’ section in each part of the site (e.g. ‘New consultations’ in ‘Consultations’; ‘New policies’ in ‘Policies’).
![Page 48: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/48.jpg)
48 / 50
Recommendation/Discussion on the top navigation
Policies
It needs to be clearer that policies include regulations
For instance:
On the policy landing page: explain it is both policies and regulations
At bottom level: identify policies and regulations (use a specific icon for instance)
Partners
Users need more explanation about what partners are, their role, how they work with BIS…
H
H
![Page 49: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/49.jpg)
![Page 50: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/50.jpg)
50 / 50
Workshop Method - Open card sort
35 cards with content description
Participants working by pairs or 3 persons together
Instruction: "Group the cards the way you think is relevant, then name the groups”
The moderator listens to discussions while participants are making the groups.
Then everyone compares with what the other participants did, and explain why. Participants then try to find a consensus.
![Page 51: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/51.jpg)
51 / 50
Workshop Method - Open card sort
The 35 cards were selected as follows:
3 policies were picked from each policy theme of the interim BIS site
Policy titles had to be as unambiguous and easy to understand as possible.
![Page 52: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/52.jpg)
52 / 50
1 to 1 Method – Task-based evaluation
Homepage review: feedback on the group ‘A-Z of policies’.
Task 4 (Find information on National Minimum Wage Pay) implicitly requires that participants use the ‘Policy’ tab. A-Z of policies on the homepage
3 ways to look for policies on the policy landing page.
![Page 53: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/53.jpg)
53 / 50
It is not easy to define what a policy is
Most of the users understand roughly what a policy is.
A policy is more perceived as a strategy than a regulation.
But not all participants can define precisely what a policy is, even participants who used the BERR or DIUS sites for several years:
Confusion on status of policy: is it an announcement, a consultation, a regulation…
Who does the policy apply to?
![Page 54: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/54.jpg)
54 / 50
Looking for policies
When participants are looking for a policy, their first idea is to look at news (half of the participants)
When the policy they look for is actually a regulation or a law, they tend to search in ‘Publications’
In the 1 to 1s, participants were asked to look for information on pay and Minimum National Wage:
![Page 55: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/55.jpg)
55 / 50
Looking for policies
A-Z of policies:
It is useful only when the user knows the exact name of policy
Generally, participants do not like it at all because they have no idea what the ‘official’ name of the policy is.
They are not confident that they will find what they are looking for this way.
Policies by theme:
On the current prototype, it is the way participants prefer to look for policies.
Recently updated:
Participants like it and find it useful.
This way of presenting policies makes sense as well on the homepage where participants expect to see the latest updates of the website.
L
![Page 56: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/56.jpg)
56 / 50
Card sorting average results
1
2
3
4
5
6
7
8
9
9 “average” groups could be defined from the 6 workshops :
1. Sectors
2. Science
3. Business
4. Skills
5. Regions
6. Trade
7. Regulations
8. Consumer protection
9. Employment
![Page 57: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/57.jpg)
57 / 50
Hard to classify
Some cards were more difficult to classify:
Intellectual Property office
Science
More generally, the innovation topics are spread in several categories.
This is not a surprise, as the Innovation part is new, compared to Business (roughly former BERR) and Skills (roughly former DIUS).
![Page 58: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/58.jpg)
58 / 50
Trees according to audiences
Political Stakeholders Specialists
Media Lobbyists Delivery
BIS staff
![Page 59: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/59.jpg)
59 / 50
Wide Vs narrow groups
Example: Classification by Lobbyists.
They made wide groups.
Also, they clearly stated that they want duplicates (topics grouped in the "Common" sections)
Creating a second level of classification would be a good idea (= merge some sections, and give a name to the merge).
Duplicates make it easier to find content without knowing the exact structure of BIS.
Lobbyists’ classification
![Page 60: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/60.jpg)
60 / 50
Recommendations on policies organization
1. Users need several classifications in order to find their content:
By sector / by theme / by profession
Sector: Transportation, Chemistry, etc.
Profession: HR, Teacher, Researcher, etc.
2. In the theme classification, group sections into larger groups, for instance:
Business : Trade, regulations, employment, consumer regulation, sectors, regional development, etc.
Innovation : Knowledge transfer, Intellectual Property Office, R&D, science, etc.
Skills: HE, FE, universities, students
3. Duplicates welcome
Content can be placed in several larger sections (e.g. Knowledge transfer in ‘Skills’, ‘Business’ and ‘Innovation’)
H
![Page 61: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/61.jpg)
61 / 50
Policies article page
It is a very good point to show who is the responsible minister for a given policy.
Having this information fulfils one of the user’s need on the website: knowing who does what.
This good practice needs to be applied at a higher level in the website: users need to have this information for BIS in general to have a big picture of what BIS is and what they are doing.
![Page 62: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/62.jpg)
62 / 50
Recommendation – Policies landing page organization Users need a definition of what a policy is
and what they can find in this section.
They do not always expect to find regulations in this section
Participants want different ways to look for policies:
By theme (« subject »)
By recent updates
By sector / industry
By profession
A to Z (but requires to know the exact name of the policy)
Participants want a dedicated search for policies (on the landing page).
![Page 63: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/63.jpg)
63 / 50
Recommendations on policy organization
1. On the homepage:
The default display should be ‘Recently updated’ rather than A-Z of policies.
2. On the policy article page:
Need to identify document types: Policies / Regulations / Speeches /
Need to know what are the related documents to a policy – e.g. the history behind a new policy, the context and what is up do date today.
H
M
![Page 64: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/64.jpg)
![Page 65: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/65.jpg)
65 / 50
Participants need more advanced search options Participants want an advanced search. This functionality should make it
possible to search by:
Date
Type (policy, publication, speech, regulation…)
Sector (transportation, chemistry…)
Profession
Participants would also like a dedicated search in each section (News, speech, policies…)
M
![Page 66: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/66.jpg)
66 / 50
Best practices on the search page
‘Search within results’ is a great feature
More than half of the participants gave very good feedback about it.
They say they would use it to refine their search rather than starting the search over again.
Participants like the recommended link
They say they would trust the recommendation and would try to click on this link
![Page 67: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/67.jpg)
67 / 50
Nobody understands the menu on the left
None of the participants could tell what the menu on the left is.
They do not understand ‘BIS corporate’ and ‘BIS mutimedia’
Actually, most of them do not even notice it.
Recommendation:
Change the menu labels to make them more relevant
If these menus are filters, they should not look like menus but rather be part of an advanced search.
M
![Page 68: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/68.jpg)
68 / 50
Page numbers and results per page can be improved
Page number:
There are only 3 page number links on the prototype
Participants wonder if this is a limitation of the prototype.
Number of results per page:
The default should be more than 10
Participants do not always notice that they can change the number of results per page
Recommendations:
Display 25 results per page by default
Duplicate the link to change number of results per page on top of the result list.
M
L
![Page 69: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/69.jpg)
69 / 50
Specify the document type in search results
Recommendation:
In the result list, the type of document should be specified (Regulation, policy, speech…)
M
![Page 70: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/70.jpg)
![Page 71: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/71.jpg)
71 / 50
Participants need targeted information. Participants find it great to have the possibility to subscribe
to email alerts and/or RSS feeds from specific pages on the website.
They want to be able to select the information they will receive.
For instance: get latest speeches alerts from a specific minister.
Many participants complained that the alerts they receive from the site today contain too many pieces of information which is not focused on their interest. They tend not to read them any more because it takes too much time to find the relevant information.
Recommendation:
The email alert being the main access point to the website, it is crucial that users keep reading them.
They have to be able to choose in details the information they want to receive and the information they are not interested in so they can quickly scan these alerts and find interesting links to click on.
H
![Page 72: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/72.jpg)
72 / 50
Subscribing to e-mail alerts
Users would like to select their topic by sector or by profession
H
![Page 73: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/73.jpg)
![Page 74: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/74.jpg)
![Page 75: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/75.jpg)
75 / 50
Landing page is too generic
Participants say the text is too generic. They want more concrete information on this page.
![Page 76: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/76.jpg)
76 / 50
Mission and priorities are not factual enough
Same as landing page: participants would like to know more about priorities: how are they going to be achieved?
Actually, ‘About BIS’ does not answer one the main questions users have when they go to this section:
H
![Page 77: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/77.jpg)
77 / 50
Recommendation
The question “what is BIS doing?” could be answered by the use of a drawing/diagram showing BIS as a network:
The domains where BIS plays a role
The departments they embed
How these departments are linked together
The responsible ministers
The partners involved
BIS
Partners
Draft example of illustration of BIS as a network
![Page 78: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/78.jpg)
![Page 79: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/79.jpg)
79 / 50
News is a popular section
All participants tend to go to ‘News’ as soon as they are looking for some recent information.
Recommendation: It would be a good idea to have a page that shows all recent additions to the website:
This page would be sorted in chronological order (most recent first)
For each item on this page, make it clear whether it is a speech, a press release, a publication, a policy…
Recommendation: Participants prefer to read plain dates rather than ‘1 hour ago’, ‘2 weeks ago’
Example: 27-08-2009
For updates of the day: Today at 1.35pm
M
M
![Page 80: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/80.jpg)
![Page 81: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/81.jpg)
81 / 50
Browse by subjects is confusing
Half of the participants would choose first to browse in the publications by subject to find one.
The other half would use the search tool (generic tool above the menu or specific to the section)
They find the list is too difficult to scan: too many items.
Recommendations:
Make subject headlines clickable: gives access to a page with all subtopics linked to the subject
Add a short explanation text below the headline (1 sentence)
Display the 2 or 3 most popular links only to help users understand the kind of publications they will find for the subject, and make it possible to expand a section to display all the links.
M
![Page 82: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/82.jpg)
82 / 50
Are printed copies free of charge?
Many participants tend to believe they have to pay to order printed copies.
There is no information about charges on the bottom page.
The ‘checkout’ button makes them think they will have to give some payment details.
Recommendations:
Add ‘free of charge’ below the button ‘Order printed copy’
Alternately, Change the label in the button: ‘Order printed copy (£0)’
L
![Page 83: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/83.jpg)
![Page 84: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/84.jpg)
84 / 50
Contact details are difficult to find
Several participants complained about the contacts being difficult to find on the current website.
They need email addresses and phone numbers, mainly.
During the 1 to 1, a participant was surprised he could not find this information in the contact section.
Recommendations:
If possible, provide more tangible contact information to users than just an e-mailing page.
Should they use the contact page to send a message, users expect to get a confirmation email telling them their request has been successfully sent, to whom it has been sent and the average time to get an answer.
M
![Page 85: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/85.jpg)
![Page 86: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/86.jpg)
86 / 50
Videos are interesting for media people
The Media group like the videos
Other groups are not interested in videos
It is too time consuming to watch them.
It usually does not bring any additional information compared to the written article.
The other groups can be interested in ministers speech videos where you can get the tone of the speech.
![Page 87: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/87.jpg)
87 / 50
Flickr is interesting for media people
Media people are much more interested in pictures than other audiences.
They like the idea of having access to more photos about a specific subject
They expect the pictures to be copyright free so they can use them in their own articles
Other audiences say it is OK to have pictures but it is not mandatory.
Like they said for videos, pictures are not really interesting for them, unless it gives information that the text cannot give.
![Page 88: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/88.jpg)
88 / 50
Media people and BIS staff are interested in Twitter
Media and BIS staff find it useful and innovative
Other participants are not interested in twitter
They think it does not fit the image of a government
They say it is for younger people.
![Page 89: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/89.jpg)
89 / 50
Having quick polls on the site is OK
Even though most of the participants did not show great interest in answering polls on the website, they are not against it.
Several participants said it is a good way to make people come back to the site
But few of them said they would answer the poll.
The ‘media’ participant said he might be interested in the results of the poll, depending on the subject and the number of respondents.
Quick polls could be interesting to get quick feedback about consultations
![Page 90: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/90.jpg)
![Page 91: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/91.jpg)
![Page 92: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/92.jpg)
92 / 50
Political stakeholders
Content they are interested in:
Speeches by ministers, news, press releases
Nominations, who does what, the BIS organisation
Expected features:
More focused but thorough email alerts: Top lines ofeach article, more capability to select the topics that will be listed in the alerts.
A section that would give an overview of all what is new since their last visit.
They would like more related links in the Policy pages to "chain" the related policies, like newspapers do.
Today, they preferably use the newspapers (e.g. Guardian, FT) over the BIS site to better understand the history of a policy and the context, like regulations in other countries or previous policies.
![Page 93: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/93.jpg)
93 / 50
Specialist managers
Strong request from this group: They would like the policies to be classified by sector and profession.
They rely heavily on the e-mail alerts.
They would like more focused, but more detailed e-mail alerts.
They made few groups (typically 5) – and so larger groups - in the policies themes, in order to simplify the organisation of the policies.
![Page 94: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/94.jpg)
94 / 50
Policy influencers and Lobbyists
They need to better understand the structure of BIS, who does what (including partners), and who to contact.
They are very interested in having updates if the structure of the BIS changes.
They try to keep updated and they need some consistency in the site across time ("New government, new site!"). They are concerned that their bookmarks will not be available anymore with the new site.
Consultations and deadlines are critical for them. An agenda would help them a lot.
![Page 95: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/95.jpg)
95 / 50
Delivery Partners
They know the structure of BIS.
They also know well the structure of BERR, DIUS and former organisations.
But they expect all these structures to be somehow temporary."Anyway, they will change it in a while".
In the card sorting exercises, they typically build an architecture of the site quite similar to the current site (11 groups).
They would group content according to the three letters of BIS: Business (close to the former BERR), Skills (close to the former DIUS) and Innovation ("the new stuff").
![Page 96: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/96.jpg)
96 / 50
BIS Staff
A group of two persons...
They would like the site to be more interactive and more innovative (e.g. videos, rolling images, social media).
![Page 97: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/97.jpg)
97 / 50
Media and journalists
Specific request: They ask for a Media Center section (!)
It would include ministers pictures, videos, people (e.g. contact person), etc.
The Media section can be placed in the "Publication" tab, according to this group.
They request videos, and ask for multimedia features to be more prominent (Twitter, Flickr, YouTube)
They expect a light homepage, mostly focused on News: Breaking news, key announcements...
They can't define precisely what is a policy.
"Does it include regulations?"
They don't know what BIS covers exactly.
This group has the highest rate of unclassified cards in the card sorting exercises.
![Page 98: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/98.jpg)
98 / 50
Top level navigation – card sorting results
Specialists
![Page 99: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/99.jpg)
99 / 50
Top level navigation – card sorting results
Political stakeholders
![Page 100: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/100.jpg)
100 / 50
Top level navigation – card sorting results
Media
![Page 101: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/101.jpg)
101 / 50
Top level navigation – card sorting results
Lobbyists
![Page 102: Bis Usability Report](https://reader033.vdocuments.us/reader033/viewer/2022061300/54c8401e4a7959246e8b458d/html5/thumbnails/102.jpg)
102 / 50
Top level navigation – card sorting results
Delivery partners