A centre of expertise in digital information management www.ukoln.ac.uk
QA For Web Sites
Brian KellyUKOLNUniversity of BathBath
URLhttp://www.ukoln.ac.uk/
UKOLN is supported by:
Contents • What Is Quality • Quality And Web
Sites• QA Case Study• Approaches To
Testing• Benchmarking• Conclusions
Contents • What Is Quality • Quality And Web
Sites• QA Case Study• Approaches To
Testing• Benchmarking• Conclusions
A centre of expertise in digital information management www.ukoln.ac.uk 2
Background To Web & QA Focus
Web Focus:• A JISC-funded post to advise the UK’s HE and
FE community• Provided by Brian Kelly
QA Focus:• An advisory service for JISC’s 5/99 projects• Provides advice and support for projects in order
to ensure that projects comply with standards and best practices
• Provided by UKOLN (Marieke Guy & Brian Kelly) and ILRT
This talk is informed by the work of Web and QA Focus
A centre of expertise in digital information management www.ukoln.ac.uk 3
Quality Control and Quality Assurance
Quality Control (QC):• Use of regular testing procedures against your
definitions of quality and more specifically the refinement of these procedures
• Relates to outputs
Quality Assurance (QA):• examines the processes that shape your Web
site in the first place and so encompasses Quality Control
• Relates to inputs• Should be involved in development stage of a
Web site and throughout its life cycle
A centre of expertise in digital information management www.ukoln.ac.uk 4
What Can Fail on Your Site?
• Broken links, broken emails• Server load – too many hits on the site• Client side performance –down load time• Security isn’t working• Content is out of date• Browser incompatibility,HTML doesn’t validate• Interface – navigation, link colour• Graphics missing or too large• Scripts don’t work - forms, databases• Isn’t accessible to those with disabilities• Browser dependencies• …
A centre of expertise in digital information management www.ukoln.ac.uk 5
What Can You Test?• Functional testing • Compatibility testing• Load/performance testing• Stress testing • Usability testing• Security testing• Integration of unit testing• Link testing• HTML Validation• Reliability testing• Regression testing• …
A centre of expertise in digital information management www.ukoln.ac.uk 6
What Procedures?
• Good documentation• Requirements• Specifications• Mission statements or statement of direction
• Define your audience • Testing suite and tools• Usability testing• Use the right authoring tools etc.• Track problems
A centre of expertise in digital information management www.ukoln.ac.uk 7
Issues
Things to be aware of:• Testing goals should relate to kind of site you have• Testing time is limited use automated tools • Automated tools can be inadequate use manual
tests• Documentation is critical to ensure practices are
repeatable• Motivating/persuading people to implement a QA
culture can be difficult• Implement your QA procedures over time to avoid
stifling productivity• QA planning at the start of development means
less time fixing things at the end
A centre of expertise in digital information management www.ukoln.ac.uk 8
Case Study: IWMW Web Sitehttp://www.ukoln.ac.uk/web-focus/events/workshops/webmaster-2002/http://www.ukoln.ac.uk/web-focus/events/workshops/webmaster-2002/
Cas
e S
tud
y
A centre of expertise in digital information management www.ukoln.ac.uk 9
Aims Of Web Site
The aims of the Web site were:• To provide information about the workshop for
participants, speakers, etc.• To provide an online booking facility• To act as a demonstrator of standards, best
practices and innovative solutions
Subject to the following constraints:• No Content Management System• Limited time• Limited software development / design expertise
Cas
e S
tud
y
A centre of expertise in digital information management www.ukoln.ac.uk 10
ApproachThe approach taken was to use:
• The design and structure used for previous two workshops
• The HTML-Kit authoring tool to edit HTML files• The Xenu link checker
Innovations this year include:• Use of HTML and CSS validation icons• Provide an XHTML Web site• Make greater use of CSS• Provide a dynamically-generated RSS news feed• Provide live usage statistics• Access to Web site on PDAs using AvantGo
Is this approach typical of yours – incremental development of an existing Web site, and introducing new features and more up-to-date standards?
Is this approach typical of yours – incremental development of an existing Web site, and introducing new features and more up-to-date standards?
Cas
e S
tud
y
A centre of expertise in digital information management www.ukoln.ac.uk 11
ToolsHTML-kit is my preferred editor (and recent version is even better, providing batch validation)
The XHTML and CSS icons act as live linksThe SiteMeter icon gives realtime info on usage
The XHTML and CSS icons act as live linksThe SiteMeter icon gives realtime info on usage
A centre of expertise in digital information management www.ukoln.ac.uk 12
Problems
Some problems:• When updating pages, links, HTML and spelling
were not checking systematically• When using the HTML and CSS checker the
browser could check an old, cached version• At one stage the SiteMeter usage statistics
service was not available
Cas
e S
tud
y
It can be a useful (and therapeutic!) process to talk about problems with your Web site.
It can be a useful (and therapeutic!) process to talk about problems with your Web site.
A centre of expertise in digital information management www.ukoln.ac.uk 13
Validating In Batch
Checking (e.g. HTML and CSS validation, links, accessibility, etc.) initial files is time-consuming and, for large or continually updated Web sites is not a scalable solutionValidation in batch:
• Familiar for link-checking• Limited set of tools available for HTML validation
(e.g. HTML Validator Pro> • Batch HTML validation validates files generated
from PowerPoint, over which I have no control• Bobby batch accessibility check is now a licensed
application – so used old copy
Cas
e S
tud
y
Can now use HTML-Kit for validating the Web site.Can now use HTML-Kit for validating the Web site.
A centre of expertise in digital information management www.ukoln.ac.uk 14
Netscape (sigh)
User feedback revealed that:• £ does not display in Netscape :-(• The Suggestions text box and the Submit button
(!) were not displayed in versions of Netscape, due to failure to support XHTML correctly
Suggestions
Submit
Cas
e S
tud
y
A centre of expertise in digital information management www.ukoln.ac.uk 15
RSS News FeedThe RSS News Feed:
• Provides a mechanism for syndication of content• News feed can be viewed on other Web sites or
in other applicationsThe News Feed was created:
• By transforming (lightly structured) HTML from the news page using a remote service
• A static version was also created in case of problems with the live transformation
Problems:• The remote service was not always available • The static version was not always in sync with the
live version
Cas
e S
tud
y
Subsequently the dynamic transformation service became unavailable
Subsequently the dynamic transformation service became unavailable
A centre of expertise in digital information management www.ukoln.ac.uk 16
Online Bookings
The online booking forms:• Were used for booking for the workshop and choosing the
parallel sessions
The backend processing:• Is not yet fully automated to store the information in a
database• Confirmation page did not echo the user’s data• When users followed link to check session, input data may
have been lost• Crashed on surnames with spaces and accents• No validation was performed
Despite messages on acknowledgements page, some users still expected a separate email confirmation
Cas
e S
tud
y
A centre of expertise in digital information management www.ukoln.ac.uk 17
What Should Be Done
Some simple things can be done for next time:• Use of SSIs or a CMS (or equivalent ) to manage
resource fragments• Improve the online booking system to provide:
• Validation (which will save time in administration)• Display of user input• Separate confirmation of entry into database
But:• I recognise that minor typos, etc. will happen and,
as long as they are not critical, can be tolerated• The goal is quality – not perfection
Cas
e S
tud
y
A centre of expertise in digital information management www.ukoln.ac.uk 18
What About The Processes?
What QA processes were used?• Automated tools• Peer review from members of the organising
committee• Alternating between Opera, IE and Mozilla when
working on Web site• Viewing the Web site on my Palm to:
• Allow me to work on the bus, train, etc.• See how the Web site works on a low-spec device
See <http://www.ukoln.ac.uk/web-focus/events/workshops/webmaster-2002/qa/>
See <http://www.ukoln.ac.uk/web-focus/events/workshops/webmaster-2002/qa/>
Cas
e S
tud
y
A centre of expertise in digital information management www.ukoln.ac.uk 19
What Else Should I Do?
Additional processes I should employ:• Reviewing by users• Peer-reviewing (others in similar situation)• Analysis of logs (esp. failure logs)• Usability testing• Multiple browser testing• Test data with unusual entries for online forms• Post-workshop review of Web site• Systematic documentation of successes and failures• Make case for extra resources, software, etc. to
implement improvements• Documenting the architecture of the Web site• Documenting the limitations of the Web site• Preparing these slides!• …
Cas
e S
tud
y
A centre of expertise in digital information management www.ukoln.ac.uk 20
Automated TestingAutomated Testing:
• Should be scalable• Only suitable approach for large Web sites• Can push (email) information about problems• Many free automated testing tools available
but:• Will not spot all usability / accessibility problems• Can provide too much information• Comprehensive and configurable testing tools
can be expensive
A centre of expertise in digital information management www.ukoln.ac.uk 21
Manual TestingManual Testing:
• Can spot usability / accessibility problems• Can make use of one’s community / user base /
peers• Can use of key pages which will identify
problems which occur elsewhere • Particularly useful for testing new Web sites• “5 users can spot most of your problems”
but:• Not suitable for testing large numbers of pages• Quality of feedback may be variable• Often not useful for testing established Web
sites
A centre of expertise in digital information management www.ukoln.ac.uk 22
Approaches To Automated Testing
Automated testing can make use of:• Desktop tools• Web-based tools
UKOLN has made use of Web-based testing tools:• Monitor accessibility, HTML, CSS, compliance,
page size, links quality, nos. of links, etc.• Benchmarking approach provides comparison
with one’s peers• See, for example,
<http://www.ukoln.ac.uk/web-focus/events/conferences/ucisa-tlig-2002/benchmarking/>
A centre of expertise in digital information management www.ukoln.ac.uk 23
Interfaces to Testing Tools (1)
Bookmarklets• Embed a testing service into your browser• Available for IE, Netscape, Opera, …browsers• Article at <http://www.ariadne.ac.uk/issue19/web-focus/>
• See <http://www.bookmarklets.ac.uk/> or Google search for “bookmarklets”, “accessibility bookmarklets”, etc.
A centre of expertise in digital information management www.ukoln.ac.uk 24
Bookmarklets Example
Bookmarklets can be accessed:
• From the browser’s normal Bookmarks / Favourites Menu
• From a side bar• By right-clicking
the mouse button
A centre of expertise in digital information management www.ukoln.ac.uk 25
Interfaces to Testing Tools (2)
URL Interface• UKOLN’s Web site has an interface to various
tools provided at the server• Easier to maintain• See <http://www.ukoln.ac.uk/,tools>
for overview
<http://www.ukoln.ac.uk/,validate><http://www.ukoln.ac.uk/,validate>
A centre of expertise in digital information management www.ukoln.ac.uk 26
Benchmarking (1)
Benchmarking:• We have discussed various tools for checking
one’s own site• But how do we compare with our peers?• Can we learn from others best practices?• Can we avoid making mistakes that others have
made?
Can the methodology to be described be used across your community – e.g.
• On a regional basis• On candidates for Best Public Library Web Site award • By government auditors for checking e-GIF compliance
Can the methodology to be described be used across your community – e.g.
• On a regional basis• On candidates for Best Public Library Web Site award • By government auditors for checking e-GIF compliance
A centre of expertise in digital information management www.ukoln.ac.uk 27
Benchmarking (2)
A centre of expertise in digital information management www.ukoln.ac.uk 28
QA Focus Web Site Benchmarking
QA Focus:• Carried out survey of 43 project Web sites• Findings freely available• Methodology can be applied to other sectors• Where best practices found, projects asked to
provide a case study• Where problems found, appropriate advice
provided
A centre of expertise in digital information management www.ukoln.ac.uk 29
HTML Compliance
W3C’s HTML Validation service is used to record the HTML compliance of project home pages
http://www.ukoln.ac.uk/qa-focus/surveys/http://www.ukoln.ac.uk/qa-focus/surveys/
A centre of expertise in digital information management www.ukoln.ac.uk 30
Accessibility
Bobby is used to record compliance with W3C WAI guidelines:
24 comply with A
5 comply with AA
21 are not compliant
Note that only compliance with guidelines which can be monitored by automated tools is recorded
Note that only compliance with guidelines which can be monitored by automated tools is recorded
A centre of expertise in digital information management www.ukoln.ac.uk 31
Repurposing Resources
Are pages from project Web sites available in the Internet Archive?
Can the project Web site be accessed on a PDA?
A centre of expertise in digital information management www.ukoln.ac.uk 32
404 Pages
404 error pages can provide an important navigational feature for Web sites
Tailored 404 Page 14 33%
Default 404 Page 28 67%
* Rating 3 21%
** Rating 3 21%
*** Rating 8 57%
**** Rating 0 0%
* Basic** Simple branding*** Additional functionality**** Full functional
A centre of expertise in digital information management www.ukoln.ac.uk 33
Critique
Limitations of this approach include:• Project Web sites may be:
The project deliverable About the project For intra-project communications
• The project deliverable may be, say, middleware • Limitations of the analysis tools• Limited number of pages surveyed• …
A centre of expertise in digital information management www.ukoln.ac.uk 34
Supporting Best Practices
QA Focus is supporting JISC 5/99 projects by providing advice on compliance with standards and best practices, based on the survey findings
http://www.ukoln.ac.uk/qa-focus/documents/
http://www.ukoln.ac.uk/qa-focus/documents/
A centre of expertise in digital information management www.ukoln.ac.uk 35
Case Studies
QA Focus is commissioning case studies:
• Document approaches projects have taken in particular areas
• Describe the problem area, the approach taken, problems experienced and lessons learnt
A centre of expertise in digital information management www.ukoln.ac.uk 36
Conclusions
To conclude:• Quality assurance is important• It will grow in importance: e.g. compliance with e-
GIF guidelines, accessibility legislation, etc.• Automated tools can help• Manual testing is needed to complement
automated approaches• Benchmarking can provide information on
approaches across communities