towards automated quality assurance for socially created information assets on the web david...
Post on 20-Jan-2016
217 views
TRANSCRIPT
![Page 1: Towards Automated Quality Assurance for Socially Created Information Assets on the Web David Davenport & Markus Schaal Computer Engineering Dept., Bilkent](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649d3f5503460f94a191ce/html5/thumbnails/1.jpg)
Towards Automated Quality Towards Automated Quality Assurance for Socially Created Assurance for Socially Created Information Assets on the WebInformation Assets on the Web
David Davenport & Markus SchaalComputer Engineering Dept.,
Bilkent University, 06800 Ankara.
![Page 2: Towards Automated Quality Assurance for Socially Created Information Assets on the Web David Davenport & Markus Schaal Computer Engineering Dept., Bilkent](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649d3f5503460f94a191ce/html5/thumbnails/2.jpg)
IntroductionIntroduction
• Quality in collaborative web-based projects • considered good • no guarantees!
Outline:• Quality & Quality Assurance• Info. Quality & User Expertise• From Actions to Assessments• A Research Testbed…
Outline:• Quality & Quality Assurance• Info. Quality & User Expertise• From Actions to Assessments• A Research Testbed…
![Page 3: Towards Automated Quality Assurance for Socially Created Information Assets on the Web David Davenport & Markus Schaal Computer Engineering Dept., Bilkent](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649d3f5503460f94a191ce/html5/thumbnails/3.jpg)
Quality isQuality is
• something good, desirable!
• hard to definemeet or exceed user expectations
• easy to spot lack offails to meet spec.,poorly designed,difficult to use, unreliable, etc.
![Page 4: Towards Automated Quality Assurance for Socially Created Information Assets on the Web David Davenport & Markus Schaal Computer Engineering Dept., Bilkent](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649d3f5503460f94a191ce/html5/thumbnails/4.jpg)
Quality isQuality is
an open-ended quest for
perfection
![Page 5: Towards Automated Quality Assurance for Socially Created Information Assets on the Web David Davenport & Markus Schaal Computer Engineering Dept., Bilkent](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649d3f5503460f94a191ce/html5/thumbnails/5.jpg)
Quality AssuranceQuality Assurance
• Assuring improvements requires a process
Vision/goal
System/product/service
Make measurements & compare
Implement changes
![Page 6: Towards Automated Quality Assurance for Socially Created Information Assets on the Web David Davenport & Markus Schaal Computer Engineering Dept., Bilkent](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649d3f5503460f94a191ce/html5/thumbnails/6.jpg)
What is Information Quality?What is Information Quality?
• Criteria: accessibility, coverage, correctness, consistency, recency, reliability, cost, etc.
• Information must “inform” / “educate”• success depends on recipient too!
• Measuring information quality• either observe user behaviour (individual & groups)• or ask their opinion… but users are not all equal.
![Page 7: Towards Automated Quality Assurance for Socially Created Information Assets on the Web David Davenport & Markus Schaal Computer Engineering Dept., Bilkent](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649d3f5503460f94a191ce/html5/thumbnails/7.jpg)
Users are not all equalUsers are not all equal
experts
novices
& subversives!
Modulate user
opinion accordingly
But, on the web, how can you know your users?
![Page 8: Towards Automated Quality Assurance for Socially Created Information Assets on the Web David Davenport & Markus Schaal Computer Engineering Dept., Bilkent](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649d3f5503460f94a191ce/html5/thumbnails/8.jpg)
From Actions to AssessmentsFrom Actions to Assessments
System model of user
User
users
Systemmodel
ofcontent
contents
updates
modulates
viewupdate
rate
Action
assessmentsof quality
![Page 9: Towards Automated Quality Assurance for Socially Created Information Assets on the Web David Davenport & Markus Schaal Computer Engineering Dept., Bilkent](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649d3f5503460f94a191ce/html5/thumbnails/9.jpg)
Content & User AssessmentsContent & User Assessments
• Update content & user models using evidence• gathered from implicit observations
& by explicitly asking users, but• modulated by current assessment of user expertise/trust!
• Everything depends on users, encourage by• making it easy to contribute and rate content• providing incentives to do so, &• showing them the results!
• What evidence should be gathered & how?• What content & user models should be used?• How should the models be updated?• How should the results be reported?
Research questions
![Page 10: Towards Automated Quality Assurance for Socially Created Information Assets on the Web David Davenport & Markus Schaal Computer Engineering Dept., Bilkent](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649d3f5503460f94a191ce/html5/thumbnails/10.jpg)
A Research Testbed…A Research Testbed…
• Target students taking CS101/2 “Java” course• Lots of Java / computing resources on the web
(some old, some too complex, some plain wrong!)• Task is to create a quality wiki-like resource.
• Module – for Moodle, Drupal, MediaWiki• Enable distributed users to collaboratively
build quality information assets on the web.
• Of note…• Log all interactions• Try out various user, content & update models• Show overall system quality & its history
![Page 11: Towards Automated Quality Assurance for Socially Created Information Assets on the Web David Davenport & Markus Schaal Computer Engineering Dept., Bilkent](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649d3f5503460f94a191ce/html5/thumbnails/11.jpg)
To Conclude…To Conclude…
• Web is increasingly important info. resource• Improvements in Quality benefit all• Need way to evaluate quality,
identify opportunities for improvements, & track changes
• Our research aims to • produce a clear understanding
& common “quality” framework• add-on “quality” modules usable by others.
![Page 12: Towards Automated Quality Assurance for Socially Created Information Assets on the Web David Davenport & Markus Schaal Computer Engineering Dept., Bilkent](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649d3f5503460f94a191ce/html5/thumbnails/12.jpg)
Thank you Thank you for your time & attentionfor your time & attention