1. Towards Automated Quality Assurance for Socially Created Information Assets on the Web David Davenport & Markus Schaal Computer Engineering Dept., Bilkent University, 06800 Ankara. david@cs.bilkent.edu.trdavid@cs.bilkent.edu.tr & schaal@cs.bilkent.edu.trschaal@cs.bilkent.edu.tr

2. Introduction Quality in collaborative web-based projects considered good no guarantees! Outline: Quality & Quality Assurance Info. Quality & User Expertise From Actions to Assessments A Research Testbed… Outline: Quality & Quality Assurance Info. Quality & User Expertise From Actions to Assessments A Research Testbed…

3. Quality is something good, desirable! hard to define meet or exceed user expectations easy to spot lack of fails to meet spec., poorly designed, difficult to use, unreliable, etc.

4. Quality is an open-ended quest for perfection

5. Quality Assurance Assuring improvements requires a process Vision/goal System/product/service Make measurements & compare Implement changes

6. What is Information Quality? Criteria: accessibility, coverage, correctness, consistency, recency, reliability, cost, etc. Information must “inform” / “educate” success depends on recipient too! Measuring information quality either observe user behaviour (individual & groups) or ask their opinion… but users are not all equal.

7. Users are not all equal experts novices & subversives! Modulate user opinion accordingly But, on the web, how can you know your users?

8. From Actions to Assessments System model of user User users System model of content contents updates modulates view update rate Action assessments of quality

9. Content & User Assessments Update content & user models using evidence gathered from implicit observations & by explicitly asking users, but modulated by current assessment of user expertise/trust! Everything depends on users, encourage by making it easy to contribute and rate content providing incentives to do so, & showing them the results! What evidence should be gathered & how? What content & user models should be used? How should the models be updated? How should the results be reported? Research questions

10. A Research Testbed… Target students taking CS101/2 “Java” course Lots of Java / computing resources on the web (some old, some too complex, some plain wrong!) Task is to create a quality wiki-like resource. Module – for Moodle, Drupal, MediaWiki Enable distributed users to collaboratively build quality information assets on the web. Of note… Log all interactions Try out various user, content & update models Show overall system quality & its history

11. To Conclude… Web is increasingly important info. resource Improvements in Quality benefit all Need way to evaluate quality, identify opportunities for improvements, & track changes Our research aims to produce a clear understanding & common “quality” framework add-on “quality” modules usable by others.

12. Thank you for your time & attention contact david@cs.bilkent.edu.tr / schaal@cs.bilkent.edu.tr david@cs.bilkent.edu.trschaal@cs.bilkent.edu.tr