1. SIMians Course Comment Forum (CoCoFo) Linda Duffy Jean-Anne Fitzpatrick Sonia Klemperer-Johnson James Reffell

2. CoCoFo Overview  Web site for SIMS students to:  Get descriptive information about courses offered at SIMS (and other classes at Berkeley of interest to SIMS students)  Find out about the opinions of other students who've taken a course in the past  Express their opinions about courses they've taken

3. Paper prototype

4. First interactive prototype

5.  Implementation: HTML and Perl/CGI scripts  Large number of static pages  Did not actually save information entered through the "add comment" function

6. Heuristic Evaluation  Largest issues:  rating scale  forced preview  Many smaller errors (such as clarity of text labels)

7. Second interactive prototype

8.  Implementation: Cold Fusion  Small number of dynamic pages  Implemented "add comment" function using Access database  Changes based on heuristic evaluation

9. Rating scale n Old:

10. Rating scale n New:

11. Preview n Old:

12. Preview n Old:

13. Preview n New:

14. Examples of other (visible) changes in second prototype… n Display login status

15. Examples of other (visible) changes in second prototype… n 'Return to top' links

16. Examples of other (visible) changes in second prototype… n Prevent invalid submissions

17. CoCoFo DeMo  I will demo the tasks we assigned to the participants in our informal usability test:  Find the course reviews for IS250.  If the tester browsed to find IS250 as part of the first task, we asked the tester to search for a class on user interface design (or vice versa).  Post a comment to IS206.

18. Results - positive n Testers liked overall design n Testers liked idea of site n Most testers understood rating system

19. Results - Negative n Inexact searches got no results n Login button misplaced n Preview didn’t highlight ratings n Course categories less than helpful n Unclear about who has access to site n Text box didn’t wrap

20. Results - Additions n More ratings n Edit/delete comments n “Rate the rater”

21. Results - Questions n Anonymity – keep identities public n Rating system – numeric is fine, more description n Threading – negative to lukewarm response

22. Results - Observations n Preference of browsing over searching n Preference of verb-noun over noun-verb method of adding comment n No search modification n Use of both “preview” and “submit” from “add a comment” page

23. Formal Experiment - Hypotheses n Presentation of rating scale: combination of both graphical and numeric scales will be understood more quickly and accurately than a strictly graphical scale or a strictly numeric scale. n Span of rating scale: A scale of 1 to 5 will come closer to capturing the user’s “true” rating than either a scale of 1 to 3 or a scale of 1 to 10.

24. Formal Experiment – Independent Variables n Factor 1: Presentation of the scale 1.Strictly numeric scale 2.Strictly graphical scale 3.Combination of numeric and graphical scale n Factor 2: Span of the scale 1.Scale of 1 to 3 2.Scale of 1 to 5 3.Scale of 1 to 10

25. Formal Experiment – Response Variables n Amount of time it takes for the user to assign ratings for course difficulty and instructor n Whether the user chooses not to rate course difficulty and/or instructor n Whether user asks for clarification of rating system n User’s satisfaction with the scale n User’s opinion regarding whether ratings assigned accurately reflected user’s opinions

26. Next Iteration: Plans n Add workload rating n Clarify difficulty rating scale n Add e-mail display option to comments n Highlight noun-verb comment option n Make active button focus behavior consistent n Get text box to wrap

27. Next Iteration: Hopes n Edit/delete functionality for old comments n Implement registration n Improve search n Improve categories n Add multi-year distinctions n Complete course listings n Graphical course rating displays n Rate the rater

28. Summary n Insightful commentary n Witty banter n Sweeping generalization n Misleading cliché n Feel-good rhetorical flourish n Snappy ending

29. Further Study n Questionnaire results: –Animated dog preference varies widely –Some prefer cats (!) –Some have multiple-dog preference n Clearly, more research required