1. Cambridge Spreadsheet Quality Foundations of Spreadsheet Workshop 2004 Margaret Burnett Oregon State University and the EUSES Consortium

2. Cambridge What are some of the right questions? (Starts here, continues via brainstorming session later) 2

3. Cambridge 3 Questions: 1. Can we “fix the system” to automatically prevent/detect (some) errors: augmented systems? 2. Can we “fix the system” to work together better with users in finding/fixing errors? 3. Can we “fix the user” to make them care more about errors? 4. Can we “fix the user” so that they get better at avoiding errors (“tidy user”)? 5. Can we enforce fixes to the user?

4. Cambridge 4 The EUSES Consortium’s Dependability Research Question Is it possible to bring the benefits of rigorous software engineering methodologies to end users?

5. Cambridge What answers are being worked on?

6. Cambridge 6 EUSES Consortium Research If we build it, will they come? Types via informal annotations Sources & meta-sources of faults by end users Incremental analyses for immediate communication Software Engineering and Languages HCI and Psychology Education

7. Cambridge 7 Building research community Joint efforts with EU group Community-building events in the works: Workshops, SIGs, BOFs, Dagstuhl K-12 education: quality control mindset Research in curriculum change Mathematics education conferences, book K-12 teachers K-12 students (Saturday Academy) High school researchers Outreach & Building Community

8. Cambridge 8 Some examples WYSIWYT & Fault localization for end users (with Gregg Rothermel, Curt Cook, Joey Ruthruff). Surprise-Explain-Reward (with Curt Cook, many students). Assertions for end users (Rogan Creswick).

9. Cambridge 9 WYSIWYT and Fault Localization What if user sees a “good” value? User can “check off” the value with a √. Increases coverage, according to an adequacy criterion. What if a “bad” value? User can “X out”. Highlights cells that may be at fault.

10. Cambridge 10 The computer’s testing caused it to wonder if this would be a good guard. Fix the guard to protect against bad values, by typing a range or double-clicking. Surprise-Explain-Reward: Enticing Users (Introduced at CHI’03).

11. Cambridge 11 Assertions User assertion System assertion Assertion conflict Value violation

12. Cambridge Now, revisiting those questions... (Brainstorming)

13. Cambridge 13 Questions 1. Can we “fix the system” to automatically prevent/detect (some) errors? 2. Can we “fix the system” to work together better with users in finding/fixing errors? 3. Can we “fix the user” to make them care more about errors? 4. Can we “fix the user” so that they get better at avoiding errors? 5. Can we enforce fixes to the user?

14. Cambridge 14 Questions (cont.)

15. Cambridge 15 And answers (at a level of fundamentals, to extent possible) A potentially very positive thing: to continue the dialog among EUSPRIG, EUSES, and other groups who are interested. EUSPRIG is tentatively July 7-8 next year. Some cooperation with PPIG? Big database of thousands of European end users who are using spreadsheets. So, there’s a set of people he can call up to ask about some of our issues. Possibility of a survey within some company as a way to get data about spreadsheet usage. (Types of errors? Application area?) Publishable! And, informs the design! Maybe a company would fund the subjects’ incentive if the people we were surveying were their users. (Steve had some specific ones in mind.) Possible follow-up by Susan and Steve. People: we need to hear and talk about actual individual users more. Connections between business questions (lots of users) and empirical experts. (200-260 new students/yr?) What if I called him next summer and said we want to try our new software out on your students? He might be able to provide the subjects, teach the use, etc.