2. Advancing Science with DNA Sequence GENEBOREE A Tool for Collaborative Gene Annotation DOE Joint Genome Institute Anya Kartavenko Rowena Luk Annette Greiner

3. Advancing Science with DNA Sequence Geneboree Geneboree is A collaborative tool for manual functional annotation of genes Prototyped as an extension to JGI’s Integrated Microbial Genomes (IMG) data analysis system

4. Advancing Science with DNA Sequence Changes Based on the Heuristic Evaluation From the Lights-On Team

5. Advancing Science with DNA Sequence Flagging Active Annotations

6. Advancing Science with DNA Sequence Flagging Active Annotations

7. Advancing Science with DNA Sequence Buttons

8. Advancing Science with DNA Sequence Buttons

9. Advancing Science with DNA Sequence Viewing Previous Versions

10. Advancing Science with DNA Sequence Viewing Previous Versions

11. Advancing Science with DNA Sequence User Information

12. Advancing Science with DNA Sequence User Information

13. Advancing Science with DNA Sequence Annotator Information

14. Advancing Science with DNA Sequence Annotator Information

15. Advancing Science with DNA Sequence Discussion Teaser

16. Advancing Science with DNA Sequence Discussion Teaser

17. Advancing Science with DNA Sequence Update Annotation Popup

18. Advancing Science with DNA Sequence Update Annotation Popup

19. Advancing Science with DNA Sequence Voting Buttons and Results

20. Advancing Science with DNA Sequence Voting Buttons and Results

21. Advancing Science with DNA Sequence Demo http://groups.sims.berkeley.edu/annotation/

22. Advancing Science with DNA Sequence Pilot Study: Participants Participant A Participant BParticipant C Age25-3535-45 GenderMMF Ed. LevelPhDMSPhD Occupation Postdoctoral Fellow Professional Annotator Field/Background Environmental Genomics Molecular Biology Exp. w/IMG Use few times a month None

23. Advancing Science with DNA Sequence Pilot Study Methodology: –Task observation (Tasks 1–5) Recorded time Errors and comments –A “qualitative” experiment (Task 6) How would you go about determining whether this annotation is a valid one or not? –Debriefing –Demographic survey

24. Advancing Science with DNA Sequence What we learned Compare and Transfer tool –Feedback upon transfer –Descriptions of the COG, EC, GO assignments –% agreement is easily confused with % identity –Annotators missed graphical representation of the multiple alignment Discussion –Comments are essential in the scientific discourse My Annotations and My Watched Annotations –What are they?

25. Advancing Science with DNA Sequence Re-Working Ratings /Discussion BEFORE AFTER

26. Advancing Science with DNA Sequence Second Heuristic Evaluation Gene sort in MyAnnotations Collapsible sections Versions with date and time CANCEL different behavior in pop-up windows

27. Advancing Science with DNA Sequence Second Heuristic Evaluation BEFORE : Update Annotation + Compare and Transfer Page

28. Advancing Science with DNA Sequence Second Heuristic Evaluation Compare Page Transfer Page Manual Entry AFTER :

29. Advancing Science with DNA Sequence Lessons Learned Working within an existing system presents serious conflicts Major design alternatives should be compared at the start Such specialized work would benefit from participatory design

30. Advancing Science with DNA Sequence Wrap-Up Work Change titles of myAnnotations Refine aesthetics Change agree/disgree buttons to radio buttons Move discussion to gene details page

31. Advancing Science with DNA Sequence Questions?