1. What does Quality Assurance Mean to Sakai?
Megan May
Sakai Quality Assurance Director

2. Role of QA Director Coordination!
Help bring parties interested in the same testing efforts together
Foster initiatives that support quality throughout the community
Assist in bridging the gap between test and design & development teams
Ensure release activities are transparent

3. WG Membership 2.2.1 23 People 10 Institutions 4 Countries 2.2.0
2.3.0
44 People
17 Institutions
5 Countries
2.4.0
93 People
37 Institutions
11 Countries
253 members on collab.
Close to 100 participants for However, there is typically a core group in the WG testing (~20)

4. QA Participants for 2.4
Vincent Siveton, Steve Swinsburg

5. QA Network Amsterdam Boston Cambridge Cape Town Columbia Indiana
rSmart
There are 8 QA servers hosted at 7 institutions.
Coming soon . . An oracle backend configured for spanish.

6. Where does QA fit in? What kinds of activities occur?
Focus on Functional Testing
Partners with accessibility WG
At end of the cycle. Usually other two cycle are beginning to occur while we’re still testing and preparing for the release.
How we’ve been releasing back to back
A lot of the release/product management is blurring into the QA process

7. Information Gathering
Integration Week/Integration Meetings
“Code Freeze”
Selective Merging
Communication
Weekly Tags
Stealth Provisional Tools
Documentation Finalized
Release!

8. Collab How to be in the know!

9. Jira

10. Confluence http://bugs.sakaiproject.org/confluence/display/QA/Home
Highlight resources available to WG members during cycle and before.

11. Year in Review 6 Releases 2.2.0 (5/12-7/19) 2.2.1 (7/15-8/15)
2.2.2 (9/18)
2.3.0 (9/15-11/3)
2.3.1 & 2.2.3
2.4.0 (3/17-5/21)

12. Summary - JIRA Activity
2.2.0 – 85% Verification
(504 JIRA Tickets)
2.2.1 – 85% Verification
(119 JIRA Tickets)
2.2.2– NA/Security Release
2.3.0 – 85% Verification
1345
(611 JIRA Tickets)
2.3.1 & – NA/Security Release
2.4.0 – 82% Verification
(1,345 JIRA Tickets)

13. Strides Forward Deployment testing on wider variety of platforms
Data retention & conversion script testing
Integration of Accessibility WG activities in general QA cycle
Push out testing earlier
Automated testing tools
Static code review
Selenium usage
Site Profiles

14. Reflection on Release process
Speak about the idea to have a phased release process.
Maintenance Branches - deliver Foundation endorsed maintenance releases every 4-6 week

15. Proposed Kernel + Bundle Release Timing
Dev
Kernel QA
Bundle Development
Patch
Week
Bundle QA
Contrib X Development QA
Contrib Y Development QA
Contrib Z Development QA
going to be more conservative about releases and freezes. Can use post-branches to get functionality out post freeze/release. Key is that we want community QA to run smoothly so our released products are nice and solid - while at the same time skilled sites can have the good stuff year-round :)
Nov
Release UI
Freeze
API
Freeze
Code
Freeze
May
Release

16. Perceived Quality Level
Various stakeholders in the community were polled. Data came from a pool of roughly 200 individuals. Mostly it’s equally

17. Quality, like beauty, is clearly in the eyes of the beholder.

18. Perceived Quality Poll

19. What we can do! Infiltration of entire development process
Formulate Working Groups
Refine processes & grow WG
Areas where QA is needed
Move more of the testing outside of the official QA cycle

20. Pledge Resources Why? Easy point of entry into Sakai
What happens in the community has a direct local impact
Easy point of entry into Sakai
One of the easier ways for people to contribute into Sakai
-- Takes time to learn the software - a good tester is an informed tester
Only 86% issue verification - and this is of BUGS
What happens in the community has a direct local impact

21. Questions

22. Other Sessions of Interest
Expanding Community: Student Involvement in QA Testing (today, immediately following)
Toward Further Automation of the Quality Assurance cycle (Thursday, 9:35-10:05 in Matterhorn 2)
Sakai Accessibility Update (Thursday, 2:05-2:35 in Zurich 1)