1. Internal QA in Open Source Development
Omer BarNir – MySQL AB

2. Agenda Background for this talk Open and Close Development Models
Internal QA and Community Testing
Q & A

3. Terminology QA Added Value
The value of the test findings
Closed / Traditional Development/Release/QA model
Not the ‘Open Source’ model

4. Open and Closed Development Comparison
Feature
Complete
Code
Complete
Development
Testing
Beta GA
External Feedback
External Testing
Internal Testing
Build
Coding
Bug Fixing
Release
Close
Build
Internal Testing
External Testing and Feedback
Alpha
Beta
RC GA
Open

5. Meaning to Internal QA Close Source Open Source
Most testing is done internally by QA
‘Large’ Testing Team
Feedback from users is arriving late.
QA is responsible to perform ‘User Scenarios’ and running tests in ‘real world’ configurations
The added value of Internal QA is in performing tests and providing feedback about the quality of the product.
Open Source
Most testing is done externally (by the community).
‘Small’ Testing Team
Feedback from users is arriving early and consistently.
Large variety of real world ‘User Scenarios’ and platform configurations provided by community testing/
The added value of Internal QA needs to be defined

6. Community and Internal QA Testing
Deterministic
Tests follow a plan and coverage status is known
Timing
Testing Schedule is known and follow development needs.
‘Bugs’/’Features
Test findings are compared to requirements / standards
Control activity
Monitoring, bug metrics, overall trend of issues, stabilizing the code
Community Testing
Not deterministic
We don’t know if things are not used or are used without a problem.
Random - No schedule to speak of.
‘Bugs’/’Features’
Is a given behavior the expected one (NIST tests found 20 issues with legacy functionality)

7. Putting It All Together
Community Testing
Internal QA
Real World User Scenarios
Configuration Variety
Large Number of participants
Determinism
Time to Market
Bug/Feature Clarity
Monitoring Process
Non Determinism
Bug/Feature Ambiguity
No Monitoring
Limited User Scenarios
Lab based Configurations
Limited testing resources
Strength
Weakness
Strength in one is weakness in the other.

8. Combining the Advantages of Both
Approach:
Positioning internal QA alongside community testing.
Leveraging the best of both worlds by adjusting Internal QA’s work
Testing Priorities
Setting QA’s test priorities / schedules to bridge the gap between the desired development testing schedule and the testing coverage done by the community.
Adjusting schedule of QA tasks based on feedback from Community Testing
Focus
Identifying features not likely to be tested by community
Planning for internal QA to have more focus on these features.

9. Examples Testing Priorities Feature Focus V5.1 V5.0 Cluster Testing
Partitioning
Row Based Replication
V5.0
Triggers / Views / Stored Procedures
Feature Focus
Cluster Testing
Specific Integration of features (e.g. views & functions)

10. Similar QA Activities Monitoring: Release Assessments Bug Reviews
Regression Testing
Quality Procedures

11. Summary Differences & similarities in the release cycle
Differences between Community testing and Internal QA
How to position Internal QA to best leverage its added value alongside community testing.

12. Questions?