Software Quality & QA Management Jatin Parekh Sr Dir of QA, Visa

What is Quality 1.Fitness of use: Quality means the product or service does what it is expected to do 2.Customer expectation: Quality is exceeding the customer expectations. Quality is delighting the customer 3.Competition: How company’s products or services compared to competitors and compared to those offered by company in the past.

Benefits of good quality Customer loyalty Strong brand Command premium price Reduced cost – Cost of maintaining – Customer support Attracting and retaining employees

Quality Characteristics Ease of use Simplicity of design Available Reliable Safe - secure Performance Responsive Maintainable - ability to make changes quickly

Software Quality Attributes Software performs as defined by accepted requirements (external quality) – Correctness (behaves per spec) – Integrity (data integrity) – Usability (ease of use) – Efficiency (system resources) – Reliability (mean time between failure

Software Quality Attributes (cont) System structure is open to change over time (internal quality – what users don’t see) – Maintainability (cost of correcting errors) – Testability (cost / ease of testing) – Flexibility (cost of product modification) – Reusability (cost of transferring a module to another application) – Efficiency (coding efficiency, system resources) – Capacity / Scalability (ability to serve at peak load) – Security (encryption / malicious injection; OWASP) C

Quality control vs Quality Assurance Quality Control: The observation techniques and activities used to fulfill requirements for quality – Design / code review, Change mgmt / release process, Measurements and metrics Quality Assurance: The planned & systematic activities implemented in a quality system so that quality requirements will fulfilled – Test strategy, test execution, defect mgmt, etc.

Software Testing Unit testing Functional testing (use cases, API) – Black box testing (end user) – Boundary testing – White / grey box testing – Testing with mock components Internationalization (I18N) / Localization L10N Integration / end to end testing Usability / UI testing Installation & upgrade Regression testing

Software Testing (contd) Rollback / recovery & Failover Security -Open Web App Security Project (OWASP) Reliability & robustness (e.g. error handling, protocol conformance) Performance (bottlenecks / memory leaks / cpu usage responsiveness) Scalability / stress / volume / capacity Compatibility (e.g. DB changes with old code) Migration Documentation (Help / User’s manual

Software Test Category Build verification testing (BVT) User acceptance testing (UAT) Regression testing Smoke / sanity testing Ad hoc testing Alpha (internal users) / bug bash Beta testing

Testing tools Test case management tools Testing tools – Commercial / open source tools – Customer tools / utilities Automation tools (e.g. Selenium for UI automation) Test framework – Execute scripts calling multiple tools – Ability to configure env – Ability manage test data – Schedule tests – Reporting – Manage results Measurement tools (code coverage, static analysis tools)

QA Activities Planning – Overall test strategy in test plan document Review use cases & spec Test case generation Test env setup Tools development Automation script development Test execution Defect logging and verifying Test results review Release checklist

Software Industry / Technology Trends Shift from Installed software to software delivered as a service (SaaS) – Vendor can upgrade software frequently Shift from physical infrastructure to cloud infrastructure – Improved scalability Desktop/PC computing to mobile / wearable devices – Need for software to provide consistent look/feel Security – Data in transit / at rest – Govt regulations

Water Fall vs Agile Method Water fall – Final product is well defined – Design, development, & testing is done in sequence – Clients won’t have the ability to make change Agile – Allows for making changes after planning – Ability to react to competition – Development & testing is done in parallel and completed in each sprint, typically 2 to 4 weeks. – Product can be launched at the end of every sprint – Ideal for SAS / web products.

Agile Methodology Scrum is software development process Daily standup of a small self contained team – 4-6 dev / 2-3 QA / Product owner / Scrum Master Sprint retrospective and planning Core Agile Process

Agile Methodology & QA Frequent releases (continuous delivery) – Upgrade features piecemeal – Different behavior for diff users (A/B testing) Reliance on communication rather than documentation Development and testing hand in hand – Think about testing before code development Continuous integration testing Regression testing in <2 days ~ 100% Automation is Key to Success

Managing Quality Manage Project Quality: Quality Control and Quality Assurance Full time job for large, complex systems – Establish organization’s quality procedures & standards – Select appropriate procedures and standards based on project scope – Ensure that standards are followed

Managing Quality (contd) Project planning – QA project plan encompasses test strategy plan, resource plan, execution plan, checklists Project tracking, Status, Reports Tools and methods – Build vs buy Cost of quality – Tradeoff in the midst of time / resource constraints – What to test, when to test, and how much to test

Managing Quality (contd) Metrics & Measurements – Test execution monitoring – Defect trends – Code coverage – Performance measurements Defect management – Defect Triage – When to push for fix and when to push back

Role of QA Lead Manage relationship with stakeholders – Product managers, Project managers, Development, IT, Ops, Customer Care, Clients, other QA teams – Negotiations (timeline, resource, expectations) – Establish win-win situation Organization of the team – Project based – Matrix organization – Geographical based Anticipate & manage change – Metrics driven approach for decision making

Role of QA Lead (contd) Key responsibilities – Owns the quality of a release – Recruiting – Goals setting – Coaching, motivating, & conflict resolution – Challenge high performers / manage low performers – Communication – Manage remote employees – Feedback & Reviews – Honesty & fairness

Recap Roles & responsibilities of QA in Org Agile / rapid deployment cycle (1 to 4 wks) – Automation & Tools – Software development / programming experience Offshore and onshore delivery Startup vs Established companies – Documentation, schedule, level of quality Perception of QA in Org

Career in QA Key traits: – Software development & programming knowledge – Ability to troubleshoot – Wanting to know how things work end to end – Curiosity: ask questions & be proactive – Communication – Wanting to break things Managerial vs Technical track

Q & A