From manual test shop to fully automated test coverage: A How-To session to speed up your journey Jayshree Bhakta ITHAKA/JSTOR

The Journey Applications 1 app 268 apps Deployments 1 / month 234 / week Automation % 0% 100% QA per team 3 / team 1 / team Testing cycle 3-6 weeks 20 - 60 mins

How?

Vision and Goals: Define BUSINESS Faster response to business needs Enhance business risk mitigations Product deficiency prevention Reduction in overall cost of achieving quality TECHNICAL Continuous delivery and testing Efficient, scalable and sustainable products and technology Immediate feedback loops Reduction of overall cycle time

Vision and Goals: Share Engagement with partnership Leadership Teams Team ownership of quality Value-add for everyone Value-add for users

Vision and Goals: Milestones Plan Act Evaluate Iterative process

Implementation

Implementation: Tool Selection Acceptance Tests Driven Development / Behavior Driven Development Language to write tests in business language Easier language to learn Open source Lower cost Dynamic and faster TOOL SELECTION TOOL EVALUATION SKILL TRAINING CODING QA TEAM CULTURE TALENT/ HIRING LESSONS LEARNED

Implementation: Tool Evaluation QTP Expensive Not very efficient Domain Specific Language Developed without QA engagement No internal staff knowledge Maintenance and upkeep Hiring challenge Cucumber/Ruby Business language / Easy to learn GitHub Code version control Jenkins Run tests remotely Schedule test runs Results – Storage and Notifications TOOL SELECTION TOOL EVALUATION SKILL TRAINING CODING QA TEAM CULTURE TALENT/ HIRING LESSONS LEARNED

Implementation: Skills Training LEADS / TASK FORCE Develop core framework Learn the tools Set up the pipeline Coach the team Adapt the training process TEAM Introduce the idea/plan Team training LeanDog Smaller group workshops Wiki creation 1-1 training TOOL SELECTION TOOL EVALUATION SKILL TRAINING CODING QA TEAM CULTURE TALENT/ HIRING LESSONS LEARNED

Implementation: Coding Prioritized business workflows Coding standards Tagging Test coverage Code review standards Sprint planning– Write cucumber Write ruby code Run and analyze - Jenkins Run times Maintainability 0 test errors Updates TOOL SELECTION TOOL EVALUATION SKILL TRAINING CODING QA TEAM CULTURE TALENT/ HIRING LESSONS LEARNED

Implementation: QA Team Culture DURING TRANSITION Shared learning Support QA and ruby - Slack Automation retrospectives ON-GOING Cross team collaboration QA and ruby - Slack Shared learning Support On-going skill development TOOL SELECTION TOOL EVALUATION SKILL TRAINING CODING QA TEAM CULTURE TALENT/ HIRING LESSONS LEARNED

Implementation: Talent Hiring Skills needed Technical aptitude Coding language knowledge Culture fit Leadership ON-BOARDING Couple of weeks Domain Technical Team Dedicated time from leads TOOL SELECTION TOOL EVALUATION SKILL TRAINING CODING QA TEAM CULTURE TALENT/ HIRING LESSONS LEARNED

Implementation: Lessons Learned Big user stories = longer to automate Too much coverage = Slow run times End to end automation = brittle Custom built Domain Specific Language Duplication of step definitions Monitoring should be done using monitoring tools – not tests TOOL SELECTION TOOL EVALUATION SKILL TRAINING CODING QA TEAM CULTURE TALENT/ HIRING LESSONS LEARNED

Our Process Pipeline Story: Pick Up Story: Preview Sprint Planning Story: Pick Up Story: Preview Story: Code and Test Story: Validation Story: Deployment Sprint Retrospectives

SPRINT/STORY: PLANNING Product Detailing Requirement Design Success criteria – First pass Technical Detailing Code approach Testing approach Success criteria – Refine PLAN STORY PREVIEW TEST + CODE VALIDATE DEPLOY RETRO

STORY: PICK UP PLAN STORY PREVIEW TEST + CODE VALIDATE DEPLOY RETRO

STORY: PREVIEW In depth discussion of the specific story between Dev QA UX PLAN STORY PREVIEW TEST + CODE VALIDATE DEPLOY RETRO

STORY: TESTING AND CODING PLAN STORY PREVIEW TEST + CODE VALIDATE DEPLOY RETRO

STORY: VALIDATION PLAN STORY PREVIEW TEST + CODE VALIDATE DEPLOY RETRO

STORY: DEPLOYMENT Feature deployment to Prod One story corresponds to one deployment. PLAN STORY PREVIEW TEST + CODE VALIDATE DEPLOY RETRO

SPRINT: RETROSPECTIVES PLAN STORY PREVIEW TEST + CODE VALIDATE DEPLOY RETRO

What did we learn?

Takeaways Quality strategy with partnership Select tools that fit your requirement Dedicated task force Set coding standards Develop team culture Iterative process

QA Engineer at ITHAKA/JSTOR Questions? Jayshree Bhakta QA Engineer at ITHAKA/JSTOR jayshree2901@gmail.com @jayshreebhakta