1. From manual test shop to fully automated test coverage: A How-To session to speed up your journey
Jayshree Bhakta
ITHAKA/JSTOR

2. The Journey Applications 1 app 268 apps Deployments 1 / month
234 / week
Automation % 0%
100%
QA per team
3 / team
1 / team
Testing cycle
3-6 weeks
mins

3. How?

4. Vision and Goals: Define
BUSINESS
Faster response to business needs
Enhance business risk mitigations
Product deficiency prevention
Reduction in overall cost of achieving quality
TECHNICAL
Continuous delivery and testing
Efficient, scalable and sustainable products and technology
Immediate feedback loops
Reduction of overall cycle time

5. Vision and Goals: Share
Engagement with partnership
Leadership
Teams
Team ownership of quality
Value-add for everyone
Value-add for users

6. Vision and Goals: Milestones
Plan
Act
Evaluate
Iterative process

7. Implementation

8. Implementation: Tool Selection
Acceptance Tests Driven Development / Behavior Driven Development
Language to write tests in business language
Easier language to learn
Open source
Lower cost
Dynamic and faster
TOOL SELECTION
TOOL EVALUATION
SKILL TRAINING
CODING
QA TEAM CULTURE
TALENT/ HIRING
LESSONS LEARNED

9. Implementation: Tool Evaluation
QTP
Expensive
Not very efficient
Domain Specific Language
Developed without QA engagement
No internal staff knowledge
Maintenance and upkeep
Hiring challenge
Cucumber/Ruby
Business language / Easy to learn
GitHub
Code version control
Jenkins
Run tests remotely
Schedule test runs
Results – Storage and Notifications
TOOL SELECTION
TOOL EVALUATION
SKILL TRAINING
CODING
QA TEAM CULTURE
TALENT/ HIRING
LESSONS LEARNED

10. Implementation: Skills Training
LEADS / TASK FORCE
Develop core framework
Learn the tools
Set up the pipeline
Coach the team
Adapt the training process
TEAM
Introduce the idea/plan
Team training
LeanDog
Smaller group workshops
Wiki creation
1-1 training
TOOL SELECTION
TOOL EVALUATION
SKILL TRAINING
CODING
QA TEAM CULTURE
TALENT/ HIRING
LESSONS LEARNED

11. Implementation: Coding
Prioritized business workflows
Coding standards
Tagging
Test coverage
Code review standards
Sprint planning– Write cucumber
Write ruby code
Run and analyze - Jenkins
Run times
Maintainability
0 test errors
Updates
TOOL SELECTION
TOOL EVALUATION
SKILL TRAINING
CODING
QA TEAM CULTURE
TALENT/ HIRING
LESSONS LEARNED

12. Implementation: QA Team Culture
DURING TRANSITION
Shared learning
Support
QA and ruby - Slack
Automation retrospectives
ON-GOING
Cross team collaboration
QA and ruby - Slack
Shared learning
Support
On-going skill development
TOOL SELECTION
TOOL EVALUATION
SKILL TRAINING
CODING
QA TEAM CULTURE
TALENT/ HIRING
LESSONS LEARNED

13. Implementation: Talent Hiring
Skills needed
Technical aptitude
Coding language knowledge
Culture fit
Leadership
ON-BOARDING
Couple of weeks
Domain
Technical
Team
Dedicated time from leads
TOOL SELECTION
TOOL EVALUATION
SKILL TRAINING
CODING
QA TEAM CULTURE
TALENT/ HIRING
LESSONS LEARNED

14. Implementation: Lessons Learned
Big user stories = longer to automate
Too much coverage = Slow run times
End to end automation = brittle
Custom built Domain Specific Language
Duplication of step definitions
Monitoring should be done using monitoring tools – not tests
TOOL SELECTION
TOOL EVALUATION
SKILL TRAINING
CODING
QA TEAM CULTURE
TALENT/ HIRING
LESSONS LEARNED

15. Our Process Pipeline Story: Pick Up Story: Preview
Sprint Planning
Story: Pick Up
Story: Preview
Story: Code and Test
Story: Validation
Story: Deployment
Sprint Retrospectives

16. SPRINT/STORY: PLANNING
Product Detailing
Requirement
Design
Success criteria – First pass
Technical Detailing
Code approach
Testing approach
Success criteria – Refine
PLAN
STORY
PREVIEW
TEST + CODE
VALIDATE
DEPLOY
RETRO

17. STORY: PICK UP
PLAN
STORY
PREVIEW
TEST + CODE
VALIDATE
DEPLOY
RETRO

18. STORY: PREVIEW In depth discussion of the specific story between DevQA UX
PLAN
STORY
PREVIEW
TEST + CODE
VALIDATE
DEPLOY
RETRO

19. STORY: TESTING AND CODING
PLAN
STORY
PREVIEW
TEST + CODE
VALIDATE
DEPLOY
RETRO

20. STORY: VALIDATION
PLAN
STORY
PREVIEW
TEST + CODE
VALIDATE
DEPLOY
RETRO

21. STORY: DEPLOYMENT
Feature deployment to Prod One story corresponds to one deployment.
PLAN
STORY
PREVIEW
TEST + CODE
VALIDATE
DEPLOY
RETRO

22. SPRINT: RETROSPECTIVES
PLAN
STORY
PREVIEW
TEST + CODE
VALIDATE
DEPLOY
RETRO

23. What did we learn?

24. Takeaways Quality strategy with partnership
Select tools that fit your requirement
Dedicated task force
Set coding standards
Develop team culture
Iterative process

25. QA Engineer at ITHAKA/JSTOR
Questions?
Jayshree Bhakta
QA Engineer at ITHAKA/JSTOR
@jayshreebhakta