1. How We Do Test Automation at HomeAdvisor
A last-minute production

2. Who Am I? Nicholas Rosevear - QA Automation Architect
At HomeAdvisor for 11 months
Previously worked at Shutterstock, Pearson, IQNavigator and EchoStar

3. A Look at Our Company Customer Facing Application 2 Sided Marketplace
Domestic US/Canada User Base
Dealing With a Rapidly Increasing User Base
Large Scale Integration With Social Media Companies

4. A Look at Our Architecture
Java
Oracle DB & ElasticSearch
Coherence Memory Cache
Kafka Data Pipelines
Node – Front End
DevOps Managed through Jenkins
Google Analytics, Adobe Omniture
Moving to a Massive Microservice Architecture

5. A Look at Our Test Architecture
Java/Spock - Unit/Integration Tests
Java/TestNg/Appium – Mobile Automated Tests
NodeJs/Nightwatch/Mocha – UI Automated Tests
NodeJs/Supertest/Mocha – API Tests
NodeJs/BenchRest - Performance Testing
NodeJs/Express/Casperjs – Performance Testing
NodeJs/Express – Assorted Other Monitoring/Assistance Tools
Java/TestNg/Selenium - Legacy Automated UI Tests
Jmeter – Legacy Performance Tests

6. How Did We Get Here? Deciding Between Needs and Wants
High Test Coverage
Usable/Maintainable Tests
Fast Performance Testing and Trending Analysis
Test Data Management for Complicated Test Setup/Teardowns
Wants
Better CI/CD Infrastructure
Useful Documentation
Simplified Test Execution and Reporting
More Thorough Security/Compatibility Testing

7. So What Did We Do?
Built a new UI framework to replace unstable long running legacy tests
Built a dynamic API test framework to generate tests on its own and enforce consistency
Built easy JSON config file driven performance testing
Built internal Test API and sites to manage data creation/validation across tools and for manual testers

8. So What Did We Do We took advantage of simplicity of Express
Easy to Display and Document
Easy to Train and Teach

9. The Blurred Line Between Automation and DevOps
At some point we drifted out of Test Engineering
Everything can be automated
Tools for the company
Monitoring CI/CD Efforts

10. UI Automation - Nightwatch
Built in NodeJs with Nightwatch
Built to focus on readability and simplicity to create
And easy to understand output and test results
Built to consider switching technologies

11. Mobile Automation – Unnamed Page Object
Built as a One-Off for the Mobile team
Basic Page Object style framework built on top of Appium
Same concepts as Nightwatch
Easy to use
Maintainable
Flexible across mobile devices

13. API Automation – Api-Tite
We are dealing with 500+ unique API Endpoints
Legacy and New Endpoints
Multiple teams, multiple standards, multiple test coverage
Created a framework to sit on top of swagger and build a database of all APIs and dynamically generate tests
Readability not prioritized
Guaranteed Coverage
Immediate Test Detection
Security Coverage
Ability to enforce consistency

15. Performance Testing - Scattershot
Had a couple of close calls with performance issues
Rapid development cycle means a need for performance testing with little advanced notice
Built a framework built around using simple config files to generate tests
Tracking performance across different numbers of concurrent connections and iterations
Allowing dynamic setup for testing to test more difficult endpoints like DELETE
Long term analytical engine for trending and comparison

17. QA/Automation Support - Zamboni
With more tools we were running the risk of duplicated effort
There were security concerns and other testing efforts that required quick access to data
We created a tool
Our own test server complete with
Endpoints for data creation/validation
Abstraction of common DB queries/API calls to consolidate and visualize information
Ability to quickly add support for testing efforts

19. Performance Testing - Ichabod
We want to do more exotic testing
Chaos and Disaster Recovery Testing
Realistic Load-Runner style load testing
We Built A Tool
To allow a network of machines to generate load on an environment
Using UI testing scripts made headless for efficiency
Manageable from a single hub site
Easy to expand scripts and configurable to choose what type of load to add to the system
Real Time results and updateability

20. Reporting/Monitoring – Jenkins Crawler
We have a lot of tests running
Across a number of Jenkins jobs
Determining overall health or specific health for certain applications teams is difficult
Building a Jenkins Crawler
Pulls data from various Jenkins jobs and stores
To allow analysis of success, number of test pass/fail historical accuracy
To allow simple reporting for overall health based on Jenkins results
Potentially to implement full CI/CD pipelines

21. QA And Automation Support - HostManager
Finally just a quick example of using automation to solve problems
Created a simple host server to manage host files across company
Gives us a way to guarantee everyone has the same host entries
Allows automation to be able to change hosts remotely

22. So What’s Next? Docker, docker, docker Integration with Sauce Labs
Simplified Test Creation – No Programming Required
RabbitMQ message queues to manage jobs, data creation and reporting
A specific security testing suite
Maybe switching technologies
And of course more dashboards