1. Diego Lo Giudice, Vice President, Principal Analyst
Webinar Continuous Testing In Agile Changes The Testing Tools Landscape
Diego Lo Giudice, Vice President, Principal Analyst
October 21, Call in at 10:55 a.m. Eastern time

2. As software applications become more differentiating business assets . . .
WHY ?
Image source: CompareBusinessProducts.com (
. . . development and testing tools evolve and the way you use them too !

3. Agenda
New practices of Agile testing improve quality and speed: How?
How do testing tools evolve to support the new testing trends?
Emerging and renewed testing practices and tools

4. Agenda
New practices of Agile testing improve quality and speed: How?
How do testing tools evolve to support the new testing trends?
Emerging and renewed testing practices and tools

5. SPEED !! Faster time-to-market!! Faster and more frequent delivery
Image source: Fotolia (
Faster time-to-market!!
Faster and more frequent delivery

6. Bugs in production biggest quality issue Wait time for testing
Optimized test coverage needs
Test environments provisioning
. . .
Image source: Nanophase Technologies (

7. The arrival of modern applications won’t make things better!
Source: February 13, 2012, “Mobile Is The New Face Of Engagement” Forrester report

8. One-size AD&D will no longer fit all!
Longer AD&D cycle, stable requirements, slow innovation
Short AD&D cycle, business feedback, change, innovation
Testing practices, process, and tools as we’ve known them won’t work any longer!
Agile, continuous
delivery
Source: February 13, 2012, “Mobile Is The New Face Of Engagement” Forrester report

9. End-to-end integration
For Agile development: Testing needs to be done continuously, early, and fast!
End-to-end integration
Integration and integration testing start early on; test data fed continuously, performance testing can’t be done only at the end.
Source: January 15, 2013, “Consistent Performance In Agile Teams Must Include Testing” Forrester report

10. Testing shifts left. . . . an old concept comes true.
Testing involved early on from sprint 0
Definition of done includes precise requirements for testing !
Functional and nonfunctional test and process automation
Performance, test data management, integration
Leaner test management with UATD, TDD, and BDD
Exploratory testing for remaining manual testing

11. Less focus on GUI automation Automation beyond the GUI API automation
Stronger need for higher levels of automation gets developers more involved in testing.
Less focus on GUI automation
Automation beyond the GUI
API automation
Services automation
Better design and reuse
SDLC to develop automation tests
A task for developers
Image source: HAAP Media (

12. In three to five years TCoEs, as we know them, will change: shifting to become practice centers
Focus of TCoE shifts to practices for:
More automation.
Performance-load testing.
Provisioning fast/better test environments.
Provisioning test data.
Provisioning virtualized assets.
Provisioning SVT solutions.
Shifting testing to the left.
Developers pick more testing up.
TPCoE: testing practice center of excellence
Or:
Community of practice
Source: January 15, 2013, “Consistent Performance In Agile Teams Must Include Testing” Forrester report

13. Agenda
New practices of Agile testing improve quality and speed: How?
How do testing tools evolve to support the new testing trends?
Emerging and renewed testing practices and tools

14. True Agile requires tools that enable the whole dev-test-ops life cycle
“Which of the following testing and release management practices does your development team currently use?” (Select all that apply)
Unit testing
58%
Exploratory testing
20%
Performance/load testing
38%
Automation/regression testing
30%
Continuous integration with multiple weekly builds
32%
Base: 698 North American, European, and Asian professional software, internal IT, game developers, and consultants; Source: Forrsights Developer Survey, Q1 2013

15. And tools must get developers more involved in testing!
“How often do you use the following tools when you develop software/applications?”
Tech-testing tools
Base: 2,038 North American, European, and Asian software developers; Source: Forrsights Developer Survey, Q

16. Five “must-haves” on testing tools to support continuous testing
Simplicity
Continuous testing
Automation
Image source: Wikibon ( Fotolia (
Integration

17. The testing tool landscape
Test management
Unit testing frameworks
TDD/BDD/
UATDD
Test automation tools
Service virtualization and testing
Performance/load
Defect and issue tracking
Test data management
Technical
code
quality

18. The landscape of testing tools is becoming more crowded and dynamic
Source: July 18, 2013, “Navigating The Agile Testing Tool Landscape” Forrester report

19. The landscape of testing tools is becoming more crowded and dynamic (cont.)
Source: July 18, 2013, “Navigating The Agile Testing Tool Landscape” Forrester report

20. The landscape of testing tools is becoming more crowded and dynamic (cont.)
Source: July 18, 2013, “Navigating The Agile Testing Tool Landscape” Forrester report

21. Agenda
New practices of Agile testing improve quality and speed: How?
Testing tools evolve to support the testing trends.
Emerging and revival of testing practices and tools

22. Testing tool landscape: emerging and revival
Test management
AUTOMATION
Unit testing frameworks
TDD/BDD/
UATDD
Test automation tools
Service virtualization and testing
25% < 10% automation
38% < 20% automation
30% between 30% to 50%
12% above 80
Performance/load
Defect and issue tracking
Test data management
Technical
code
quality

23. What is service virtualization?
Forrester definition:
The practices and tools of simulating interfaces, APIs, services, and behavior of application, data, and infrastructure to help application development teams and testers perform early virtual integration testing and comprehensive functional and nonfunctional testing (e.g., performance testing) and source test data all in an automated way

24. Integration testing in complex environments is part of the challenge of today’s composite apps
Web browser
Mobile apps
Tablet Internet apps
Composite
applications
Existing infrastructure
Applications under test
SWIFT
SAP/
legacy
LDAP
SOAP
Rest DB
JDBC MQ
Mainframe
Web services
Security server
Application
services
Online cloud third-party services
INTEGRATION TESTING
Unavailable for testing or
expensive to use for testing

25. More testing challenges addressed by SVT . . .
Increase
levels of automation.
Performance testing presents many struggles.
Test environments are an issue in many ways.
Creating and provisioning test data is ad hoc or not done.
Integration testing is hard, just getting harder.
Developers develop ad hoc stubs, which don’t necessarily mimic production precision and are not reusable.

26. Virtualization and testing market
Watch out for the Forrester Wave™ on service virtualization and testing tools, coming Q
Grid-Tools
Shunra
SmartBear
IBM CA HP
Parasoft
Services
Network
Data
Open source stubs: Mockito, jMock

27. Testing is the weak link between dev-and-ops but SVT . . .
“Le trade-d’union”
SVT
Dev
Dev
Ops
Ops
Testing
Testing

28. Test data management: the Cinderella of testing . . .
Test data cranked together ad hoc by developers/testers
Test data provisioned in a secure way, sub-setted, masked from production
Have a strategy around TDM; don’t make it happen “ad hoc.”
Low cost dev
Tactical
Poor quality
High cost dev
Skills lack
Slow
High cost for business
Poor quality
Higher quality
Compliance
Tools requirement:
Look at the five musts.
Source: July 18, 2013, “Navigating The Agile Testing Tool Landscape” Forrester report

29. Diego Lo Giudice

30. Our Agile and Lean Playbook has a wealth of additional information on testing
Source: Forrester Research, The Agile And Lean Playbook