1. Vendor Landscape: Software Testing Solutions
Testing tools accommodate increasing application complexity in the mobile and cloud space.
Info-Tech's products and services combine actionable insight and relevant advice with ready-to-use tools and templates that cover the full spectrum of IT concerns.© Info-Tech Research Group

2. Introduction
Software testing tools range from point solutions to sophisticated suites. Choose the right tool for your team.
This Research Is Designed For:
This Research Will Help You:
IT Leaders within mid-sized to entry-level enterprises seeking comprehensive software testing functionality.
Use cases that may include:
Improving the quality of software.
Improving efficiency of software testing.
Achieving process maturity.
Align vendor strengths and limitations to your current and projected testing needs.
Determine which products are most appropriate for particular scenarios.
Understand available functionality for current software testing solutions.
Select the vendor that is the best fit for your organization.

3. Guided Implementation points in the Software Testing Vendor Landscape
Book a Guided Implementation Today: Info-Tech is just a phone call away and can assist you with your evaluation. Our expert Analysts can guide you to successful technology selection.
Here are the suggested Guided Implementation points for the Software Testing Vendor Landscape:
Section 1: Shortlist Assistance and Requirements
Get off to a productive start: discuss the market space and how vendors are evaluated. Decide on which deployment option suits you best and narrow down the options based on customized requirements.
Section 2: RFP and Budget Review
Interpret and act on RFP results: review vendor RFPs and ensure the solution is meeting your needs. Discuss average pricing of solutions and what can fit into your budget.
Section 3: Negotiation and Contract Review
Purchase optimization: review contracts and discuss best practices in negotiation tactics to get the best price for your solution.
This symbol signifies when you’ve reached a Guided Implementation point in your project.
To enroll, send an to or call and ask for the Guided Implementation Coordinator.

4. Market overview How it got here Where it’s going
Mobile devices have necessitated a suite of testing tools that can properly address the mobile context, not just traditional testing on a single form factor.
Downward pressure on development cycle times means that virtualization, reusability, and automation, especially for regression, become critical features.
Continued maturity in cloud use cases means that convenience and accuracy can both be achieved through virtual provisioning and teardown of test systems.
With more organizations using Agile for development, reports from testing are an essential component in determining whether a set of failed or incomplete tests are an acceptable business risk for launch.
Applications will continue their trajectory around adding more features to help test complex applications.
Testing complexity will increase as more devices are interconnected.
Device and service virtualization continue to play a role in test cycle reduction.
Business policy and compliance management permit stronger business/development cohesion in the testing phase.
New use cases for cloud and mobile are driving the need for increased business awareness and virtualization capabilities to help properly test applications. The objective now is to test effectively across multiple distributed systems.

5. Software testing vendor selection / knock-out criteria: market share, mind share, and platform coverage
The software testing space continues to address several use cases around cloud and mobile applications. Traditionally, these types of applications would rely exclusively on manual testing, which was not scalable in the long term.
For this Vendor Landscape, Info-Tech focuses on those vendors that offer broad capabilities across multiple platforms and that have a strong market presence and/or reputational presence among mid and large-sized enterprises.
Seapine. A tool aimed at a process-centric testing methodology.
T-Plan Ltd. A test case management and automation solution.
HP. A test management suite that supports the entire SDLC.
Borland. Aimed at addressing the needs of every stakeholder within software testing.
Microsoft. Manual and automated testing tool inside the application testing domain.
Parasoft. Tool that emphasizes automated defect prevention.
IBM. An end-to-end solution with a focus on user experience and consumability.
Included in this Vendor Landscape:

6. Software testing criteria & weighting factors
Product Evaluation Criteria
Criteria Weighting
Features
The solution provides basic and advanced feature/functionality.
Features
Usability
Usability
The end-user and administrative interfaces are intuitive and offer streamlined workflow.
Affordability
Implementing and operating the solution is affordable given the technology.
Affordability
Architecture
Architecture
Multiple deployment options and extensive integration capabilities are available.
Product
Vendor Evaluation Criteria
Viability
Vendor is profitable, knowledgeable, and will be around for the long term.
Vendor
Strategy
Vendor is committed to the space and has a future product and portfolio roadmap.
Viability
Strategy
Reach
Vendor offers global coverage and is able to sell and provide post-sales support.
Channel
Vendor channel strategy is appropriate and the channels themselves are strong.
Channel
Reach

7. Table Stakes represent the minimum standard; without these, a product doesn’t even get reviewed
The Table Stakes
What does this mean?
The products assessed in this Vendor LandscapeTM meet, at the very least, the requirements outlined as Table Stakes.
Many of the vendors go above and beyond the outlined Table Stakes, some even do so in multiple categories. This section aims to highlight the products’ capabilities in excess of the criteria listed here.
Feature
What it is:
Record & Script Tests
Ability to capture a test for replay or modification.
Graphical User Interface
User-friendly interface into software testing product.
Reporting
A reporting suite containing canned reports with the ability to export in multiple formats.
Traceability
Testing solution maps to requirements and links to code, artifacts, and test cases.
If Table Stakes are all you need from your software testing solution, the only true differentiator for the organization is price. Otherwise, dig deeper to find the best price to value for your needs.

8. Advanced Features are the capabilities that allow for granular market differentiation
Scoring Methodology
Advanced Features
Info-Tech scored each vendor’s features offering as a summation of its individual scores across the listed advanced features. Vendors were given one point for each feature the product inherently provided. Some categories were scored on a more granular scale with vendors receiving half points.
Feature
What we looked for:
Record and reuse test scripts across all applications, available as SaaS offering, conduct testing in multiple test labs that are running concurrently; functionality, security, and performance testing of a cloud-based application, automated testing, automatically create test suite from schema.
Cloud Testing
Performance testing (network-related, response time, memory usage testing), security testing (client-side and server-side security), functionality testing (form factor, gestures, internationalization/localization, interruptions), compatibility testing, automated testing, record and reuse test scripts across different applications, testing for mobile applications (web, native, hybrid).
Mobile Testing
Testing for desktop framework APIs (e.g. Java, .NET), cloud SaaS APIs, cloud PaaS APIs, and cloud IaaS APIs.
API Testing
For an explanation of how Advanced Features are determined, see Information Presentation – Feature Ranks (Stoplights) in the Appendix.

9. Advanced Features are the capabilities that allow for granular market differentiation (continued)
Scoring Methodology
Advanced Features
Info-Tech scored each vendor’s features offering as a summation of its individual scores across the listed advanced features. Vendors were given one point for each feature the product inherently provided. Some categories were scored on a more granular scale with vendors receiving half points.
Feature
What we looked for:
Trace business processes to test cases, run business components in business process testing.
Business Process Testing
Creation, execution, and removal of virtual test labs.
Test Lab Management
Ability to simulate the application’s intended environment with virtual data or infrastructure/network images.
Service Virtualization
Requirements Traceability
Errors, queries, defects logged and tracked throughout testing lifecycle, test cases/scripts mapped to high-level requirements, ability to trace progress on resolving issues connected to requirements.
Business Risk Analysis
Ability to perform impact analysis based on test results on the business risk of an application going live.
For an explanation of how Advanced Features are determined, see Information Presentation – Feature Ranks (Stoplights) in the Appendix.

10. Advanced Features are the capabilities that allow for granular market differentiation (continued)
Scoring Methodology
Advanced Features
Info-Tech scored each vendor’s features offering as a summation of its individual scores across the listed advanced features. Vendors were given one point for each feature the product inherently provided. Some categories were scored on a more granular scale with vendors receiving half points.
Feature
What we looked for:
Ability to test the impact of code changes, notifications sent when code changes fail/succeed to respective stakeholders.
Regression Testing
Ability to test individual code modules, notifications sent when testing for code modules fail/succeed to respective stakeholders.
Unit Testing
Ability for users to find the required data using context-based filtering, sorting and reporting, reports offer real-time data based on report criteria, create dashboards to track progress of issue management-related tasks and project activity.
Reporting and Analytics
For an explanation of how Advanced Features are determined, see Information Presentation – Feature Ranks (Stoplights) in the Appendix.

11. Leverage solutions with extensive virtualization capabilities for test lab environments and device types
Use virtualization capabilities as a means of achieving greater test coverage for your applications and to simulate production environments. 1
Virtualization Capacity
Exemplary Performers 2 3
Why Scenarios?
In reviewing the products included in each Vendor LandscapeTM, certain use cases come to the forefront. Whether those use cases are defined by applicability in certain locations, relevance for certain industries, or as strengths in delivering a specific capability, Info-Tech recognizes those use cases as Scenarios, and calls attention to them where they exist.
Well-Suited Candidates
For an explanation of how Scenarios are determined, see Information Presentation – Scenarios in the Appendix.

12. Info-Tech Research Group Helps IT Professionals To:
Quickly get up to speed with new technologies
Make the right technology purchasing decisions – fast
Deliver critical IT projects, on time and within budget
Manage business expectations
Justify IT spending and prove the value of IT
Train IT staff and effectively manage an IT department
Sign up for free trial membership to get practical
solutions for your IT challenges
“Info-Tech helps me to be proactive instead of reactive – a cardinal rule in a stable and leading edge IT environment.
- ARCS Commercial Mortgage Co., LP
Toll Free: