Jeff Tonkovich

Today’s Topic A (brief) comparative analysis of Excel and TestLink as Test Management Tools across five categories: Setup Usability Reporting Efficiency Unique Features

Research Methodology 30 minute interviews with 5 engineers with hands-on experience with both Excel and TestLink Used standard questionnaire with each engineer plus an open-comments section Obtained access to over 100 presentation slides for previous effort to transition test from Excel to a formal Test Management System (TMS) TestLink was the eventual winner

Setup Excel Deployment time varies but is generally very low May be longer if deploying via SharePoint w/ user accounts Manual population of test cases / requirements TestLink Web-based w/ central DB Requires a server with Apache, MySql, PHP & Testlink Deployment time varies and increases with project size Import test cases, setup user accounts, import requirements Approx. 2 hours for medium size project

Usability Excel Used extensively in school, minimal learning-curve, extensive help and documentation available Not ideal for multiple-user scenarios Merging worksheets, check-in/out via SharePoint TestLink Perfect for collaboration between multiple testers User Management: able to assign roles users E.G. Test Lead, Tester, Program Manager, etc Poor documentation, particularly for new features Rule of thumb: 30+ requirements: TestLink, otherwise Excel

Reporting Excel Very simple to generate test reports with graphs Pass/fail statistics, weekly reports, etc TestLink No major gain over Excel for simple tasks Built-in report generators Auto-generated report showing requirements missing TCs Customizable via PHP RVTM auto-generated with click of a button!

Efficiency TestLink Rapid Test Report generation “days” -> “hours” Huge time savings in requirements maintenance 4x time savings vs Excel Rapidly generate Test Plans (TP) TPs for different platforms, functionality (e.g. UI), etc Test reporting less prone to human error Bugs in Excel Macros, inconsistent reporting, forgot who tested what, etc

Unique Features Excel Simple to use but amazingly powerful Customizable via macros, VB script and add-ons TestLink notifications Notify test engineers which test cases / suites are assigned to them Version control Test Cases and requirements Metadata tagging Bugzilla integration

Summary All interviewees unanimously agreed that TestLink is superior to Excel for Test Management Exceptions for very small projects and projects using Excel that are almost over Use in CS577 Projects are small and overhead may not be worth it Mitigate by setting up / configuring prior to semester start Auto-generate RVTM / Traceability Matrix (portions) Fully-functional Demo available on TestLink website