Quality Assurance CS Pace University Jim Leonardo Milo Auguste Jr. Ritu Mehrotra

Mission Statement The Quality Assurance team provides assurance to stakeholders in CS- 615/616 projects that their projects meet their requirements for correctness, usability, reliability, and maintainability The Quality Assurance team provides assurance to stakeholders in CS- 615/616 projects that their projects meet their requirements for correctness, usability, reliability, and maintainability

General Tasks Establishment of Best Practices Establishment of Best Practices Testing for Correctness Testing for Correctness Concurrency Testing for Server Based Applications Concurrency Testing for Server Based Applications Assuring Maintainability Assuring Maintainability Evaluating Usability Evaluating Usability Tracking of Compliance to All of the Above Tracking of Compliance to All of the Above

Best Practices Guidelines Guidelines When applicable, RDBMS databases should be used. Local databases such as MS Access should be avoided (Performance, Reliability) When applicable, RDBMS databases should be used. Local databases such as MS Access should be avoided (Performance, Reliability) If a given language makes it optional to force variable declaration, it should be turned on. (Reliability, Maintainability) If a given language makes it optional to force variable declaration, it should be turned on. (Reliability, Maintainability) Code should be commented before or during typing. (Maintainability) Code should be commented before or during typing. (Maintainability)

Guidelines continued Guidelines continued When public classes are created, all public interfaces should be documented separately from internal code documentation. (Maintainability, Usability) When public classes are created, all public interfaces should be documented separately from internal code documentation. (Maintainability, Usability) Browser based applications should be compatible with more than one browser (Reliability, Usability) Browser based applications should be compatible with more than one browser (Reliability, Usability) Any user input should be pre-filtered to prevent database errors (Reliability, Usability) Any user input should be pre-filtered to prevent database errors (Reliability, Usability) Best Practices

Guidelines continued Guidelines continued Ensure that you can’t get lost on web pages… You should always be able to navigate the website as easily as navigating any application. Ensure that you can’t get lost on web pages… You should always be able to navigate the website as easily as navigating any application. Websites requiring login should also allow logout and should redirect any restricted page back to the login screen if you are not currently logged in. Websites requiring login should also allow logout and should redirect any restricted page back to the login screen if you are not currently logged in. Fields, buttons, etc. should be labeled consistently Fields, buttons, etc. should be labeled consistently

Correctness Testing Step 1: Review of requirements/design documents Step 1: Review of requirements/design documents Step 2: Use Case Testing Step 2: Use Case Testing Step 3: Break/Load testing. Step 3: Break/Load testing.

Technologies Pre-made software Pre-made software OpenSTA automated virtual user scripting and load testing for HTTP/HTTPS OpenSTA automated virtual user scripting and load testing for HTTP/HTTPS Mozilla general purpose web browser used for secondary testing Mozilla general purpose web browser used for secondary testing

Questionnaire Result Highlights Determining and Setting up technology took on average 2.7 weeks, median was 2 weeks…or 5 weeks if you drop out the projects with strong tech. requirements. Determining and Setting up technology took on average 2.7 weeks, median was 2 weeks…or 5 weeks if you drop out the projects with strong tech. requirements. Arcview, MySQL, J2ME, Matlab, and Java were specifically requested to be used by the customers. Arcview, MySQL, J2ME, Matlab, and Java were specifically requested to be used by the customers. Our projects are using such technologies as PHP, Oracle, MySQL, Java, Apache, ColdFusion, C++, and Perl…and just about everything else. Our projects are using such technologies as PHP, Oracle, MySQL, Java, Apache, ColdFusion, C++, and Perl…and just about everything else. At the middle of this semester, only about 45% of the projects felt that they were more than 80% complete. At the middle of this semester, only about 45% of the projects felt that they were more than 80% complete. Everyone felt that they supplied 70% or more of the requirements for the project to the customer, not the other way around. Everyone felt that they supplied 70% or more of the requirements for the project to the customer, not the other way around. Half of the projects had settled on more than 70% of their requirements at the end of the first semester. Half of the projects had settled on more than 70% of their requirements at the end of the first semester. More than 60% of project teams felt that there would be additional work beyond general maintenance required at the end of the semester) More than 60% of project teams felt that there would be additional work beyond general maintenance required at the end of the semester)

Case Studies System Web Based? Phone Based? GenealogyX Absentee System XX Yellow Pages X Complaint Desk X

Genealogy System First testing and feedback in November/December First testing and feedback in November/December System was entirely rewritten since the original tests were performed System was entirely rewritten since the original tests were performed Seven major issues uncovered in the first version, these were not repeated in the second version. This demonstrates the value of early testing. Seven major issues uncovered in the first version, these were not repeated in the second version. This demonstrates the value of early testing. Despite early testing, more issues were uncovered, demonstrating the need for ongoing testing. Despite early testing, more issues were uncovered, demonstrating the need for ongoing testing.

Absentee System Original testing in November/December revealed three errors, these were addressed shortly thereafter. Original testing in November/December revealed three errors, these were addressed shortly thereafter. QA was able to identify several areas of the old ( ) system that were not in conformance with best-practices. QA was able to identify several areas of the old ( ) system that were not in conformance with best-practices. A redesigned web-interface was introduced A redesigned web-interface was introduced Reasons for this: Usability/Cosmetic Appeal and Conformance. Reasons for this: Usability/Cosmetic Appeal and Conformance. QA verified that no further errors were introduced and best-practices related issues were addressed during this transition. QA verified that no further errors were introduced and best-practices related issues were addressed during this transition.

Yellow Pages This was a new undertaking for Spring ’03 This was a new undertaking for Spring ’03 Testing performed using both “corded”, cordless and cellular telephones. No noticeable impact was observed as a result of the phone type. Testing performed using both “corded”, cordless and cellular telephones. No noticeable impact was observed as a result of the phone type. Unable to perform concurrency verification due to limit of 1 phone line into the system. Unable to perform concurrency verification due to limit of 1 phone line into the system. The overall quality of this system was impaired by the (third party) speech engine that was used. It had a number of problems with recognizing some words and needed to be spoken to very slowly. The overall quality of this system was impaired by the (third party) speech engine that was used. It had a number of problems with recognizing some words and needed to be spoken to very slowly.

Complaint Desk This was a new project to Spring ’03, it was hoped it would share much with a previously developed help desk system. This was a new project to Spring ’03, it was hoped it would share much with a previously developed help desk system. The team generally avoided repeating the same types of errors from the prior system, but there were a few that did creep in. The team generally avoided repeating the same types of errors from the prior system, but there were a few that did creep in. Errors that were repeated centered mostly around data validation items like checking length of text that needed to be implemented on new fields. Errors that were repeated centered mostly around data validation items like checking length of text that needed to be implemented on new fields.

Conclusions Too much time was spent determining technologies to be used…this could be alleviated by standardizing. Too much time was spent determining technologies to be used…this could be alleviated by standardizing. Standardizing would also allow for component reuse between projects and successive years. (No need for everyone to create their own login form) Standardizing would also allow for component reuse between projects and successive years. (No need for everyone to create their own login form) A forum for knowledge sharing should be created and utilized. A forum for knowledge sharing should be created and utilized. One more milestone should be added to each semester for code delivery. One more milestone should be added to each semester for code delivery. QA could be provided in the future by requiring teams to cross test each others’ systems. QA could be provided in the future by requiring teams to cross test each others’ systems.