CS 160: Software Engineering November 24 Class Meeting Department of Computer Science San Jose State University Fall 2014 Instructor: Ron Mak

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 2 Review: Unit Testing  Each unit test focuses on components created by a developer.  Developers create test cases to do unit tests. Done by a developer before committing the code to the source repository. Easier to find and fix bugs when there are fewer components. Bottom-up testing.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 3 Review: Web Application Testing  Automate website testing with a Java program.  The program enters values into the text fields, chooses menu items, clicks on links, and presses buttons. The server code (servlets) don’t know that it’s talking to a program.  Use the open-source tool HttpUnit Emulates browser behavior, including form submission. Download from

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 4 Review: Regression Testing  Make a collection of unit tests.  Run all the tests whenever you make a code change. Make sure your application did not lose functionality or “regress”. Make sure a bug fix did not introduce new bugs.  Programmers are much less reluctant to improve their code if they can run regression tests to verify their changes.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 5 Review: Stress Testing  Push your application until it breaks.  Know what the breaking point is. If you don’t find out before deployment, the users will!  Understand the behavior of your application under stress. Recognize when it’s about to break so you have time to mitigate.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 6 Integration Testing  After your code has passed all the unit and regression tests, you must do integration testing to see how your unit works with other units  Integrate other units either written by you or by other developers.  Not too hard to do if you’ve been doing continuous integration all along.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak Profiling  Collect statistics about your program execution. How long does your program spend in each method? How many bytes and how many objects of each type does your program create?  NetBeans has a built-in profiler. Read the documentation at Demo

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 8 Usability Testing: The German Newton  The Apple Newton MessagePad was an “iPad” developed in the early 1990s. Besides the English version, there was a German and a Japanese version.  It was too far ahead of its time and was killed by Steve Jobs shortly after he returned to Apple.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 9 Usability Testing: The German Newton, cont’d  A key feature of the Newton was its handwriting recognition.  The Newton recognized successive words in a sentence using an algorithm that tracked the movement of the stylus. Not by optical pattern recognition.  Handschrifterkennung in the German version.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 10 Usability Testing: The German Newton, cont’d  Cultural differences between Americans and Germans.  The way the Germans write their letters (e.g., the letters h and J).  The way the Germans write words (e.g., Geschäftsreise “business trip”).  Philosophy about personal handwriting.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 11 Beta Testing: NASA Mars Rover Mission  Beta testing for the Collaborative Information Portal (CIP) software system.  NASA’s Jet Propulsion Laboratory (JPL) conducted several Operational Readiness Tests (ORTs) before the actual rovers landed.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 12 Beta Testing: NASA Mars Rover Mission, cont’d  Operational Readiness Tests (ORTs) A working rover was inside of a large “sandbox” in a separate warehouse building at JPL. Mission control personnel communicated with and operated the sandbox rover as if it were on Mars.  A built-in delay simulated the signal travel time between Earth and Mars. Mission scientists accessed and analyzed the data and images downloaded by the sandbox rover.  All mission software, including CIP, was intensely tested in this simulated environment.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 13 System Testing  Test the entire application in its operating environment. Rerun all regression tests Installation testing Performance testing Stress testing Usability testing  Part of beta testing

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak Acceptance Testing  Can be performed by the clients.  Signoff by stakeholders, clients, and customers. 14

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 15 When can you stop testing?  “Testing can prove the presence of bugs, but never their absence.”  Stop testing when: All the regression tests pass. Testing finds only “acceptable” bugs.  Put on the Known Bugs list.  Have workarounds. When you’ve run out of time.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 16 Test Plan  Part of the product design documentation.  Description of each functional, performance, and stress test case. Each test case should be traceable to a functional or nonfunctional requirement or to a use case.  How the unit tests will be generated.  Description of each regression test script.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 17 Test Plan, cont’d  How to run each test: Input data Manual operations (text input, menu selection, button press, etc.) Expected output.  Acceptance criteria (success/failure).

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 18 Test Plan, cont’d  Integration testing procedure.  Description of each usability test. Who will be the guinea pigs?  Description of each system test. What are the beta test sites?  Description of each acceptance test. Signoff checklist

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 19 Example Test Case Specification  For the “Antique Square Root Engine” website.  Name Compute Square Root of Nonnegative Number  Purpose To test that the engine can compute the square root of a value that is zero or positive.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 20 Example Test Case Specification, cont’d  Manual operations 1. Enter 0 or a valid positive number into the text field.  The number may have a + sign and a single decimal point.  The maximum value of the number can be Press the Submit button.  Expected results THE SQUARE ROOT OF number IS sqrt where number is the number entered into the text field and sqrt is the correct square root of the number.  Acceptance criteria The square root is accurate to within 8 significant digits.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 21 Test-Driven Development (TDD)  A programming practice originally from the Extreme Programming (XP) methodology but which is starting to catch on elsewhere.  Basic idea: Before adding a new feature to your application, first write a test case for it. Can be a JUnit test case. Of course, the test will fail, because you haven’t written the code for the feature yet.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 22 Test-Driven Development (TDD), cont’d  Writing the test case first forces you to: Understand what the feature is supposed to do based on its requirements and use cases. Design a good API for the feature. … before writing the code.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 23 Test-Driven Development (TDD), cont’d  “Traditional” agile programming cycle:  TDD programming cycle: TEST CODE COMMIT DESIGN TEST CODE COMMITDESIGN  Your goal as a developer:  Make your code pass its test.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 24 Logging  Add special logging statements to your application. Generate output to a log file. Can be just simple print statements.  Each logging statement logs (writes) a message containing useful information about the application’s execution. “I am here.” – Execution reached this point. Status information at that point. The log file provides a history of the application’s execution.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 25 Logging, cont’d  Logs are useful if a failure occurred. Status of the erroneous condition leading up to the failure. Status at the point (or just before) the failure occurred.  Turn logging off before deployment, or leave it on. Perform “log mining” to glean useful information about how the application is used and how it’s behaving.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 26 Logging Package: Log4J  Open-source logging services package for Java. Download from  Add logging statements to your application without incurring a heavy performance penalty.  Logging levels: DEBUG < INFO < WARN < ERROR < FATAL Set a logging level and only log messages at that level or greater. Example: If the logging level is WARN, then you will see only WARN, ERROR, and FATAL messages.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 27 Logging Package: Log4J, cont’d  Hierarchical logging. Logger names like Java packages: X, X.Y, X.Y.Z, etc. Loggers inherit logging levels from their parents and ancestors. Each logger “rolls up” its messages to its parent.  Send logging messages to appenders. Consoles (standard out), files, remote socket connections, etc. Can have multiple appenders per logger.

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 28 Logging Package: Log4J, cont’d  Control logging externally from the configuration file log4j.properties. Logging level Loggers and appenders Layout (format) of the logging messages

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 29 Log4J Demo package cs160; import org.apache.log4j.Logger; public class Calculator { static final String CLASS_NAME = Calculator.class.getName(); static Logger mainLogger = Logger.getLogger(CLASS_NAME); static Logger addLogger = Logger.getLogger(CLASS_NAME + ".add"); static Logger subtractLogger = Logger.getLogger(CLASS_NAME + ".subtract");... }

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 30 Log4J Demo, cont’d public double add(double first, double second) { addLogger.info("Called add"); addLogger.debug("first=" + first + ", second=" + second); return first + second; } public double subtract(double first, double second) { subtractLogger.info("Called subtract"); subtractLogger.debug("first=" + first + ", second=" + second); return first - second; } DEBUG < INFO < WARN < ERROR < FATAL

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 31 Log4J Demo, cont’d public static void main(String args[]) { mainLogger.info("Start application."); Calculator calc = new Calculator(); double result = calc.add(40, 30); mainLogger.debug("add() returned " + result); result = calc.subtract(40, 30); mainLogger.debug("subtract() returned " + result); mainLogger.info("End application."); } DEBUG < INFO < WARN < ERROR < FATAL

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 32 log4j.properties LOGGER Calculator LOGGER Calculator.add LOGGER Calculator.subtract roll up messages # Loggers log4j.logger.Calculator=INFO,Console,Main log4j.logger.Calculator.add=DEBUG,Add log4j.logger.Calculator.subtract=DEBUG,Subtract DEBUG < INFO < WARN < ERROR < FATAL

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 33 Log4J Demo, cont’d # Appenders log4j.appender.Console=org.apache.log4j.ConsoleAppender log4j.appender.Main=org.apache.log4j.FileAppender log4j.appender.Main.File=main.log log4j.appender.Add=org.apache.log4j.FileAppender log4j.appender.Add.File=add.log log4j.appender.Subtract=org.apache.log4j.FileAppender log4j.appender.Subtract.File=subtract.log LOGGER Calculator APPENDER Main main.log APPENDER Add add.log APPENDER Subtract subtract.log LOGGER Calculator.add LOGGER Calculator.subtract APPENDER Console roll up messages

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 34 Log4J Demo, cont’d # Layouts log4j.appender.Console.layout=org.apache.log4j.PatternLayout log4j.appender.Console.layout.ConversionPattern=%d %-5p %c - %m%n log4j.appender.Main.layout=org.apache.log4j.PatternLayout log4j.appender.Main.layout.ConversionPattern=%d %-5p %c - %m%n log4j.appender.Add.layout=org.apache.log4j.PatternLayout log4j.appender.Add.layout.ConversionPattern=%d %-5p %c - %m%n log4j.appender.Subtract.layout=org.apache.log4j.PatternLayout log4j.appender.Subtract.layout.ConversionPattern=%d %-5p %c - %m%n LOGGER Calculator APPENDER Main main.log APPENDER Add add.log APPENDER Subtract subtract.log LOGGER Calculator.add LOGGER Calculator.subtract APPENDER Console roll up messages Demo

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 35 Log4J Example: Mars Rover Mission :23:33,435 INFO : mjane: Streamer.putDataFile(/opt/bea/user_projects/cip/ conf/preferences/m.preferences) :23:33,439 DEBUG: Begin upload of file '/opt/bea/user_projects/cip/conf/ preferences/sthompso.preferences' :23:39,140 DEBUG: Completed upload of file '/opt/bea/user_projects/cip/conf/ preferences/sthompso.preferences': bytes '/opt/bea/user_projects/cip/conf/global.properties': bytes :28:57,516 INFO : jdoe: Streamer.getPreferences(user) :29:29,721 INFO : jdoe: Streamer.getFileInfo() :29:30,784 INFO : jdoe: Streamer.getFileInfo(/oss/merb/ops/ops/surface/ tactical/sol) :17:43,320 INFO : jqpublic: Streamer.getDataFile(/global/nfs2/merb/ops/ops/ surface/tactical/sol/120/opgs/edr/jpeg/1P ETH2809P2845L2M1.JPG) :17:43,324 DEBUG: Begin download of file '/global/nfs2/merb/ops/ops/surface/ tactical/sol/120/opgs/edr/jpeg/1P ETH2809P2845L2M1JPG' :17:44,584 DEBUG: Completed download of file '/global/nfs2/merb/ops/ops/ surface/tactical/sol/120/opgs/edr/jpeg/1P ETH2809P2845L2M1.JPG': 1876 bytes  Timestamp, user name, method name  Information about the uploaded or downloaded file

Computer Science Dept. Fall 2014: November 24 CS 160: Software Engineering © R. Mak 36 “Log Mining” Example: Mars Rover Mission