Testing and Monitoring at Penn Testing and Monitoring Model-based Generated Program Li Tan, Jesung Kim, and Insup Lee July, 2003

Testing and Monitoring at Penn Outline 1. Motivations 2. Overview of our methodology 3. Creating tester 4. Generating monitor 5. Case Study and Performance Issues 6. Related Work and Future direction

Testing and Monitoring at Penn Motivations Challenges of testing and monitoring an implementation with respect to its model. 1. Testing and monitoring integrated hardware/software platforms, e.g., embedded systems. 1. The architecture of platforms are different from general-purpose computing environments: Industrial processors, real-time operation systems. 2. Limited resources. 3. Testing and monitoring tasks should be executed in real systems. 2. Reducing the development cost of monitoring/testing platform. 3. Generating tester and monitor from the model and its requirement specification.

Testing and Monitoring at Penn Our goal Building self-testing and self-monitoring executable programs. Given, A model-based code generator for hybrid automata. Targeted platform: Sony robotic dog, a typical embedded system. Yield, A miniaturized tester and monitor directly works on the targeted platform. The ability to synthesize concrete tester and monitor from the high-level specification. The path Modeling testing and monitoring tasks. Generating tester and monitor from the models.

Testing and Monitoring at Penn Overview of our approach Model monitoring and testing jobs …… Generating monitors and testers from models!

Testing and Monitoring at Penn Modeling testing task Testing task is modeled as a deterministic hybrid automata. Textual Description of Testing Task Testing automata System Model + Simulator/ Coverage Checker Not meet requirement Refine test automata Code Generator Embedded Tester

Testing and Monitoring at Penn Testing Sony Dog: requirements 1. Testing platform: 2. Testing requirements can be a combination of coverage criteria and system properties. 1. (Coverage-based Testing) Testing should cover all the modes in system model (mode coverage). 2. (Specification-based Testing) Dog should not lose the track of ball if ball is visible. 1. Practically, the dog will lose track of the ball if the ball moves too fast, 1. What is the threshold of ball’s speed? 2. What is the reaction of the dog?

Testing and Monitoring at Penn Modeling requirements 2.Translate to a two-mode hybrid automata, 3.Simulator/Coverage Checker is used to refine testing automata and check the required coverage criteria 1.What value of a, b, c, d will make testing meet the requirement? 2.Simulation/coverage checker will execute the models on the simulation level and check the coverage. Vision=false Vision=true Position of ball: Pos=d ¢ sin(a t 2 +b t+c)

Testing and Monitoring at Penn Modeling Monitor Monitoring task is modeled as a deterministic hybrid automata. Textual Description of System properties Deterministc Time automata Hybrid Automata Code Generator Monitor

Testing and Monitoring at Penn Monitoring Sony Dog System specification: 1. Dog should not lose the track of ball if ball is visible. 1. Dog doesn’t lost the ball: |ball-head| < Dog should been given a fair chance to make its efforts: |ball-head|<10 five seconds after the ball is visible. Monitoring automaton is a deterministic timed automaton. Vision indicates visibility of the ball, and 10 is the threshold of a “visible” ball

Testing and Monitoring at Penn Synthesizing monitor Translate to hybrid automaton Time is handled as linear differential equation Generating monitor from the hybrid model.

Testing and Monitoring at Penn Put it together: a case study Monitoring automaton System Model Testing automaton Modular compilation MonitorGenerated CodeTester Link as needed

Testing and Monitoring at Penn Performance Issues 1. Testing and monitoring can be simulated on model level. 2. The space overhead of tester and monitor.

Testing and Monitoring at Penn Conclusions We proposed a framework of generating tester and monitor via code generation: 1. It allows the rapid-prototyping of testing and monitoring programs. 1. Monitor and tester tasks are specified in the high-level modelling language. 2. Tester and monitor can be re-targeted to a different platform for which code generation process is available. 2. Testing and monitoring may be preformed both on simulation level and implementation level. 1. Tester and monitor are executable on targeted platform. 3. Resource saving, 1. Each tester and/or monitor is a customer job. 2. (for modularity compilation) tester and monitor can be linked and applied as needed

Testing and Monitoring at Penn Related works and future directions 1. Related works: “platform-specific” approach v.s. “general purpose” approaches. 1. NASA Ames: Java PathExplorer 2. UPenn: MaC tools 3. Works on synthesizing finite model/test-oracle from the formal specification [DilRam96,GiaHav01] 2. Ongoing and future researches 1. Incorporating existing specification language to the framework. 1. MEDL to CHARON translator 2. Approximating general linear temporal property as monitoring automaton. 3. Testing and monitoring the code from conventional sources.