1. 1 RealProct: Reliable Protocol Conformance Testing with Real Nodes for Wireless Sensor Networks Junjie Xiong, Edith C.-Ngai, Yangfan Zhou, Michael R. Lyu 2011-11-18

2. 2 Outline Motivation RealProct Design Evaluation Conclusion

3. 3 Motivation Current main methods to fight back the software bugs in Wireless sensor networks (WSNs): Simulation: different from real execution. Testbeds: designed for network performance evaluation rather than software bug detection. Large-scale real deployment: expensive. uses a small number of real sensor nodes to mimic large-scale WSNs and test the protocol implementation against the specification -> RealProct

4. 4 Outline Motivation RealProct Design Evaluation Conclusion

5. 5 Challenges Sensor node is difficult to control than a computer.  Limited CPU and inconvenient interface. How to test the protocol with various topologies and events when RealProct only employs a few real sensor nodes. Volatile wireless environment in WSNs will lead to random packet loss, and cause problems in testing.

6. 6 RealProct Solutions to the Challenges An architecture that enables testing with real sensor nodes. Topology virtualization and event virtualization. Dynamic Test Execution.

7. 7 Background of RealProct Protocol conformance testing (PCT) process  IUT (Implementation Under Test)

8. 8 RealProct Architecture SUT (System Under Test) Lower Tester Upper Tester Point of Control & Observation

9. 9 Topology Virtualization Use the tester to virtualize a 3-node topology for SUT

10. 10 Event Virtualization Use the tester to create a packet disorder event at the SUT.

11. 11 Reason to Use Dynamic Test Execution Suppose packet loss probability is L 0, a test case is executed n times, and it passes n 1 times and fails n 2 times. If n 1 > n 2, declare as pass, the FN (false negative) probability is If n 1 < n 2, declare as fail, the FP (false positive) probability is

12. 12 Dynamic Test Execution To guarantee that both the FN and FP error rates are lower than a required value E, the minimum time to execute each test case is: The actual execution times are dynamic.  Repeat the test case execution until its FN and FP error rates are lower than E.

13. 13 Outline Motivation RealProct Design Evaluation Conclusion

14. 14 Performance Evaluation Two real TelosB sensor nodes and a PC. Contiki 2.4: μIP TCP/IP protocol. Two new bugs and previous bug repetition. Two new bugs and previous bug repetition.  Bug 1 & 2 – Connect to opened & unopened TCP ports.  Bug 3 – SYN/ACK packet loss.  Bug 4 – SYN packet duplication.

15. 15 Performance Evaluation Bug 1 – Connect to opened TCP ports.  Test opened port 0 & 80 (within 0 to 65535).

16. 16 Performance Evaluation Bug 1 – Client (Tester) connects to opened TCP port 0 of Server (SUT). Bug: Client expects SYN/ACK response while it receives no reply.

17. 17 Performance Evaluation Bug 2 – Client (Tester) connects to unopened TCP port 0 of Server (SUT). Bug: Client expects RST response while it receives no reply.

18. 18 Outline Motivation RealProct Design Evaluation Conclusion

19. 19 Conclusion As a protocol testing tool, RealProct finds two new bugs, discovers all the previously detected bugs in the TCP/IP stack of WSNs, and validates the Rime mesh routing protocol. Propose two techniques, topology virtualization and event virtualization, for testing. Design an algorithm to tackle the inaccuracy problem caused by non-deterministic events in test execution.

20. 20 Q & A Thank you!

21. 21 Codes that Cause Bugs

22. 22 Repeat Bug – SYN Packet Loss

23. Repeat Bug – Duplicate SYN Packet

24. 24 Dynamic Test Execution