Presentation is loading. Please wait.

Presentation is loading. Please wait.

CS4723 Software Validation and Quality Assurance Lecture 7 Non-Functional Testing.

Similar presentations

Presentation on theme: "CS4723 Software Validation and Quality Assurance Lecture 7 Non-Functional Testing."— Presentation transcript:

1 CS4723 Software Validation and Quality Assurance Lecture 7 Non-Functional Testing

2 2  Performance Testing  Test whether the efficiency (time and space) of a software meets requirements  Security Testing  Test whether the software is vulnerable to attacks (special invalid inputs designed to control the software or reveal info from the software)

3 3 Performance Testing  Load Testing  Soak Testing  Stress Testing  Spike Testing

4 4 Input loads  Size of inputs  An extremely long SQL query  An extremely large html file for a browser  Number of inputs provided  Number of students supported in a school management system  Number of web pages opened in a browser  Frequency of inputs provided  Number of SQL queries made per second  Number of Http requests made per second

5 5 Performance Measures  Input Lag  Response Time  Throughput

6 6 Performance Measures

7 7 Load Testing  Provide input under the maximal designed load of software and observe behavior  Purpose:  See whether the software works normally  Find potential bottlenecks of performance: Profiling  Instrument each major component (e.g., method) to see how much time / memory is spent on it  Sampling is sometimes used to reduce overhead

8 8 Load Testing  Test steps  Determine the content of inputs  Usually can be a large amount of identical or similar inputs  The input can be simple or very complex (to check the performance of software when handling complex input)  Determine the frequency of input feeding  Determine how long the input feeding lasts time Design Load Input Load

9 9 Demo  Usage of JMeter to perform load testing for web application and databases

10 10 Input feeding  A multi thread program to feed inputs randomly in a given period of time  Sometimes require multiple machines to feed inputs  Usually only consider valid inputs

11 11 Stress Testing  Provide input OVER the maximal designed load of software and observe behavior  The software is expected to fail  Purpose:  Observe when (how much load) the software is going to fail  Observe the how the failure looks like: crash? CPU or memory used up? Can be recovered or not?  Observe whether the system can partially work when failure happens

12 12 Stress Testing  Illustration time Design Load Input Load

13 13 Soak Testing  Provide heavy input load (slightly under designed maximal load) for a long time  Purpose:  Testing for how long time the software can work normally under heavy input load  Usually memory and disk oriented  Observe the memory / disk usage trend (abnormal increase in the usage)

14 14 Soak Testing  Illustration time Design Load Input Load

15 15 Spike Testing  Provide extremely heavy input load (OVER designed maximal load) for a very short time  Purpose:  Test how the software can handle a input load burst  Probable expected behavior:  Temporarily refuse inputs that cannot be handled  Provide some temporary services for the inputs to wait until the burst ends

16 16 Spike Testing  Illustration time Design Load Input Load

17 17 Performance Diagnosis  Find out why performance problems happen  Figure out how to optimize software to achieve higher performance

18 18 Profilers

19 19 Memory Profilers

20 20 Performance Testing: Review  Load Testing  Stress Testing  Soak Testing  Spike Testing

21 21 Tools for Test Measurement  EclEmma  Update site:  Install in eclipse  Setup coverage configuration: do not check the test folder  Run test using Emma coverage as…  Read test coverage  Enhance test coverage

22 22 Tools for Test Measurement  VisualVM   Start VisualVM  Start a Java software  Open the tab for the process  Do profiler for memory and computation

23 23 Security Testing  Major security concerns  Vulnerabilities  Penetration Testing

24 24 Major Security Concerns  Undermine usability  DOS attacks  Peculiar inputs causing crashes, bloats, …  Information Leaking  SQL Injection, Cross-site Scripting, unencrypted data, side channels, …  Command and Control  OS Injection, Cross-site Scripting, Return Oriented Programming, …

25 25 Vulnerabilities  Avoid common vulnerabilities  Buffer Overflow  Injection  Cross-Site Scripting

26 26 Buffer Overflow  Quite many languages (C, C++) are memory unsafe  You define a buffer, and it is your responsibility to keep your data in the buffer  If you read or write to the place out of a buffer  Semantic errors  Crashes  What else? Anything related to security? char buffer[12];

27 Review of OS course: call stacks  Function calls are traced by call stacks main f f f g int main(int argc, char args**){ int result; if(argc >= 1){f(args[0]);} } void f(char* data){ char buffer[12]; strcpy(buffer, data) if(g()){return;} else{…} } bool g(){... }

28 Call stack of the function f  The local variable buffer  The parameter data  The return address to go back to the call-site at main function char[12] buffer

29 Feed in a valid input  Example “username” char[12] buffer

30 Feed in an invalid input  Example “usernameusername”  The parameter data is covered  So it is no longer usable  The return value is covered  So can not return normally  Still just a bug  Minor security problem  Undermines usability char[12] buffer

31 Feed in a malicious input  Idea to do the trick  Feed in an input with 20 chars  Cover the return address  f() will return to the code we Specify  Consider the program is on a server, accessing user requests  How to make it possible?  Where to put the code?  How to specify the return value to our code? char[12] buffer

32 Feed in a malicious input  Use the buffer itself to store the code  Set the return value to the buffer address  Example  Run exec(“/bin/sh”) to open a shell  Translate to machine code char[12] buffer mov $a0 15 mov $a1 data syscall data: /, b, i, n, /, s, h 0x20, 0x42, 0x00,...

33 Feed in a malicious input  Other issues  How to know the address of buffer[]: Programs are executed in virtual memory, so install the software and check memory state  Buffer is too small to hold your code? Jump through return value to the stack frame of parent function char[12] buffer

34 34 The state of practice  Buffer overflow is very common in C / C++ programs  About 50% of new attacks are related to buffer overflow  Known bugs are being exploited from time to time

35 35 How to deal with buffer overflow  Boundary check for input-reachable buffers  Not so easy in practice  Check too many places: slow the software down  Check too few places: buffer overflow risk  Automatic supports  Buffer Overflow Detection: libsafe, stackguard, …  Runtime protection: weak memory safe  Runtime protection: split stack

36 36 A real-world example  If you are interested  Here is a real world example:  -overflow-a-real-world-example/

37 37 Injection  Directly inject user input into code to be executed  SQL Injection  Inject code to SQL queries  OS Injection  Inject code to OS commands

38 38 SQL Injection  An example  A student information system  You can query your grade for certain course, year, …  You login to your session, and say you are going to search for the grade of “CS4723”  What does the server do?

39 39 SQL Injection  The malicious Input  We want to inject code into the SQL query  Say we want it to be “select * from Grade”  It is the same with “select * from Grade where username = ‘you’ and course = ‘CS4723’ or ‘a’ = ‘a’”

40 40 OS Injection  Quite Similar  Consider a server is going to make a dir for you as a new user, and it will execute exec(“mkdir path/to/” + username)  What username you should make up? An example: HahaGotyou | \bin\sh

41 41 Injection Protection  Injection works by passing user inputs into back- end engines  Can we simply cut off the path?  Definitely NO  We have to do some filtering  We are going to work on the example: select * from Grade where username = ‘you’ and course = ‘CS4723’/**/or/**/‘a’=‘a’

42 42 User Input Filtering  What to filter?  or ? => “oorr” can bypass it  Space? => use /**/ can bypass it  Quotes? A little bit difficult, we can search by year, and use year = 2009 or 1=1  Want more?  See select * from Grade where username = ‘you’ and course = ‘CS4723’ or ‘a’ = ‘a’

43 43 User Input Filtering: Other Issues  Functioning of the software  Filter ‘or’ will affect username ‘George’  Cannot filter space when space is allowed for the field  Other string manipulations  In web applications, there are HTML/URL escape characters  " for “, ' for ‘,   / %20 for space, …  Replacing escaping characters are common  So ' may be used if quotes are disabled…  Always be clear about how user inputs flows in your code

44 44 Cross Site Scripting  One of the most popular attacks to web applications  Everything is about where the input goes to  This time it goes to a web page  This becomes more popular with so-called web 2.0 (let users do the work, e.g., wiki, youtube, blogs)

45 45 XSS: Scenario

46 46 Example  Bob wants to get all the login information of his friends in a social network  So Bob writes a blog, in the blog, he writes: xxxxxxx, xxxx, email(“”, getcookie())  Now Mary reads the blog, so the script runs, Bob will get the cookie, and will be able to login with Mary’s cookie…

47 47 Protection against XSS  Limit the usage of cookies: may result in much inconvenience  Quite similar to SQL Injection  Try to filter dangerous things such as “ ” from user’s input  Also quite similar to SQL Injection  There are a lot of ways to bypass the filtering, so always hard to do a correct filtering  Even harder because HTML is more complex than SQL, and much more web page generations than database query points…

48 48 Core idea: Devil inputs  Software Security is almost all about the malicious inputs  Buffer Overflow, Injection, and XSS accounts for 70% to 80% of attacks each year…  Also for DOS (Denial of Service) attacks  An example: you may want to filter ‘\’ for security reasons, but if so, handling a input like ‘\\\....\\\\’ with n ‘\’s will take n 2 CPU time…  Consider all possible values of user inputs  Never make assumptions to user inputs

49 49 Penetration Testing  Random testing (or fuzzing) is often useful for security testing, because it can generate inputs that you cannot imagine  Have security checks during the testing  Buffer Overflow: whether any “out of boundary” happens  Use boundary checker in testing, and disable them in distribution  SQL Injection & XSS: whether user inputs reach syntax tree part of the HTML or SQL code  Use taints during testing to track the user inputs along the execution

50 50 Review of Non-Functional Testing  Performance Testing  Test whether the efficiency (time and space) of a software meets requirements  Security Testing  Test whether the software is vulnerable to attacks (special invalid inputs designed to control the software or reveal info from the software)

Download ppt "CS4723 Software Validation and Quality Assurance Lecture 7 Non-Functional Testing."

Similar presentations

Ads by Google