Presentation on theme: "1 GREY BOX TESTING Web Apps & Networking Session 3 Boris Grinberg"— Presentation transcript:
1 GREY BOX TESTING Web Apps & Networking Session 3 Boris Grinberg
2 Session 3 (4 Hours) Here are the things that we will cover: –Batch Files –Web Application Testing Functionality, Usability, Server side Interface, Client Side Compatibility, Performance, Security –Special tools for Web Application Testing –A methodology for WebApp testing
3 Batch Files Most powerful ways to customize and simplify the management of your computer: batch files These text files are easy to create, and only as complex as you want them to be, but they can perform many useful operations from file backups to system configuration quickly and automatically
4 Understanding & Creating Batch Files What are batch files? Batch files are not programs, per se, they are lists of command line instructions that are batched together in one file. For the most part, you could manually type in the lines of a batch file and get the same results, but batch files make this work easy
5 Backbone of the Windows OS If you look in your C:\ or C:\WINDOWS folder you will see a multitude of.BAT,.SYS,.CFG,.INF and other types. These are all kinds of batch files. This may shock you, but while most applications are written in OOP Languages they sit on a mountain of batch files. Batch files are the backbone of the Windows operating system, delete them and you've effectively disabled the OS.
6 The power of a batch file At their simplest, batch files are text files which execute one or more command prompt commands in a specific order. The power of a batch file lies in the way that it allows you to combine multiple commands into one batch file 'program' and customize the way that each command operates
7 LAB Exercise: Creating Batch files 1. Open text editor notepad (NOT word or wordpad) 2. Type or copy this OFF ECHO This is a batch file ECHO PAUSE CLS EXIT 3. Save this as batchfile.bat, make sure there is no.txt extension after the.bat 4. Double-click the file icon
8 LAB Exercise: Second Batch files Use with "errorlevel" 1. Open a notepad, type or copy this OFF COPY C:\file1.txt C:\file2.txt ECHO PAUSE EXIT 2. Save this as copy.bat and run it. You get the error, something like: COULD NOT FIND C:\FILE1.TXT or The system cannot find the file specified.
9 Parameter errorlevel The generic parameter errorlevel refers to the output another program or command an errorlevel of 1 means there was an error, errorlevel of 0 means there was no error. You can see these levels by adding this line after any line of commands: ECHO errorlevel: %errorlevel%
10 Last LAB Exercise: Create copy1.bat OFF :START COPY C:\file.txt C:\file2.txt IF errorlevel 1 GOTO MKFILE GOTO :END :MKFILE ECHO file text>C:\file.txt GOTO START :END ECHO Quitting PAUSE Lets Run and discuss our results
11 Reference Materials for BAT files Download the file batch.doc from here: batch.doc Helpful Links: Batch Programming Creating Batch Files Sample Batch Files
12 Interview Question: How are you testing the Web Application? Usual Answers: 1.Testing for broken links 2.Testing Website with different Screen resolutions 3.Testing Login 4.Testing Graphical Elements 5.Testing on Different Browsers Which Answer is Correct? Your Answer?
13 6 Main Areas: Web Application Testing 1.Functionality 2.Usability 3.Server side Interface 4.Client side Compatibility 5.Performance 6.Security
14 Sub Areas: Web Application Testing Functionality: Links, Forms, Cookies, Web Indexing, Programming Language, Dynamic Interface Components, Databases Usability: Navigation, Graphics, Content, General Appearance Server side Interface: Server Interface, External Interface Client side Compatibility: Platform, Browsers, Settings, Printers Performance: Connection Speed, Load, Stress Security: General security
15 Sub Areas: Web Application Testing Functionality: Links, Forms, Cookies, Web Indexing, Programming Language, Dynamic Interface Components, Databases Usability: Navigation, Graphics, Content, General Appearance Server Side Interface: Server Interface, External Interface Client Side Compatibility: Platform, Browsers, Settings, Printers Performance: Connection Speed, Load, Stress Security: General security
16 Functionality testing: Links Links may be the main feature on web sites. They constitute the mean of transport between pages and guide the user to certain addresses without the user knowing the actual address itself. Linkage testing is divided into three sub areas. –First - check that the link takes you to the page it said it would. –Second – That the link isnt broken i.e. that the page youre linking to exists. –Third – Ensure that you have no orphan pages at your site. An orphan page is a page that has no links to it, and may therefore only be reached if you know the correct URL.
17 Functionality testing: Links - Summary Remember that to reduce redundant testing, there is no need to test a link more than once to a specific page if it appears on several pages; it needs only to be tested once. –This kind of test can preferably be automated and several tools provide solutions for this. –Link testing should be done during integration testing, when connections between pages subsist. Summary: –Verify that you end up at the designated page –Verify that the link isnt broken –Locate orphan pages if present
18 LAB Exercise: Testing Broken Links 1.Download, unzip and install Xenus toolXenus tool 2.Go to Options and set Check external URLs
19 LAB Exercise: Testing Broken Links 3.To check a site, click the toolbar icon on the left and enter a WWW address. If the address finishes with a directory name, don't forget to put a / at the end or you will possibly get the whole parent directory spidered. Incorrect: Correct: 4.Check for broken links; use three random websites.
22 Functionality testing: Forms – Summary Information hits the server in correct form Acceptance of invalid input Handling of wrong input (both client an server side) Optional versus mandatory fields Input longer than field allows Radio buttons Default values
23 Form Input Validation Tools It is quite essential to have input validations in the server side form processing script. Having the right set of validations prevent bad data entering your database. Form validations can, to some extend, prevent hacking attacks too.
24 Form Input Validation Tool: PHP Form Validator 1.0 The PHP form validation script contains a set of commonly required form validations. You can download PHP Form ValidatorPHP Form Validator Documentation and code samples are included in the download.
26 Cookies Verification Verify that the information that is to be retrieved is there. If login information is stored in cookies check for correct encryption of these. If your applications require cookies, how does it respond to users that disabled the use of such? Does it still function or will the user get notified of the current situation. How will temporary cookies be handled? What will happen when cookies expire? Depending on what cookies are used for, one should examine the possibilities for other solutions.
27 Functionality testing: Cookies Summary Encryption of e.g. login info Users denying or accepting Temporary and expired cookies
28 Cookies Validation Tool: MAXA Cookie Manager 4.0 First Cookie Manager supporting the new browser independent cookies (Flash Cookies, Silverlight Isolated Storage) together with conventional cookies of Internet Explorer, Firefox and many more browsers. Allows deleting, viewing, blocking and more.
29 MAXA Cookie Manager 4.0
30 MAXA Cookie Manager 4.0
31 MAXA Cookie Manager 4.0 Most people are unaware, that beside the normal browser cookies, websites using Flash have the possibility to save own cookies. These cookies are not deleted together with the browser cookies, and can be used through all browsers.
32 MAXA Cookie Manager 4.0 MAXA Cookie Manager lists new generation cookies like Flash and Silverlight cookies together with conventional cookies of Internet Explorer, Firefox, Opera, Safari, Google Chrome, K-Melon and Flock browsers. You can explore their contents and delete them. MAXA Cookie Manager provides you with the possibility to block any traffic to these sites. It is also possible to block particular cookies, or all cookies of a specific type.
33 MAXA Cookie Manager 4.0 Screen
34 LAB Exercise: Testing Cookies Download, unzip and install MAXA Cookie ManagerMAXA Cookie Manager Go through the setup wizard and run the app on your local PC. Select and delete few, newly added cookies. Find Flash cookies or look at the picture on the next slide and tell me the difference between FLASH and other type of cookies.
35 LAB Exercise: Testing Cookies
36 Functionality testing: Web Indexing There are a number of different techniques and algorithms used by different search engines to search the Internet. Depending on how the site is designed using Meta tags, frames, HTML syntax, dynamically created pages, passwords or different languages, your site will be searchable in different ways.
37 Web Indexing - Summary Meta tags Frames HTML syntax Passwords Dynamically created pages
39 Programming Language - Summary There are several tools on the market for validating different programming languages. For languages that need compiling e.g. C++, this kind of check is often done by the compiling program. Language validation tools can be found in compilers, online as well as for download, free or by payment. Resources: Summary: Language specifications Language syntax (HTML, C++, C#, Java, Scripting languages, SQL etc.)
41 Dynamic Interface Components The issue here is to test and verify the function of the components, not compatibility issues. –An example of what to test can be a Java applet constructing and displaying a chart of company statistics, where the information first have to be retrieved and then interpreted and displayed on the screen. Since server-side components dont have user interface, event logging (logfiles) can be used to record events by applications on the server side in order to determine functionality. Resources: Java Specific tools: JavaSpec and JavaStar
43 Functionality testing: Databases Databases play an important role in web application technology, housing the content that the web application manages, running queries and fulfilling user requests for data storage. The most commonly used type of database in web applications is the relational database and its managed by SQL to write, retrieve and editing of information.
44 Two types of DB errors In general, there are two types of errors that may occur, data integrity errors and output errors. Data integrity errors refer to missing or wrong data in tables and output errors are errors in writing, editing or reading operations in the tables. The issue is to test the functionality of the database, not the content and the focus here is therefore on output errors. Verify that queries, writing, retrieving or editing in the database is performed in a correct way.
45 Functionality testing: Databases - Summary Issues to test are: 1.Creation of tables 2.Indexing of data 3.Writing and editing in tables (for example valid numbers or characters, input longer than field etc.) 4.Reading from tables
46 Usability testing: Navigation (1) Navigation describes the way users navigate within a page, between different user interface controls (buttons, boxes, lists, windows etc.), or between pages via e.g. links. To determine whether or not your page is easy to navigate through consider the following. Is the applications navigation intuitive? Are the main features of the site accessible from the main page? Does the site need a site map, search engine, or other navigational help?
47 Usability testing: Navigation (2) Be careful though that you dont overdo your site. Too much information often has the opposite effect as to what was intended. Users of the web tend to be very goal driven and scan a site very quickly to see if it meets their expectations. If not, they quickly move on. They rarely take the time to learn about the sites structure, and it is therefore important to keep the navigational help as concise as possible.
48 Usability testing: Navigation (3) Another important aspect of navigation is if the site is consistent in its conventions regarding page layout, navigation bars, menus, links etc. Make sure that users intuitively know that they are still within the site by keeping the page design uniform throughout the site. As soon as the hierarchy of the site is determined, testing of how users navigate can commence. Have real users try and navigate through ordinary papers describing how the layout is done.
49 Usability testing: Navigation - Summary Intuitive navigation Main features accessible from main page Site map or other navigational help Consistent conventions (navigation bars, menus, links etc.)
50 Usability testing: Graphics (1) The graphics of a web site include images, animations, borders, colors, movie clips, fonts, backgrounds, buttons etc. Issues to check are: –Make sure that the graphics serve a definite purpose and that images or animations dont just clutter up the visual design and waste bandwidth –Suitable background colors combined with font- and foreground color. Remember that a computer display exceptionally well presents contrasts apposed to printed paper
51 Usability testing: Graphics (2) Issues to check are: –Three-dimensional effects on buttons often gives useful clues –When displaying large amount of images, consider using thumbnails. Check that the original picture appears when a thumbnail is clicked –Verify that fonts are consistent in style –Size – quality of pictures, usage of compressed formats (JPG or GIF) –Mouse-over effects
52 Usability testing: Content Content testing is done to verify the correctness, accuracy and relevancy of information on the site, or in a database, in forms of text, images or animations. –Correctness is whether the information is truthful or contains misinformation. For example wrong prices in a price list may cause financial problems or even induce legal issues. –The accuracy of the information is whether it is without grammatical or spelling errors. These kinds of verifications are often done in e.g. Word or other word processors. –Remove irrelevant information from your site. This may cause misunderstandings or confusion
53 Usability testing: Content - Summary Content testing should be done as early as possible, i.e. when the information is posted. Summary: –Correctness –Accuracy –Relevancy
54 Usability testing: General Appearance Does the site feel right when using it? Do you intuitively know where to look for information? Is the design consistent throughout the site? Important to all kinds of usability tests is to involve external personnel that have little or no connection to the development of the site. Its easy to get caring of ones own solution, so having actual users evaluating the site may be critical.
55 Usability testing: General Appearance Summary Intuitive design Consistent design If using frames, make sure that the main area is large enough Consider size of pages. Several screens on the same page or links between them Do features on the site need help systems or will they be intuitive
56 Server Side Interface: Server Interface Due to the complex architecture of web systems, interface and compatibility issues may occur on several areas. The core components are web servers, application servers and database servers (and possibly mail servers) –Web servers normally hosts HTML pages and other web services. Application severs typically contains objects such as programs, scripts, DLLs or third party products, that provide and extend functionality and effects for the web application.
57 Server Interface Test Test the communication between the different servers by making transactions and view logfiles to verify the result. Depending on the configuration of the server side compatibility issues may occur depending on, for example, server hardware, server software or network connections. Database compatibility issues may occur depending on different database types (SQL, Oracle, Sybase etc.)
58 Server Side Interface: Issues to Test: Verify that communication is done correctly, web server-application server, application server-database server and vice versa. Compatibility of server software, hardware, network connections Database compatibility (SQL, Oracle, Sybase etc.)
59 Server Side Interface: External Interface Several web pages have external interfaces, such as merchants verifying credit card numbers to allow transactions to be made or a site like that compares prices and delivery times on different merchants on the web. Verify that is sent and retrieved in correct form.
60 Client Side compatibility: Platform There are several different operating systems that are being used on the market today, and depending on the configuration of the user system, compatibility issues may occur. Different applications may work fine under certain operating systems, but fail under another. The most commonly used: Windows (XP, Server2003, Server2008, Windows 7), Unix, Mac OS X, Linux (Red Hat, SUSE, etc)
62 Client Side compatibility: Browsers This substantiates the fact that compatibility problems commonly occur. Frames and Cascading style sheets may display differently on different browsers, or not at all. Different browsers also have different settings for e.g. security or Java support. A good way to test browser compatibility is to create a compatibility matrix where different brands and versions of browsers are tested to a certain number of components and settings, for example Applets, scripting, ActiveX controls or cookies.
63 Client Side compatibility: Browsers - Summary Internet Explorer (6.X, 7.X, 8.X), Firefox (3.X), Opera, Goggle Chrome, etc Browser settings (security settings, graphics, Java etc.) Frames and Cascade Style sheets Applets, ActiveX controls, DHTML, client side scripting HTML specifications Graphics
64 Client Side compatibility: Settings, Preferences Depending on settings and preferences of the client machine, web applications may behave differently. Try and vary the following: Screen resolution (check that text and graphic alignment still work, font are readable, etc.) Color depth (256, 16-bit, 32-bit)
65 Client Side compatibility: Printing Despite the paperless society the web was to introduce, printing is done more than ever. Verify that pages are printable with considerations on: –Text and image alignment –Colors of text, foreground and background –Scalability to fit paper size –Tables and borders
66 Performance: Connection speed Users may differ greatly in connection speed. They may be on a cable modem or on a T1 connection. Users expect longer download times when retrieving programs, but not when requesting a homepage. –If the transaction response time is to long, user will leave the site. Other issues to consider are time-out on a page that request logins. If load time is too long, users may be thrown out due to time-out. Database problem may occur if the connection speed is two low, causing data loss. Summary: Connection speed & Time-out
67 Performance : Load What is the estimated number of users per time period and how will it be divided over the period? Will there be peak loads and how will the system react? Can your site handle a large amount of users requesting a certain page?
68 Performance : Load - Summary Load testing is done to measure the performance at a given load level to assure that the site work within requirements for performance. The load level may be a certain amount of users using your site at the same time or large amount of data transactions from user such as online ordering. Summary: –Many users requesting a certain page at the same time or using the site simultaneously –Large amount of data from users
69 Performance : Stress Stress testing is done in order to actually break a site or a certain feature to determine how the system reacts. Stress tests are designed to push and test system limitations and determine whether the system recovers gracefully from crashes. Hackers often stress systems by providing loads of wrong in-data until it crashes and then gain access to it during start-up. Typical areas to test are forms, logins or other information transaction components.
70 Performance : Stress - Summary Performance of memory, CPU, file handling etc. Error in software, hardware, memory errors (leakage, overwrite or pointers)
71 Performance : Continuous use Is the application or certain features going to be used only during certain periods of time or will it be used continuously 24 hours a day 7 days a week? Test that the application is able to perform during those conditions. Will downtime be allowed or is that out of the question? Verify that the application is able to meet the requirements and does not run out of memory or disk space.
72 Security Security is an area of immense extent, and would need extensive writing to be fairly covered. First make sure that you have a correct directory setup. You dont want users to be able to brows through directories on your server.
73 Security Logins are very common on todays web sites, and they must be error free. Make sure to test both valid and invalid login names and passwords. Are they case sensitive? Is there a limit to how many tries that are allowed? Can it be bypassed by typing the URL to a page inside directly in the browser?
74 Security Is there a time-out limit within your site? What happens when its exceeded? Are users still able to navigate through the site? Logfiles are a very important in order to maintain security at the site. Verify that relevant information is written to the logfiles and that the information is traceable.
75 Security When secure socket layers are used, verify that the encryption is done correctly and check the integrity of the information. Scripting on the server often constitute security holes and are often used by hackers. Test that it isnt possible to plant or edit scripts on the server without authorization.
77 A methodology for WebApp testing The process of establishing a methodology for web application testing resulted in the Test Priority Sheet. This methodology helps determine the most important areas to test in any web application, as well as being a tool for prioritizing when time is short. The methodology is general to be useful for any web application. For this kind of tool to be used, it is required that it is short and easy to use. The Test Priority Sheet is all this.
78 The Test Priority Sheet Test Priority Sheet, gives you a good idea on the overall complexity of your application and the testing effort needed throughout the development.
79 Three factors for prioritization Prioritizing depends on the resources available, but this is mostly concerning to which extent prioritizing needs to be done. The question to answer at this point is; how may these three factors, Complexity, Purpose and Target Group, be used to be able to determine what to test? As certain test areas might be overlooked if using one of a small number of predefined methodologies, we will use the three factors, not to categorize web sites, but to determine the need of certain test areas when having a specific web application in mind.
80 Considerations… Consider any web application, but only one. –To what extent does it depend on links, forms or cookies? –Is the content valid according to the purpose of the site? –Does the site contain features sensible to slow connection speeds? –How will the target group react to long download times?purpose and target group, are combined into to a new factor, Aim Purpose and target group, are combined into to a new factor, Aim
81 Calculation: Test Priority Sheet Giving numerical answers to these questions and multiplying them creates a testing need value. The values may then be compared to each other and the highest values are assigned the highest testing need and should therefore be prioritized. This approach gives you a good idea on the overall complexity of your application and the testing effort needed throughout the development.
82 HomeWork Select Web Application Use the set of questions provided by me and feel in the empty matrix with your answers Bring your Matrix with results on our next session Example
83 Interviews… Boriss Advice # 3 Add in Resume a few lines with provocation which should lead to the expected question on interview Be ready to impress your interviewers!