Presentation on theme: "GREY BOX TESTING Web Apps & Networking"— Presentation transcript:
1GREY BOX TESTING Web Apps & Networking Session 3Boris Grinberg
2Session 3 (4 Hours) Here are the things that we will cover: Batch FilesWeb Application TestingFunctionality, Usability, Server side Interface, Client Side Compatibility, Performance, SecuritySpecial tools for Web Application TestingA methodology for WebApp testing
3Batch FilesMost powerful ways to customize and simplify the management of your computer: batch filesThese text files are easy to create, and only as complex as you want them to be, but they can perform many useful operations from file backups to system configuration quickly and automatically
4Understanding & Creating Batch Files What are batch files? Batch files are not programs, per se, they are lists of command line instructions that are batched together in one file. For the most part, you could manually type in the lines of a batch file and get the same results, but batch files make this work easy
5Backbone of the Windows OS If you look in your C:\ or C:\WINDOWS folder you will see a multitude of .BAT, .SYS, .CFG, .INF and other types. These are all kinds of batch files. This may shock you, but while most applications are written in OOP Languages they sit on a mountain of batch files. Batch files are the backbone of the Windows operating system, delete them and you've effectively disabled the OS.Continue: There is a reason for this. The system batch files on each computer are unique the that computer and change each time a program is loaded. The operating system must have access to these files and be able to add and delete instructions from them.
6The power of a batch file At their simplest, batch files are text files which execute one or more command prompt commands in a specific order. The power of a batch file lies in the way that it allows you to combine multiple commands into one batch file 'program' and customize the way that each command operates
7LAB Exercise: Creating Batch files 1. Open text editor notepad (NOT word or wordpad)2. Type or copy this text:@ECHO OFFECHO This is a batch fileECHOPAUSECLSEXIT3. Save this as batchfile.bat, make sure there is no .txt extension after the .bat4. Double-click the file icon
8LAB Exercise: Second Batch files Use with "errorlevel"1. Open a notepad, type or copy this text:@ECHO OFFCOPY C:\file1.txt C:\file2.txtECHOPAUSEEXIT2. Save this as copy.bat and run it.You get the error, something like: “COULD NOT FIND C:\FILE1.TXT” or “The system cannot find the file specified.”
9Parameter errorlevelThe generic parameter errorlevel refers to the output another program or commandan errorlevel of 1 means there was an error, errorlevel of 0 means there was no error. You can see these levels by adding this line after any line of commands:ECHO errorlevel: %errorlevel%
10Last LAB Exercise: Create copy1.bat : Lets Run and discuss our results @ECHO OFF:STARTCOPY C:\file.txt C:\file2.txtIF errorlevel 1 GOTO MKFILEGOTO :END:MKFILEECHO file text>C:\file.txtGOTO START:ENDECHO QuittingPAUSELets Run and discuss our results
11Reference Materials for BAT files Download the file batch.doc from here:batch.docHelpful Links:Batch ProgrammingCreating Batch FilesSample Batch Files
12Interview Question: How are you testing the Web Application? Usual Answers:Testing for broken linksTesting Website with different Screen resolutionsTesting LoginTesting Graphical ElementsTesting on Different BrowsersWhich Answer is Correct?Your Answer?
136 Main Areas: Web Application Testing FunctionalityUsabilityServer side InterfaceClient side CompatibilityPerformanceSecurity
14Sub Areas: Web Application Testing Functionality: Links, Forms, Cookies, Web Indexing, Programming Language, Dynamic Interface Components, DatabasesUsability: Navigation, Graphics, Content, General AppearanceServer side Interface: Server Interface, External InterfaceClient side Compatibility: Platform, Browsers, Settings, PrintersPerformance: Connection Speed, Load, StressSecurity: General security
15Sub Areas: Web Application Testing Functionality: Links, Forms, Cookies, Web Indexing, Programming Language, Dynamic Interface Components, DatabasesUsability: Navigation, Graphics, Content, General AppearanceServer Side Interface: Server Interface, External InterfaceClient Side Compatibility: Platform, Browsers, Settings, PrintersPerformance: Connection Speed, Load, StressSecurity: General security
16Functionality testing: Links Links may be the main feature on web sites. They constitute the mean of transport between pages and guide the user to certain addresses without the user knowing the actual address itself. Linkage testing is divided into three sub areas.First - check that the link takes you to the page it said it would.Second – That the link isn’t broken i.e. that the page you’re linking to exists.Third – Ensure that you have no orphan pages at your site. An orphan page is a page that has no links to it, and may therefore only be reached if you know the correct URL.
17Functionality testing: Links - Summary Remember that to reduce redundant testing, there is no need to test a link more than once to a specific page if it appears on several pages; it needs only to be tested once.This kind of test can preferably be automated and several tools provide solutions for this.Link testing should be done during integration testing, when connections between pages subsist.Summary:Verify that you end up at the designated pageVerify that the link isn’t brokenLocate orphan pages if present
18LAB Exercise: Testing Broken Links Download, unzip and install Xenu’s toolGo to Options and set “Check external URLs”
19LAB Exercise: Testing Broken Links To check a site, click the toolbar icon on the left and enter a WWW address. If the address finishes with a directory name, don't forget to put a / at the end or you will possibly get the whole parent directory spidered.Incorrect:Correct:Check for broken links; use three random websites.
22Functionality testing: Forms – Summary Information hits the server in correct formAcceptance of invalid inputHandling of wrong input (both client an server side)Optional versus mandatory fieldsInput longer than field allowsRadio buttonsDefault values
23Form Input Validation Tools It is quite essential to have input validations in the server side form processing script. Having the right set of validations prevent bad data entering your database.Form validations can, to some extend, prevent hacking attacks too.
24Form Input Validation Tool: PHP Form Validator 1.0 The PHP form validation script contains a set of commonly required form validations.You can download PHP Form ValidatorDocumentation and code samples are included in the download.
26Cookies VerificationVerify that the information that is to be retrieved is there. If login information is stored in cookies check for correct encryption of these.If your applications require cookies, how does it respond to users that disabled the use of such? Does it still function or will the user get notified of the current situation.How will temporary cookies be handled?What will happen when cookies expire?Depending on what cookies are used for, one should examine the possibilities for other solutions.
27Functionality testing: Cookies Summary Encryption of e.g. login infoUsers denying or acceptingTemporary and expired cookies
28Cookies Validation Tool: MAXA Cookie Manager 4.0 First Cookie Manager supporting the new browser independent cookies (Flash Cookies, Silverlight Isolated Storage) together with conventional cookies of Internet Explorer, Firefox and many more browsers. Allows deleting, viewing, blocking and more.
31MAXA Cookie Manager 4.0Most people are unaware, that beside the normal browser cookies, websites using Flash have the possibility to save own cookies. These cookies are not deleted together with the browser cookies, and can be used through all browsers.
32MAXA Cookie Manager 4.0MAXA Cookie Manager lists new generation cookies like Flash and Silverlight cookies together with conventional cookies of Internet Explorer, Firefox, Opera, Safari, Google Chrome, K-Melon and Flock browsers. You can explore their contents and delete them.MAXA Cookie Manager provides you with the possibility to block any traffic to these sites.It is also possible to block particular cookies, or all cookies of a specific type.
34LAB Exercise: Testing Cookies Download, unzip and install MAXA Cookie ManagerGo through the setup wizard and run the app on your local PC.Select and delete few, newly added cookies.Find Flash cookies or look at the picture on the next slide and tell me the difference between FLASH and other type of cookies.
36Functionality testing: Web Indexing There are a number of different techniques and algorithms used by different search engines to search the Internet.Depending on how the site is designed using Meta tags, frames, HTML syntax, dynamically created pages, passwords or different languages, your site will be searchable in different ways.
37Web Indexing - Summary Meta tags Frames HTML syntax Passwords Dynamically created pages
39Programming Language - Summary There are several tools on the market for validating different programming languages. For languages that need compiling e.g. C++, this kind of check is often done by the compiling program. Language validation tools can be found in compilers, online as well as for download, free or by payment. Resources:Summary:· Language specifications· Language syntax (HTML, C++, C#, Java, Scripting languages, SQL etc.)
41Dynamic Interface Components The issue here is to test and verify the function of the components, not compatibility issues.An example of what to test can be a Java applet constructing and displaying a chart of company statistics, where the information first have to be retrieved and then interpreted and displayed on the screen. Since server-side components don’t have user interface, event logging (logfiles) can be used to record events by applications on the server side in order to determine functionality.Resources: Java Specific tools: JavaSpec and JavaStar
43Functionality testing: Databases Databases play an important role in web application technology, housing the content that the web application manages, running queries and fulfilling user requests for data storage. The most commonly used type of database in web applications is the relational database and it’s managed by SQL to write, retrieve and editing of information.
44Two types of DB errorsIn general, there are two types of errors that may occur, data integrity errors and output errors. Data integrity errors refer to missing or wrong data in tables and output errors are errors in writing, editing or reading operations in the tables. The issue is to test the functionality of the database, not the content and the focus here is therefore on output errors. Verify that queries, writing, retrieving or editing in the database is performed in a correct way.
45Functionality testing: Databases - Summary Issues to test are:Creation of tablesIndexing of dataWriting and editing in tables (for example valid numbers or characters, input longer than field etc.)Reading from tables
46Usability testing: Navigation (1) Navigation describes the way users navigate within a page, between different user interface controls (buttons, boxes, lists, windows etc.), or between pages via e.g. links. To determine whether or not your page is easy to navigate through consider the following. Is the application’s navigation intuitive? Are the main features of the site accessible from the main page? Does the site need a site map, search engine, or other navigational help?
47Usability testing: Navigation (2) Be careful though that you don’t overdo your site. Too much information often has the opposite effect as to what was intended. Users of the web tend to be very goal driven and scan a site very quickly to see if it meets their expectations. If not, they quickly move on. They rarely take the time to learn about the sites structure, and it is therefore important to keep the navigational help as concise as possible.
48Usability testing: Navigation (3) Another important aspect of navigation is if the site is consistent in its conventions regarding page layout, navigation bars, menus, links etc. Make sure that users intuitively know that they are still within the site by keeping the page design uniform throughout the site.As soon as the hierarchy of the site is determined, testing of how users navigate can commence. Have real users try and navigate through ordinary papers describing how the layout is done.
49Usability testing: Navigation - Summary Intuitive navigationMain features accessible from main pageSite map or other navigational helpConsistent conventions (navigation bars, menus, links etc.)
50Usability testing: Graphics (1) The graphics of a web site include images, animations, borders, colors, movie clips, fonts, backgrounds, buttons etc.Issues to check are:Make sure that the graphics serve a definite purpose and that images or animations don’t just clutter up the visual design and waste bandwidthSuitable background colors combined with font- and foreground color. Remember that a computer display exceptionally well presents contrasts apposed to printed paper
51Usability testing: Graphics (2) Issues to check are:Three-dimensional effects on buttons often gives useful cluesWhen displaying large amount of images, consider using thumbnails. Check that the original picture appears when a thumbnail is clickedVerify that fonts are consistent in styleSize – quality of pictures, usage of compressed formats (JPG or GIF)Mouse-over effects
52Usability testing: Content Content testing is done to verify the correctness, accuracy and relevancy of information on the site, or in a database, in forms of text, images or animations.Correctness is whether the information is truthful or contains misinformation. For example wrong prices in a price list may cause financial problems or even induce legal issues.The accuracy of the information is whether it is without grammatical or spelling errors. These kinds of verifications are often done in e.g. Word or other word processors.Remove irrelevant information from your site. This may cause misunderstandings or confusion
53Usability testing: Content - Summary Content testing should be done as early as possible, i.e. when the information is posted.Summary:CorrectnessAccuracyRelevancy
54Usability testing: General Appearance Does the site feel right when using it? Do you intuitively know where to look for information? Is the design consistent throughout the site?Important to all kinds of usability tests is to involve external personnel that have little or no connection to the development of the site. It’s easy to get caring of ones own solution, so having actual users evaluating the site may be critical.
55Usability testing: General Appearance Summary Intuitive designConsistent designIf using frames, make sure that the main area is large enoughConsider size of pages. Several screens on the same page or links between themDo features on the site need help systems or will they be intuitive
56Server Side Interface: Server Interface Due to the complex architecture of web systems, interface and compatibility issues may occur on several areas. The core components are web servers, application servers and database servers (and possibly mail servers)Web servers normally hosts HTML pages and other web services. Application severs typically contains objects such as programs, scripts, DLLs or third party products, that provide and extend functionality and effects for the web application.
57Server Interface TestTest the communication between the different servers by making transactions and view logfiles to verify the result.Depending on the configuration of the server side compatibility issues may occur depending on, for example, server hardware, server software or network connections. Database compatibility issues may occur depending on different database types (SQL, Oracle, Sybase etc.)
58Server Side Interface: Issues to Test: Verify that communication is done correctly, web server-application server, application server-database server and vice versa.Compatibility of server software, hardware, network connectionsDatabase compatibility (SQL, Oracle, Sybase etc.)
59Server Side Interface: External Interface Several web pages have external interfaces, such as merchants verifying credit card numbers to allow transactions to be made or a site like that compares prices and delivery times on different merchants on the web. Verify that is sent and retrieved in correct form.
60Client Side compatibility: Platform There are several different operating systems that are being used on the market today, and depending on the configuration of the user system, compatibility issues may occur. Different applications may work fine under certain operating systems, but fail under another. The most commonly used: Windows (XP, Server2003, Server2008, Windows 7), Unix, Mac OS X, Linux (Red Hat, SUSE, etc)
62Client Side compatibility: Browsers This substantiates the fact that compatibility problems commonly occur. Frames and Cascading style sheets may display differently on different browsers, or not at all. Different browsers also have different settings for e.g. security or Java support.A good way to test browser compatibility is to create a compatibility matrix where different brands and versions of browsers are tested to a certain number of components and settings, for example Applets, scripting, ActiveX controls or cookies.
63Client Side compatibility: Browsers - Summary Internet Explorer (6.X, 7.X, 8.X), Firefox (3.X), Opera, Goggle Chrome, etcBrowser settings (security settings, graphics, Java etc.)Frames and Cascade Style sheetsApplets, ActiveX controls, DHTML, client side scriptingHTML specificationsGraphics
64Client Side compatibility: Settings, Preferences Depending on settings and preferences of the client machine, web applications may behave differently. Try and vary the following:Screen resolution (check that text and graphic alignment still work, font are readable, etc.)Color depth (256, 16-bit, 32-bit)
65Client Side compatibility: Printing Despite the paperless society the web was to introduce, printing is done more than ever.Verify that pages are printable with considerations on:Text and image alignmentColors of text, foreground and backgroundScalability to fit paper sizeTables and borders
66Performance: Connection speed Users may differ greatly in connection speed. They may be on a cable modem or on a T1 connection. Users expect longer download times when retrieving programs, but not when requesting a homepage.If the transaction response time is to long, user will leave the site. Other issues to consider are time-out on a page that request logins. If load time is too long, users may be thrown out due to time-out. Database problem may occur if the connection speed is two low, causing data loss.Summary:Connection speed & Time-out
67Performance : LoadWhat is the estimated number of users per time period and how will it be divided over the period?Will there be peak loads and how will the system react?Can your site handle a large amount of users requesting a certain page?
68Performance : Load - Summary Load testing is done to measure the performance at a given load level to assure that the site work within requirements for performance. The load level may be a certain amount of users using your site at the same time or large amount of data transactions from user such as online ordering.Summary:Many users requesting a certain page at the same time or using the site simultaneouslyLarge amount of data from users
69Performance : StressStress testing is done in order to actually break a site or a certain feature to determine how the system reacts. Stress tests are designed to push and test system limitations and determine whether the system recovers gracefully from crashes. Hackers often stress systems by providing loads of wrong in-data until it crashes and then gain access to it during start-up.Typical areas to test are forms, logins or other information transaction components.
70Performance : Stress - Summary Performance of memory, CPU, file handling etc.Error in software, hardware, memory errors (leakage, overwrite or pointers)
71Performance : Continuous use Is the application or certain features going to be used only during certain periods of time or will it be used continuously 24 hours a day 7 days a week? Test that the application is able to perform during those conditions. Will downtime be allowed or is that out of the question?Verify that the application is able to meet the requirements and does not run out of memory or disk space.
72SecuritySecurity is an area of immense extent, and would need extensive writing to be fairly covered.First make sure that you have a correct directory setup. You don’t want users to be able to brows through directories on your server.
73SecurityLogins are very common on today’s web sites, and they must be error free. Make sure to test both valid and invalid login names and passwords.Are they case sensitive?Is there a limit to how many tries that are allowed?Can it be bypassed by typing the URL to a page inside directly in the browser?
74SecurityIs there a time-out limit within your site? What happens when it’s exceeded? Are users still able to navigate through the site?Logfiles are a very important in order to maintain security at the site.Verify that relevant information is written to the logfiles and that the information is traceable.
75SecurityWhen secure socket layers are used, verify that the encryption is done correctly and check the integrity of the information.Scripting on the server often constitute security holes and are often used by hackers.Test that it isn’t possible to plant or edit scripts on the server without authorization.
76Security: Summary Directory setup Logins Time-out Logfiles SSL Scripting Languages
77A methodology for WebApp testing The process of establishing a methodology for web application testing resulted in the Test Priority Sheet. This methodology helps determine the most important areas to test in any web application, as well as being a tool for prioritizing when time is short. The methodology is general to be useful for any web application. For this kind of tool to be used, it is required that it is short and easy to use. The Test Priority Sheet is all this.
78The Test Priority Sheet Test Priority Sheet, gives you a good idea on the overall complexity of your application and the testing effort needed throughout the development.
79Three factors for prioritization Prioritizing depends on the resources available, but this is mostly concerning to which extent prioritizing needs to be done. The question to answer at this point is; how may these three factors, Complexity, Purpose and Target Group, be used to be able to determine what to test? As certain test areas might be overlooked if using one of a small number of predefined methodologies, we will use the three factors, not to categorize web sites, but to determine the need of certain test areas when having a specific web application in mind.
80Considerations… Consider any web application, but only one. To what extent does it depend on links, forms or cookies?Is the content valid according to the purpose of the site?Does the site contain features sensible to slow connection speeds?How will the target group react to long download times?purpose and target group, are combined into to a new factor, AimPurpose and target group, are combined into to a new factor, Aim
81Calculation: Test Priority Sheet Giving numerical answers to these questions and multiplying them creates a testing need value. The values may then be compared to each other and the highest values are assigned the highest testing need and should therefore be prioritized. This approach gives you a good idea on the overall complexity of your application and the testing effort needed throughout the development.
82HomeWork Example Select Web Application Use the set of questions provided by me and feel in the empty matrix with your answersBring your Matrix with results on our next session
83Interviews… Boris’s Advice # 3 Add in Resume a few lines with provocationwhich should lead to theexpected question on interviewBe ready to impress your interviewers!