We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byYesenia Stukey
Modified over 2 years ago
Automating Coded UI Testing using Microsoft Testing Tools in Visual Studio 2012 Clive G Hermann
Name Title/Function Job Responsibility Testing Experience C# Experience Expectations for the Course Introductions © GSAtrain2Custom Coded UI Training for Microsoft
Course Materials Lab Manual - Printed Course Notes - PowerPoint Slides Virtual Machines for TFS and VS 2012 Links and Whitepapers (Favorites) © GSAtrain3Custom Coded UI Training for Microsoft
1)Overview of Automated Code UI Testing 2)Walkthrough Steps from Beginning-to-End 3)Creating Manual Tests 4)Creating a Coded UI Test Using Coded UI Test Builder 5)Automating Coded UI Test Cases Course Contents © GSAtrain4Custom Coded UI Training for Microsoft
6)Data Driven Coded UI Tests 7)Anatomy of Code UI Tests 8)Introduction to Lifecycle Management 9)Extending Coded UI Tests and Action Recordings 10) Testing Large Applications with Multiple UI Maps Course Contents (Continued) © GSAtrain5Custom Coded UI Training for Microsoft
11) Using HTML5 Controls in Coded UI Tests 12) Visual Studio 2012 ALM Features 13) Upgrading Coded UI Tests from Visual Studio ) Using WPF with Coded UI Tests 15) Best Practices for Coded UI Tests Course Contents (continued) © GSAtrain6Custom Coded UI Training for Microsoft
16) Using the Coded UI Test Logs 17) Using SharePoint with Coded UI Tests 18) ExcelCodedUIHelper Course Contents (optional) © GSAtrain7Custom Coded UI Training for Microsoft
Overview of Automated Coded UI Testing Module 1 © GSAtrain8Custom Coded UI Training for Microsoft
UI Testing o Functional Testing Controls User Interface Whole Application The earlier we catch a “bug” or potential problem the less it costs! o Image o Manpower Automated Code UI Testing © GSAtrain9Custom Coded UI Training for Microsoft
70% of all Testing is manual Reality of Automated Testing © GSAtrain10Custom Coded UI Training for Microsoft
Replacing a Manual User Interface Test with an Automated Test. o To Allow: Replication Consistency Regression Testing For: Multiple Environments Multiple Versions What is Coded UI Testing? © GSAtrain11Custom Coded UI Training for Microsoft
IT Has a Problem © GSAtrain12Custom Coded UI Training for Microsoft
Catching defects as early as possible is the least expensive way to ensure software quality Kent Beck and Cynthia Andres wrote o "Here is the dilemma in software development: defects are expensive, but eliminating defects is also expensive.” o “However, most defects end up costing more than it would have cost to prevent them." Test Early and Often © GSAtrain13Custom Coded UI Training for Microsoft
Test-Driven Development o Design the Test o Then Write the Code The Testing Trends Forwards © GSAtrain14Custom Coded UI Training for Microsoft
Linking it all together o Help-Desk o Test o Story Board o Task o Unit o Programmer The Testing Trends: Backwards and Circular © GSAtrain15Custom Coded UI Training for Microsoft
To find the bugs before your users do! o UI Testing is END USER TESTING o OR TOTAL SYSTEM TESTING o OR INFRASTRUCTURE TESTING The Ultimate Goal of Software Testing © GSAtrain16Custom Coded UI Training for Microsoft
Manual to Coded UI © GSAtrain17Custom Coded UI Training for Microsoft
Guarantee SOFTWARE QUALITY Regression Test each Version or Build Test all Environments Provide Developers with all the information required: o To REPRODUCE THE “BUG” o Trace the steps to get the BUG o Screen shots, IntelliTrace, Logs and Stack Traces NO-REPRO must be ELIMINATED Testers’ Multiple Roles © GSAtrain18Custom Coded UI Training for Microsoft
Clear Separation of Functions Clear Definition of Communication and Collaboration TEAM of Software Quality Tiers of Software Testing © GSAtrain19Custom Coded UI Training for Microsoft
Walkthrough Steps from Beginning-to-End Module 2 © GSAtrain20Custom Coded UI Training for Microsoft
Lesson 1 Manual Testing o Using Microsoft Test Manger to create and manage Test Plans o Running Test Cases and Publishing Results o Conducting Exploratory Testing o Fast Forward for Manual Testing Lesson 2 Coded User Interface Testing o Automated Functional Tests – Using CUIT o Create a CUIT from Scratch o Create a CUIT from action recording o Making CUIT more Robust and Flexible Walkthrough Steps from Beginning-to-End © GSAtrain21Custom Coded UI Training for Microsoft
A Manual Testing Process PlanDesignTestTrackVerifyAutomate © GSAtrain22Custom Coded UI Training for Microsoft
Demo of Simple Calculator Building an App For Manual Testing © GSAtrain23Custom Coded UI Training for Microsoft
Demo Automating the Calculator © GSAtrain24Custom Coded UI Training for Microsoft
Demo Adding Data © GSAtrain25Custom Coded UI Training for Microsoft
Demo Introducing an Error © GSAtrain26Custom Coded UI Training for Microsoft
Walkthrough Steps from Beginning-to-End o Examine the code to be tested o Create a Coded UI o Test the Coded UI o Add a data file to drive the UI Tests o Run all of the tests o Create errors in the data file o Examine failed tests Hands-On Lab © GSAtrain27Custom Coded UI Training for Microsoft
Creating Manual Tests Module 3 © GSAtrain28Custom Coded UI Training for Microsoft
Visual Studio 2012 Supported Test Types © GSAtrain29Custom Coded UI Training for Microsoft
Provides a single environment to author and manage Tests Microsoft Test Manager © GSAtrain30Custom Coded UI Training for Microsoft
Testing Center © GSAtrain31Custom Coded UI Training for Microsoft
Generate Code for Coded UI Test © GSAtrain32Custom Coded UI Training for Microsoft
us/library/dd aspx us/library/dd aspx Quick Start Guide for Manual Testing using Microsoft Test Manager © GSAtrain33Custom Coded UI Training for Microsoft
Introducing Managed Exploratory Testing The 5 Minute “elevator” Video o Studio-2012-Premium-and-Ultimate- Overview/Visual-Studio-Ultimate Easily-reproducing-issues-through-manual- testing Studio-2012-Premium-and-Ultimate- Overview/Visual-Studio-Ultimate Easily-reproducing-issues-through-manual- testing Easily Reproducing Issues Through Manual Testing © GSAtrain34Custom Coded UI Training for Microsoft
Creating a Coded UI Test Using Coded UI Test Builder Module 4 © GSAtrain35Custom Coded UI Training for Microsoft
Test Types Microsoft Test Manager Exploratory Testing Easily reproducing issues through manual testing Creating a Coded UI Test Using Coded UI Test Builder © GSAtrain36Custom Coded UI Training for Microsoft
The first time you run MTM it will prompt o Add Team Foundation Manager “If you don’t know the Name, contact your administrator” It might require “ServerName:Port Number” Microsoft Test Manager © GSAtrain37Custom Coded UI Training for Microsoft
TFS and MTM © GSAtrain38Custom Coded UI Training for Microsoft
MTM is always connected to TFS Connecting Microsoft Test Manager to TFS © GSAtrain39Custom Coded UI Training for Microsoft
Test Plan Properties © GSAtrain40Custom Coded UI Training for Microsoft
Static suite: the content of this suite is manually added test cases Query-based suite: a query-based suite lists all test cases matching a given work item filter Requirements-based suite: this suite shows the test cases associated with a selected TFS requirement MTM Test Suites © GSAtrain41Custom Coded UI Training for Microsoft
© GSAtrain42 Custom Coded UI Training for Microsoft
With Visual Studio 2012 Lab Management it is possible to create an environment based on existing infrastructure without SCVMM o This is great because we can now start using Lab Management without any additional installation required o The lab machines in a standard environment can be physical or virtual (or both) and the virtual machine does not have to be managed by SCVMM or Hyper- V Creating a Lab Environment © GSAtrain43Custom Coded UI Training for Microsoft
To create a standard environment you only need to know the names of the machine you want to add to the environment and a user account to be used to log in and deploy the test agent service onto the machine A standard environment can be connected to directly from MTM and the Test Runner making the integration with lab environments really smooth Creating a Lab Environment (continued) © GSAtrain44Custom Coded UI Training for Microsoft
The alternative to standard environments is to use a SCVMM environment The key features of a SCVMM environment include o Create new machines from templates New lab machines can automatically be created by anyone (having permissions) running MTM It is no longer required to send a request to the operations department to get machines created for a new environment or to do other maintenance operations such as starting, stopping, snapshotting, or restoring machines Creating a Lab Environment (continued) © GSAtrain45Custom Coded UI Training for Microsoft
Support for snapshots o It is possible to work with snapshots from MTM, which enable testers to save state during testing o This is a powerful feature both to unlock the tester if a test case fails (no need to block an environment while waiting for a resolution) but also to give a developer a fixed point in time to reproduce a bug Creating a Lab Environment (continued) © GSAtrain46Custom Coded UI Training for Microsoft
Connection inside MTM o A SCVMM environment can be connected to MTM and the Test Runner just like with the standard environment Creating a Lab Environment (continued) © GSAtrain47Custom Coded UI Training for Microsoft
Number of bugs per state o How many bugs do we have that are active, resolved, or closed? o Are the number of active bugs increasing and number of resolved and closed bugs constant? o Then we need to look into how we perform our testing Number of bugs sent back from testers for more information o A large number might indicate that communication between developers and testers must improve KPIs For Software Testing © GSAtrain48Custom Coded UI Training for Microsoft
Code coverage o This shows us how much of the code has been covered by our automated unit tests o We get the value as a percentage of the whole code Tests run results o How are our tests performing? o Do we have many failed tests? o If so, what can we do to improve this? KPIs For Software Testing (continued) © GSAtrain49Custom Coded UI Training for Microsoft
Percent Requirements covered by test cases o Do we write test cases for all our requirements? o If not, what is the reason? Percent Requirements covered by testing o Do we actually run the tests we have test cases for? o If this figure is low and the figure for Percent Requirements covered by test cases is high we might have an issue we need to deal with KPIs For Software Testing (continued) © GSAtrain50Custom Coded UI Training for Microsoft
First we find all web services for team project collections We all recognize this so no further explanation is necessary: o Version control o Build service o Lab management o Work Item tracking o Team Foundation Framework services Application Tier: What Does TFS Offer? © GSAtrain51Custom Coded UI Training for Microsoft
This set of services is running on the Application Tier of TFS and allows us to access the different aspects of Team Foundation Server, including administration, security, and events They enable us to extend Team Foundation functionality by developing TFS extensions and adapters to integrate third-party tools with TFS 2012 Application Tier: What Does TFS Offer? (continued) © GSAtrain52Custom Coded UI Training for Microsoft
© GSAtrain53 Custom Coded UI Training for Microsoft
© GSAtrain54 Custom Coded UI Training for Microsoft
© GSAtrain55 Custom Coded UI Training for Microsoft
After you have run your tests, you can report on your progress When you use test plans to structure your testing approach, you can answer the following questions: o How many tests have passed or failed? o How many tests still have to be run for this iteration? o When will the testing be complete? o Which areas of the product have high test failure rates? Tester’s Benefits © GSAtrain56Custom Coded UI Training for Microsoft
o Which test configurations have high test failure rates? o Who has the most tests left to be run? o Can resources be reallocated to better balance the remaining testing? o Which build should the testers be using based on code changes and bug fixes? Tester’s Benefits (continued) © GSAtrain57Custom Coded UI Training for Microsoft
Essential Guide for Running Automated Tests from a Test Plan © GSAtrain58Custom Coded UI Training for Microsoft
You perform the test manually while CUIT Test Builder runs in the Background You can specify the Values that should appear in specific fields The CUIT Test Builder Records actions and generates Code Your Main Skills are in Testing not Coding Creating a CUIT © GSAtrain59Custom Coded UI Training for Microsoft
You can extend the CUIT code The CUIT Test Builder builds Structured Code Straightforward to copy and adapt If You Are a Developer… © GSAtrain60Custom Coded UI Training for Microsoft
Visual Studio Ultimate or Visual Studio Premium Required Software © GSAtrain61Custom Coded UI Training for Microsoft
Planned Exploratory Load and Muli-user The 3 Major UI Test Types © GSAtrain62Custom Coded UI Training for Microsoft
Defining Your Testing Effort Using Test Plans Microsoft Test Manager enables you to do three important tasks, as shown in the following illustration: © GSAtrain63Custom Coded UI Training for Microsoft
Exploratory testing seeks to find out how the software actually works, and to ask questions about how it will handle difficult and easy cases The quality of the testing is dependent on the tester's skill of inventing test cases and finding defectstest cases defects The more the tester knows about the product and different test methods, the better the testing will betest methods Exploratory Testing © GSAtrain64Custom Coded UI Training for Microsoft
The main advantage of exploratory testing is that less preparation is needed, important bugs are found quickly, and at execution time, the approach tends to be more intellectually stimulating than execution of scripted tests Another major benefit is that testers can use deductive reasoning based on the results of previous results to guide their future testing on the fly deductive reasoning Benefits of Exploratory Testing © GSAtrain65Custom Coded UI Training for Microsoft
Another benefit is that, after initial testing, most bugs are discovered by some sort of exploratory testing This can be demonstrated logically by stating, "Programs that pass certain tests tend to continue to pass the same tests and are more likely to fail other tests or scenarios that are yet to be explored“ WikiPedia Benefits of Exploratory Testing (continued) © GSAtrain66Custom Coded UI Training for Microsoft
Disadvantages are that tests invented and performed on the fly can't be reviewed in advance (and by that prevent errors in code and test cases), and that it can be difficult to show exactly which tests have been run Freestyle exploratory test ideas, when revisited, are unlikely to be performed in exactly the same manner, which can be an advantage if it is important to find new errors; or a disadvantage if it is more important to repeat specific details of the earlier tests o Can be controlled with specific instruction to the tester, or by preparing automated tests where feasible, appropriate, and necessary, and ideally as close to the unit level as possible DrawBacks of Exploratory Testing © GSAtrain67Custom Coded UI Training for Microsoft
Exploratory testing is particularly suitable if requirements and specifications are incomplete, or if there is lack of timerequirements specifications Same approach can also be used to verify that previous testing has found the most important defects Exploratory Testing © GSAtrain68Custom Coded UI Training for Microsoft
© GSAtrain69 Custom Coded UI Training for Microsoft
Performing Exploratory Testing Using Microsoft Test Manager Performing Exploratory Testing Using Microsoft Test Manager o In Microsoft Test Manager: Open Testing Center Test Do Exploratory Testing o If you don't see the option to “Do Exploratory Testing”, then your team is using a version of TFS prior to 2012 Visual Studio and Exploratory Testing © GSAtrain70Custom Coded UI Training for Microsoft
Automated Functional Tests – Using CUIT Create a CUIT from Scratch Create a CUIT from action recording Making CUIT more Robust and Flexible Coded User Interface Testing © GSAtrain71Custom Coded UI Training for Microsoft
When you create a Coded UI Test, the Coded UI Test Builder creates a map of the user interface under test, and also the test methods, parameters, and assertions for all tests It also creates a class file for each test Contents of a Coded UI Test © GSAtrain72Custom Coded UI Training for Microsoft
Conduct exploratory test sessions Locate, view, and edit your test methods, UI actions and their associated controls in the UI control map Quickly track your test plan progress using the Results tab in Microsoft Test Manager Clone your test plan to work on different releases in parallel Testing - What is New in VS 2012? © GSAtrain73Custom Coded UI Training for Microsoft
Test Windows Store apps running on a remote device using manual tests or exploratory test sessions Install and configure agents more easily for Lab Management for Visual Studio 2012 Include SCVMM 2012 and clustered Hyper- V hosts in Lab Management for Visual Studio 2012 Testing - What is New in VS 2012? (continued) © GSAtrain74Custom Coded UI Training for Microsoft
Contents – Test Configurations © GSAtrain75Custom Coded UI Training for Microsoft
Establish Environments Diagnostic Connectors Best Practice o Work with Development Teams o The Developers need to have the same tools to view the test data you collected for them Microsoft released a “Test Attachment Cleaner” available at o “Aka.ms./TFPowerTools” Configuring Test Settings © GSAtrain76Custom Coded UI Training for Microsoft
Scout o Installing Software and running some initial tests o After the SCOUT is successful it can become a “template for other Testers” There is an available builds DROPDOWN Click “ASSIGN TO PLAN” to add a new build to a plan Using Builds © GSAtrain77Custom Coded UI Training for Microsoft
Quickly identify tests that need to be run based on changes to code Can Run in the background “Track” menu => Recommended Tests o Does not recognize “DATA” changes o Does not recognize other library changes o Does not recognize other Application changes Best Practice - Periodically run all Tests Analyzing Impacted Tests © GSAtrain78Custom Coded UI Training for Microsoft
Environments may differ: o IE 8,9,10 o Ipad Horizontal/Vertical o BYOD o HTML5 o Etc. Defining Multiple Test Configurations © GSAtrain79Custom Coded UI Training for Microsoft
Automating Coded UI Test Cases Module 5 © GSAtrain80Custom Coded UI Training for Microsoft
Requirements-Based test suite Query-Based test suite Static Test Suite o Status of tests in TEST SUITE In planning In Progress Completed Test Cases Are Organized Into Suites © GSAtrain81Custom Coded UI Training for Microsoft
Unit and Component Tests Functional Tests Exploratory Testing Capability Testing Types of Tests © GSAtrain83Custom Coded UI Training for Microsoft
Automated tests written to help the team develop software effectively With good suites of unit and component level tests we have the safety net that helps us develop software incrementally in short iterations without breaking existing functionality Invaluable when refactoring code o With good test coverage a developer should feel confident to make a change without knowing all about every dependency o The tests should tell us if we did anything wrong Unit and Component Tests © GSAtrain84Custom Coded UI Training for Microsoft
Functional tests are mainly our traditional scripted system tests in different flavors It is hard to avoid running these tests manually at first, but we should try to find ways to automate them as we learn more about our product and how it needs to be tested Many functional tests can be automated and then we can focus on early testing for the manual tester Functional Tests © GSAtrain85Custom Coded UI Training for Microsoft
Exploratory testing is a form of software testing in which the individual tester can design and run tests in a freer form Instead of following detailed test scripts, the tester explores the system under tests based on the user stories As the tester learns how the system behaves the tester can optimize the testing work and focus more on testing than documenting the test process Exploratory Testing © GSAtrain86Custom Coded UI Training for Microsoft
Lastly we have the capability tests These tests are run against the behavior of the system; o We test non-functional requirements Performance Security These tests are generally automated and run using special purpose tools, such as load test frameworks and security analyzers Capability Testing © GSAtrain87Custom Coded UI Training for Microsoft
We should leave this category of tests as manual tests The focus should be to catch bugs that would fall through the net of automated tests A key motivation for automated testing is to let do more of exploratory and usability testing because these tests validate how the end-user feels when using the product Exploratory Testing © GSAtrain88Custom Coded UI Training for Microsoft
© GSAtrain89 Custom Coded UI Training for Microsoft
A user story can be stated as simply as o As an Employee I want to have an efficient way to manage my expenses The conversation around this statement between the product owner and the development team can raise questions such as: o Who can submit expense reports? Story Boards © GSAtrain90Custom Coded UI Training for Microsoft
o What states can an expense report be in? o When is it possible to change or remove an expense report? o What data is required in an expense report to register it correctly? o Where are the expense reports stored? For how long? Story Boards (continued) © GSAtrain91Custom Coded UI Training for Microsoft
An expense report has the following state model: o New when created o Pending after submitted for approval o Approved o Rejected Adding Detail to User Stories © GSAtrain92Custom Coded UI Training for Microsoft
Allows the Tester to Verify what should happen If this does not Happen… o See the Tasks that were assigned to this requirement o See the Programmers assigned to Each Task o Notify the Developer(s) Provide the necessary info to avoid “NO REPRO” Add this Test to the “Suite” Story Boards Are Vital to Testing © GSAtrain93Custom Coded UI Training for Microsoft
Introduction to Test Planning with Microsoft Test Manager 2012 o Authoring Test Cases Hands-On Lab © GSAtrain94Custom Coded UI Training for Microsoft
Data Driven Tests Module 6 © GSAtrain95Custom Coded UI Training for Microsoft
Any Files Used in The Tests Example XML to load Variable and Expected Data Could be: o SQL o Business Connectivity Services o SharePoint Lists o SQL Reports o Excel / PowerPivot o SSAS Data Driven Tests © GSAtrain96Custom Coded UI Training for Microsoft
Create errors Add new Cases Modify the rows Automation can be changed by using different files Uses of Data Driven Tests © GSAtrain97Custom Coded UI Training for Microsoft
How to Create a Data-Driven Coded UI Test Hands-On Lab © GSAtrain98Custom Coded UI Training for Microsoft
70% of all Application Testing is MANUAL The UNFORTUNATE REALITY © GSAtrain99Custom Coded UI Training for Microsoft
Anatomy of Coded UI Tests Module 7 © GSAtrain100Custom Coded UI Training for Microsoft
When you create a Coded UI Test, the Coded UI Test Builder creates a map of the user interface under test, and also the test methods, parameters, and assertions for all tests It also creates a class file for each test o us/library/ff aspx us/library/ff aspx Anatomy of Coded UI Tests © GSAtrain101Custom Coded UI Training for Microsoft
FileContentsEditable? UIMap.Designer.cs Declarations section UIMap classUIMap class (partial, auto-generated) Methods Properties No UIMap.csUIMap classUIMap class (partial)Yes CodedUITest1.cs CodedUITest1 class Methods Properties Yes UIMap.uitest The XML map of the UI for the test. No Contents of a Coded UI Test © GSAtrain102Custom Coded UI Training for Microsoft
UIMap.Designer.cs o This file contains code that is automatically created by the Coded UI Test Builder when a test is created o This file is re-created every time that a test changes, so that it is not a file in which you can add or modify code Contents of Coded UI Test (continued) © GSAtrain103Custom Coded UI Training for Microsoft
using System; using System.CodeDom.Compiler; using System.Collections.Generic; using System.Drawing; using System.Text.RegularExpressions; using System.Windows.Input; using Microsoft.VisualStudio.TestTools.UITest.Extension; using Microsoft.VisualStudio.TestTools.UITesting; using Microsoft.VisualStudio.TestTools.UITesting.WinControls; using Microsoft.VisualStudio.TestTools.UnitTesting; using Keyboard = Microsoft.VisualStudio.TestTools.UITesting.Keyboard; using Mouse = Microsoft.VisualStudio.TestTools.UITesting.Mouse; using MouseButtons = System.Windows.Forms.MouseButtons Declarations section © GSAtrain104Custom Coded UI Training for Microsoft
Introduction to Lifecycle Management Module 8 © GSAtrain105Custom Coded UI Training for Microsoft
Visual Studio Ultimate, Visual Studio Premium and Test Professional include a Microsoft Test Manager o To help you define and manage your testing effort by using test plans o You create a test plan and add any test suites, test cases, or configurations that you need Visual Studio 2012 © GSAtrain106Custom Coded UI Training for Microsoft
By using the suite of tools in Visual Studio 2012, and combining those tools with Visual Studio Team Foundation Server, you can apply proven practices to manage your application's lifecycle, from understanding customer needs through code design and implementation to deployment You can use the instrumentation in these tools to trace requirements to checked-in code, builds and test results Lifecycle Management © GSAtrain107Custom Coded UI Training for Microsoft
These practices can help your team create software that is valued by your customers and that is faster and more reliable. You can use these tools to achieve the following results: o Visual Studio Application Lifecycle Management (ALM) provides you and your team with the tools to effectively manage the lifecycle to develop applications o As the following illustration shows, you and your team can use these tools to tell user stories, to manage the backlog, plan a sprint, and run a sprint, to engage stakeholders to provide continuous feedback, and to easily incorporate feedback to support the next iteration of software Lifecycle Management (continued) © GSAtrain108Custom Coded UI Training for Microsoft
ALM Cycles © GSAtrain109Custom Coded UI Training for Microsoft
us/library/bb aspx What's New for ALM in Visual Studio 2012? © GSAtrain110Custom Coded UI Training for Microsoft
In 2012 Visual Studio Improves: o Integration between testing and the rest of the Software Development lifecycle o Example – Better handoff from tester to developer o Less NO REPRO o Improved build-deploy-test workflow o Improved Virtualization Visual Studio 2010 to 2012: Completely Restructured Software Testing © GSAtrain111Custom Coded UI Training for Microsoft
Manual Tests Coded UI Tests Unit Tests Web Performance Tests Load Tests Generic Tests Ordered Tests Types of Tests Supported by VS 2012 © GSAtrain112Custom Coded UI Training for Microsoft
Testing Pyramid © GSAtrain113Custom Coded UI Training for Microsoft
Testing Structures © GSAtrain114Custom Coded UI Training for Microsoft
Demo LAB Using Shared Steps © GSAtrain115Custom Coded UI Training for Microsoft
Demo Assigning Configurations © GSAtrain116Custom Coded UI Training for Microsoft
Require a Human to Interact with an application Verify some expected result Report on the success of the test Manual Tests © GSAtrain117Custom Coded UI Training for Microsoft
Provides the Capability to author tests that automatically interact with user interface o Verify Results o File Bug Report o It can be run Frequently without human intervention o Typically more expensive to author and maintain than manual test Coded UI Tests © GSAtrain118Custom Coded UI Training for Microsoft
Used by Programmers for Programmers Quickly becoming the standard practice A vital part of Test-Driven Development Generally focus on a lower level than other Testing Ideally a unit is the smallest atomic part - One method of One object Unit Tests © GSAtrain119Custom Coded UI Training for Microsoft
Test written before the object It will initially fail o Correct o Re-test o Refactor o Repeat Essential component of Regression Testing Unit Tests (continued) © GSAtrain120Custom Coded UI Training for Microsoft
Verify that the web site is behaving as expected o Add load to see if the application is still capable of Scaling Web Performance Tests © GSAtrain121Custom Coded UI Training for Microsoft
Tests that Application Scales and Performs as Necessary Load Tests © GSAtrain122Custom Coded UI Training for Microsoft
Calling external testing systems Results can be automatically parsed to determine success Could be simple console application to XML document exported from LOB Generic Tests © GSAtrain123Custom Coded UI Training for Microsoft
Container for other tests Specific order of tests Some run more than once Ordered Tests © GSAtrain124Custom Coded UI Training for Microsoft
The Key challenge of Testers Provide Developers with sufficient information to reproduce the problem Most Bugs result in No Repro “It Works on my machine” Diagnostic Data Adapters © GSAtrain125Custom Coded UI Training for Microsoft
A Diagnostic Data Collector is responsible for collecting data about the machine under test o Action Log o ASP.NET Profiler o Code Coverage o IntelliTrace o Event Log o System Information o Test Impact o Video Recorder o Network Emulation Diagnostic Data Adapters © GSAtrain126Custom Coded UI Training for Microsoft
Test-Driven Development Unit Testing Developers view © GSAtrain127Custom Coded UI Training for Microsoft
Is a software development process that relies on the repetition of a very short development cycle:software development process o First the developer writes an (initially failing) automated test casetest case That defines a desired improvement or new function Then produces the minimum amount of code to pass that test Refactors the new code to acceptable standards Refactors Test-driven development (TDD) WikiPedia © GSAtrain128Custom Coded UI Training for Microsoft
Not Unit Testing Not Integration Testing Not Regression Testing Whole Application Testing Tester’s View © GSAtrain129Custom Coded UI Training for Microsoft
When requirements or user stories, or features are ready to be tested, you can run your tests for each configuration that you specified UI – User Interface Testing Usability - Testing Requirements-Driven Testing © GSAtrain130Custom Coded UI Training for Microsoft
Test Runner: o The Test Runner from MTM can create action recordings from a manual test o The action recording can later be used to play back the test session in MTM or to generate a Coded UI test from the recorded steps Coded UI Test: o The implementation of the actual test written in C# or VB.NET o The Coded UI Test is either generated by the Coded UI Test builder code generator or coded directly against the Coded UI Test API Components of Coded UI Testing © GSAtrain131Custom Coded UI Training for Microsoft
Test Recorder: Records actions from a test session, which then can be used by the Test Runner to automate a generated test or by the coded generator to create a Coded UI test Coded UI Test API: The Coded UI Test API can be used to create UI tests which are more tightly connected to the UI elements Components of Coded UI Testing (continued) © GSAtrain132Custom Coded UI Training for Microsoft
Technology Managers: These are the core components responsible for mapping UI elements to the underlying technology (i.e. web browser or a rich client) We can implement custom plug-ins to support platforms which the native plug-ins does not support Components of Coded UI Testing (continued) © GSAtrain133Custom Coded UI Training for Microsoft
© GSAtrain134 Custom Coded UI Training for Microsoft
These tests are critical to project success The number keeps growing Regression testing is vital Sometimes hundreds or even thousand of tests Managing Automated Tests © GSAtrain135Custom Coded UI Training for Microsoft
Internet Explorer 8, 9, 10 Windows Forms 2.0 and later WPF 3.5 and later Windows Win32 (not officially supported but may work) MFC (partially) SharePoint (partially) Dynamics CRM Citrix/Terminal Services (partially) PowerBuilder (partially) Supported Platforms © GSAtrain136Custom Coded UI Training for Microsoft
Coded UI Test Builder Interface © GSAtrain137Custom Coded UI Training for Microsoft
A common requirement of automated tests is to be able to run the test a number of times with different parameters o We have already looked at how we can work with data- driven test in details o The same techniques are valid for coded UI tests as well except that the data source for a UI test often points to the data set managed in the associated test case o Using the test case to manage test data for the automated test is nice because then the tester can update the data directly from inside MTM Parameterized Testing © GSAtrain138Custom Coded UI Training for Microsoft
Coded UI tests are probably the ideal type of test for regression testing because they give us the opportunity to automatically test the system just like a real user would But there are a number of issues that make UI tests difficult as a general purpose solution for automation solution, including o Requires a UI o Requires a desktop session to run o UI tests are more complex to design and maintain o Difficult to scale Regression Testing © GSAtrain139Custom Coded UI Training for Microsoft
To work around these challenges we generally recommend considering “under- the-UI” tests as a complement to the UI tests These tests could be written against the APIs in the system or against the controller layer in an MVC style of application “Under-the-UI” Testing © GSAtrain140Custom Coded UI Training for Microsoft
Another option for running tests from the command-line is to use the Test Case Management command-line tool, TCM.exe o This tool is installed under C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\TCM.exe There are several options for us to start a test run, for instance: o Run all tests in a test suite o Locate the Suite Id from the test suite in MTM tcm run /create /title:"UI Test Run" /planid:5 /suiteid:11 /configid:2 /collection:http://tfs.kangreen.com:8080/tfs /teamproject:"Expense Reporting" Test Case Management Command-Line Tool © GSAtrain141Custom Coded UI Training for Microsoft
You can use tcm.exe to figure out the Ids for plans, suites, and configurations as well: o List plans tcm plans /list /collection:http://tfs.kangreen.com:8080/tfs /teamproject:"Expense Reporting" o List suites tcm suites /list /collection:http://tfs.kangreen.com:8080/tfs /teamproject:"Expense Reporting" o List configurations tcm configs /list /collection:http://tfs.kangreen.com:8080/tfs /teamproject:"Expense Reporting" Test Case Management Command-Line Tool (continued) © GSAtrain142Custom Coded UI Training for Microsoft
Studio-2012-Premium-and-Ultimate- Overview/Visual-Studio-Ultimate Easily-testing-user-interfaces-with- Coded-UI-tests Studio-2012-Premium-and-Ultimate- Overview/Visual-Studio-Ultimate Easily-testing-user-interfaces-with- Coded-UI-tests Demo Creating a Test Project © GSAtrain143Custom Coded UI Training for Microsoft
Components of a Test Plan © GSAtrain144Custom Coded UI Training for Microsoft
Automated tests can quickly test whether your application is still working correctly after code changes have been made to your application Automating Test should not take more than about 40% of team members' time Benefits of Automated Testing © GSAtrain145Custom Coded UI Training for Microsoft
The problem with user interface tests is not in how to execute them, but what to test Since human testers would often check the whole round-trip of information, from the GUI to the back-end report First instinct is to replicate that and try to verify business domain rules. This is the Achilles’ heel of UI tests, and it ultimately leads to enormous waste of time The Achilles’ Heel of Automated Tests © GSAtrain146Custom Coded UI Training for Microsoft
UI Testing Without Shooting Yourself in the Foot without-shooting-yourself-in-the-foot-2/ © GSAtrain147Custom Coded UI Training for Microsoft
The idea of thinking about these different levels is good because it allows us to write o UI-level tests that are easy to understand o Efficient to write o Relatively inexpensive to maintain Thinking About These Different Levels © GSAtrain148Custom Coded UI Training for Microsoft
Checking that delivery is available for two books involves putting a book in a shopping cart o Putting a book in a shopping cart involves a sequence of technical steps o Entering address details does as well Breaking things down like that and combining lower level concepts into higher level concepts reduces the cognitive load and promotes reuse Thinking About These Different Levels (continued) © GSAtrain149Custom Coded UI Training for Microsoft
You can plan, manage and execute both manual and exploratory tests You can also automate your manual tests after they are stabilized While you are performing manual and exploratory tests, you can log bugs o The bug will contain a trace of your recent actions, a snapshot of the state of the system, and a copy of any notes you made while exploring the system o You can record your actions in the test case, so that they can be played back on later occasions Using Microsoft Test Manager © GSAtrain150Custom Coded UI Training for Microsoft
Manual to Coded UI © GSAtrain151Custom Coded UI Training for Microsoft
Extending Coded UI Tests and Action Recordings Module 9 © GSAtrain152Custom Coded UI Training for Microsoft
Supported Configurations o us/library/dd aspx Extending Coded UI Tests and Action Recordings © GSAtrain153Custom Coded UI Training for Microsoft
Upgrading Coded UI Tests from Visual Studio 2010 Module 10 © GSAtrain154Custom Coded UI Training for Microsoft
Extending Coded UI Tests and Action Recordings Module 11 © GSAtrain155Custom Coded UI Training for Microsoft
The testing framework for coded UI tests and action recordings does not support every possible user interface o It might not support the specific UI that you want to test o For example, you cannot immediately create a coded UI test or an action recording for a Microsoft Excel spreadsheet However, you can create your own extension to the coded UI test framework that will support your specific UI by taking advantage of the extensibility of the coded UI test framework Extending Coded UI Tests and Action Recordings to Support Microsoft Excel © GSAtrain156Custom Coded UI Training for Microsoft
The following topic gives an example of how to extend the framework to support the creation of coded UI tests and action recordings for Microsoft Excel o For more information about the platforms that are supported, see Supported Configurations and Platforms for Coded UI Tests and Action RecordingsSupported Configurations and Platforms for Coded UI Tests and Action Recordings This section presents a coded UI test extension that can record and play back tests of Excel Worksheets o Each part of the extension is explained in this section and in the code comments for developers who want to create just such an extension Extending Coded UI Tests and Action Recordings to Support Microsoft Excel (continued) © GSAtrain157Custom Coded UI Training for Microsoft
Architecture Overview © GSAtrain158Custom Coded UI Training for Microsoft
Testing Large Applications with Multiple UI Maps Module 12 © GSAtrain159Custom Coded UI Training for Microsoft
Using HTML5 Controls in Coded UI Tests Module 13 © GSAtrain160Custom Coded UI Training for Microsoft
Using WPF with Coded UI Tests Module 14 © GSAtrain161Custom Coded UI Training for Microsoft
Best Practices for Coded UI Tests Module 15 © GSAtrain162Custom Coded UI Training for Microsoft
Use the Coded UI Test Builder whenever possible Do not modify the UIMap.designer.cs file directly. If you do this, the changes to the file will be overwritten Create your test as a sequence of recorded methods. For more information about how to record a method, see Creating Coded UI TestsCreating Coded UI Tests Each recorded method should act on a single page, form, or dialog box Create a new test method for each new page, form, or dialog box Best Practices For Coded UI Tests © GSAtrain163Custom Coded UI Training for Microsoft
When you create a method, use a meaningful method name instead of the default name A meaningful name helps identify the purpose of the method When possible, limit each recorded method to fewer than 10 actions This modular approach makes it easier to replace a method if the UI changes Create each assertion using the Coded UI Test Builder, which automatically adds an assertion method to the UIMap.Designer.cs file Best Practices For Coded UI Tests (continued) © GSAtrain164Custom Coded UI Training for Microsoft
If the user interface (UI) changes, re-record the test methods, or the assertion methods, or re-record the affected sections of an existing test method Create a separate UIMap file for each module in your application under test. For more information, see Testing a Large Application with Multiple UI MapsUIMapTesting a Large Application with Multiple UI Maps In the application under test, use meaningful names when you create the UI controls This gives more meaning and usability to the automatically generated control names Best Practices For Coded UI Tests (continued) © GSAtrain165Custom Coded UI Training for Microsoft
If you are creating assertions by coding with the API, create a method for each assertion in the part of the UIMap class that is in the UIMap.cs file UIMap Call this method from your test method to execute the assertion If you are directly coding with the API, use the properties and methods in the classes generated in the UIMap.Designer.cs file in your code as much as you can These classes will make your work easier, more reliable, and will help you be more productive Best Practices For Coded UI Tests (continued) © GSAtrain166Custom Coded UI Training for Microsoft
User interfaces frequently change during development Here are some ways to reduce the effect of these changes: Find the recorded method which references this control and use the Coded UI Test Builder to re- record the actions for this method You can use the same name for the method to overwrite the existing actions What to Do If Your User Interface Changes © GSAtrain167Custom Coded UI Training for Microsoft
If a control has an assertion that is no longer valid: Delete the method that contains the assertion Remove the call to this method from the test method Add a new assertion by dragging the cross- hair button onto the UI control, open the UI map, and add the new assertion What to Do If Your User Interface Changes (continued) © GSAtrain168Custom Coded UI Training for Microsoft
// Set the playback to wait for all threads to finish Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.AllThreads; // Press the submit button this.UIMap.ClickSubmit(); // Reset the playback to wait only for the UI thread to finish Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.UIThreadOnly; What To Do If A Background Process Needs to Complete Before The Test Can Continue © GSAtrain169Custom Coded UI Training for Microsoft
archive/2012/06/05/enabling-coded-ui- test-playback-logs-in-visual-studio release-candidate.aspx archive/2012/06/05/enabling-coded-ui- test-playback-logs-in-visual-studio release-candidate.aspx Using the Coded UI Test Logs © GSAtrain170Custom Coded UI Training for Microsoft
Evolution of Testing Module 17 © GSAtrain171Custom Coded UI Training for Microsoft
In-Line Programs Sub-Routines Functions Time Sharing On-Line Terminals Client Sever 3 Tier Architecture Thin Client MVC SOA Evolution of Testing © GSAtrain172Custom Coded UI Training for Microsoft
Developers – View Individual Units of Code Administrators Build Environments and Versions Testers Test Complete Applications De Coupling of Software Testing © GSAtrain173Custom Coded UI Training for Microsoft
Inability of TESTERS to test BlackBox Code o 10,000 lines of code could result in 1 Million Routes o Impossible to Test Interface – Contract testing – is practical CLOUD increases the environment exponentially o Interfaces are constant Primary Drivers of SOA © GSAtrain174Custom Coded UI Training for Microsoft
Microsoft Windows 2012 Windows 8 Office 2013 Office365 SQL 2012 SharePoint 2012 LINQ 2012 o are all Cloud – SOA Oriented Microsoft’s Commitment To The Cloud © GSAtrain175Custom Coded UI Training for Microsoft
The platform includes five services: o Live Services for communication and collaboration services o SQL Azure (formerly SQL Services) for data-storage services o AppFabric (formerly.NET Services) for cloud-computing services o SharePoint Services for web-content-management and document-management services o Dynamics CRM Services for customer-relationship- management services Azure Services © GSAtrain176Custom Coded UI Training for Microsoft
Microsoft TFS is now Cloud Based Testing is Moving To The CLOUD © GSAtrain177Custom Coded UI Training for Microsoft
© GSAtrain178Custom Coded UI Training for Microsoft
Collaboration is the Key to TFS o Developers o Testers – Creating Reproducible Automated Tests o Administrators – Builders of (Virtual Machines and Versions) Using TFS in the cloud can help us enhance collaboration It can also enhance collaboration in an organization with geographically spread locations Benefits of TFS in the Cloud © GSAtrain179Custom Coded UI Training for Microsoft
With TFS in the cloud is that we can neither add our own process template nor make adjustments to the existing templates Concerns with TFS in the Cloud © GSAtrain180Custom Coded UI Training for Microsoft
Interface Testing Viewing Objects from the outside Leads to Unit Testing Test-Driven Design Service Oriented Architecture © GSAtrain181Custom Coded UI Training for Microsoft
IT Has a Problem! ACCELERATING RATE OF CHANGE © GSAtrain182Custom Coded UI Training for Microsoft
Reasons for SOA Rapid Adoption © GSAtrain183Custom Coded UI Training for Microsoft
The short answer is: Cloud is an initial application of SOA to infrastructure --its setup, configuration, monitoring, management, with the essential elements of metering and billing added to satisfy the goal of a charge model of e.g., pay-as-you-go -- the specific requirements for XaaS: resource pooling and virtualization, elasticity and multi-tenancy, dynamic configuration and provisioning Cloud paradigm leverages SOA to deliver a charge-based model for non-business specific services; -- infrastructure/hardware -- software --platform https://www.ibm.com/developerworks/mydeveloperworks/blogs/AliArsanjani/?order=asc&maxresults =15&sortby=0&lang=en https://www.ibm.com/developerworks/mydeveloperworks/blogs/AliArsanjani/?order=asc&maxresults =15&sortby=0&lang=en What is the Relationship Between Cloud and SOA © GSAtrain184Custom Coded UI Training for Microsoft
SIMM https://www2.opengroup.org/ogsys/publicati ons/viewDocument.html?publicationid=12450 &documentid=11340www2.opengroup.org/ogsys/publicati ons/viewDocument.html?publicationid=12450 &documentid=11340 © GSAtrain185Custom Coded UI Training for Microsoft
Vulnerabilities Security Testing DOD is Insisting on SOA © GSAtrain186Custom Coded UI Training for Microsoft
Discoverable Self-Describing Composable Loose Coupling Governed by Policy Independent of Location, Language and Protocol Definition of a Service © GSAtrain187Custom Coded UI Training for Microsoft
SOA Maturity Model © GSAtrain188Custom Coded UI Training for Microsoft
IT Benefits Of SOA © GSAtrain189Custom Coded UI Training for Microsoft
SOA © GSAtrain190Custom Coded UI Training for Microsoft
SOA © GSAtrain191Custom Coded UI Training for Microsoft
SOA © GSAtrain192Custom Coded UI Training for Microsoft
© GSAtrain193Custom Coded UI Training for Microsoft
SOA © GSAtrain194Custom Coded UI Training for Microsoft
Automating Coded UI Testing using Microsoft Testing Tools in Visual Studio 2012 Clive G Hermann.
Jan 25 th 2013 Test Automation: Coded UI Test 1.Introduction 2.Internal Class Structure (Generic Structure) 3.Program architecture 4.Framework function.
03 | Manage Team Foundation Server Steven Borg | Co-founder & Strategist, Northwest Cadence Anthony Borton | ALM Consultant, Enhance ALM.
QWise software engineering – refactored! Testing, testing A first-look at the new testing capabilities in Visual Studio 2010 Mathias Olausson.
Software Testing with Visual Studio 2013 & Team Foundation Server 2013 Benjamin Day.
Developing Workflows with SharePoint Designer David Coe Application Development Consultant Microsoft Corporation.
Ognjen Bajić Ana Roje Ivančić Ekobit Efficient Application Testing.
©Ian Sommerville 2000Software Engineering, 6th edition. Chapter 29Slide 1 Configuration management l Managing the products of system change l Objectives.
MTA EXAM Software Testing Fundamentals : OBJECTIVE 6 Automate Software Testing.
Test Case Management and Results Tracking System October 2008 D E L I V E R I N G Q U A L I T Y (Short Version)
©2011 Quest Software, Inc. All rights reserved. Steve Walch, Senior Product Manager Blog: November, 2011 Partner Training Webcast.
Configuration management l Managing the products of system change l Objectives To explain the importance of software configuration management (CM) To describe.
Continuous Delivery and Team Foundation Server 2013 Ognjen Bajić Ana Roje Ivančić Ekobit.
Classroom User Training June 29, 2005 Presented by:
University of Palestine software engineering department Testing of Software Systems Fundamentals of testing instructor: Tasneem Darwish.
Presenter: Byron Goodman National Director of Quality Assurance Neudesic.
A Taste of Visual Studio 2005 David Grey. Introduction In this session we will introduce Visual Studio 2005 and its features and examine those features.
04b | Manage Test Execution (2 of 2) Steven Borg | Co-founder & Strategist, Northwest Cadence Anthony Borton | ALM Consultant, Enhance ALM.
| | Tel: | | Computer Training & Personal Development Microsoft Office PowerPoint 2007 Expert.
1 HP Quality Center Overview. 2 Deliver optimal business results through high-quality applications HP Quality Center core Supporting key stakeholders.
06 | Develop Customer Value with High Quality Anthony Borton | ALM Consultant, Enhance ALM Steven Borg | Co-founder & Strategist, Northwest Cadence.
Program Management Practice Program Management Portal: Overview for the Client © 2011, 2014 IBM Corporation 1.
Introduction to KE EMu Training Page 1 Introduction to KE EMu
Microsoft Office Excel 2010 ® ® Tutorial 8: Developing an Excel Application.
V7 Foundation Series Vignette Education Services.
Mike Azocar Sr. Developer Technical Specialist Microsoft Corporation
Microsoft Management Seminar Series SMS 2003 Change Management.
Microsoft Office Word 2013 Expert Microsoft Office Word 2013 Expert Courseware # 3251 Lesson 4: Working with Forms.
Understand Application Lifecycle Management Software Development Fundamentals LESSON 3.1.
Microsoft Office Illustrated Fundamentals Unit C: Getting Started with Unit C: Getting Started with Microsoft Office 2010 Microsoft Office 2010.
Windows Vista Inside Out Chapter 22 - Monitoring System Activities with Event Viewer Last modified am.
3/12/2013Computer Engg, IIT(BHU)1 CLOUD COMPUTING-1.
© 2004, The Trustees of Indiana University 1 OneStart Workflow Basics Brian McGough, Manager, Systems Integration, UITS Ryan Kirkendall, Lead Developer.
User Group 2015 Version 5 Features & Infrastructure Enhancements.
Module 7: Fundamentals of Administering Windows Server 2008.
Power BI Sites and Mobile BI. What You Will Learn Sharing and Collaboration Introducing Power BI Exploring Power BI Features and Services Partner Opportunities.
Microsoft ® Office Excel 2003 Training Using XML in Excel SynAppSys Educational Services presents:
Microsoft ® Official Course Monitoring and Troubleshooting Custom SharePoint Solutions SharePoint Practice Microsoft SharePoint 2013.
Introduction Lesson 1 Microsoft Office 2010 and the Internet Microsoft Office 2010 Advanced Cable / Morrison 1.
Maintaining and Updating Windows Server 2008 Lesson 8.
ECATS The Honeywell Web-based Corrective Action Solution Overview Last Revised: January 14, 2002 Honeywell Confidential & Proprietary.
1 The Software Development Process Analysis Design Implementation Testing Documentation Evaluation Maintenance.
Architecture View Models A model is a complete, simplified description of a system from a particular perspective or viewpoint. There is no single view.
Designing, Deploying and Managing Workflow in SharePoint Sites Steve Heaney Product Development Manager OBS
XP 1 Microsoft Access 9 Automating Tasks With Macros.
MCTS Guide to Microsoft Windows Server 2008 Applications Infrastructure Configuration (Exam # ) Chapter Four Windows Server 2008 Remote Desktop Services,
1 Shawlands Academy Higher Computing Software Development Unit.
@GPUG WHAT’S NEW IN DYNAMICS GP 2016 ? Presented by Beat Bucher, Dynamics GP MVP 1 Tweet During today’s #GPUG, #GPUG.
Testing Relational Database. Overview Once the design of a database system has been completed, the developers are ready to move into the implementation.
Troubleshooting Windows Vista Lesson 11. Skills Matrix Technology SkillObjective DomainObjective # Troubleshooting Installation and Startup Issues Troubleshoot.
© 2017 SlidePlayer.com Inc. All rights reserved.