Offshore Testing Pitfalls and Strategies to Help Succeed Tracey Clark Michael Cookson SWOSQG Meeting October 2nd 2012

Agenda Why offshore testing? Why offshore testing? Choosing the team Choosing the team Four areas: Four areas: –Getting Started –Executing the project –Finalizing the project –Lessons Learned Final Q&A Final Q&A –If at any time, you want to ask a question or debate a point, shout it out

Common pitfalls What do you think some of the common pitfalls of offshore testing?

Offshore Testing Why Offshore?

Offshore Testing Definition Definition –Using a test team located in another country to perform some or all aspects of a projects testing –May or may not be part of your company –May or may not be a one time project

Offshore vs. Nearshore Nearshore is usually considered same geographical region Nearshore is usually considered same geographical region –Waterloo -> Newfoundland Offshore usually different geo Offshore usually different geo –Waterloo – India / China

Offshore vs. Out-sourcing Offshore – is the act of moving business functions, that is in house and a part of your region, to another lower cost region Offshore – is the act of moving business functions, that is in house and a part of your region, to another lower cost region Outsourcing – loosely defined, but basically means use of a third party company outside your organization to perform a function for your company Outsourcing – loosely defined, but basically means use of a third party company outside your organization to perform a function for your company

Out-sourcing vs. In-sourcing Out-sourcing – contracting of an entire function to another company Out-sourcing – contracting of an entire function to another company In-sourcing – moving an entire function to another area of the company In-sourcing – moving an entire function to another area of the company Out-sourcing and in-sourcing can involve offshore, nearshore or resources in same locale as you Out-sourcing and in-sourcing can involve offshore, nearshore or resources in same locale as you

Out-SourcingIn-Sourcing Offshore Having a 3rd party company take over your whole IT Support function and running it with resources in Brazil Moving your IT support function to in house resources located in Brazil Nearshore Having a 3rd party company take over your whole IT support function and running it with resources In the same city or region Moving your IT Support function to in house resources located in Newfoundland

Offshore Testing

Why is your company doing this? Why is your company doing this? –Critical to understand this A one time project to deal with a resource shortage A one time project to deal with a resource shortage A cost saving, short term solution A cost saving, short term solution A long term strategic initiative A long term strategic initiative –Where does your project relate to the goals above One of the first to be performing this One of the first to be performing this Part of a well-established process Part of a well-established process

Offshore Testing Choosing the Team

Offshore Testing Choosing the team / company you will be working with Choosing the team / company you will be working with –Many pitfalls here –How do you verify capabilities? –How can you verify claims made? –The path you choose will depend on why you are doing this in the first place

Offshore Testing Selecting Your Company Selecting Your Company –If your company has been doing this for a while, is there a knowledge base you can tap into? Has this company been used before by your company? Has this company been used before by your company? References – any other company you can go to? References – any other company you can go to? –What is the turnover rate of this company in the last year?

Offshore Testing If this is strategic and long term, do things like any other new strategy If this is strategic and long term, do things like any other new strategy –Start small – pick a pilot project that: Is not a ‘break the company’ project Is not a ‘break the company’ project Does not have extremely short deadlines Does not have extremely short deadlines Does not have many other high-risk items Does not have many other high-risk items

Offshore Testing Getting Started

Offshore Testing Getting started - Questions to ask Getting started - Questions to ask –How do you get the offshore team up to speed on: How the project is to be run How the project is to be run Documentation standards and storage Documentation standards and storage Reporting standards Reporting standards Other internal standards and procedures that are critical to your project success Other internal standards and procedures that are critical to your project success

Offshore Testing Getting Started Getting Started –How many people are on the offshore team and who are they? Need to set up usernames, security Need to set up usernames, security –Do they have people who can handle hardware / installation and network issues? If you ship equipment, is there someone there who can make sure the set up works? If you ship equipment, is there someone there who can make sure the set up works? Is that part of the contract? Is that part of the contract?

Offshore Testing Getting Started Getting Started –Licensing issues Many vendors have geo specific licenses – if you give access are you in violation Many vendors have geo specific licenses – if you give access are you in violation –Import / Export regulation Software and hardware (if it needs to be shipped) are subject to import / export regulations for both countries. Do you understand the implications if any? Software and hardware (if it needs to be shipped) are subject to import / export regulations for both countries. Do you understand the implications if any? –Intellectual property What can you share and what can not? What can you share and what can not?

Offshore Testing Getting Started Getting Started –Where will the testing environments be? Do you need to ship equipments – are there specific hardware / operating system / software Do you need to ship equipments – are there specific hardware / operating system / software –What internal systems will the offshore team need access to? Mail; defect tracking; automated script repositories; documentation sites Mail; defect tracking; automated script repositories; documentation sites 24 x 7 IT / Support desk? 24 x 7 IT / Support desk?

Offshore Testing Getting Started Getting Started –Is there training on the applications / tools required Are all your trainers local and do you need to send them out? Was that budgeted for? Are all your trainers local and do you need to send them out? Was that budgeted for? –Network connectivity Firewall and bandwidth issues (performance) Firewall and bandwidth issues (performance) Especially if offshore team needs to access tools remotely Especially if offshore team needs to access tools remotely –System Availability Is there a regular maintenance window for the systems you are using that will conflict with the hours of operation for the offshore team Is there a regular maintenance window for the systems you are using that will conflict with the hours of operation for the offshore team

Offshore Testing Communication Strategies Communication Strategies –Webex –Video conferencing –Conference calls – can you dial internationally to all locales –Messaging – MSN, gmail, Sametime … –Project repositories / team rooms

Offshore Testing Executing the Project

Offshore Testing Executing the Project Executing the Project –Ensure you understand the contract What is explicitly included / excluded? What is explicitly included / excluded? –Make sure everyone is on the same page before the project actually begins regarding these expectations

Offshore Testing Executing the Project Executing the Project –Most companies have some governance model for running projects Does yours’ include sections on dealing with offshore? Does yours’ include sections on dealing with offshore? –Yes – what do you need to do differently? –No – what needs to be added? –Communication Keeping the team in the loop on what has been done, what the issues are and what the focus is Keeping the team in the loop on what has been done, what the issues are and what the focus is

Offshore Testing Executing the Project Executing the Project –Do you need an onshore resource? Someone who represents the offshore team but is situated locally. Understand labour laws and what is allowed. This person is usually part of the offshore company, not a resource on your team. Someone who represents the offshore team but is situated locally. Understand labour laws and what is allowed. This person is usually part of the offshore company, not a resource on your team. –Staggering start times for all teams –Measurements Productivity vs. Status Productivity vs. Status –You will get what you measure –What is in the contract?

Offshore Testing Executing the Project Executing the Project –Capturing knowledge As the offshore test team works through the product, they will acquire knowledge about the product – how do you capture this? As the offshore test team works through the product, they will acquire knowledge about the product – how do you capture this? Turn over rate is currently very high with many offshore companies Turn over rate is currently very high with many offshore companies –Sharing knowledge As turnover happens, how do you get new people up to speed? As turnover happens, how do you get new people up to speed?

Offshore Testing Noon GMT8:00 EST5:00 PST17:00 IST19:00 CST20:00 JST WaterlooVancouverLondon UKBangalore INBeijing CNTokyo JP

Offshore Testing Finalizing the Project

Offshore Testing Finalizing the Project Finalizing the Project –Do you need to close out the project or will it be an ongoing effort? If closing: Retrieve equipment; remove access; ensure intellectual property is accounted for If closing: Retrieve equipment; remove access; ensure intellectual property is accounted for If ongoing: are there budget changes need to ensure proper accounting for resources moving forward? Includes new hardware needed, old hardware returned If ongoing: are there budget changes need to ensure proper accounting for resources moving forward? Includes new hardware needed, old hardware returned

Offshore Testing Lessons Learned

Offshore Testing Lessons Learned Lessons Learned –Even if you have been doing this as a company for an extended period of time, there are always lessons learned –Is there a process in place to capture this and share it with the next project –How did this company fair compare to the last project or if new, the last company –What improvements can be made based on the lessons you just learned?

Offshore Testing Final Questions

Offshore Testing Your Presenters Tracey Clark and Michael Cookson

Tracey Clark Tracey Clark is currently a QA manager at Research In Motion. She was worked there for over 2 years and manages a team of over 60 resources including both onshore and offshore. Tracey previously worked at Sun Life for almost twenty years with ten years of experience in quality assurance and testing activities. She has supported a variety of business units including Individual and Group Retirement Services and has tested many different technologies including desktop applications, web sites, and mainframe systems. Tracey also champions quality, leads process improvement initiatives, and consults on large, complex and challenging projects. Tracey also participates regularly in local SQA forums such as SWOSQG and KWSQA. Tracey can be reached at

Mike Cookson Mike Cookson has been working in the software industry for over 20 years, the majority Of that time in the Quality Assurance and Testing area. He has worked in the financial Sector and most recently testing and deploying e-commerce applications for a world Wide audience. Mike is currently working at IBM in the Tivoli Software Group, working as the Manager of Operations for the Services Hosting Center. Mike can be reached at: or Linkedin or through the South Western Ontario Software Quality Group