Presentation on theme: "Test Data Privacy Best Practices Methodology Bill Mackey Subject Matter Expert."— Presentation transcript:
Test Data Privacy Best Practices Methodology Bill Mackey Subject Matter Expert
Introduction Why Do Companies Care About Data Privacy? 2
Worldwide Data Privacy Drivers Regulatory Compliance… – United States Gramm-Leach-Bliley Act, Sarbanes-Oxley Act – European Union Personal Data Protection Directive, 1998 – Health Insurance Portability and Accountability Act (HIPAA) – Australia Privacy Amendment Act of 2000 – Japanese Personal Information Protection Law – Canadian Personal Information Protection and Electronic Documents Act (PIPEDA) Internal auditors are forcing data protection controls and procedures, especially for offshore use/outsourcing arrangements Risk of exposure can cause significant damage – Corporate embarrassment, lawsuits, negative press, fines/penalties, loss of customers, etc.
Data Breaches Reported Since the ChoicePoint Incident 2846 Incidents Reported Between – ,066,426 Consumers Impacted The catalyst for reporting data breaches to the affected individuals has been the California law that requires notice of security breaches, the first of its kind in the nation, implemented July Personal information compromised includes data elements useful to identity thieves, such as Social Security numbers, account numbers, and driver's license numbers. A Chronology of Data Breaches Reported Since the ChoicePoint Incident Privacy Rights Clearinghouse, January 19, 2012
How are Companies Addressing this Issue? Signing non-disclosure agreements Restricting security access to sensitive/ confidential data Applying minimal “de-identifying” rules Implementing a complete data disguise solution with processes and procedures Low Effectiveness High Effectiveness
Best Practices Approach to Data Privacy 6
Technology alone is not the answer 7 Services Repeatable Best Practices Assessment Implementation Superior Expertise with o 3rd Party Software o Financial o Healthcare o Government Meet dates within high risk projects Services Repeatable Best Practices Assessment Implementation Superior Expertise with o 3rd Party Software o Financial o Healthcare o Government Meet dates within high risk projects Technology Related Data Extraction Data Sub-setting Data Format Conversion Disguise Rules Definition Common Rules Across the Enterprise Unified Rules Repository Support for Mainframe and Distributed Environments Roles Based Authorization Audit and Reporting Technology Related Data Extraction Data Sub-setting Data Format Conversion Disguise Rules Definition Common Rules Across the Enterprise Unified Rules Repository Support for Mainframe and Distributed Environments Roles Based Authorization Audit and Reporting Methodology Data Analysis o Analyze metadata o Discover PII o Classify data Design o Associate disguise rules o Define extract criteria o Identify target environment(s) o Identify load method(s) o Define population strategy Develop o Extract data and relationships o Apply rules across data sources o Load data Deliver o Produce reports o Audit results o Enable best practices Methodology Data Analysis o Analyze metadata o Discover PII o Classify data Design o Associate disguise rules o Define extract criteria o Identify target environment(s) o Identify load method(s) o Define population strategy Develop o Extract data and relationships o Apply rules across data sources o Load data Deliver o Produce reports o Audit results o Enable best practices Comprehensive Solution
Deliver – Deploy and maintain data protection processesDevelop – Build the processes to disguise test dataDesign – Define strategies for disguising test data Process: Data Privacy Methodology Analyze – Understand each application’s sensitive information
Data Privacy Best Practices 9
Data Privacy Project Plan 10
Data Privacy Best Practices Process Overview 11
Deployment Approaches Two project approaches: – Progressive: Organizations that have large numbers of applications and multiple lines of business benefit more from a progressive approach. The progressive approach builds upon the success of early efforts, building up a library of disguise routines and process definitions that align with existing projects within the organization. – Parallel: Organizations that have small to medium numbers of applications benefit more from the parallel approach. The parallel approach covers a wider range of applications at the same time, which is possible when the applications are less intertwined or more independent. Both approaches use a risk based methodology.
Operational Structure Centralized- A single team responsible for performing the data masking function for all lines of business or application areas. This organization is also often referred to as a center of excellence model. Benefits Fewer resources need to be trained on the data disguise software and activities; Increased control over consistency of the disguise techniques and behavior; and Increased productivity of these resources as they work across applications. Drawbacks Increased effort during the Analyze phase as these resources gain the necessary application centric knowledge; Increased duration as there are typically less of these resources, so more effort with less people results in long duration. Decentralized- Each application group is responsible for the data masking functions. Benefits Existing application domain knowledge can be leveraged; The duration of Analyze phase may be shortened as activities can be performed in parallel; and This model streamlines the communication model between the groups. Drawbacks Increased effort related to training; and Increased demand on communications in order to maintain consistency.
Process: How we get there Establish an actionable roadmap Determine the scope Establish a strategy Identify constraints (internal and external) Select the technology Recognized and adaptable Support multiple environments, platforms, & techniques Partner to gain the experience Minimize first time hurdles, pit-falls, & dead-ends Maximize analysis and design efficiency
Project Overview – Planning 15
Project Phases 16 Perform the Analyze methodology phase Data Model Analysis Function Model Analysis Perform the Design methodology phase Design extract process Design disguise techniques Design load process Perform the Develop methodology phase Creation and population of Translation/Association tables Creation and population of Encryption keys Development and Unit Testing of Extract/Disguise/Load tasks Perform the Deliver methodology phase Create the repeatable process
Data Privacy Analysis Phase 17
Analysis 18 Analysis phase can be broken down into two major activities: – Identification and documentation of the data model (DM), – identification and documentation of the functional model (FM) components of the application. These two activities provide the cornerstone for a Data Privacy initiative, and as such, are arguably the most critical of the entire project scope.
Managing Analysis Tasks 19
Data Model Analysis 20 The goal of the Data Model Analysis activities is to provide knowledge about the environment’s data. determine the elements that are considered sensitive define their association to other data objects.
Data Privacy_ _Data_Model_Analysis
Function Model Analysis 22 identifies and documents information about the application processes. determine what business rules and logic apply to the data considered sensitive or private. Outline how the affected data should be changed. Identify all data validations and checks done against sensitive fields within the application programs.
Design Overview 28 Design is the second phase of the Compuware Data Privacy Best Practices methodology and it is broken down into three major activities: – Documentation of the Data Extracts to be created – Identification and documentation of the data disguise rules to be created/implemented – Documentation of the Data Loads to be created These activities provide the background for the creation of the actual rules and specifications to create a Disguised copy of the data
Design 29 Define application disguise strategy and process – Field-level disguise rules (encrypt, translate, age, generate) – Source extract criteria for data (filters, naming conventions, etc.) – Security rules for supporting files – Structure, value domain (content), population strategy for translate table(s) – Target environment(s) and load method(s) to be used
Managing Design Tasks
Data Extract Design 31 Identifies the required information to extract the data from the original source tables/files/environments. Includes the following: – environmental data (region, subsystem, server, etc), – driving object identification (which table/file do we drive the extract from), – selection criteria information, – extract specific information needed to pull the needed information from the source tables/files. Finally, the overall extract execution strategy will be documented (when to execute, frequency of execution, etc)
Data Disguise Design 32 Takes the fields to be disguised and begin to scope out what exactly will be done to these fields to create a disguised test environment. Identifies the specific disguise technique selection criteria to be applied field masking to be applied If any translations will be done, the Translation Table information is also documented (creation data, fields to be created, etc).
Data Disguise Techniques Replace sensitive values with meaningful, readable data using a translation table Generate fictitious data from scratch or from some other source Replace sensitive values with formulated data based on a user- defined key Replace sensitive dates consistently while maintaining the integrity of a date field Conceal partial fields Encrypt Translate Age Mask Generate
Data Privacy_ _Disguise Rule Design 34
Data Privacy_ _Disguise Rule Design 35
Data Privacy_ _Data Load Design 36
Data Privacy_ _Data Load Design 37
Data Privacy Develop Phase 38
Develop Phase 39
Develop 40 Subset Extract Load Maintain Integrity Build Test Validate z/OS Distributed Test z/OS Distributed Production Data Privacy Manager
Develop - z/OS Relationships 41 AR/RI Production z/OS
Develop - z/OS Extract 42 z/OS Production Subset Extract
Develop - Distributed Related Extract 43 Distributed Production Subset Extract
Develop - Disguise 44 Build Test Validate Test Data Privacy Manager
Deliver Production Test System Test Unit Test QA Test Acceptance Test Apply Privacy Rules Subset Extract Load Maintain integrity Data Privacy Manager z/OS Distributed z/OS Distributed z/OS Distributed z/OS Distributed z/OS Distributed z/OS Distributed Privacy Audit Reports
Managing Delivery Tasks System Unit QA Acceptance Fictionalized Data Privacy Audit Reports
Deliver - Disguise Rule Administration 53 Disguise Rules Test Data Privacy Manager
Document - Extract & Disguise Reports 54
Document - Audit Reports 55
Data Privacy_1.4.1_Deliver Execution Sequence 56
Data Privacy_ _Deliver Execution Sequence 57
Data Privacy Solution Product Technology Tools that can deliver quality data that meets the integrity, consistency and usability demands of your data privacy requirements Process A clear strategy backed up by a methodology that serves as a roadmap or blueprint for an enterprise-wide data privacy initiative Expertise The knowledge and experience to effectively manage the process and drive the technology to implement data privacy assurance in the application testing environment