Presentation is loading. Please wait.

Presentation is loading. Please wait.

Data Integrity Charlie Appleby, U.S. EPA Region 4 SESD Quality Assurance Section.

Similar presentations


Presentation on theme: "Data Integrity Charlie Appleby, U.S. EPA Region 4 SESD Quality Assurance Section."— Presentation transcript:

1 Data Integrity Charlie Appleby, U.S. EPA Region 4 SESD Quality Assurance Section

2 Integrity “There can be no friendship without confidence [trust], and no confidence [trust] without integrity.” – Samuel Johnson “Transparency is the key to trust.” – Steven Hill “Real integrity is doing the right thing, knowing that nobody’s going to know whether you did it or not.” – Oprah Winfrey Oprah Winfrey

3 Data with Integrity Data of known and documented quality Representative, Comparable and Complete Defensible and Usable for its intended purpose, the first time. –Best Practices for the Detection and Deterrence of Laboratory Fraud, 1997

4 Data Integrity Requirements Careful planning prior to sample collection Coordination between stakeholders Communication lines

5 Planning For Success DQO Process Decisions Data Needs Data Quality Requirements

6 Planning For Success Take time to plan Define the the project boundaries Include all stakeholders in the entire process Establish lines of communication providing essential information to all personnel involved in the project

7 Data Quality No Task for a One-Man Band Project Leader Project QA Contractor Support Management RSCC Laboratory Peer Reviewer QA Manager

8 Coordination For Success Between –Branches –Divisions –Agencies

9 Planning for Quality Quality Management Plan Quality Assurance Project Plan

10 It Comes Down to People and Communication

11 Corporate Policies Reflect Values Hiring practices - checking references Ethics training Data integrity training Complete technical training A quality system

12 In the future, employees will either be superstars or perspiration wipers. Those who aren’t qualified to do either will become managers. – Scott Adams

13 Ethics Policy –Conduct all business with integrity and in an ethical manner –Responsibility of each staff member and manager to hold to the highest ethical standard of professional conduct in the performance of all duties

14 Data Integrity Policy –To ensure work is of highest integrity –Employees responsible and accountable for the integrity and validity of their own work –Employees to respect and adhere to the principles of ethical conduct –Fabrication or falsification of work results are direct assaults on the integrity of the laboratory and will not be tolerated

15 Documenting the Quality System, The QA Manual The Corporate Mission, Values & Vision Organizational structure and responsibilities Procedures for documenting lab ops –Sample receipt –Stds/reagent prep –Completing Training Document control Corrective action

16 The QA Manual (continued) Data verification, approval, and reporting Facility/data security Emergency procedures Corrective action policies and procedures Index of SOPs Reports to management

17 Cracks in the Quality System Entropy – Newton?

18 Cracks in the Quality System Lax documentation in sample receiving, Poor hiring decisions, Failure to complete or document training, Lack of cross-training, Missed SOP updates, internal audits,

19 Cracks in the Quality System Lax peer review, Poor document control, Poor housekeeping, Turnover, Drop in data quality

20 Whither the Quality System Vulnerabilities Inappropriate practices Lost business/revenue Excessive turnover Fraudulent activities

21 What is Laboratory Fraud? Intentional misrepresentation of lab data to hide known or potential problems Make data look better than it really is Dr. Bruce Woods

22 Potential Areas of Lab Fraud Procedural Deceptions: –Not following critical steps of methodology –Short-cutting sample prep, calibration, analysis Measurement Deceptions: –Directly altering results –Time and date, conditions of experiment

23 Examples of Procedural Fraud Leaving out hydrolysis step in herbicide analysis to avoid hassle. Not prepping the PE sample before analysis (direct injection). Not digesting metal samples for analysis due to heavy workload

24 More Examples Selectively background subtracting spectra from other peaks to make tuning criteria pass in GC/MS analysis. Using calibration procedures that are not allowed by the required method.

25 Examples of Measurement Fraud Deceptive GC Peak Integration Time Travel Dry Labbing

26 Preventing QA System Pitfalls What can the lab do? Independent QA Officer, Ethics Policy, Internal audits, Certifications, Managers who keep the vision fresh

27 Detecting QA System Problems What can EPA do? Independent data validation, Monitoring PT sample performance, Data tape audits, On-site laboratory audits –Announced –Unannounced Follow-up audits

28 Preventing QA System Weaknesses Contract language, –Clear QA/QC requirements, –Incentives / Disincentives Pre-award audits, Past performance assessment, Performance Testing

29 Case Studies Let’s Test your Knowledge

30 Challenges for EPA Why do we need a vision for data integrity? “Though leaders in the middle may not always be the inventors of the vision, they are almost always its interpreters.” –John C. Maxwell We are what we repeatedly do. Excellence then is not an act, it is a habit. –Aristotle

31 Vision Statement Creation First, identify the mission –Protect Human Health and the Environment Identify values –science-based policies and programs –adherence to the rule of law –overwhelming transparency Distill and refine

32 Elements of vision Clarity Connection of Past, Present, and Future Purpose Goals Challenge Stories Passion

33 Vision Exercise Imagine the ideal state Identify needed skills/competencies Evaluate strategy Clarify the forces you will face Be realistic

34 Data Integrity Starter Quiz Are you DI Savvy? Questions borrowed from SHOQ Quality Assurance Manuals Inc. ISO Culture QuizSHOQ Quality Assurance Manuals Inc.

35 Management and technical personnel should have the necessary: A. A. Personnel B. B. Authority and resources C. C. Instrumentation

36 The laboratory’s quality system policies and objectives should be defined in a: A. A. Quality Policy Statement B. B. Quality Manual C. C. Standard Operating Procedure

37 Document control means: A. A. Ways to reduce paper B. B. Documents are identified, authorized, reviewed C. C. Give all documentation to supervisor

38 A laboratory is not responsible to the client for the work of a subcontractor A. A. True B. B. False

39 Records should be maintained of all client complaints and of: A. A. How angry the client was B. B. The investigations and corrective actions taken by the lab C. C. How loud the client complained D. D. To CYA in court

40 The procedure for corrective action must start with: A. A. Finger Pointing B. B. Risk Assessment C. C. An investigation to determine the root cause(s) of the problem D. D. A judicious application of CYA

41 Controlled records should be: A. A. Legible, readily retrievable, and in a suitable environment B. B. Designed for auditors C. C. Controlled by IT personnel

42 Internal audits are conducted to verify: A. A. Compliance of operations with quality system B. B. We won’t get caught again! C. C. The cost vs benefit of each test offered D. D. Compliance of operations with EPA requirements

43 Management reviews determine: A. A. Continuing suitability and effectiveness of the quality system B. B. That there will always be another Dilbert cartoon C. C. Employee requirements are met through 365 degree feedback

44 Training records are essential to: A. A. Writing Job Descriptions B. B. Accrediting the analyst C. C. Ensure competence and authorize personnel

45 Methods must be sufficiently validated as well as: A. A. Maximize profits B. B. Meet the needs of the client and appropriate for the test C. C. Easily implemented by the lab

46 Primary measurement standards must be traceable by means of an unbroken chain of calibrations or comparisons linking them to: A. A. The last standard entered in the log B. B. NTIS C. C. Check samples

47 Sampling generally happens prior to reception at a lab, and therefore has little affect on final results: A. A. True B. B. False

48 Three levels of data/peer review are necessary to: A. A. Keep the analysts feeling insecure B. B. Give the manager something to do C. C. Ensure the data are accurate, defensible, and meet the clients’ needs

49 Case Studies Let’s Test your Knowledge

50 Case Study 1 An analyst is preparing a method blank associated with a batch of samples which will be digested for metals determinations. The analyst selects a specific beaker which he/she always uses to digest the blank because it seems to produce non-detect or very low results. This practice is: A.Perfectly acceptablePerfectly acceptable B.A deceptive lab practiceA deceptive lab practice C.An improper lab practiceAn improper lab practice D.None of the aboveNone of the above E.Both B & CBoth B & C

51 Case Study 2 An analyst discovers that a water sample has accidentally been left out of a sample batch which was extracted for pesticides analysis. He/she included the sample in a different extraction batch which was prepared eight days later. He/she records the sample extraction date as the date of the first batch because the sample holding time was exceeded when the second batch was extracted. And besides, the pesticides supervisor does not like to qualify data for missed HTs. This practice is: A.A good reason for a time off awardA good reason for a time off award B.An improper lab practiceAn improper lab practice C.An example of good sample managementAn example of good sample management D.Lab fraudLab fraud E.All of the aboveAll of the above

52 Case Study 3 An analyst who performs total suspended solids analyses remembers on his/her way home that the balance calibration check for that day’s sample batch was forgotten. He/she performs the balance calibration as soon as he/she arrives at work the next day. The 100g, 1000 mg and 500mg weights are all within acceptable tolerances. The analysts is unsure how to proceed. He/she should: A.Record the calibration check and not mention it to anyoneRecord the calibration check and not mention it to anyone B.Repeat the analyses on the entire batchRepeat the analyses on the entire batch C.Request that the TSS secondary analyst repeat the calibration check to confirm it was done properly.Request that the TSS secondary analyst repeat the calibration check to confirm it was done properly. D.Ask his/her co-worker for adviceAsk his/her co-worker for advice

53 Case Study 4 The TSS analyst asked a co-worker what to do about the balance calibration check. The co- worker says “No harm, no foul” and advises him/her to just record the calibration results and keep quiet. The TSS analyst should: A.Follow the co-workers adviceFollow the co-workers advice B.Turn the co-worker in to the EPA IG.Turn the co-worker in to the EPA IG. C.Tell his/her supervisor that the co-worker is unethical.Tell his/her supervisor that the co-worker is unethical. D.Go to upper management and spill everythingGo to upper management and spill everything E.Seek counsel from the lab supervisor and QA manager.Seek counsel from the lab supervisor and QA manager.

54 Case Study 5 Near the end of the day, a VOA analyst notices all surrogate standard recoveries are running about 40% low for the day’s batch and are outside control limits. The analyst does not have time to prepare a new surrogate standard solution before the shift ends, but at his/her previous job he/she was taught that it was OK to increase the volume of surrogate standard in such instances. The analyst increases the volume of surrogate injected from the normal 1.0 uL to 1.4 uL and sets up a run to reanalyze the batch during the night. This is: A.A good way to make sure the QC is within limits.A good way to make sure the QC is within limits. B.An improper lab practice.An improper lab practice. C.Lab fraud.Lab fraud. D.A failure to follow the SOP.A failure to follow the SOP. E.Difficult to assess without more information.Difficult to assess without more information.

55 Case Study 6 During the preparation of soil SV samples, the method blank was lost due to an instrument malfunction during cleanup. It has been 48 hours since the batch was extracted. The correct thing to do is: A.Re-extract the entire batch of samples.Re-extract the entire batch of samples. B.Prepare and extract another method blank only.Prepare and extract another method blank only. C.Go ahead and analyze the batch w/o a MB and flag the data.Go ahead and analyze the batch w/o a MB and flag the data. D.Perform maintenance on the clean-up instrument so the next batch will be OK.Perform maintenance on the clean-up instrument so the next batch will be OK. E.None of the aboveNone of the above

56 Correcto!! Click here to return

57 Sorry that is incorrect! Click here to return


Download ppt "Data Integrity Charlie Appleby, U.S. EPA Region 4 SESD Quality Assurance Section."

Similar presentations


Ads by Google