Download presentation
Published byJourney Winslow Modified over 9 years ago
1
Charlie Appleby, U.S. EPA Region 4 SESD Quality Assurance Section
Data Integrity Charlie Appleby, U.S. EPA Region 4 SESD Quality Assurance Section
2
“Transparency is the key to trust.”
Integrity “There can be no friendship without confidence [trust], and no confidence [trust] without integrity.” Samuel Johnson “Transparency is the key to trust.” Steven Hill “Real integrity is doing the right thing, knowing that nobody’s going to know whether you did it or not.” Oprah Winfrey
3
Data with Integrity Data of known and documented quality
Representative, Comparable and Complete Defensible and Usable for its intended purpose, the first time. Best Practices for the Detection and Deterrence of Laboratory Fraud, 1997
4
Data Integrity Requirements
Careful planning prior to sample collection Coordination between stakeholders Communication lines
5
Planning For Success DQO Process Decisions Data Needs
Data Quality Requirements
6
Planning For Success Take time to plan
Define the the project boundaries Include all stakeholders in the entire process Establish lines of communication providing essential information to all personnel involved in the project
7
Data Quality No Task for a One-Man Band
Project Leader Project QA Contractor Support Management RSCC Laboratory Peer Reviewer QA Manager
8
Coordination For Success
Between Branches Divisions Agencies
9
Planning for Quality Quality Management Plan
Quality Assurance Project Plan
10
It Comes Down to People and Communication
11
Corporate Policies Reflect Values
Hiring practices - checking references Ethics training Data integrity training Complete technical training A quality system
12
In the future, employees will either be superstars or perspiration wipers. Those who aren’t qualified to do either will become managers. – Scott Adams
13
Ethics Policy Conduct all business with integrity and in an ethical manner Responsibility of each staff member and manager to hold to the highest ethical standard of professional conduct in the performance of all duties
14
Data Integrity Policy To ensure work is of highest integrity
Employees responsible and accountable for the integrity and validity of their own work Employees to respect and adhere to the principles of ethical conduct Fabrication or falsification of work results are direct assaults on the integrity of the laboratory and will not be tolerated
15
Documenting the Quality System, The QA Manual
The Corporate Mission, Values & Vision Organizational structure and responsibilities Procedures for documenting lab ops Sample receipt Stds/reagent prep Completing Training Document control Corrective action
16
The QA Manual (continued)
Data verification, approval, and reporting Facility/data security Emergency procedures Corrective action policies and procedures Index of SOPs Reports to management
17
Cracks in the Quality System
Entropy – Newton?
18
Cracks in the Quality System
Lax documentation in sample receiving, Poor hiring decisions, Failure to complete or document training, Lack of cross-training, Missed SOP updates, internal audits,
19
Cracks in the Quality System
Lax peer review, Poor document control, Poor housekeeping, Turnover, Drop in data quality
20
Whither the Quality System Vulnerabilities
Inappropriate practices Lost business/revenue Excessive turnover Fraudulent activities
21
What is Laboratory Fraud?
Intentional misrepresentation of lab data to hide known or potential problems Make data look better than it really is Dr. Bruce Woods
22
Potential Areas of Lab Fraud
Procedural Deceptions: Not following critical steps of methodology Short-cutting sample prep, calibration, analysis Measurement Deceptions: Directly altering results Time and date, conditions of experiment
23
Examples of Procedural Fraud
Leaving out hydrolysis step in herbicide analysis to avoid hassle. Not prepping the PE sample before analysis (direct injection). Not digesting metal samples for analysis due to heavy workload
24
More Examples Selectively background subtracting spectra from other peaks to make tuning criteria pass in GC/MS analysis. Using calibration procedures that are not allowed by the required method.
25
Examples of Measurement Fraud
Deceptive GC Peak Integration Time Travel Dry Labbing
26
Preventing QA System Pitfalls What can the lab do?
Independent QA Officer, Ethics Policy, Internal audits, Certifications, Managers who keep the vision fresh
27
Detecting QA System Problems What can EPA do?
Independent data validation, Monitoring PT sample performance, Data tape audits, On-site laboratory audits Announced Unannounced Follow-up audits
28
Preventing QA System Weaknesses
Contract language, Clear QA/QC requirements, Incentives / Disincentives Pre-award audits, Past performance assessment, Performance Testing
29
Let’s Test your Knowledge
Case Studies Let’s Test your Knowledge
30
Challenges for EPA Why do we need a vision for data integrity?
“Though leaders in the middle may not always be the inventors of the vision, they are almost always its interpreters.” John C. Maxwell We are what we repeatedly do. Excellence then is not an act, it is a habit. Aristotle
31
Vision Statement Creation
First, identify the mission Protect Human Health and the Environment Identify values science-based policies and programs adherence to the rule of law overwhelming transparency Distill and refine
32
Elements of vision Clarity Connection of Past, Present, and Future
Purpose Goals Challenge Stories Passion
33
Vision Exercise Imagine the ideal state
Identify needed skills/competencies Evaluate strategy Clarify the forces you will face Be realistic
34
Data Integrity Starter Quiz
Are you DI Savvy? Questions borrowed from SHOQ Quality Assurance Manuals Inc. ISO Culture Quiz
35
Management and technical personnel should have the necessary:
A. Personnel B. Authority and resources C. Instrumentation
36
The laboratory’s quality system policies and objectives should be defined in a:
A. Quality Policy Statement B. Quality Manual C. Standard Operating Procedure
37
Document control means:
A. Ways to reduce paper B. Documents are identified, authorized, reviewed C. Give all documentation to supervisor
38
A laboratory is not responsible to the client for the work of a subcontractor
A. True B. False
39
Records should be maintained of all client complaints and of:
A. How angry the client was B. The investigations and corrective actions taken by the lab C. How loud the client complained D. To CYA in court
40
The procedure for corrective action must start with:
A. Finger Pointing B. Risk Assessment C. An investigation to determine the root cause(s) of the problem D. A judicious application of CYA
41
Controlled records should be:
A. Legible, readily retrievable, and in a suitable environment B. Designed for auditors C. Controlled by IT personnel
42
Internal audits are conducted to verify:
A. Compliance of operations with quality system B. We won’t get caught again! C. The cost vs benefit of each test offered D. Compliance of operations with EPA requirements
43
Management reviews determine:
A. Continuing suitability and effectiveness of the quality system B. That there will always be another Dilbert cartoon C. Employee requirements are met through 365 degree feedback
44
Training records are essential to:
A. Writing Job Descriptions B. Accrediting the analyst C. Ensure competence and authorize personnel
45
Methods must be sufficiently validated as well as:
A. Maximize profits B. Meet the needs of the client and appropriate for the test C. Easily implemented by the lab
46
Primary measurement standards must be traceable by means of an unbroken chain of calibrations or comparisons linking them to: A. The last standard entered in the log B. NTIS C. Check samples
47
Sampling generally happens prior to reception at a lab, and therefore has little affect on final results: A. True B. False
48
Three levels of data/peer review are necessary to:
A. Keep the analysts feeling insecure B. Give the manager something to do C. Ensure the data are accurate, defensible, and meet the clients’ needs
49
Let’s Test your Knowledge
Case Studies Let’s Test your Knowledge
50
Case Study 1 An analyst is preparing a method blank associated with a batch of samples which will be digested for metals determinations. The analyst selects a specific beaker which he/she always uses to digest the blank because it seems to produce non-detect or very low results . This practice is: Perfectly acceptable A deceptive lab practice An improper lab practice None of the above Both B & C
51
Case Study 2 An analyst discovers that a water sample has accidentally been left out of a sample batch which was extracted for pesticides analysis. He/she included the sample in a different extraction batch which was prepared eight days later. He/she records the sample extraction date as the date of the first batch because the sample holding time was exceeded when the second batch was extracted. And besides, the pesticides supervisor does not like to qualify data for missed HTs. This practice is: A good reason for a time off award An improper lab practice An example of good sample management Lab fraud All of the above
52
Case Study 3 An analyst who performs total suspended solids analyses remembers on his/her way home that the balance calibration check for that day’s sample batch was forgotten. He/she performs the balance calibration as soon as he/she arrives at work the next day. The 100g, 1000 mg and 500mg weights are all within acceptable tolerances. The analysts is unsure how to proceed. He/she should: Record the calibration check and not mention it to anyone Repeat the analyses on the entire batch Request that the TSS secondary analyst repeat the calibration check to confirm it was done properly. Ask his/her co-worker for advice
53
Case Study 4 The TSS analyst asked a co-worker what to do about the balance calibration check. The co-worker says “No harm, no foul” and advises him/her to just record the calibration results and keep quiet. The TSS analyst should: Follow the co-workers advice Turn the co-worker in to the EPA IG. Tell his/her supervisor that the co-worker is unethical. Go to upper management and spill everything Seek counsel from the lab supervisor and QA manager.
54
Case Study 5 Near the end of the day, a VOA analyst notices all surrogate standard recoveries are running about 40% low for the day’s batch and are outside control limits. The analyst does not have time to prepare a new surrogate standard solution before the shift ends, but at his/her previous job he/she was taught that it was OK to increase the volume of surrogate standard in such instances. The analyst increases the volume of surrogate injected from the normal 1.0 uL to 1.4 uL and sets up a run to reanalyze the batch during the night. This is: A good way to make sure the QC is within limits. An improper lab practice. Lab fraud. A failure to follow the SOP. Difficult to assess without more information.
55
Case Study 6 During the preparation of soil SV samples, the method blank was lost due to an instrument malfunction during cleanup. It has been 48 hours since the batch was extracted. The correct thing to do is: Re-extract the entire batch of samples. Prepare and extract another method blank only. Go ahead and analyze the batch w/o a MB and flag the data. Perform maintenance on the clean-up instrument so the next batch will be OK. None of the above
56
Correcto!! Click here to return
57
Sorry that is incorrect!
Click here to return
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.