1. Charlie Appleby, U.S. EPA Region 4 SESD Quality Assurance Section
Data Integrity
Charlie Appleby,
U.S. EPA Region 4
SESD
Quality Assurance Section

2. “Transparency is the key to trust.”
Integrity
“There can be no friendship without confidence [trust], and no confidence [trust] without integrity.”
Samuel Johnson
“Transparency is the key to trust.”
Steven Hill
“Real integrity is doing the right thing, knowing that nobody’s going to know whether you did it or not.”
Oprah Winfrey

3. Data with Integrity Data of known and documented quality
Representative, Comparable and Complete
Defensible and Usable for its intended purpose, the first time.
Best Practices for the Detection and Deterrence of Laboratory Fraud, 1997

4. Data Integrity Requirements
Careful planning prior to sample collection
Coordination between stakeholders
Communication lines

5. Planning For Success DQO Process Decisions Data Needs
Data Quality Requirements

6. Planning For Success Take time to plan
Define the the project boundaries
Include all stakeholders in the entire process
Establish lines of communication providing essential information to all personnel involved in the project

7. Data Quality No Task for a One-Man Band
Project Leader
Project QA
Contractor Support
Management
RSCC
Laboratory
Peer Reviewer
QA Manager

8. Coordination For Success
Between
Branches
Divisions
Agencies

9. Planning for Quality Quality Management Plan
Quality Assurance Project Plan

10. It Comes Down to People and Communication

11. Corporate Policies Reflect Values
Hiring practices - checking references
Ethics training
Data integrity training
Complete technical training
A quality system

12. In the future, employees will either be superstars or perspiration wipers. Those who aren’t qualified to do either will become managers.
– Scott Adams

13. Ethics Policy
Conduct all business with integrity and in an ethical manner
Responsibility of each staff member and manager to hold to the highest ethical standard of professional conduct in the performance of all duties

14. Data Integrity Policy To ensure work is of highest integrity
Employees responsible and accountable for the integrity and validity of their own work
Employees to respect and adhere to the principles of ethical conduct
Fabrication or falsification of work results are direct assaults on the integrity of the laboratory and will not be tolerated

15. Documenting the Quality System, The QA Manual
The Corporate Mission, Values & Vision
Organizational structure and responsibilities
Procedures for documenting lab ops
Sample receipt
Stds/reagent prep
Completing Training
Document control
Corrective action

16. The QA Manual (continued)
Data verification, approval, and reporting
Facility/data security
Emergency procedures
Corrective action policies and procedures
Index of SOPs
Reports to management

17. Cracks in the Quality System
Entropy – Newton?

18. Cracks in the Quality System
Lax documentation in sample receiving,
Poor hiring decisions,
Failure to complete or document training,
Lack of cross-training,
Missed SOP updates, internal audits,

19. Cracks in the Quality System
Lax peer review,
Poor document control,
Poor housekeeping,
Turnover,
Drop in data quality

20. Whither the Quality System Vulnerabilities
Inappropriate practices
Lost business/revenue
Excessive turnover
Fraudulent activities

21. What is Laboratory Fraud?
Intentional misrepresentation of lab data to hide known or potential problems
Make data look better than it really is
Dr. Bruce Woods

22. Potential Areas of Lab Fraud
Procedural Deceptions:
Not following critical steps of methodology
Short-cutting sample prep, calibration, analysis
Measurement Deceptions:
Directly altering results
Time and date, conditions of experiment

23. Examples of Procedural Fraud
Leaving out hydrolysis step in herbicide analysis to avoid hassle.
Not prepping the PE sample before analysis (direct injection).
Not digesting metal samples for analysis due to heavy workload

24. More Examples
Selectively background subtracting spectra from other peaks to make tuning criteria pass in GC/MS analysis.
Using calibration procedures that are not allowed by the required method.

25. Examples of Measurement Fraud
Deceptive GC Peak Integration
Time Travel
Dry Labbing

26. Preventing QA System Pitfalls What can the lab do?
Independent QA Officer,
Ethics Policy,
Internal audits,
Certifications,
Managers who keep the vision fresh

27. Detecting QA System Problems What can EPA do?
Independent data validation,
Monitoring PT sample performance,
Data tape audits,
On-site laboratory audits
Announced
Unannounced
Follow-up audits

28. Preventing QA System Weaknesses
Contract language,
Clear QA/QC requirements,
Incentives / Disincentives
Pre-award audits,
Past performance assessment,
Performance Testing

29. Let’s Test your Knowledge
Case Studies
Let’s Test your Knowledge

30. Challenges for EPA Why do we need a vision for data integrity?
“Though leaders in the middle may not always be the inventors of the vision, they are almost always its interpreters.”
John C. Maxwell
We are what we repeatedly do. Excellence then is not an act, it is a habit.
Aristotle

31. Vision Statement Creation
First, identify the mission
Protect Human Health and the Environment
Identify values
science-based policies and programs
adherence to the rule of law
overwhelming transparency
Distill and refine

32. Elements of vision Clarity Connection of Past, Present, and Future
Purpose
Goals
Challenge
Stories
Passion

33. Vision Exercise Imagine the ideal state
Identify needed skills/competencies
Evaluate strategy
Clarify the forces you will face
Be realistic

34. Data Integrity Starter Quiz
Are you DI Savvy?
Questions borrowed from SHOQ Quality Assurance Manuals Inc. ISO Culture Quiz

35. Management and technical personnel should have the necessary:
A. Personnel
B. Authority and resources
C. Instrumentation

36. The laboratory’s quality system policies and objectives should be defined in a:
A. Quality Policy Statement
B. Quality Manual
C. Standard Operating Procedure

37. Document control means:
A. Ways to reduce paper
B. Documents are identified, authorized, reviewed
C. Give all documentation to supervisor

38. A laboratory is not responsible to the client for the work of a subcontractor
A. True
B. False

39. Records should be maintained of all client complaints and of:
A. How angry the client was
B. The investigations and corrective actions taken by the lab
C. How loud the client complained
D. To CYA in court

40. The procedure for corrective action must start with:
A. Finger Pointing
B. Risk Assessment
C. An investigation to determine the root cause(s) of the problem
D. A judicious application of CYA

41. Controlled records should be:
A. Legible, readily retrievable, and in a suitable environment
B. Designed for auditors
C. Controlled by IT personnel

42. Internal audits are conducted to verify:
A. Compliance of operations with quality system
B. We won’t get caught again!
C. The cost vs benefit of each test offered
D. Compliance of operations with EPA requirements

43. Management reviews determine:
A. Continuing suitability and effectiveness of the quality system
B. That there will always be another Dilbert cartoon
C. Employee requirements are met through 365 degree feedback

44. Training records are essential to:
A. Writing Job Descriptions
B. Accrediting the analyst
C. Ensure competence and authorize personnel

45. Methods must be sufficiently validated as well as:
A. Maximize profits
B. Meet the needs of the client and appropriate for the test
C. Easily implemented by the lab

46. Primary measurement standards must be traceable by means of an unbroken chain of calibrations or comparisons linking them to:
A. The last standard entered in the log
B. NTIS
C. Check samples

47. Sampling generally happens prior to reception at a lab, and therefore has little affect on final results:
A. True
B. False

48. Three levels of data/peer review are necessary to:
A. Keep the analysts feeling insecure
B. Give the manager something to do
C. Ensure the data are accurate, defensible, and meet the clients’ needs

49. Let’s Test your Knowledge
Case Studies
Let’s Test your Knowledge

50. Case Study 1
An analyst is preparing a method blank associated with a batch of samples which will be digested for metals determinations. The analyst selects a specific beaker which he/she always uses to digest the blank because it seems to produce non-detect or very low results . This practice is:
Perfectly acceptable
A deceptive lab practice
An improper lab practice
None of the above
Both B & C

51. Case Study 2
An analyst discovers that a water sample has accidentally been left out of a sample batch which was extracted for pesticides analysis. He/she included the sample in a different extraction batch which was prepared eight days later. He/she records the sample extraction date as the date of the first batch because the sample holding time was exceeded when the second batch was extracted. And besides, the pesticides supervisor does not like to qualify data for missed HTs. This practice is:
A good reason for a time off award
An improper lab practice
An example of good sample management
Lab fraud
All of the above

52. Case Study 3
An analyst who performs total suspended solids analyses remembers on his/her way home that the balance calibration check for that day’s sample batch was forgotten. He/she performs the balance calibration as soon as he/she arrives at work the next day. The 100g, 1000 mg and 500mg weights are all within acceptable tolerances. The analysts is unsure how to proceed. He/she should:
Record the calibration check and not mention it to anyone
Repeat the analyses on the entire batch
Request that the TSS secondary analyst repeat the calibration check to confirm it was done properly.
Ask his/her co-worker for advice

53. Case Study 4
The TSS analyst asked a co-worker what to do about the balance calibration check. The co-worker says “No harm, no foul” and advises him/her to just record the calibration results and keep quiet. The TSS analyst should:
Follow the co-workers advice
Turn the co-worker in to the EPA IG.
Tell his/her supervisor that the co-worker is unethical.
Go to upper management and spill everything
Seek counsel from the lab supervisor and QA manager.

54. Case Study 5
Near the end of the day, a VOA analyst notices all surrogate standard recoveries are running about 40% low for the day’s batch and are outside control limits. The analyst does not have time to prepare a new surrogate standard solution before the shift ends, but at his/her previous job he/she was taught that it was OK to increase the volume of surrogate standard in such instances. The analyst increases the volume of surrogate injected from the normal 1.0 uL to 1.4 uL and sets up a run to reanalyze the batch during the night. This is:
A good way to make sure the QC is within limits.
An improper lab practice.
Lab fraud.
A failure to follow the SOP.
Difficult to assess without more information.

55. Case Study 6
During the preparation of soil SV samples, the method blank was lost due to an instrument malfunction during cleanup. It has been 48 hours since the batch was extracted. The correct thing to do is:
Re-extract the entire batch of samples.
Prepare and extract another method blank only.
Go ahead and analyze the batch w/o a MB and flag the data.
Perform maintenance on the clean-up instrument so the next batch will be OK.
None of the above

56. Correcto!!
Click here to return

57. Sorry that is incorrect!
Click here to return