Determine Scope of Assessment Select Assessment Team Notify Laboratory & Request Records Determine Team Assignments Sample Receipt and Storage Assessment SOPs Process Demonstration Sample Receipt and Storage Records Sample Custodian Interviews Security Sample Storage Glassware and Bottle Preparation Data Transfer (from technical areas to reporting) ELECTRONIC DATA Audit Trail HARD COPY DATA Archives - Original - Copies Summary Reports Case Narratives Conduct Pre- Assessment Records Review Plan On-Site Assessment PRE-ASSESSMENT ACTIVITIES ON-SITE ASSESSMENT PROCESS DATA ASSESSMENT Process Waste Disposal and Material Throughput SOPs Process Demonstration Logbooks, Bench Sheets, Maintenance Records Analyst Interviews Access and Computer Security Data Reduction and Review (Process) Equipment and Facilities Technical Area Assessments Quality Systems Documentation Organization and Functions Training Records Management Interviews Records Management & Archives Computer Access & Security (LIMS, Data Systems) Data Integrity Program (Records/ Policy) Internal Audits Corrective Action Document Control Quality Systems and Management Assessment ASSESSMENT PROCESS FLOW CHART RED FLAGS: DATA ASSESSMENT F Incomplete case narratives F Unexpected sample results F Too-perfect QC results F Reports missing secondary reviewer signatures or dates F Discrepancies between CoC and reports F Unexplained gaps/changes in records F Discrepancies in QC performance between analysts F Calibration records missing reviewer signature/dates F “Reappearing” quality control results F Screen-printed reports F Routine dead time on auto- injection run logs F Samples or blanks diluted without apparent justification F Too few quality control records to support data output F Chronological chromatograms that do not display prevalent background RED FLAGS: PRE-ASSESSMENT F Lack of documented assessment follow- up or corrective action F Questionable proficiency F Questionable technical depth F Recent, significant, organizational changes F Lack of responsiveness F Lack of management commitment to data integrity RED FLAGS: OPENING MEETING & LAB TOUR F Time wasters (e.g. lengthy health and safety overview, non-relevant briefings (e.g. history, sales) F Selected staff not available for interviews F Housekeeping practices F Unexplained, restricted access Opening Meeting & Laboratory Tour On-Site Assessment RED FLAGS: SAMPLE RECEIPT & TECHNICAL AREA ASSESSMENTS F Denials of access or conditional access F Attempts to distract the assessment team F Records not readily accessible F Discrepancies between method and SOP F Referenced methods out of date F Presence of pencils, white-out F Extremely clean, neat logbooks F Uncontrolled records F Logbook entries all aligned F Discrepancies between SOP and practice F Expired standards F Disabled audit trails F Shared log-on and password access F Unlabeled containers or illegible labels F Sample volume discrepancies F Materials inventory does not match throughput F Sample throughput exceeds time required to process the samples F Managers translating or clarifying analysts’ responses to questions F Emphasis on production F Inadequate sample-handling procedures for evening or weekend deliveries F No mechanism for reporting problems F Analysts not allowed to use their judgment F Analysts unable to describe data review and/or oversight RED FLAGS: QUALITY SYSTEMS AND MANAGEMENT ASSESSMENT F Lack of data integrity policy F Unclear communication of either expected behavior or improper practices F Lack of a “no-fault” reporting mechanism F Emphasis on production F Inadequate back-up (personnel and equipment) F Poor coordination for accepting new work F Excessive overtime F “Bottleneck” departments F Lack of technical depth F High staff turnover F Gaps in training records F QA Staff lacks direct access to senior management F Unclear roles and responsibilities F QA staff performing competing functions F Inadequate internal assessments F Lack of data surveillance F Incomplete assessment files F Repeat assessment findings F Inadequate procedures for handling complaints F Corrective action fails to address root cause F Lack of timely follow-up or documentation of corrective action F Lack of record verification in archiving procedures F Inadequate procedures to ensure data retrieval