Presentation on theme: "Anne Sholander, MT(ASCP), International QA/QC Coordinator, SMILE"— Presentation transcript:
1Anne Sholander, MT(ASCP), International QA/QC Coordinator, SMILE DAIDS clinical trials outside the US: Furthering the quality of laboratory operations The NIAID SMILE resource (November 2006)Anne Sholander, MT(ASCP), International QA/QC Coordinator, SMILE
2Who are the partners?DCLOTPPDNon-NetworkSMILESitesNetworks
3How does DAIDS manage clinical laboratory oversight for HIV trials? There are currently nearly 170 laboratories outside the US performing DAIDS funded HIV clinical trials.
4Patient Safety Monitoring in International Laboratories SMILEPatient Safety Monitoring in International LaboratoriesOur Mission:To Provide Quality Assurance Support forDAIDS-Funded HIV Clinical TrialLaboratories (Non-US)To Maintain a Web-Based Repository ofRecords and Resources
5“This contract will provide a critically needed comprehensive, uniform program to evaluate the capability of laboratories in developing countries, advise and train when deficiencies are identified, and ensure the ongoing quality of study test results and ultimately the outcome of clinical research performed in these countries.”--original contract language--outlined the goals of the project--we remain true to these original goals
6Clinical Laboratory Research Resources We aim to achieveSafety of trial subjects.Safety of those performing tests.Quality of the laboratory data.Increased appreciation of labs for good laboratory practices that translates to better local service.What are the goals of clinical laboratory research resources?What do we hope to achieve with laboratory oversight?
7Who is SMILE?Contract resource between NIH/NIAID-DAIDS & Johns Hopkins UniversityStaff of 11 with a wide range of laboratory experienceCombined total of 185 years of experienceWide variety of backgrounds and areas of expertisewe come from clinical labs, research labs, public health labs and from DAIDS funded HIV lab (Zimbabwe).Depth of experience in variety of lab disciplines. SMILE staff is experienced in:Microbiology/TB , MalariaChemistry/Hematology/ImmunologyValidation of new methods, instrument troubleshootingManual differentials/urinalysisScientific writing and editingWe are especially fortunate to have a PI with a strong background in Bioinformatics. This has lead to development of extensive computer and web based tools used by SMILE in this project.
8How does SMILE work to achieve the goals of our contract? Primarily from our office in Baltimore.Extensively throughRegular teleconferences with our site labs, networks and DAIDS.Each lab is assigned a primary and a back-up coordinator.regular (sometimes daily) communication with the labs--develop a personal relationship with the labs.
9A year in the life of a SMILE supported lab… The best way to show you the tools that we use at SMILE is to walk through a typical year in the life of a SMILE supported lab.How our tools work to achieve these goals.
10Laboratory AuditsIndependent assessments are performed annually by PPD or DAIDSBased on the standards of Good Clinical Laboratory Practice (GCLP)The “year” for the lab can be said to begin with the audit.Independent of the SMILE contract—avoids any conflict of interest.Audits performed by DAIDS contractor, PPD or by DAIDS.PPD (or DAIDS) performs the audits and SMILE assists with remediation of audit findings.Audits based on GCLP
11Good Clinical Laboratory Practice (GCLP) Organization and PersonnelTesting Facilities OperationVerification of Performance SpecificationsQuality ManagementEquipment, Physical FacilitiesPersonnel SafetyWhat is GCLP?US has a set of accepted guidelines and standards for clinical labs, there was a need for a unifying set of standards for international labs.DAIDS AND contractor PPD developed GCLP guidelines.GCLP are a set of standards based on FDA, CLIA, CAP, ISO, SANAs guidelines to ensure the quality of study data and allow for accurate reconstruction of experiments.Some of the sections of the audit include:Audit gives a comprehensive picture of the lab—and covers all aspects of Good Clinical Laboratory Practice.:
12Audits and Action Plans Audit report sent to SMILEAction Plan is preparedNetworks designate critical itemsAction Plan and audit sent to labWhat do we do with the audit information?Audit reports sent to SMILE for review and preparation of an Action Plan.Action plan contains the audit finding or observation, and SMILE prepares a suggested action for remediation of the finding.Affiliated networks review Action Plan and designate items critical to their studies. This process also works for non-network affiliated labs.Once the Action Plan is finalized it is sent to the lab along with the Audit.
13The Action Plans are “Keyed” to the audit shell The Action Plans are “Keyed” to the audit shell. Lab can cross-reference original audit report.SMILE suggests an action for remediation and provides resources/training.and we track the history of that action item.The AP is color coded. Can see at a glance where the lab stands.Generates feedback and interaction --between the lab and SMILE to resolve items.SMILE is able to provide guidance and training based on our SMILE resources and experience of SMILE staff to address specific problems in the lab.Used by networks and DAIDS as a tool to assess lab readiness for opening of new studies.Also used as a tool to assess the readiness of a brand new lab to open it’s first study.Goal is to close the Action Plan before the next audit.
14SMILE Site Assistance and Training PhoneWeb conferenceAdobe ConnectAt conferencesOn siteThe Action Plan often leads to opportunities to help our labs by teaching or training.Many times this takes place in or telephone conversations.Our computer expertise and support has also allowed us to use web tools and create computer based training programs.We utilize the Adobe Connect software which allows us to connect with our sites via the web and show them any presentations or walk them through any trainings that we have.We have also presented trainings at conferences. TB Safety training was given at the last ACTG meeting and will be presented in Thailand in January.We also are occasionally asked to visit a lab to provide specific training or assistance. Accomplish a specific goal or need—focused visit.Our goal is to empower the labs through training. By doing this the knowledge and the experience stays with the lab even after the DAIDS study is finished.
15Another type of lab assistance or training has been the SMILE TB lab observational visits. SMILE has travelled extensively to assess and assist our labs with their TB testing capabilities.Through these SMILE visits and the support of DAIDS and the cross-network TB working group, the quality, capacity, and safety of TB laboratories in developing countries has improved dramatically.(We have already been to 17 network TB labs, 4 National Labs one of which is going to be added to network oversight and will have a PPD visit, 3 more will be in process during the presentation, 1 is scheduled for January.)(This picture was taken in Uganda.)
16Audits occur only once per year. How do we monitor laboratory performance continuously throughout the year?Back to the year in the life of our SMILE lab….and how we are managing laboratory oversight.The lab has had an audit and is working on their action plan with SMILE. But the audit only occurs once per year.How do we know throughout the year that the test results released from the lab are accurate?
17Proficiency TestingProficiency Testing (PT) is a means of evaluating a laboratory's performance through analysis of unknown samples provided by an external source.
18What is proficiency testing? Also called PT or EQA (External Quality Assurance)Blinded samples sent on a regular scheduleRun as patient samplesResults are compared to peersThe proficiency testing process is independent of SMILE—we cannot influence the outcome.It is similar to audit where SMILE uses tool provided by an independent group to learn information about the lab’s quality.SMILE provides assistance with ordering EQA.Based on our experience with different providers and the requirements of the different network protocols, we are able to customize the PT order to the needs of the lab and the study.
19Before we order EQA we need more information Protocol analytesInstrumentation in useBack-up plansSpecimen flowIn order to be able to create a customized PT order for a laboratory, we need detailed information about the testing that will take place.We first need to know which analytes will be needed for the protocol—both for safety testing and for study monitoring and endpoint data.We also need to know which instruments and methods the lab will be using.We need to know what the lab plans to do if their primary method of testing fails—what is their back-up plan.It is also helpful to understand the specimen flow in the lab.All of this information is captured on a Protocol Analyte List (or PAL) spreadsheet.Protocol Analyte List (PAL)
20This form was originally developed jointly by the networks/DAIDS and SMILE and it has become a critical tool for SMILE.Specific to a particular protocol.It shows the details of analytes are needed for a protocol—and how and where they will be tested.It also shows how we intend to cover those analytes with EQA—who the provider is and how frequently the panels will be delivered.It is a way to communicate what the specific needs of a protocol are.
21How does SMILE use proficiency testing? Monthly review and monitoring of EQA resultsEvaluationReviewTracking and summaryReportingUsing the PAL, we have ordered EQA and the lab has begun testing the samples and submitting the results.What does SMILE do with this information?PT results are obtained from the EQA provider directly (usually).PT provider has evaluated results.SMILE provides the lab with a review of their survey results. Review process is a required element of GCLP.Summarize the information and notify of investigations needed.SMILE tracks PT performance of each lab on an EQA Summary Spreadsheet.SMILE sends a monthly report to the lab and all affiliated networks (copying the DCLOT POC)detailing the performance on that month’s surveys and any pending investigations.
22EQA Summary ScheduleOn this first page of the spreadsheet is the year’s schedule for EQA shipments.Lab knows a year in advance when to expect their survey shipmentsThis page also summarizes the laboratory’s status at a quick glace.By color coding we can quickly see that there is an outstanding investigation required for Gram stain.
23The second page of the EQA spreadsheet gives detailed information about each analyte tested in the lab.The EQA history for 3 years is maintained on this sheet.Color coding also gives a quick picture of how well the lab is performing.So, for example, in 2008 this lab had difficulty with their platelet counts. This result would require an investigation—any result less than 100% successful (for a protocol analyte) requires a documented investigation.In the case of this platelet count—the problem appears to have been corrected on the next survey.
24Investigations Required for EQA failures SMILE provides assistance, tools and resourcesRoot cause and prevention of future failureDocumentation of corrective action is end productSMILE assists with investigations of EQA failure and provides labs with tools and resources for corrective action.The process of troubleshooting the cause of EQA failure is another example of how important our extensive experience in the laboratory is.The process often involves a lot of back and forth and research and discussion of possible causes.Our goal is to teach the labs how to investigate problems and prevent them in the future by the use of good laboratory practices.
25SMILE Investigation Form The SMILE investigation form is designed to aid in the process of troubleshooting.It identifies the analyte, and then asks a series of questions designed to lead the lab through a thorough investigation.
26Root cause analysisThe goal of an investigation is root cause analysis and prevention of further failures.We often help labs discover problems by prompting them to probe deeper into EQA failures.We push them to find the cause of a failure and fix it, so that the integrity of patient testing is ensured.
27Where do we store all of this documentation? We are generating a lot of documentationWhat do we do with all of this documentation—where is it kept?One of our missions was to create a document repository.We file all of each lab’s documentation on our website.Psmile.org—this is our home page
28Document RepositoryEQA surveys, reviews and investigations onAvailable to site labs (privately by log-in)Available to DAIDS and the NetworksAlso onAudits and Action PlansSite visit reportsProtocols and Protocol Analyte Lists (PAL)Currently there are approximately 18,000 site lab documents stored on psmile and we upload an average of about 400 new documents each month.
29Each site lab is updated at least monthly with their most current EQA summary, and new PT surveys and reviews.Each lab can see only their own information—DAIDS and the affiliated networks also have access to this information.Any investigations submitted during the month are also stored here. We also store the PPD Audits, action plans and PALs on psmile.However, Psmile is much more than just a document storage website.
30Resources on pSMILE.org One of the things we are proudest of is our collection of resources posted on psmile.SMILE resources are open to all—no log in is required.This area of psmile is searchable by keyword. For example, if a lab is looking for information on Water Quality, they can simply type this in the search box.There are currently ~1400 documents and weblinks posted in the resources section.Many of these resources were written or developed by SMILE staff. We have devoted a considerable amount of our efforts to improving and perfecting our web resources.Audit remediation, Training and Competency, Instrument validationLaboratory move planningGCLP compliance and SafetySpecific instrument/kit informationThe success and evolution of our webpage is a credit to our PI Dr. Miller and his experience in the application of computer technology to the laboratory.We are constantly encouraged to enhance and improve our use of computer and web-based resources.
31Monthly reports sent DAIDS Let’s recap the year…AuditAction PlanEQAMonthly reports sent DAIDS1. The lab has had an audit2. SMILE provides resources and works with the lab individually on their audit remediation by working to resolve action plan items.3. SMILE reviews and evaluates EQA on a monthly basis.SMILE is tracking the laboratory’s EQA and Action Plan progress and uploading all updates to4. Monthly reports are sent to DAIDS that summarize all 170 SMILE labs
32Summarizing the data EQA Exceptions Delinquent Investigations Tracking LogNearly 170 labs-- how do we summarize the month for DAIDS ?How do we keep DAIDS informed of what is going on with the labs, without overburdening them with the detailed information?While we maintain all the detailed information, and it is available at any time—we also summarize the information on a monthly basis and send it to DAIDS.The EQA Exceptions list shows labs that have two out of three unsuccessful rounds of EQA.Alerts DAIDS and the networks that there may be a problem and testing may need to be moved to back-up.Delinquent Investigations is a list of labs that are >60 days delinquent in submitting investigation.Tracking log gives a brief summary of the month’s activities for each lab.
33All of these activities create an ongoing dialogue between: SMILE and the labsSMILE and the networksSMILE and PPDSMILE and DAIDSSMILE and IQA and VQAAll of these activities create an ongoing dialogue between:This dialogue leads to constant, continuous improvement for our labs, for our processes, for our resources.Although our focus at SMILE is on patient safety testing, we become more than just a resource for safety testing.We also become a key contact and facilitator. We triage the questions to the appropriate organization.We provide support to practically every test that appear in protocols, except for CD4 testing, where the IQA is responsible to monitor performance and for HIV virological tests where the VQA is responsible.
34Additional benefits of SMILE ExperienceDiscounted costs and improved servicesConsistencyFlexibilityCollaborationExperience—we have extensive experience in all aspects of laboratory oversight. We utilize every member of the staff and draw on his or her strengths to give the best possible service to our labs.Discounts--Because of the number of labs we assist, we have an advantage in dealing with proficiency providers. We are the third largest user of CAP proficiency testing products, which gives us a tremendous advantage. We have points of contact and a relationship with the PT providers—we are able to work with them to solve problems unique to our labs. Ability to negotiate and manage premium shipping where needed.Consistency across the SMILE staff so that all labs are treated in a uniform manner. Extensive internal SOPs, constantly refining the process and identifying areas for improvement.Flexibility—we are not locked into the use of one PT provider, one shipper, etc. We have the flexibility to look at what will work best for the labs and customize their coverage accordingly.Use of SMILE creates a sense of collaboration—between all affiliated networks, the labs and DAIDS. A unifying force that ensures that the requirements for each network and DAIDS are met and maintained in the lab.
35In conclusion…How does SMILE work to achieve the goals of Clinical Laboratory Research Resources?To sum up—let’s look back at the original goals outlined for Clinical Laboratory Research Resources.There were 4 goals outlined for laboratory oversight.How does SMILE help achieve these goals?
36Safety of trial subjects We have an obligation and responsibility to protect the safety of the patients in our clinical trialsBy ensuring that laboratory tests are accurate we can ensure the safety of trial subjects.
372. Safety of those performing tests Good Clinical Laboratory Practices place emphasis on laboratory safety.Audit and Action Plan focus on safety of laboratory staff.
383. Quality of the laboratory data. Quality and integrity of study data.Why conduct research if the data is not reliable?Laboratory oversight ensures that study data is valid and reproducible.Documentation is created that allows reconstruction of study experiments and shows corrective action for any problems that occur during the study.
394. Increased appreciation of labs for good laboratory practices that translates to better local service.When we improve the quality in the labs, it creates a ripple effect.The culture of emphasis on quality spreads—to other labs, to clinics, to hospitals.Education and job training for laboratorians improves.Infrastructure and resources are improved.All of this translates into improved quality of healthcare in developing countries.
40Some of the things we are proud of… Laboratories taking the initiative to improve their quality measures on their ownLaboratories striving for (and achieving) CAP, ISO and SANAs accreditationLaboratories publishing/receiving awardsRecognition from local authorities of improved laboratory qualityImproved safety and increased testing capability and quality of TB diagnostic laboratoriesSome of the things we are most proud of are the side-effects, or by products of our involvement with labs.We strive to improve the lab as a whole—not to narrowly focus on only those things that directly affect our studies.The SMILE approach to lab oversight reflects a shift in the paradigm, away from the old model of placing Westerners on the ground to manage the labs.We aim for lab empowerment and improvement of local staff and infrastructure.We are very careful to train, mentor, assist and encourage—without doing the work FOR the labs.We want the quality of these labs to persist long after the DAIDS study has closed.
41DAIDS clinical trials outside the U. S DAIDS clinical trials outside the U.S. -furthering the quality of laboratory operations The NIAID SMILE resourceDr. Robert E. Miller, M.D.—Principal InvestigatorKurt L. Michael, M. Ed., MT (ASCP)—SMILE Project Manager