1. Martin Beran – QA John Roncoroni – FRD Van Starner – SD IR-4 National Education Conference Phoenix, AZ - March 1, 2006 Perspectives on QA Audits/ Responses

2. Martin Beran IR-4 Western Region Quality Assurance officer U.C. Davis

3. Those Darn Findings The Pseudo-Antagonistic and Poly- Unsaturated Architecture of QA Findings I Have No Idea What QA is Asking Me? How Am I Supposed to Respond? Who Gets a Copy of My Responses?

4. QA Obligations Researchers Need to Know What the Issue Is Findings Should be Clear Avoid Accusatory Tones Offer Solutions to Problems

5. These Findings Say it All You did not have your SOPs on site at the time of the application. According to the GLP regulations, SOPs should be immediately available. I recommend a copy of your current SOPs be placed in your truck.

6. One More: Part 4E, Test Substance Records - I cannot seem to locate test substance storage records for the period of 2/11 – 3/29/05. Test substance storage records are needed from the date of receipt to the date of the last application. Please insert additional records if available into the FDB.

7. Findings that are Weak Sauce Luke Skywalkers’s training records did not include any reference to GLP training. Any impromptu GLP training Skywalker might have had should be added to Part 2B of the FDB. This can be in the form of a copy of his ongoing Jedi training record or a separate sheet of paper with any informal GLP training documented.

8. Weak Sauce Continued Your Crop Destruct statement is insufficient. Please explain how and when the treated apples were destroyed. You simply stated that the row was taped off. Please document the measures you took to ensure that the apples did not enter the food chain.

9. Responses Did they answer the question Did they provide the necessary information Has the FRD carried his corrections over to the FDB

10. Responses I get a lot of these: OK Noted Study director will Address How dare you

11. Note to Study Director The plot layout consists of one row of 14 plants. The Protocol states that samples should not be taken from hops on the ends or the edges and they must be taken from at least five hills. Since the researcher has only one row to sample from, this seems to be a Protocol Deviation. Did you mean to write in the Protocol that hops could not be taken from plot edges? This means that plots need to be more than two rows of hop hills. No response needed from the researcher.

12. More Notes: The Chicory samples from the TA01 trial seemed to be quite different in appearance from the –DS05 samples. The Tatooine root samples were at most finger thickness compared to the Death Star samples which were round as a turnip. Even though Amendment #4 mentioned a potential difference in residue, the physical difference in the Tatooine samples was stark enough to be brought to your attention.

13. Routing FDB Findings

14. Routing CPI Findings

15. Additional Points Sending Corrected Pages Along with the Audit The Quality Control Review Communicate with the Auditing QA

16. Response To My Responses?

17. John Roncoroni IR-4 Western Region Field Research Director U.C. Davis

18. Finding #1 The GLP status of the test substance was not know “at the time of receipt,” and there was no record of a communication to the Study Director, who sent an email on July 28. Suggest changing to “No” (not known at time of receipt). Most time there is a GLP assurance in the letter from the supplier, but not in this case.

19. My Response “Technically I was taking the registrant’s word that they would actually send a Certificate of Analysis to Dan Kunkel at HQ - see page 3 part 4. Section 13 of the protocol states that ‘the registrant will provide a Certificate of Analysis to IR-4 headquarters.’ Not necessarily to the FRD.”

20. Registrant Letter “This test substance, which was obtained from production lot no. ABC123, has been found to have an active ingredient concentration of 9.3%. A certificate of quality has been forwarded to the attention of Dr. Dan Kunkel at IR-4 Headquarters.”

21. Finding #2 Part 6, page 14: 48.4 x 233.823 = 11317.033 Please correct this and the GPA and send the corrected page back to HQ.

22. My Response The numbers in part 6 page 14 (and actually on the page before also) are: 48.4 x 2333.823 = 112957.03 (reviewer is off by a factor of 10) !

23. Finding #3 Part 6 page 28 of 30: Show % deviation from Protocol rate. This is asked for in instructions for this page.

24. My Response Clarification added: Note: I think that 100.39% indicates that I was 0.39% over the protocol rate and is clear for the reviewer to recognize.

25. Finding #4 Part 5 page 14 of 20: The non-applicable ID# has been lined out but there is no error code.

26. My Response This is a field ID that is on a page that is copied – my answer is I have found in the past that there really is no appropriate error code - I will use WE for wrong entry.

27. Finding #5 Page 5, pages 7-9: On page 7, there is a notation of the GPS reading used for the water valve used as a permanent landmark. If you will be relying on that reading, versus the valve itself, we should add the GPS unit to the list of GLP exceptions in Part 1C. If the valve has some kind of unique ID, that could be used for future trials.

28. My Response I am relying on the valve as the permanent marker. The valves are far enough apart that the GPS will identify this valve as opposed to others.

29. Van Starner IR-4 Headquarters Study Director Rutgers Univ.

30. Current IR-4 Guidance San Antonio Results Document (9/27/02): Q: How do we correct raw data that we no longer have in our possession? A: Make the correction on a copy of the raw data and forward with the responses to HQ QA (updated as per new procedures). Contact the Study Director if there are any questions about responding to QA findings, or contact the QA who performed the audit with questions of intent. IR-4 Advisory #2003-01 (4/25/03): “Responding to FDB QA Findings”

31. Current IR-4 Guidance Instructions from QA auditor in audit cover letter: “Return the audit with your original responses and any corrections, clarifications or altered copies of your data pages to IR-4 HQ QA. The HQ QA will then forward the responses and any corrected pages to the Study Director and Testing Facility Management. Please keep a copy of the audit for yourself, and send copies of the responses to your Regional Field Research Coordinator. Only one mailing, your original response, needs to be sent to HQ.”

32. Who is the target for your responses? QA? Field Coordinators? Management? EPA?

33. The Study Director!

34. Why? By 40 CFR Part 160.33, the Study Director : Represents the single point of study control Has overall responsibility for the technical conduct of the study Is responsible for interpretation, analysis, documentation, and reporting of study results

35. So…….. When you receive an audit for review and response: Don’t take it personally that QA has “dinged” your work, BUT…. Consider how you can best respond to assist the Study Director in finalizing the study!

36. And Remember! Before a final report can be finalized, signed and submitted to EPA, EVERY QA finding must be completely addressed to the satisfaction of the SD, QA and Management!

37. FINDING : “Meteorological data not complete; last data is for 31 July 2003. Please update meteorological data.” RESPONSE : None – FRD only signed the bottom of the page of audit findings.

38. FINDING : “Part 6H – a brief description of procedure used in cleaning spray equipment is asked for here, not the SOP followed. Please provide this information. ” RESPONSE : “Prior to use, equipment is washed with clean potable water. Tanks are triple rinsed and flushed as are the lines, nozzles, strainers, and nozzle screens. Equipment is inspected for leaks or obvious wear prior to calibration and use. Clamps are ………………….”

39. FINDINGS : “  Part 6C – incomplete error documentation; all cross-out entries need to be initialed and dated and the error code added.  Lined out spaces need to be initialed and dated on Part 5G, 5H, 7B, 7D.  Part 7B – crop destruct; entry needs to be initialed and dated.” RESPONSE : For all three: “Acknowledged”

40. FINDING : “Parts 4 and 9 contain several photocopies that lack one or all: Field ID #, date, initials, location of originals” RESPONSE : “ No action needed, thanks for the suggestion.” OR “No response necessary from Field Research Director – Study Director should address.”

41. FINDINGS : “  Please line-out unused parts of Parts 1C, 2A, 4C (page 1 and 3), 5G, and 5H.”  A minor error is seen in Part 6, page 27 of 35. Total pass time was 41.5 seconds. 41 seconds has been used in calculations. This has given a lower % deviation of +3.65. It should be +4.76%.” RESPONSE : For both: “ OK”

42. FINDING : “Part 7A, sample collection/storage; you failed to mention whether row ends were sampled from, and whether five or more hills were sampled from. This needs to be added to the sampling description.” RESPONSE : “ Protocol states: ‘Avoid sampling from plot edges/ends,’ so we complied with the protocol.”

43. FINDING : “Part 6C, pg. 2 and 4 of 32: please cross out field ID of trial that does not belong in this FDB. In the future, only one field ID per page, please.” RESPONSE : “ Opinion – not written rule. If it’s not written, it doesn’t exist.”

44. FINDING : “Part 7B. Crop destruction section needs to be filled out with appropriate information.” RESPONSE : “ Acknowledged.”

45. - Overall, responses to QA audits have been improving significantly, as has the consistency of QA findings, etc. - There is still room for improvement on both sides, and it’s through training events like these that we can re- emphasize, face-to-face, the need to communicate, provide clear direction, and use a neutral (vs adversarial) tone in order to expedite responses and the entire review process. Final Thoughts