1. Navigating the Research Misconduct Process Kenneth L. Busch, Ph.D. NSF OIG Research Integrity and Administrative Investigations Division

2. Who is NSF OIG? – Independent office reporting to the Congress and NSB. – Promote economy, efficiency, and effectiveness. – Prevent and detect fraud, waste, and abuse. – Accomplishes mission through: Audits Investigations – Criminal and Civil (e.g., false claims, false statements, embezzlement). – Administrative (e.g., regulatory and policy violations). Where does research misconduct fit in? OIG is delegated the responsibility for investigating research misconduct allegations involving NSF programs.

3. Research Misconduct (RM) Federal definition and procedural framework (OSTP Dec. 2000). RM means “fabrication, falsification, or plagiarism in proposing or performing research …, reviewing research proposals … or in reporting research funded by [the agency].” (45 C.F.R. 689.1(a)) Not honest error or differences of opinion. Must be “reckless, knowing, or intentional” and not careless. Must be “a significant departure from the accepted practices of the relevant research community.”

4. Why care about RM? Fairness – An NSF proposal is a request to obtain Federal funds (ultimately) from the taxpayer. – NSF program officers/reviewers assess the proposers’ ability to carry out the proposed work and the proposers’ understanding of the current state of the field. – False representation of data and plagiarism misleads reviewers’ assessment of the PI’s (and Co-PI’s capabilities/knowledge. Economy, efficiency, and effectiveness – Do not want to pay for research already completed. – Do not want subsequent work to be based on misrepresented work.

5. OIG’s Investigative Scientists

6. In an investigation- Good Practices, Bad Practices, and Things that make us say “Hmmmm.” 45 CFR Part 689

7. The Basic Process  Allegations  Inquiry &  Investigation  Adjudication  Appeal  Final Decision  Define a problem  Ask questions & collect evidence  Draw conclusions  Reconsider  Finalize conclusions Looks like the scientific method Looks like the legal process

8. Take home messages I.Be prepared- II.Every case is unique- III.Document, document, document- IV.Go where the evidence takes you- V.Make a balanced, and supportable decision-

9. Research Misconduct Management Plan BeforeAllegationInquiryInvestigation Report & Decision After Not even a little bit

10. Research Misconduct Management Plan BeforeAllegationInquiryInvestigation Report & Decision After Policy Up-to-date. Integration with other policies. Not overly agency specific. OGC has designated someone specific. Available online. Out-of- date. Multiple conflicting policies. Agency specific. OGC never heard of the policy. Buried in a subpage in a corner of the web. Does your community know your policy? And where to find it? Will they act according to policy? To whom does your policy apply? Do they know that it applies to them? Is there buy-in at all levels? How is it integrated with other policies? (e.g., data management, authorship, IRB, IACUC, financial COI) Does your General Counsel know you yet?

11. RIO prevails and handles the allegation under RM policy with notification to NSF. When faculty don’t know about the policy, they make it up as they go. PI (junior faculty) raises fabrication allegation against student. Graduate coordinator gives student choice – leave or be expelled. Student leaves and PI enlists help of students to investigate. No formal process started until the Dean found out a week later. COI and gap in evidence chain-of-custody. Policy Mid- Investigation Policy Change Shhh... Don’t tell NSF University receives an allegation. Subject gets a lawyer. GC wants to handle as personnel matter and not tell NSF. Poof! Now we have a Policy!

12. Research Misconduct Management Plan BeforeAllegationInquiryInvestigation Report & Decision After Intake Process Complainant’s role is clear. Complainant’s role is neutral. Multichannel intake mechanism. Intake at neutral level. Complainant as adversary. Lack of anonymity. High evidentiary burden. Single intake point. Intake in zone of conflict. Can the allegation be anonymous? Must it be written? Is the complainant an entity (a journal or an agency)? Who receives the allegation? What should the allegation contain? Does the allegation impact research supported internally or externally? Is there any imminent harm or immediate need for action?

13. Seamless Handoff- Student Academic Integrity Committee adjudicates cheating allegation and passes allegation of plagiarism to RIO to handle federal funder requirements. Why be so formal about it? Dean receives RM complaints from faculty members in one department about another department’s PI. He does not begin university process (could impact active awards). Keeps it within the college for an informal process with no documentation. Intake Process OIG contacts university about allegation and does onsite inquiry. University GC learns for the first time of informal process below. University-level investigation process begins. But he didn’t say the magic word- RIO receives plagiarism allegation from student but immediately classifies it as an authorship dispute without further probing. Not our problem anymore - Student found to have committed Academic Dishonesty under student policy. RIO in the dark until student has been expelled.

14. Research Misconduct Management Plan BeforeAllegationInquiryInvestigation Report & Decision After Finding Substance Committee role defined. Committee products defined. Evidence is secured. Single person decision (possible bias). Evidence unsecured/lost. What evidence do you secure? When? Who conducts your inquiry? What are the requirements for confidentiality? What is the end product of the inquiry? What are the notification requirements? What is the timeframe for the inquiry? What do you do with admissions of guilt?

15. The easy way out - Dean notifies foreign student by letter which ends with a pre- written admission and a place for her to sign. The student is told if she does not sign, the Dean will begin the inquiry process. She signs and leaves the university. Finding Substance Something to work with here- Inquiry committee determines the allegations could be proven with further development of the evidentiary record and recommends detailed investigation. We changed our minds- Committee recommends investigation. Subject gets attorney. Committee reconvenes and notes many other possible explanations for the data and rescinds recommendation. OIG investigates; University out of the loop. Why bother? Because the subject admitted the wrong-doing, the RIO did not see any need to form a committee or conduct an investigation and proceeded to take care of it himself. Research Integrity Officer

16. Research Misconduct Management Plan BeforeAllegationInquiryInvestigation Report & Decision After All affected funders identified and notified. Non-RM allegations identified and passed along. Not notifying funders until process is complete. Ignoring non-RM allegations and not reporting them. What evidence do you secure? When? Who conducts your inquiry? What are the confidentiality requirements? What is the end product of the inquiry? What are the notification requirements? What is the timeframe for the inquiry? What do you do with admissions of guilt? Notifying Funders

17. Way to go! University RIO notifies OIG that an inquiry committee has determined an investigation is warranted and that the inquiry report is in the mail. Delays = Complications University correctly identifies NSF as a funder in the inquiry report, but does not notify NSF until investigation is wrapping up 2 years later. University missed out on opportunity for committee briefing and shared documentation. Criminal issues complicated by passage of time. Notifying Funders Through other channels RIO notifies only one federal agency, but has identified funding from multiple agencies. OIG learns through blog. But she said she’s sorry- Subject admitted extensive data manipulation but could not “remember” every place she did. University asks OIG for a multiparty settlement to avoid complete investigation. Subject wants to avoid debarment. OIG requests full investigation to fully document the extent of the fabrication and falsification. OIG finds link to high impact award; leads to debarment. Subject was employed by a government contractor but would not disclose this to OIG.

18. Research Misconduct Management Plan BeforeAllegationInquiryInvestigation Report & Decision After The Committee From relevant community COIs identified and managed Tenured Legal counsel available Experienced Policy briefing by OIG case lead Not from relevant community COIs before or during Untenured No legal counsel available Inexperienced No OIG briefing What are the confidentiality requirements? What is the timeframe for investigation? How are COIs identified and managed? What are the elements of a finding? Do they differ from NSF’s? What is the process for adding Subjects? How is testimony corroborated? Standing committees v. Ad Hoc committees

19. The Committee Committees are usually ad hoc and composed of faculty members without any specialized training in investigating. They are out of their comfort zone. They are worried about getting sued. They will gravitate back to their comfort zone at any opportunity. They are like juries, except they can ask the questions. Like juries, they need instructions. A RIO (with the help of counsel) should: guide them back into the investigation zone; not influence their assessment; provide the context for consistency. OIG has resources you can use. The OIG Case Lead The OIG investigation briefing The OIG website Semiannual reports Case closeout memoranda and research misconduct reports online.

20. Can’t you wait? Following a subject interview, a committee member tells the RIO that he has an interest in collaborating with the subject. RIO manages the new COI and tells the OIG case lead. The Committee Counsel should avoid representing both the subject and the university. Consistency – not always a hobgoblin +1 University; +2 similar allegations of RM; +2 different Committees operating in parallel; +2 very different investigations and outcomes. = Little to no consistency. Better Late than Never? Committee does not schedule its first meeting until day 175 of 180-day referral period.

21. Best Made Plans - University community is small and contains numerous sets of spouses/partners as well as interdisciplinary collaborations. Committee members realize during OIG briefing that they have COIs and cannot serve. Mum’s the word - Subject tells colleagues and reporters about the allegations against him. The case plays out in multiple newspapers. Committee members have finished report; maintain confidentiality. Institution acknowledges that process completed and that it is now with feds. The Committee Ask and Ye shall Receive RIO schedules the OIG briefing on the same day as the initial committee meeting as a way of setting the guidelines and policy expectations. Outside the box- University retains outside committee members to manage potential COIs and add technical expertise. No Committee is an island - Committee turns down OIG policy briefing. RIO is hands off. Committee report is haphazard and unsupported. No finding of RM. OIG does not find it accurate, complete, or in accord with reasonable procedures. OIG proceeds independently and recommends a finding and debarment.

22. Research Misconduct Management Plan BeforeAllegationInquiryInvestigation Report & Decision After Interviews Interviews recorded and transcribed Trust but verify statements Safeguards for vulnerable persons No interview documentation Statements not corroborated No safeguards for vulnerable persons How do you address confidentiality? How do you balance confidentiality with the number of witnesses called? What are your options for hostile witnesses? Can you compel participation of witnesses? If so, what rights do they have? How do you handle subjects and witnesses at other institutions?

23. He said, She said- Committee members interview subject and students without notes, recordings, or transcripts. Subject later contests committee’s interpretation of his statements. Go Ahead, Make our Day- Committee records the interviews, introducing all who are present and their roles. Individuals speak one at a time. Questions are open, eliciting free form answers initially. Closed, direct questions follow to clarify points. Committee remembers to ask directly “Did you do it?” Transcriptions are shared with witnesses to ensure accuracy (edits made in track changes). Original and edited versions included in case materials. Minimal Effort- Committee relies only on the evidence provided by OIG in referral letter. Committee does not interview the subject or any other witnesses, although they are available. Interviews It’s not what you ask, it’s how you ask it- Committee asks leading or Socratic questions prompting specific answers from the subject.

24. Research Misconduct Management Plan BeforeAllegationInquiryInvestigation Report & Decision After Evidence Chain of custody. Forensic images of digital media. Emails appropriately obtained. Minimal research impediments. Trust but verify statements. REP violated. No chain of custody. Unverifiable digital files. Unsupervised access. Inhibition of ongoing research. What can you secure? Data Notebooks Instrument records Samples Email(?) What is the subject’s reasonable expectation of privacy (REP)? How should you secure it? Can the research proceed? Where else may the evidence be?

25. Evidence You wanted us to investigate? Committee accepted the software generated similarity reports at face value without manually reviewing the individual sources, some of which were the subject’s own publications. Looking under the next rock Committee fails to look at other proposals and publications for additional misconduct. It’s all interrelated Committee fails to look at associated financial COI disclosures when the subject is known to have SBIR or other corporate involvement. The Devil in the Details University forensically analyzed hard drives and found that computer science professor manipulated metadata in proposal drafts to change dates to support his claims. Border Crossings - Gathering evidence in investigations across international lines has unique challenges.

26. Research Misconduct Management Plan BeforeAllegationInquiryInvestigation Report & Decision After Assess the act. Assess proposals & products. Develop the evidence. Connect the dots. Consistent standards. Assess the science. Ignore proposals or products. Assess only what OIG provides. Non-sequitors. Double standards. Are you analyzing the science or the misconduct? What are the elements of a finding, per policy or referral? What is pattern evidence and where do you look? What can you use it for? Can the IC identify additional Subjects and allegations? How does the IC corroborate testimony? How do you explain/define intent levels? Analysis

27. Reductio ad absurdum- Under the guidance of counsel, the committee analyzed each segment of text as discrete units without looking at the totality of the copying. Committee used copyright (author centric) instead of plagiarism (reader centric) analysis for material copied from instrument brochure. Chain-link Plagiarism- Committee found that PI copied from student, and that student copied from sources. Committee found that the actions of both met the definition for plagiarism. Analysis Mixed Signals- Committees mix the levels of intent: “carelessly knowing” “at least knowingly and most probably carelessly” “carelessly, recklessly, and unintentionally” Being right doesn’t make it right- Although the subject argued extensively that he ultimately proved his conclusions correct, the committee correctly found that at the time of the misconduct, the data were falsified to support the hypothesis.

28. Research Misconduct Management Plan BeforeAllegationInquiryInvestigation Report & Decision After Report Evidence-based conclusions. Connect the dots. Specific recommendations. Active voice. Addresses referral questions. Subject opportunity to comment. Assumptions not conclusions. Non-sequitors. Vague institutional response. Passive voice. No questions answered. No opportunity for subject to comment. To whom is the draft distributed for comments? Who is the deciding official? What if individuals have left the institution? How are reputations rehabilitated? What are potential legal implications? Are the final conclusions reasonable?

29. Close but no cigar- Deciding official does not make an explicit finding of research misconduct but takes action. And the Oscar for best report goes to – Report clearly address each element as outlined in university policy and responds to OIG referral questions with specificity, including INTENT. Each conclusion is supported by the evidence and the committee’s rationale is clear. An unambiguous statement of finding (or no finding) is made. Report Where’s the Evidence? Report submitted to OIG does not contain all the evidence on which the committee relied. The other side of the story- Report submitted to OIG but the subject was never offered an opportunity to comment or rebut the finding at the institution.

30. Research Misconduct Management Plan BeforeAllegationInquiryInvestigation Report & Decision After Notifying OIG 45 C.F.R. 689.4(c) (risk to health or safety, civil or criminal violations, etc.) Subject retains counsel. Ancillary allegations. Deviations from policy. Any surprises. Not notifying OIG until the end, if ever. What can the OIG case lead do for me? How do you know if there is a potential civil or criminal violation? What will NSF or OIG think if we tell them something is wrong?

31. It’s in the mail – RIO fails to deliver subject a copy of the referral packet at the time records are secured (or at any other time during investigation). And then there were two - During an interview the witness disclosed information that made him an additional subject. RIO immediately notified OIG. Notifying OIG They said what? In separate interviews, two subjects each admit voluntarily to knowingly misstating the experimental conditions in a publication. A witness also admits to knowingly misstating the conditions. The admissions are not part of the originally alleged misconduct. OIG learns of the admissions when reading transcripts. On the same side of the “v” - RIO receives the subject’s comments to the report which have a cc to Bob Smith, Esq. of Dewey, Cheatham, and Howe. OIG case staff notified immediately. This triggers represented party rules for the attorneys involved.

32. Research Misconduct Management Plan BeforeAllegationInquiryInvestigation Report & Decision After Implement Actions Consult Debrief Review policies Read the OIG report Wait Do nothing Assume it will never happen again Don’t read the OIG report How long do consequences last and who monitors them? Are policy revisions necessary? How do completed cases affect RCR training? How is public disclosure and/or complaint handled? What if the subject has left the institution?

33. It’s a work in progress- University officials with oversight responsibility are told of the university decision and of any requirements imposed by NSF (e.g., assurances, data and mentoring plan reviews). He’s not our problem anymore - University fails to notify NSF that it has replaced the PI on and NSF award because the PI was found to have committed RM. OIG investigation is ongoing. NSF must approve replacement PIs – OIG cannot do this Implement Actions Learn, Practice, Repeat- University RCR program uses redacted report as a starting point for case studies in live-instructor training; subject required to be involved in teaching the class. Debunk the myth of “it doesn’t happen here.”

34. I.Be prepared- Have a comprehensive, up-to-date policy to guide you. II.Every case is unique- A good policy will get you through twists and detours. III.Document, document, document- This is the support for your direction and decision. IV.Go where the evidence takes you- Who, What, Where, When, Why, and How. V.Make a balanced and supported decision- Make it timely, focused, and clear.

35. Contact Information www.nsf.gov/oig Hotline:1-800-428-2189 E-mail:oig@nsf.gov Fax:(703) 292-9158 Mail: 4201 Wilson Boulevard Suite II-705 Arlington, VA 22230 ATTN: OIG HOTLINE