Promoting the Responsible Conduct of Research for College and University Leaders

Office of Research Integrity & Loyola Marymount UniversityHilton Garden Inn, Marina del Rey, Los Angeles

April 14-15, 2016

ContentsDay 1: Thursday.....................................................................................................................................................................1

I. Welcome and Introductions, John Carfora, EdD, CCEP, RIO; Associate Provost, Research Advancement and Compliance, Research Integrity Officer, Loyola Marymount University...........................................................................1

A. Zoë Hammatt, JD, MPhil; Director, Office of Research Integrity, Division of Education and Integrity................1B. Susan Garfinkel, PhD; Director, Division of Investigative Oversight.....................................................................1C. Scott Moore, National Science Foundation (NSF), Office of Inspector General (OIG).........................................2D. Lisa Buchanan, MAOM; Office for Human Research Protections; Division of Compliance Oversight................5E. Patricia Brown, VMD, MS, DACLAM; Director, Office of Laboratory Animal Welfare, National Institutes of Health...............................................................................................................................................................................5

III. The Role of Senior Institutional Leaders in Promoting Responsible Research...............................................6A. Fostering Research Integrity & Providing Incentives; John Carfora, EdD; CCEP, RIO, Loyola Marymount University.........................................................................................................................................................................6B. Ideal & Problematic Scenarios; Lauran Qualkenbush; Director, Office for Research Integrity, Research Integrity Officer, Jay Walsh, PhD; Vice President for Research, Professor Biomedical Engineering, Northwestern University. 8

IV. Responsible Research in Cases Involving Humans & Animals............................................................................10A. Coordination Between VPs & Research Integrity Officers, Institutional Counsel, IRB, IACUC, Jennifer K. Yucel, PhD; Director, Office of Research Compliance, Research Integrity Officer; Emily Schriver, JD; Special Assistant to the Senior Vice President and General Counsel, The Ohio State University............................................10

B. Noncompliance in Clinical Research Settings & Protecting Participants Lisa Buchanan, MAOM; OHRP.............11C. Protocol Violations & Protecting Animal Subjects, Patricia Brown, VMD, MS, DACLAM; OLAW.................12

V. Responsible Conduct of Research Within and Beyond the Institution......................................................................13A. Drivers of Misconduct & Biased Proceedings: The Power of Institutional Leaders to Ensure Integrity, Raju Tamot, PhD; ORISE Fellow, Office of Research Integrity...........................................................................................13B. Establishing an Office of Research Compliance: Challenges & Opportunities, Gary K. Ostrander, PhD; Vice President for Research and President, FSU Research Foundation, Florida State University........................................15C. Building Networks: Responsible Research in Diverse Community Settings, Keith C. Norris, MD, PhD; Clinical Professor, Department of Medicine, University of California, Los Angeles.................................................................16D. Perspectives from the National Institutes of Health, Patricia Valdez, PhD; NIH Extramural Research Integrity Officer, National Institutes of Health.............................................................................................................................17

Day 2: Friday........................................................................................................................................................................18VI. Tips on Handling Research Misconduct Allegations & Promoting Integrity................................................18

A. Office of Research Integrity, Alex Runko, PhD; Scientist Investigator, Division of Investigative Oversight......18B. National Science Foundation, Scott Moore, Office of Inspector General.............................................................20C. Institutional Approaches, David Hudson, PhD; Associate Vice President for Research, Research Integrity Officer, University of Virginia.......................................................................................................................................22

VII. The Role of Senior Institutional Leaders in Handling Research Misconduct Allegations............................23VIII. Small Group Discussion: Topics of Interest for Senior Institutional Leaders...........................................28

A. Group 1: Handling Allegations of Research Misconduct......................................................................................28B. Group 2: Fostering Research Integrity: Incentives and Monitoring......................................................................28C. Group 3: Misconduct Involving Human and Animal............................................................................................29D. Group 4: Responsible Conduct of Research Within and Beyond the Institution...................................................29

IX. Concluding Remarks..............................................................................................................................................30

Day 1: Thursday

I. Welcome and Introductions, John Carfora, EdD, CCEP, RIO; Associate Provost, Research Advancement and Compliance, Research Integrity Officer, Loyola Marymount University Theme of senior leadership first developed several years ago. This is a vision I’ve had for a long

time: How do we get senior university leaders to take more of an articulated stand on the promotion and belief on ethics and integrity and not think in strictly in terms of compliance? Compliance will keep you out of jail; integrity will get you into heaven. By gathering such a diverse group – the representatives from federal agencies; animal and human protections people, lawyers – we’ll make a lot of sense here.

Our job is to think about how we can engage senior leaders to take a more active role in what we all do for a living.

II. Definitions and Regulatory Requirements Related to Responsible Research

A. Zoë Hammatt, JD, MPhil; Director, Office of Research Integrity, Division of Education and Integrity ORI consists of the Office of the Director (OD) and two separate divisions: the Division of

Investigative Oversight (DIO), and the Division of Education and Integrity (DEI), which I have been director of for past two years.

As condition of applying for/receiving PHS funds, we 42 CFR 93. Most important part: falsification, fabrication, and plagiarism committed intentionally, knowingly, recklessly. Hoping to issue specific guidance that more clearly defines recklessness soon. Must have preponderance of evidence, and must be significant departure from accepted practices.

As condition of receiving federal funds, we require institutions to apply for assurances. DEI also conduct policy reviews – working to make this more sophisticated and streamlined. Susan and DIO investigators provide, through Rapid Response Technical Assistance (RRTA)

program, assistance for those asking hypothetical questions and assistance with operating forensic tools.

Also have exciting new grants programs: conference grants and research-driven. As part of the funding opportunity announcement, one goal was to produce tangible outcomes. Research-driven grants aim to help people understand factors that contribute to research misconduct (RM) and factors that help instill integrity at the highest levels.

DEI also produces education and communication products that aim to simplify and convey complicated processes surrounding RM and questionable research practices – both for institutional officials and students. Goal has been to produce shorter videos and case studies. The Lab was wonderful resource, but we also wanted shorter, more easily digestible videos on separate topics.

ORI carries out federal regulations globally to institutions receiving PHS funding through Assurance and Annual Report programs. We have assurances 400 international institutions throughout Africa, Asia, Latin America, Europe.

Just formed Asia-Pacific Research Integrity (APRI) network.

B. Susan Garfinkel, PhD; Director, Division of Investigative Oversight When institution receives allegations, they conduct assessment, then possible inquiry and

investigation depending on severity. If institution goes from inquiry to investigation, ORI becomes notified at that point. We’ve seen increasing number of allegations coming to ORI.

Some allegations come directly to ORI. Our case-tracking system monitors every allegation. By regulation, the allegation must be specific and credible.

If we find allegation doesn’t have merit, it’s closed for us. If it does need to be referred to institution, institution starts its process.

When ORI gets investigation, DIO communicates with RIO for more information. Investigation report comes in with supporting documentation. The more you send ORI, the better; we review everything. After we decide to make finding, we settle the case. 90% of cases where we make findings of RM, we settle. If we can’t, we go through Office of General Counsel – which puts scientific review process into legal proceedings.

If we can’t settle a case, goes to charge letter with our attorneys; they have right for appeal and it goes to judge. Can take years depending on how obstructive respondent becomes.

The reason we’re all here today is to keep you all out of the media. Been at ORI for 13 years, and publicity around RM is at all-time high – more in mainstream media than ever before, especially with human subjects research.

C. Scott Moore, National Science Foundation (NSF), Office of Inspector General (OIG)

From NSF OIG. Independent office part of NSF. As independent office, we serve as internal watch dogs of agency and report directly to

Congress. Goal: promote effectiveness and efficiency, and prevent fraud, waste and abuse. Inspector General: Allison Lerner. Office mostly functions through two arms: Office of Audits

and Office of Investigations. We’re unique among government OIG offices in that we have entire division of investigative

scientists. NSF has delegated investigations of RM to OIG. We investigate, don’t decide. We are law enforcement office. Agents can do search warrants (usually happens at small

businesses). All of us are cross trained. Throughout office, we have experience in research, grant contract

administration. Similarities and differences in what OIG does and ORI does:

Both conduct investigations, both promote research integrity. However at OIG, we provide independent oversight of NSF operations, we’re not a

programmatic office. We are law enforcement office with direct subpoena authority and search warrant

capability. ORI is not law enforcement office. At OIG we can conduct independent investigations, don’t have to refer to investigation to

the institution. ORI has limitations on its ability to investigate independently. OIG does not make decisions regarding findings, ORI can. NSF conducts investigations behind a firewall, NSF does not know until we give them

the report and investigation is complete. Only know of two involuntary exclusions at NSF.

Settlements do not happen before we complete the report. Reports available online, but are heavily redacted and sanitized. We only release names if there’s been a debarment (on sam.gov). In contrast, ORI publishes in federal register.

Limited outreach activities. We can’t award conference grants. ORI has much heartier infrastructure for education and oversight.

At NSF, very few regulations that govern what we do. Refer to America COMPETES Act of 2007: mentoring plans, RCR training plans, and sharing research results (investigator who fails to comply with data sharing policy prohibited from receiving NSF award).

Regulations: We follow common rule for human subjects (45 CFR part 690) and NSF research misconduct regulation (45 CFR part 689). Definitions of policy come from OSTP guidance issued in December 2000.

Same basic framework that ORI uses with a few tweaks: inquiry phase is 90 days and investigation phase is 180 days because we have different trigger points throughout the process.

We operate under policies and grant terms. Animals in research, financial COIs all handled by policy incorporated into grant agreement. Refer to Proposal & Award Policies and Procedures Guide (PPAPPG).

RM is our main regulation, finalized out of OSTP in December 2000; adopted and finalized in April 2002. NSF is second agency to adopt RM defined as falsification, fabrication, or plagiarism in proposing or performing research funded by NSF, reviews of proposals, or in reporting research results. Proposal doesn’t have to be awarded, only signed by AOR and entered into system.

RM doesn’t include mistakes, honest error, difference of opinions. By regulation and grant terms, institution must inform OIG if initial inquiry supports formal investigations. Even better if you come to us at inquiry phase, or about hypotheticals. More communication we have, the better we can get through process.

If we need additional info, may be initial back and forth, but we often conduct supplemental investigations on our own (interviews, subpoenas), especially with multidisciplinary research functions, where we may not have access to other institutions.

Must have preponderance of evidence, and subject must have acted at threshold level of intent Once we have final report, goes to Deputy Director, who can request additional investigations or

move forward. There is opportunity for hearing; only has happened a couple times. It is NSF’s final decision, but can be reviewed in federal district court.

Fabrication – making up data plagiarism. Falsification – manipulating research materials, equipment, process, or changing or omitting data or results. Different from copyright infringement.

Must be significant departure from accepted practices. Question from journal editor to Scott Moore: We’ve had problems with self-plagiarism where

people rehash previously published info, and we consider self-plagiarism plagiarism. How does that work at NSF? Answer: We don’t consider it plagiarism. From journal editors perspective, it’s copyright infringement. We don’t get involved.

Question to Scott Moore: Is the number of cases at NSF involving plagiarism being higher than ORI’s because of the difference in staffing support? Answer: No; it’s because of the difference in type of research we handle. ORI deals with biomedical; NSF deals with physical science.

Discussion around authorship disputes: ORI does not view authorship disputes as plagiarism. If we had to handle all authorship

disputes, we would not have adequate staffing levels to carry out. ORI has definition of plagiarism on our website. If there’s an argument between

collaborators, it doesn’t rise to issue of plagiarism. Authorship is not RM case – could be something that institution handles as questionable research practice, but does not reach federal definition for RM.

NSF funds “everything else,” including ground-based astronomy. Authorship disputes not plagiarism, we assess whether there’s a fraud or criminal aspect to it, because we have authority to investigate – criminal and civil analysis. Different way of treating investigation based on our different missions.

Question to Scott Moore: Big data and confidentiality aren’t addressed under these three definitions – how does NSF handle breaches of confidentiality in review process? Answer: We handle those cases, but they’re not RM cases. They’re policy violations where we go back to OHRP and coordinate with them. We issue warning letters/questionable practices letters to institution.

Susan Garfinkel (ORI): Experience that ORI has had dealing with RIOs – if you ask some whether they think jurisdiction should be broader, they say no because university has its own policies in place to deal with questionable research practices. Other RIOs at smaller institutions believe that the government should have broader authority. It takes a long time to change those

regulations, and you have to have buy-in from community, but there’s not a clear consensus among community.

Research record: everything that federal money touched. Physical and electronic data in the least processed form possible. Also looking at data and specific analysis of results published, and the data that was excluded (bad data).

“Intentionally, knowingly, or recklessly.” Intent tough to explain – [NSF] developed written guidance just approved. When we give referral, we offer up chance for your committee to ask us questions gain insight into our process. We try to take fear factor of working with government out of investigations. A lot of universities don’t take us up on this briefing, but it’s helpful because we don’t have to come back to you.

Federal RM findings have legal consequences subject to review by federal district court. Final decisions are reviewable by state or local board. If a subject is not happy, they can always challenge in court.

“Intentional intent” – confusing term, often qualified as “purposeful/willful.” Court will use these definitions to review these cases if they go to court.

Definitions will vary by jurisdiction, so we use “Black’s Law” dictionary – homogenized version of levels of intent. We treat it as starting point for assessment of intent. There are a lot of factors that you put together and weigh to determine level of intent. When you deal with faculty, they tend to overanalyze. Don’t overthink.

Intent treated as continuum. No finding for careless, finding for reckless, knowing, intentional. Careless and reckless “reasonable person standard.” Our guidance is always to consult with legal counsel.

Carelessness: Subject did not exercise level of care that reasonable person would have exercised under similar circumstances.

Recklessness: consciously aware of a substantial risk that FFP will result from his/her action or inaction and they fail to exercise level of care that someone of ordinary prudence would exercise under those circumstances.

Knowing: Subject had awareness and understanding of his actions. (subject highlights content, then hits CTRL C, CTRL V). Almost all plagiarism cases are immediately considered “knowing.”

“Should have known” – refers to the fact that they’ve gone through entire US education system and claims they didn’t know. Comes under reasonable person standard.

Negligence: Would a reasonable person under same circumstance acted similarly or with greater care to avoid the harm? Evidence of patterned behavior can be used to assess intent. Pattern of behavior can demonstrate state of mind or ongoing plan to commit RM. Pattern does weigh; depends on extent of pattern.

Question to Scott Moore: Regarding timing of investigations (180 days), do you have data that indicates what timing actually is? Answer: We’re in process of changing over case-tracking system. We have discretion to lengthen time period from six months to a bit longer. The investigator does have to go to OIG and explain the case. Not uncommon to give 30- or 60-day extension. Anything beyond that, we’ll ask to see what progress you’ve made.

D. Lisa Buchanan, MAOM; Office for Human Research Protections; Division of Compliance Oversight

Overview of office: Mission is to provide leadership in protection of rights, welfare, and wellbeing of human subjects research. Anything supported by Department of HHS is regulated by 45 CFR 46. Our organizational structure: Office of Director, Division of Education and Development, Division of Policy and Assurance, and Division of Compliance and Oversight.

Division of Policy and Assurance: When institutions receive funding, they have to have file an assurance with our office.

Division of Compliance and Oversight: Charged with assessing compliance with 45 CFR 46.

Regulations on human research protections: Goes back to Tuskegee syphilis study: 1933-1972. Subjects not offered penicillin as the standard of treatment; some even prevented access to it.

Three basic principles: respect for persons (promoting autonomy), beneficence (maximize benefits, minimize harm); justice (equitable distribution of burden and benefits).

45 CFR Part 46 Subpart A – The Common Rule: IRB and informed consent. B: pregnant women and fetuses, C: Prisoners, D: children subpart E: IRB registration. Not the only vulnerable subjects in research, and IRBs are charged with having protections in place for others – students in research.

ABCs of HSP: A: Assurance B: Institutional Review Boards (IRB), and C: Informed Consent Need to provide more accessible records, remote reviews, more reliances and agreements among

institutions. More centralized IRBs. New proposed rule: one of the key elements includes requirement of independent IRB for multi-site trials.

Institutions are grappling with costs of proposed changes. Accreditation from Association for the Accreditation of Human Research Protection Programs (AAHRPP). Increasing expenses for institutions aiming to stay current with technology and how IRBs operate.

Our office only covers research funded by HHS that involves human subjects. For your assurances, you’re allowed to apply HHS regulations to all research regardless of

funding. If you have checked the box, anything you conduct can be investigated by OHRP. If you have not checked the box, we cannot investigate or make findings on research noncompliance for that research.

Must have assurance if you receive HHS funding. One of the first people we speak with is signatory or institutional official who signed assurance. In more recent years, we have evolved to establish a more conscientious culture in protecting human subjects. We urge institutions contact us to help us walk you through issues you’re concerned about.

OHRP oversight would include all clinical research; the new proposed rule eliminates and changes what would have come through IRB. Change in definition expands coverage that would eliminate “checking the box” issue.

E. Patricia Brown, VMD, MS, DACLAM; Director, Office of Laboratory Animal Welfare, National Institutes of Health

Federally supported scientists are accountable from planning to completion of research to protect welfare of research with animals used in research. From start to end, they’re accountable.

Legislation that authorized our oversight: Health Research Extension Act of 1985. USDA was also amending its regulations in response to Congressional demand to have more oversight with creation of Institutional Animal Care and Use Committee (IACUC). Though we sit at NIH, we have authority and responsibility for all PHS-funded activities.

Animal welfare assurance mechanism – commitment between your institution and us. Requires IACUC at your institution and says that institutions should have opportunity to correct policy before NIH takes action.

“Public Health service policy and humane care and use of laboratory animals” – applies to CDC, FDA, NIH. MOUs with NSF and VA.

Different from animal welfare act regulations: PHS covers all vertebrae. Any live vertebrae animal intended for use in research, research training, experimentation or biological testing.

Self-regulation by institution. There’s specific language about self-regulation at the local level. Policy requires institution to self-report when things go wrong versus us inspecting on a yearly basis like USDA.

PHS policy standards for animal care and use: your institution must base its use on “guide for the care of use of laboratory animals.”

U.S. government principles: similar to Belmont report in how animals should be cared for.

Principle II: procedures involving animals should be designed and performed with due consideration of the relevance to human and animal health. If you’re going to do something to an animal, there better be a good reason.

Principle III: appropriate species and quality and the minimum number required to obtain results. Replacement, reduction, refinement. Methods, such as mathematical models, computer simulation, should be considered.

Principle IX: IACUC makes decisions on where there should be exceptions to principles in terms of care of animals.

When it comes to scientists themselves, they need to obtain IACUC approval of proposed activities. If they make changes, they must continue to receive IACUC approval.

Conducting research according to protocol and adhering to institutional processes (biohazards, etc.)

Other responsibilities: if making significant changes to research scope, NIH must know. Must obtain prior permission from NIH for use of animals involving change in of scope, including change in performance site. Good communication is essential. Site could be in foreign country.

Post-award missteps: conducting research in absence of IACUC approval or implementing significant change without IACUC approval. Reportable to us through IACUC to OLAW and the funding component.

Many times these changes are open and closed. You found it, issued corrective action plan, we send letter back.

Animal activity going on without IACUC approval – appropriate adjustments must be made to remove charges associated with unauthorized animal research without IACUC approval.

We look at this as collaborative – here to assist in process in overseeing program, provide guidance on things that are noncompliant, and ensure appropriate corrective action.

III.The Role of Senior Institutional Leaders in Promoting Responsible Research

A. Fostering Research Integrity & Providing Incentives; John Carfora, EdD; CCEP, RIO, Loyola Marymount University Overseeing your staff in the complex array of issues that arise in research. Bringing in those that represent federal government, those from universities, and institutional

leaders (Vice Presidents/Provosts of Research) represents an intersection of regulation, compliance, ethics, behavior, among other things.

Thoughts on research and scholarly integrity along with reflection of experiences implementing RCR training at higher learning institutions: How do we engage senior leadership?

What strategies can we help introduce and enhance to ensure that our colleges and universities have adequate systems in place for research and scholarly integrity, RCR, the responsible conduct of scholarly activity, and larger compliance-related themes?

What if you’re in the arts and humanities—how do we address RM in these disciplines? The intersection between compliance and ethics, compliance and integrity. How might we better

educate and align our efforts with college/university constituents and senior academic leaders especially?

How do we engage, not only leadership, but institutions as well? How do we use our voices to talk to students, chairs?

Recommended article by Geoff Grant, Odell Guyton: “Creating an Effective Research Compliance Programs in Academic Institutions.”

[John Carfora] established research compliance committee at Dartmouth College, then became director of research at Boston College. Currently serve as Associate Provost for Research advancement and compliance at Loyola Marymount.

Presented on subject of ethics, integrity, RCR and compliance throughout the USA and abroad.

Recommended book: “University Ethics: How Colleges can Build and Benefit from a Culture of Ethics”

Take a values-based approach toward ethics, integrity and compliance. We have to take institutional responsibility that considers what federal agencies mandate.

Codes of conduct, code of ethics. Clearly articulated standards and procedures. Compliance oversight (compliance officer). How do we educate Board of Regents? Start at the top, engage board/president/chancellor. Educate them, don’t intimidate. Mentoring and auditing. Reporting an investigation. Enforcement, incentives, and response and prevention, education and awareness.

If research involving human subjects research has been proposed, it’s a flag for training. If it involves animals, has IACUC application been submitted? Flag for training. Do we have

highly trusted veterinarian on staff? Every year, put together compliance plan for president and provost.

Confidentiality. All opportunities for training and training leadership to get them on board. Connect with director of risk management, director of internal audits. Review institutional codes of conduct, ask how they are shared across your institution and

viewed and received across campus. If you don’t have compliance officer, talk to leadership about getting one.

We have to take a proactive stand, domestically and internationally, with the leaders of our institutions. Don’t wait for an adverse event before you’re forced to call someone about an issue.

Work to develop “institutional trust” and a “culture of compliance.” Education and training is key – allows us to dialogue with campus about what we’re doing. Brief senior leaders, presidents, provosts, on ethics, RCR, integrity. Show them what you’re

doing well, what needs improvement, where you need resources. What are the barriers to promoting culture of research integrity?

Question to John Carfora: In most institutions, who is the compliance officer, and what is their background? In some institutions, the compliance office is a totally separate office that reports directly to U.S. president. Could be a combination of financial compliance along with research compliance. Research administration, research compliance, all housed within one office. Answer: When we first started out, compliance became a hot topic. There is a difference between institutional compliance and research compliance. Institutions typically appoint a compliance director. It comes down to institutional context and the scope of research you conduct. Smaller institutions may take their own approach. There’s no one correct answer. The questions you should ask: Are we doing the right thing? Can we do it better?

We have to create institution-wide culture of integrity, ethics. If we’re not talking about it, not preparing undergrads about difference between FFP, how are they going to behave in graduate school? We need to educate people on all of these topics. The graduate students need to have RCR funding. They’re going to go into the classroom to educate, and we want them to have the level of integrity they need to do it right.

B. Ideal & Problematic Scenarios; Lauran Qualkenbush; Director, Office for Research Integrity, Research Integrity Officer, Jay Walsh, PhD; Vice President for Research, Professor Biomedical Engineering, Northwestern University. As RIO, I think about how difficult is to do our job regardless – it’s great to have VP of research

to collaborate with. Function of RIO: all about increasing and enabling the research that occurs on our campus. That

includes not just the volume of the research and resources available, but also ensuring that that research is done ethically, morally, responsible, safely, reproducibly. Extraordinary research done safely. This is about doing research safely, not university pride/competition.

Messaging to board members: we send a note to every board meeting explaining what is happening broadly in compliance area; what we do across the entire office for training. We hire faculty, and this is what we want them to do: impactful research. Enabling research and discovery has many different dimensions, and explaining that to a board member is challenging.

We have to explain at the top (Boards) and then bring it down. Public trust: Most of us get money from the public. The taxpayers should feel like were doing

this work well and correctly. There’s a public trust component even when we’re not using money directly from taxpayers. Training is an opportunity to touch students, particularly grad students, as soon as they come in the door. At Northwestern, one of the first things grad students encounter is the safety office – relays a culture of research integrity that hits them immediately.

Commitment to compliance: If you see something wrong, you can close the door in the lab. I will back you up every single time. Backed by institutional policy and by president/provost.

Compliance person has to be able to quarterback a wide range of folks. Being able to work well with a wide range of people and range of personalities is extremely important. Somebody who can play well in a team with everybody.

Lauren: Being the person that needs to feel empowered, we need policies there to support us. Sometimes people encounter me because I’m there to sequester. I can show them the policy and expect that they’re going to call Jay. We work as a team, we coordinate efforts, and point back to our policy.

The culture and expectation at institution: This is how we do things, and here are the policies. My contribution to promoting research is ensuring that we have the proper processes in place: protecting the process and protecting the people in the process. Much more difficult with people in vulnerable situations.

One of the most difficult points of this job: people coming forward who are vulnerable. Grad students and post-docs coming forward, reporting on someone in the food chain. That person was someone they looked up to, who could write letters for them. Would love to hear how others work with students and post-docs and how they deal with this.

The timing is very important. It needs to be faster. Would love to see data on how quickly things move through the process.

Part of the issue is the trust of the community. Ensuring that people know there’s someone they can come to. Be visible, go to RCR classes.

We don’t talk about how we reward good behavior. How can we reward good behavior? That’s a big part of how we create this culture of ethics and integrity.

Methods for incentivizing and rewarding RCR/service on committees: John Carfora: We hand out awards every year. Awards for good behavior, ethical

behavior. Circulate a booklet with recognition. What are the metrics, standards for getting an award? Criteria: good at what they do, trust of the community. It’s a qualitative assessment.

We also need to recognize service on committees. Good conduct is not being appropriately awarded for service on committees.

Lauren Qualkenbush: We’re doing an analysis on service on IRB and IACUC panels – looking to provide a different framework that includes incentives and recognition.

Susan Babcock: We have persuaded chairs in departments of medicine that service on IRB panel should equal service on review board.

Lauren Qualkenbush: we pay IRB and IACUC members, but not research integrity chairs.

We annually ask our institutional official to send personal email thank you letters. The letter includes formal evaluation of their performance. The department chairs and directors use a certain set of information to evaluate.

Take a personal approach informally, when you run into people, tell them in person. Want them to hear it from me, not just on a letter once a year.

Provide food and coffee. Annual letters. John: when we have our recognition luncheon, we invite the provost and president. Make

sure that the provost comes. The RM piece is often only one piece of a much larger issue. When we don’t deal things, they

snowball. Need to deal with noncompliance in a practical way. RI issues are the tip of an iceberg. There are a lot more issues going on – disharmony among

faculty, etc. Challenge is: how do you solve these challenges earlier so that they don’t become RI issues?

Question to Jay Walsh: How do you deal with dean and president calling you about a case? Answer: You have to build credibility. In contentious situations, having clear evidence of what your policies/procedures are is what you need. Your policies are what’s guiding you, and that’s the protection for your office. Sometimes, explaining exactly what the process is starts to calm them down. When we don’t deal with these things and put good policies and procedures into place, it’s a free for all.

We put cases into three buckets: 1) people who think rules don’t apply; 2) people under pressure and 3) those who are untrained, unqualified, unsupervised.

Group 1: often the loudest, most litigious, highly connected, might call the president. One message we’re trying to get out: The quality of your work does not allow you to be

noncompliant. Doesn’t matter how good your work is; how many awards you’ve gotten – it doesn’t justify noncompliance.

The purpose of this meeting is get everyone in a room in talk about what we need. The community can come to a group consensus to effect change. There are ways that the community can catalyze the conversation.

Proactive measures: creating clear expectations. What is your job as a principal investigator? Do you understand your responsibilities? What does it really mean? Building a framework, setting expectations.

We need appropriate resources; staff who are able to do their jobs effectively. The major problem isn’t that they don’t know what they should do – it’s that they don’t have the

resources to do it correctly and deploy the resources to do so. We have created a guidebook with tools, tips, resources.

Tips: Train faculty to become mentors, because they set the tone, regardless of policies, for how research is done. Help them understand what expectations are. Get the disciplines to all agree on basic things that they want to cover.

Question: How do you evaluate whether these things are working? Answer: We’re doing some analyses and it looks like students are taking these things with them into the lab. There are a lot of students who just want it.

What about people who grew up in different countries; people with different cultural expectations? People may have their own convictions about how things should happen; cultural context affects RM processes.

When I think about whether I’m succeeding in promoting RCR, the biggest thing I can do is conduct outreach, awareness, be seen. Go to every RCR class. Every time I go out there, I get a call or email.

Question: Curious about pushback on RCR training – did you ask them exactly why they’re not receptive to it? Answer: There are a lot of things required of our grad students, and this isn’t the highest priority. Problems and concerns with how mentors view requirements. The minimum we can do is make sure that educational requirements are met and that they are sensitized to the idea.

As a an administrator, I do not control curriculum. If I can’t convince people to implement specific curriculum, it’s my fault for not presenting a convincing argument.

IV. Responsible Research in Cases Involving Humans & Animals

A. Coordination Between VPs & Research Integrity Officers, Institutional Counsel, IRB, IACUC, Jennifer K. Yucel, PhD; Director, Office of Research Compliance, Research Integrity Officer; Emily Schriver, JD; Special Assistant to the Senior Vice President and General Counsel, The Ohio State University Topic to address: How do you remove siloes and bridge gaps between general counsel and

RIOs? Having a counsel completely dedicated to research arms is beneficial. Collaboration, coordination, and communication and maintaining confidentiality critically

important for addressing RM allegations. Scenario 1:

Receive anonymous complaint about faculty members doing unapproved procedures on animals – housing animals overnight, using rooms that aren’t approved.

Research study involves materials developed by company associated with university. Complainant having trouble reproducing the data. Faculty members uncooperative with complainant; shuts them down. How do you disentangle these areas, who works on these various issues, and how do you

maintain coordination? Includes dealing with OLAW regulations, ensuring safety of the animals, commercialization, conflict of interest issues, HR-related concerns. How do you tackle each piece of this mess?

How we work together to resolve issues: Jen receives complaint. Counsel then has ability to contact each area that helps investigate these processes. Can

be coordination point or move it to office of compliance. Having legal affairs involved provides a 360 degree view around the university.

Assess: Is there any immediate need for us to do something to save people/animals. Legally, what do we need to do next?

We started out with a conversation every week, every other week, now it’s much more constant. What am I hearing, what is she hearing? What are some outside pressures? Public interest; where is this going to land downtown in the government? Media?

The fact that we can coordinate this at the very beginning is very helpful. We have to make sure that we’re addressing all of these issues as an institution. By communicating and coordinating, we can peel off these pieces and give them to relevant groups to deal with, then put the pieces back together. It allows us to help coordinate. Regulations have different reporting requirements.

Scenario 2: IRB sees discrepancies between continuing review documents and originally approved

protocol. IRB conducts audit and finds that protocol is incorrect, consent forms appear to be falsified. Important to know that IRB knows that this is a research misconduct issue.

Counsel sits in on forum on IRB noncompliance to identify issues that should be considered and investigated as research misconduct. Legal counsel can work with RIO and IRB to ensure confidentiality throughout these discussions. A lot of times you see legal counsel saying, “That’s not particularly a legal question,” but it’s important to know that a lot of these touch on legal issues down the round. Assisting with the compliance function often helps with assisting with the legal function at a later date. If IRB is conducting routine audit then asks more specific questions of PI, by the time it gets to us, those records might have been compromised. If IRB has concerns, hold on to those records.

It’s rare that there’s not a negative interpersonal interaction between complainant and respondent. In cases where there’s a more neutral component, is the dynamic of the processing of the misconduct case different? Sometimes yes/sometimes no. Even when it seems like complainant should be disinterested, they may not be. IRB has made finding and made recommendation – when questions come up, maybe a student has done the work and is no longer here; faculty throws the student under the bus. PI wasn’t providing oversight.

B. Noncompliance in Clinical Research Settings & Protecting Participants Lisa Buchanan, MAOM; OHRP

Discussing practical implications of auditing and how you facilitate that process at your applications. Signatory officials know what is required of you in terms of assurance

Need to change cultures of institution at the top-down. Signatory official should be someone who has the authority to do that. OHRP expects that there’s

someone who can make changes at the institutional levels and penetrate the program. Two of the nine responsibilities on the assurance: 1) set the tone for the investigation and 2)

ensure that investigators fulfill their responsibilities Compliance risks: issues that OHRP sees when it goes out to institutions and talks to researchers,

IRB members. Inadequate IRB staff and resources. Inadequate IRB documentation. Failure to conduct auditing. When there’s crossover and overlap from audits. It is okay to let OHRP know that you’ve

learned of something, there’s an issue, you’re looking into it, and you’ll follow up. Another thing that’s important: If someone contacts us after institution has told us about

an issue (either complainant, media) we will look at the fact that the institution has already notified us because we know that it’s being addressed. It’s one of the ways that OHRP views institution is taking issues seriously.

Inadequate written procedures: policies in place for addressing noncompliance. Inappropriate IRB reporting lines: never want institutions with IRBs that make findings

that institutional officials didn’t know about. Reporting processes should be part of your procedures

Inadequate communication. Ensuring that you have not just written policy, but that you’ve published and circulated it.

Inadequate resources for researchers. Insufficient support from leadership – most leaders want to do the right thing, but they

have competing demands, and money is always limited. You have to develop ways to be more efficient with limited resources.

Inadequate education. Training is the thing most often used to remedy the situation (make bad actor serve on IRB).

Failure to use regulatory flexibility – appreciate who’s responsible for what. Understand why you’re doing what you’re doing and what’s driving it. Look at HHS policies, then institutional policies.

Reducing risk – motivates us to have auditing programs. Researchers need to understand that it’s a changing landscape; how things are done now is different from 20 years ago. Everyone in the human research protections program must know their role and responsibilities. Everyone in the HRPP at your institution has a role and needs to understand it.

Reporting requirements. Most adverse events are not unanticipated problems. There are a lot of requirements to navigate. All unanticipated problems that involve risk to harm to subjects must be reported. Some overlap with adverse events, which would need to be reported.

What’s an Unanticipated Problem (UP)? Incident, experience, or outcome that’s unexpected in nature, severity or frequency. Related or possibly related to research. Suggests risk of harm occurred.

Compliance procedures: receipt, inquiry, investigation. May lead to restriction or suspension of assurance – very rare. Pick up the phone.

If you report, it’s a positive thing for OHRP not to come out to your institution and pay you a visit. Having only three reports in five years could be a red flag. Reporting is a good sign that your institution has components in place to report what’s happened. 1-2 reports in 20 years end up in an investigation. You do not need resolution to all issues before reporting – follow-ups are fine.

Question to federal agencies: Is there any plan to have coordination among federal agencies, plan for common rule? Answer: Working collaboratively on interagency issues. We do try to harmonize and collaborate where we can.

Adverse events versus unanticipated problems: reporting requirements include suspensions, terminations. There are a lot more adverse events that you have to deal with at an institutional level, but there’s a small slice of those that are considered UPs that would be reported.

C. Protocol Violations & Protecting Animal Subjects, Patricia Brown, VMD, MS, DACLAM; OLAW

Institutional reporting assistance. Robust; dates back to 2005 when we issued specific guidance on what needs to be promptly reported to us. OLAW webpage devoted to noncompliance; includes sample report. Because of the general public’s interest in research, our office gets FOIA’d a lot. We don’t want you to give us too much information about things that may impact confidentiality.

If you’re not sure whether something’s reportable, call us. Division of Compliance is staffed by three experienced veterinarians. Call us and report when something happens so that we’re aware of it. If someone from the media or Congressional inquiry contacts us, we can defend you and say that you’re addressing it. We want to stand by you and be partners.

Opened 5,000 cases between 2010 and 2015. Most of them are institutional self reports. Receive allegations from employees, other oversight agencies. OLAW review of assurance and annual report. Receive allegations from funding components, other sources (like PETA or Physicians Committee for Responsible Medicine).

Sometimes allegations involve insider who was wearing a camera and videotaped. We then try to figure out if it’s real or manipulated, whether animals are in distress or receiving proper care.

We have slightly under 1,000 assured domestic institutions. Types of reportable issues: animal study protocol issues (highest number). Investigator research

team (5). Clinical issues (13). Animal protocol issues: Failure to follow the protocol. Unapproved significant change.

Inadequate oversight. Work before protocol approval. Working under expired protocol. Investigator research team issues: unauthorized animal housing. Unauthorized personnel. Food

and water restriction issues. Other: training failure, equipment failure, natural disasters, human error, neglect, abuse, and

crime (very limited). Failure to follow institutional policies. Clinical issues. Animal husbandry. Food and water issues.

Other husbandry (not cleaning cages). Escaped animals (repeated instances need to be reported). Physical plant – occasionally issues with heating or ventilation. Not many problems with

IACUCs. Sometimes we get a self-report with no violation found. Types of animals involved: rodents (77%), fish (3%), non-human primates (7%) Individual responsible for reportable issue: investigator and research team (73%) Animal care

staff (12%).

It is a self-reporting, self-correcting system. If it involves investigative staff, it involves retraining. Counseling, reprimand, or termination of employment. Institutions will modify their policies, repair or modify the facility, IACUCs conduct enhanced oversight of lab, put people on probation, or suspend animal study/protocol itself.

At end of case, you’ll get a letter that attests to what you told us happened and what your corrective actions were.

Implications of reportable issues: corrective actions and improved systems. Enhanced reporting requirements. Suspension or termination of award. Restriction/withdrawal of assurance. Criminal prosecution. Last three do not happen often – institutions have improved drastically over the years.

Other possible ramifications: Negative publicity for institution (newspaper calls us, then we come to institution). Damage to reputation of institution or individual. Media is adept at targeting

investigators. NIH works to stand behind scientists. If there’s evidence that research was conducted without IACUC approval, we request that

publication be retracted. Inability to attract new staff. Inquiries from Congress, other Federal regulators.

IACUC is well set up for working with animals in a biomedical context. Question: Are you getting many allegations regarding field use? Answer: We’ve had a few NSF

cases. NIH does cover field studies. Because NIH or PHS agencies fund teaching agencies, those teaching activities would fall under

PHS policies as a requirement. Institutions want to have a standard of oversight, so they will cover research performed under other federal agencies.

Question: Regarding types deviations from approved protocol that did not result in animal misconduct – are these reportable? Answer: It depends. Call us and we’ll tell you.

The most complicated complaints we’ve ever received in our animal program are ones concerning teaching.

In case of human subjects research, its important to distinguish between investigative purpose versus food consumption. We’re tied to biomedical research because of where our responsibilities are. If it’s not research-related or teaching-related, it’s not subject to oversight.

V. Responsible Conduct of Research Within and Beyond the Institution

A. Drivers of Misconduct & Biased Proceedings: The Power of Institutional Leaders to Ensure Integrity, Raju Tamot, PhD; ORISE Fellow, Office of Research Integrity

1980s: John Long – “The system has problems, but we cannot relate that to my misconduct.” As an anthropologist, we have to understand what’s happening behind the scenes.

The U.S. biomedical/life sciences ecosystem. How is pressure to attend higher education affecting our biomedical system? Doctorates awarded in scientific fields increased from 25K to 38K from 1994 to 2014.

Stay rate of temporary visa holders with definite commitments: 59% to 79%. Retraction notices increased from 467 to 684 from 2013 to 2015 Average age to obtain RO1, securing faculty position, 7.5 years Full-time tenured and tenure track facility – decreased from 45 to 24. We are seeing a lot of pressure on young investigators coming out of graduate school. Alberts: former EIC of science: “unsustainable hypercompetitive system” Postdoctoral fellows: well-educated, cheap, highly motivated (Gould, Bourne) Take advantage of indirect cost reimbursement on salaries paid by grants (Bourne, Alberts) Cash incentives for publishing in Asian countries. Bunton and Mallon: no financial guarantee associated with tenure.

Drivers of Research Misconduct: Analyzed narratives of RM respondents and analyzed 50 close-cased files containing admissions on RM. Made by respondents or mentors or colleagues at the lab. Analyzing narratives of these stakeholders involved in RM proceedings.

Closed about 800 cases at ORI. Inverse relationship between rank of respondent and the findings made against them. Senior people in much more powerful position to hide and destroy information and

intimidate junior people into submission. Causes of research misconduct: when we analyzed these narratives, five major themes

emerged: 1) individual circumstances; 2) poor mentorship; 3) competitive pressures; 4) inadequate training; and 5) individual psychology.

Competitive pressures: Junior scientists have to publish, produce data. Graduate student: needed to get paper and put pressure on himself. Another wanted to

appease reviewer. Did you get a chance to look at environment in which this occurred? Inquiry final report. “I still don’t know how to properly publish western blot data.” Mentors can sometimes be the real problems in RM cases. Graduate student: “My data

was never once reviewed” Offered up several times, and was declined. ORI: His mentor did not review his work on a regular basis = oversight failure. Mentor drew diagram of what data should look like. Scared at what PI would do; scream and yell.

Biased proceedings. There were 16 cases in which mentors were the problem, nine of the committees refracted past events.

Power struggle between respondent and committee. Kaiser Permanente: what does institution do to prevent harassment in the workplace?

NSF starting to rise in harassment allegations accompanying RM investigations. Students don’t know where to go. Something is falling down in the process where they don’t feel like they have someone to talk to. Folks in vulnerable positions either don’t know where to go or are afraid to go somewhere.

Deciding official: review research misconduct proceedings: ensure that committees have conducted fair, thorough, competent, and objective review.

Develop internal mechanisms to promote research integrity. Review data, supervise junior scientists effectively, help prepare young scientists to survive in a competitive environment. Make efforts to increase the number of tenure and tenure-track faculty. Discourage self-protective behaviors. Make efforts to reform the tournament style job market, reform disturbing trends in the training of new scientists

B. Establishing an Office of Research Compliance: Challenges & Opportunities, Gary K. Ostrander, PhD; Vice President for Research and President, FSU Research Foundation, Florida State University

Most faculty find research compliance to be unnecessary, intrusive, waste of money, bureaucratic, not relevant. Until you actually have to work with them, then they understand why we need these policies and procedures. Selectively pulling out data, blotting out images was the culture of RM in the 1980s. It was the beginning of people feeling “publish or perish” pressure. When something happened at public university, we didn’t talk about it.

What changed? Funding became more competitive. Technological revolution allows you can go back and see what really happened. More federal oversight sparked by key cases. Easier to catch offenders.

Establishing your research compliance office: 1 – 1.5 FTEs, 100-125K per year; .5 FTE performs scheduling, transcriptions, making PDFs, travel (2-3) meetings per year Secure office for handling sensitive materials; out of main thoroughfare on campus.

Computer, cell phone, etc. – must be able to take a call from me at 2 or 3 in morning (work involves life or death issues).

Training, website, reserve/back-stop. Educate whoever you’re making this presentation to that sometimes a case will cost a lot

of money. A lot of misconduct issues are bumping against a technological issue. Sell the idea to the faculty. Most to lose and the least to gain in their minds. You will

curtail their current behavior. Make them a partner/advocate. Articulate their role and how the research compliance office will function within your university. Regular access to legal counsel (this is protecting the university). Most important reason – protecting faculty, staff, and students.

Recruit your first director – experienced compliance officer from another university OR train an internal staff member. They need to really understand research compliance. Send these people to workshops. If you’re hiring someone internally, they must be respected by faculty/staff.

Whether you have wired internal candidate or not, launch search committee. Staff member, someone from IT, someone from faculty, someone from IRB, animal care and use, tech transfer, security, and finance and admin. There will be times when you have to confiscate material from lab.

Secret to making whole office work: Hire a good cop. Someone who is advocate for every faculty, staff member, and student on campus. They must respect everybody that they interact with regardless of what behavior they’re accused of. You don’t want someone who only sees “black and white” because these RM issues are not black and white. If you’re accused of RM, whether you knowingly did anything bad or not, it’s a trainwreck. Your livelihood may be gone; there may be criminal prosecution. Even if innocent, you’re looking at a long process. This is the one moment where you need a friend, advocate.

Why do you need a good cop? It’s the right thing to do. Everything you do is going to be observed and noted by faculty, staff and students, who will be looking at how you treated their colleague. It’s about how you treat faculty members throughout process. If you do it right, you will develop allies and get support. When you hire your first director, make sure they meet key people in your key departments. Not likely they will have a backlog of work. Begin with having them update your policies. Have them go out and talk to stakeholders in advance. Circulate drafts. Follow your procedures.

ROI: educated faculty (creating culture of integrity), staff and students, preventing misconduct, promoting integrity.

Question: At what point did you decide you needed to start an office? Answer: I looked at it and realized that it didn’t exist, or was disjointed and we needed to pull it together.

C. Building Networks: Responsible Research in Diverse Community Settings, Keith C. Norris, MD, PhD; Clinical Professor, Department of Medicine, University of California, Los Angeles

Being proactive in addressing RM issues in collaborations across institutions, communities, countries.

Nuances of research networks involving diverse communities. Who are those stakeholders, and what are their roles? Some of these ethical considerations can be interesting when they arise.

These perspectives arise from two networks: Translational Research Network and the NIH Biomedical Workforce Diversity Consortium. Teaching-intensive institutions looking at students and institutions and different interventions that will help them succeed in biomedical science. National research mentoring network has a lot of programs around mentor training; they have outreach that includes entire nation. Assisting each and every institution with strategies around mentoring. MOUs for data sharing.

Considering data collection strategies; being cognizant of language used in different materials. Diverse communities: 15 or more concepts of diversity. Low income or educationally

disadvantaged backgrounds only apply to HS or undergraduates. Why these issues have become more important: more engagement in the community, more

institutions working together; trying to address issues to ensure that all U.S. citizenships are receiving benefits of science.

Why partner? New approaches to achieve new results; new insights to be gained from broader pool of institutions.

Important to have broad perspective when we think about ethical issues. Investigator has a perspective, community has a perspective, institution has a perspective. Need to consider these perspectives in network-based research.

What are barriers to clinical research? For many of our communities, it’s about understanding what clinical research is. We don’t have communities that understand what clinical research is. Many members of communities don’t understand that they are the benefactors.

Communication is extremely important. Convey information; confirm understanding. Need to make sure that we have extra time to fully develop and convey message.

Do the members of the network believe that institutions are really concerned about ethical issues? For local communities: Has our institutional research offices and leadership appreciated our community or the voice of our community and partners? Are we sensitive to issues that they have? Are we acting in a way where we’re being patronizing when we think we’re being protective?

We need to make sure that we’re not being a barrier to the types of engagements that we should be facilitating with other research partners, whether they are community partners, organizations, or institutions.

Memorandums of understanding (MOUs): Help to establish, early on, what shared vision is and what guiding principles are. It’s a

memorandum, not official legal document. Because it has everyone at the table to agree, it promotes mutual respect and builds trust. Process helps to reinforce actual role of MOU. Data sharing agreements can be more challenging. Having MOU in place for any relationship helps. As you go forward, people go in and

out, you can refer to this document for common values and shared goals for this project. Many times advisory boards are used, but you can’t bypass having other documents in place.

Other aspect: conflict resolution. There should be a formal plan put in place early on. Don’t enter into these partnerships with assumptions. It’s powerful to try to work from

perspective that each partner is bringing something unique with intense value, which helps to frame relationship in a positive way. Value each partner for their strengths and unique contributions.

Establish advisory boards, but recognize while they are important, they support, but do not replace the need for full partnership.

Don’t put investigator in position where they feel compelled to cut corners. Changes in common rule: use of single IRB for multi-site studies in the U.S.

Public trust/utility important in network/partner research. What our communities are looking for is confidence in our institution that research is being conducted in responsible way, and that participants in research being treated with highest level of respect and authority.

Notion of single IRB – a local IRB would know local issues, and similarly, issue of trust in community, they may be happier with local IRB than one halfway across company. If local IRB has ceded to prime IRB, particularly when you’re looking at multi-site research study, it’s almost impossible to develop language in consent form that will resonate with a specific community.

Carl Henry: In UK, we have National Health Service (NHS). Despite size of UK, there are still disparities in healthcare provision. Our National Institute for Health Research finances researchers on the ability to recruit patients into their trials. The only way to make progress is to start to look at bigger data sets and bring communities together. Greater participation from institutions. Need to educate communities and give back so that they have a sense of trust; understand what it means to participate. How we do put “research” in terms that everyone understands to communicate how essential it is to everyday life? Those with specific disease of condition much more likely to participate if there’s a drug to provide them with opportunity for treatment that current drug cannot provide.

Most common responses: “Aren’t you already doing this?” There’s a disconnect between what research does to make scientific advances. Only bad research

makes news.

D. Perspectives from the National Institutes of Health, Patricia Valdez, PhD; NIH Extramural Research Integrity Officer, National Institutes of Health

Will cover how NIH handles allegations of RM and the NIH initiative to enhance reproducibility from rigor.

NIH research falls into two categories: intramural and extramural. Intramural is about 10%. We have 26 extramural IC RIOS.

Our office receives 100 allegations per year. When allegation comes in, we conduct initial inquiry then send to ORI for investigation. If it

involves human subjects, we send to OHRP. We also receive allegations that involve breaches of confidentiality during peer review – seeing more and more of those.

When there’s been a resolution, they contact us, and we report that to IC RIOs. “Enhancing Reproducibility through Rigor and Transparency” initiative just went into effect on

1/25/16. Why do we care? Over the past couple years, there’s been a reproducibility challenge across all

research areas, particularly pre-clinical research (Prinz and Begley). The efforts of NIH alone will not be enough to make enough change. As funders, we can only

control so much. There are a lot of factors. Science often viewed as self-correcting; immune from reproducibility problems. Reproducibility

problem leads to loss of time, money, careers, and public confidence. Factors that erode reproducibility:

Hypercompetitive environment sparked by historically low funding rates. Grant review and promotion decisions depend too much on high-profile publications.

Another factor that short circuits self-correction: publication practices. Poor training. Inadequate experimental design, p-hacking, incomplete reporting of

resources used and/or unexpected variability in resources. RM’s contribution to irreproducible research is relatively small. Not a focus on reproducibility

policy. Over past two years, we have developed policy with following guidelines: clarify NIH’s long-

standing expectations on rigor and transparency in applications. Raise awareness and begin culture shifts in the scientific community. Improve the way applicants describe their work. Provide sufficient information for reviewers. Demonstrate to our public stakeholders that NIH is considering their concerns.

Four factors of rigor that have been added to grant applications: 1) Scientific premise: Applicant needs to describe previous research/preliminary data

supporting application and needs to discuss strengths/weaknesses in how rigorous that research was.

2) Scientific rigor: Asking for more information – What are the analysis/statistical plans? What kinds of animals are they using?

3) Consideration of relevant biological variables, such as sex. A lot of research is done in female mice; want to make sure that there are sex differences in outcomes. All we’re asking is that this is considered in human subject and vertebrae animals. Need to propose to use both sexes.

4) Authentication of key biological and/or chemical resources (driven by problem with misidentified cell lines). Applicant must provide attachment that includes plans for authenticating key resources. First three areas scored, last is not. If application is deemed meritorious, program officer works with applicant to develop and implement actual authentication plan.

Training: Need for more training that focuses on rigor and transparency; would be separate from RCR. We envision that it will happen on a day-to-day basis – how to report data transparently; how to conduct power analysis. Additional guidance on training requirements will be forthcoming for FY17. This will be another attachment in grant application. Required for institutional training grants, institutional career development grants, and individual fellowships.

National Institute of General Medical Sciences (NIGMS) developed four rigor and transparency training modules. NIGMS funded 25 applications to develop new modules. NIGMS also developed supplements for T-32 grants.

Rigor and reproducibility website: grants.nih.gov/reproducibility. Comment: What about expenses to conduct research? There aren’t incentives to reproduce in

research.

Day 2: Friday

VI. Tips on Handling Research Misconduct Allegations & Promoting Integrity

A. Office of Research Integrity, Alex Runko, PhD; Scientist Investigator, Division of Investigative Oversight Our goal is to protect health of the public; promote the integrity of PHS-supported research;

remain steward of research. Complainants: researchers, co-authors, co-PIs. Complaints come in through blogs, websites, peer

reviews, publications. Challenges with receiving allegations: maintaining confidentiality (complainants might tell their

colleagues, PI, other people on campus). Quality of information provided. Need institutional policy on communication, email, tweets. Institutional policy on evidentiary standards, training of RIO.

Assessment need policy to determine when inquiry/investigation is warranted. Consultation: guidance from institutional officials re: training of RIO, compliance director,

IACUC, IRB, IBC, other staff. Protection/retaliation policies to protect whistleblowers. Institutional policies on good/bad faith

allegations. Be prepared to ACT: Be Aware (confidentiality; HR policies in place), be Comprehensive,

assess in Timely manner. Sequestration:

Happens early on; early as possible after assessment. Team: RIO, IT staff, forensic experts, legal counsel, security.

Documentation: chain of custody forms. Itemized/detailed. Photographs and documentation. Detailed descriptions of locations of documents. Lab door locked? Who had computer access? Was office locked? Who had access to the key? Also look at receipts signed by respondent/lab manager.

Evidence: lab notebooks, computer evidence, hard drives, email accounts/servers. Once you’ve collected evidence, make sure it’s secure.

Anticipate additional research misconduct. Upon closer look, might reveal further suspicious data in other publications. Data that’s falsified in publication may be present in application as well.

Anticipate additional respondents involved; additional sequestration from these people. Challenges: incomplete sequestration. Additional records recognized later. Communicate

to core facilities manager that you may need access to these records. RIO’s coordination with sequestration team and other staff (IACUC). Poor sequestration methods: non-forensic electronic copies, loss of metadata, corrupted

files, poor training of sequestration team. Chain of custody deviations – unsecured evidence; evidence intentionally or negligently

destroyed. Sequestration (staff training, documentation): ACT Be Aware(additional misconduct), be

Comprehensive, be Timely. Integrity of scientific record: NIH policies require researchers to maintain their data on file for

three years. PHS policy says that research data is owned by the grantee institution, not the researcher. Challenges associated with integrity of scientific record: researchers unaware of data retention

policies; intentional or negligent destruction of records. Remedies: mandatory training of PIs or staff; be aware of grants management guidelines.

Institutional data retention requirements—professional and personal computers, emails, cloud storage.

ACT: Aware (management); Comprehensive (sufficient training); Timely (monitor). Conflicts of interest:

Free from bias and personal, professional, or financial, or other ties to respondent and complainant. Management: hire outside experts if need be; use consortium. Make sure they’re aware of confidentiality. Give proper notice to respondent and opportunity to object.

Challenges: risk of poor judgement, case integrity compromised; violation of terms of awarded grants/contracts.

International respondents: language/cultural barriers. Mentor/mentee relationships are special, unique; they don’t want to go against their PI. Visa issues put them in vulnerable positions.

Remedies: communication: establish and maintain dialogue; institutional policy on interpreters/translators.

Collaboration & network: liaison offices/agents, such as APRI, World Conference on Research Integrity, ENRIO, ARIO.

Be aware of barriers, be comprehensive in communication, act in timely manner and use your network.

Be proactive; be aware; need to update institutional policies? Are they comprehensive; is there enough training in place?

Question: How far does ORI take the COI web out from the actual allegation to individuals who are exterior to it? Answer: Every case is different. Try to have unbiased review of COI. If respondent doesn’t object, you proceed. Also have opportunity to hire outside, objective help.

Question: How much variability in policies are there in getting access to respondents’ private info? Answer: It varies. You have to go though counsel and figure out what you have access to. You have to be aware of personal devices being used. State law has implications on policy, personal computers, emails addresses. Institution needs policies in place to deal with it – check with counsel to work through issues. A lot of institutions will have faculty sign agreements that include accessibility to data upon employment.

Susan: When you’re staffing an office, what you can think about is whether paralegal will be useful to have. When cases move through investigation and findings are made, RIOs and VPs get sued. A paralegal could be very useful. The regulations state “on or before notifying respondent.” Sequestration data means taking it, not examining it. Look through email for information regarding allegations.

Question: What about RM complainants that extend beyond three year NIH restrictions? Answer: Every single piece of data may not have to be stored in three years, but published data is different—narrow down set of data that you want to store.

UK has 25-year policy for data storage.

B. National Science Foundation, Scott Moore, Office of Inspector General

Our process for handling RM investigation looks like scientific method. When looking at RM process, it looks more like legal process where we have process for inquiry/investigation, have adjudication, final decision. For federal/state entities, we have additional rules: fourth and fifth amendment. If you’re a state institution, be really clear what is yours to access/sequester, and what is not. Every federal circuit has its own tweak. Ninth circuit has different rules that tend to limit access, specifically emails. Fourth amendment applies to search and seizure. For us to seize something, we need search warrant – there are ways to work around that with internal policies, which help. If you’re starting to think about sequestering, give us a call. If it’s not done properly initially, it can cause problems later on.

Private institutions: Constitutional amendments may not apply directly to you, but you have other things to worry about: trespass, invasion of privacy. Torts: someone can say that you went too far. Consult early to make sure that we’re getting good evidence to move forward. Any federal actor reaching out and taking something that belongs to someone else – make sure you know what’s yours and what’s not.

Take home message on handling allegations: be prepared and have comprehensive, up-to-date policy.

Every case is unique – a good policy gets you through twists and turns. During process, document as much as possible. You’re securing evidence, but also documenting your process – this will save you if there’s a challenge. Err on the side of precaution; prepare for the worst.

Go where the evidence takes you. Nothing in referral restricts you from adding subjects, allegations.

Make a final balanced decision: focused, clear, and consistent. Don’t treat faculty different from students.

Encouraging institutions to have research investigation management plan in place before starting, making sure community knows, who it applies to, and how it integrates with other policies (IRB, IACUC, data management).

Make sure general counsel knows you before you do anything. That line of communication helps you though the process so that you’re not scrambling at the last minute.

Some institutions do not accept anonymous allegations. Some institutions require interview of complainant. NSF believes complainant is just a witness without a vested interest in outcome.

Do your folks know who receives allegation? Is there any immediate harm or immediate need for action? A plan in place beforehand makes

this easier. Issue with committees is that they try to solve case during inquiry phase. Inquiry committees

may drag it out. NSF simply looks: is there enough there to warrant further investigation? More than allegation assessment, but much less than formal investigation. Need to be thorough in determining that there is an issue – is there a material fact in dispute? Are there facts not lining up that need to be resolved? This warrants investigation.

What to do with admissions of guilt? We have seen cases where RIO drafts letter to foreign student subject: here’s the allegation, but if you sign this confession, we’ll drop the matter and you leave university. That borders on coerced admission. There needs to be a trust, but verify. We may ask you to go back and do more after.

Committees, usually ad hoc committees, composed of faculty without RM training. They are out of their comfort zone, worried about getting sued, will bury themselves in science. Your committees are like juries, but they can ask the questions. A good RIO and legal counsel guides committee back to investigation zone, providing context for consistency, not influencing their assessment. We offer OIG committee briefing to let them talk to us and see that we’re not there to intimidate; simply there to investigate. We can walk them through thornier issues, working in partnership with RIO and general counsel. Our website: semiannual reports and case closeout memoranda are posted online – allows you to see our logic.

Investigation: We see committee analyzing the science – did data support what was said in research record? What can you secure, and how can you secure it? Digital forensics field is changing rapidly – there are requirements we have to meet in order for evidence to be used in criminal and administrative proceedings (currently being assessed by DOJ).

Because we have subpoena authority, think about where other evidence may be. If it’s not under your control, like in the cloud, can be tough to get data out of cloud. If you need to recover data from someone else, talk to us because we have subpoena authority.

Report and decision process: know who deciding official is, and it’s important that they actually explicitly state, “I find this person guilty of research misconduct. We made a finding.”

After investigation: know how long consequences/corrective actions last, and who will monitor them – there need to be monitoring process in place. Always important to review policies after going through process to make tweaks.

Question: What if subject has left institution? Answer: If award goes to institution and RM occurred there, NSF presumes that you have the data. NSF has same three-year rule as NIH; comes from OMB circulars. If subject left, do your best. Ask them to be interviewed; participate. If they don’t, they don’t. The fact that they’re refusing to participate in administrative process can be used to draw inferences. Work with evidence you have and make more reasonable decision you can. This is another reason to engage us early.

Question: Why do initial inquiry committees take so long? Answer: Inquiry committees can be hesitant to move to investigation because stake are high. We try to get committees over angst about high stakes – tell them that this could also save your colleagues’ career. Putting them in ‘good cop’ mode.

Question: How do you select committee members in terms of experience, internal/external, size of committee? Answer: Size is up to you. Typical size for inquiry is three. For investigation, three to five. May not be original three from inquiry, depending on policy.

In initial phase, we ask for all committee CVs. Check for conflicts. Required to manage conflict, document how you’re managing conflict. If you can’t manage it, going to outside world may be necessary to pull in expert. Have to be careful about specific words that may have legal implications. Variety in committees is nice, helps to get other perspective – needs to consist of relevant scientific committee, but the definition of that is flexible – what are the customs and norms that you expect in your campus, your department?

Question: We’re seeing more work in multidisciplinary areas: NSF, NIH, DOE money. There are issues of harmonization and cooperation – talk about harmonization within federal government? Answer: Challenging for us to have consistency. At ORI, the voluntary settlements and ALJ process are very different from NSF process. NSF more streamlined. When you bring DOD, DOE in, they have their own variation as which reflect different programmatic natures of those agencies. The earlier you can identify agencies involved the better chance you have for coordination and figuring out which agency is leading. Once case comes in and there’s overlap,

we figure out who the case is most important to, who has the resources to do it – one of us will take the lead. We try to make a SPOC to simplify things. When you send us a copy, CC someone else from other agency. If you feel like you have too many people you’re reporting to – go to one person who’s been designated lead, and make it our problem.

Susan: There are cases where there’s equal jurisdiction, and ORI and NSF both have equal interests in case. There, we have joint jurisdiction and make our own separate findings – has to do with resolution of cases for the government. It’s been difficult to find people at other agencies who are responsible. We have inter-agency working group to figure out who to contact.

C. Institutional Approaches, David Hudson, PhD; Associate Vice President for Research, Research Integrity Officer, University of Virginia

Handled handful of big cases and many little cases. Attended San Diego APRI meeting. Have had a subtle change of appreciation of what role of RIO is.

Role of institution is to ensure that each allegation is handled in thorough, competent, objective, and fair manner. I do not represent institution; I represent process, and it’s my job to handle it in this manner. To protect institution, faculty, students, staff and integrity of research process, process has to be carried out in this manner.

Be prepared: training for everyone involved, particularly for RIO. Powerful process. Association of RIO (ARIO): meeting 9/26 in NYC. Also ARIO regional meetings and

conference calls. If you don’t have a good RIO, hire one. Role of RIO: Great attention to detail. Learned importance of mastering policies and key phrases.

Nobody has job like RIO. Event-driven. Being RIO in your organization has to be thought of like you’re the earthquake preparedness and operations manager. Operationally, this comes very close to that. You don’t know when a case is going to show up, how big it will be, what the damage will be, how long clean-up will take. You can prepare all you want, but you have no idea what you’re preparing for.

Must demand a good budget – how can you propose a budget without knowing what’s going to happen?

Career damages from RM cases could be substantial; may have to make arrangements for graduate communities.

The cost of grievance side alone could be substantial, let alone cost of investing in RM case. Always possible that there will be a big case.

If you’re the VPR, you need a good RIO. There aren’t a lot of excess RIOs looking for jobs. Great way to proceed might be to hire a #2 or train your own.

Characteristics of a good RIO: Knows how to manage effectively. Handle stressful situations while remaining calm dealing with difficult people. Must know how to rigorously follow procedures and will see that others do so as well. Figuring out when you follow the first of those two and when to follow the second. Don’t be easily intimidated. Sequestering data from world-renowned scientists is not

easy, but this is not confrontational, you’re representing a process. Must approach Nobel prize winner and discuss what problems you’re trying to solve. It doesn’t look good to make it up later – figure it out now, pull out procedure and use it.

Question: What do you look for in a RIO – scientist, JD, research compliance background? Answer: Any could be competent.

Question: How do you support RIO? Directly – you have to provide RIO with budget and resources needed. As VP for Research, you have to provide range for that budget. Protect and defend them. Must not appear on 6 o’clock news.

Establish a culture of compliance; climate of integrity. If we’re doing our job correctly, we can decrease the number of allegations, number of cases. Need effective training program for faculty and students. Has to be continuous and ongoing. Research integrity should be the first thing mentioned in every lab meeting you have with student, part of everything we do, everything we think about.

Question: What if I’m an aspiring RIO? What steps can I take to prepare for this job? Answer: Talk to ORI about attending boot camps. Find out who your local RIO is and ask if you can intern them; hang out with them.

Question: Are there ever cases where institute might outsource or hire RIO? Answer: Don’t know if that can work because it takes very intense, hands-on collaboration. Maybe if they were in town or across town. Might be able to make it work.

Question: How do you assemble subcommittee when an accusation comes in and who’s a deciding official? I have standing committee at school of medicine that’s experienced. Try to find good SMEs who are fair and impartial. We carefully consider whether to have someone from outside. Always ask whether we need someone from the community to represent community viewpoint. Standing committee, but may add additional SMEs or members as necessary.

VII. The Role of Senior Institutional Leaders in Handling Research Misconduct Allegations

A. Ideal & Problematic Scenarios, Cheryl Cameron, PhD, JD; Professor and Vice Provost for Academic Personnel, Research Integrity Officer and Anne Ackenhusen, JD; Director of Office for Research Misconduct Proceedings University of Washington

Cheryl Cameron: We’ve separated accusations of RM from Office of Research and put them in office of academic personnel. Became in RIO in 2002. Long history of misconduct proceedings.

Still holds RIO title, but held in administrative capacity, because administration believes faculty members need to hold that position.

Anne Ackenhusen: Came on board in 2006. Litigator with a background in science. RIO has to develop understanding of science, but you don’t need to be the expert because you have SME access. We had outside attorney who conducts investigations who shadowed me – if there’s ever a situation where we need to bring someone else in, we can.

Challenges inherent in RM process: unforeseen and urgent. Volcanic eruptions. Need to have process in place where you can immediately respond. Senior institutional official needs to understand how difficult this work is. Need a RIO who can stay calm, who’s commanding, collaborative, coordinative, and compliant.

It’s a peer review process that we’re engaging in. Try to make it as collegiate as possible. Finger-pointing: complainant pointing at respondent, respondent point back. It’s critical to avoid

RM team including the RIO in engaging finger-pointing; need to maintain order of the process. High stakes process with reputations at stake. Can lead to threats of legal action very early on.

Stay focused, follow your process, and discover the facts. Each case is different and fact dependent. Each case takes on life of its own. There are always surprises.

Never know where they’re going to come from. Will always be surprises. Full range of respondents. Local administrators who you think are on your team may have

difficulty with these processes and facing pressure that they face with respondents or collaborators.

Other regulatory processes may be triggered, which may have different requirements with different timelines (IRB, IACUC, FDA). Need to be sure that we’re prepared to respond immediately, sequester immediately, and maintain confidentiality throughout entire process.

Even the best efforts at organization and structure do not ensure that chaos will not ensue.

How a senior IO can support and properly handle RM allegations: be sure that we’re promoting culture of compliance where expectation is that everyone is conducting ethical, high-quality research. Need to ensure that RM is well-defined at the institutional level. (At a minimum, should include FFP).

Definition is extremely important. UWashington just clarified our policy, where we tried to focus on maintaining flexibility. Can never anticipate every possible circumstance coming up. Encourage continuous review and improvement process and lessons learned meetings to see where we can improve for future cases.

Ensure institution has method to funnel RM allegations to proper venue and direct non-RM cases to other appropriate offices. We look to manage cases at least intrusive, least complex process as possible. Complainants want respondents to go through most rigorous process possible.

Process needs to be clearly defined for all individuals. There are complainants who bring up something that’s not FFP seeking it to go through RM process. Having clarity in our definition is absolutely essential.

Ensure that RM cases are handled uniformly throughout institution. Ensure institution has clear and transparent procedures for handling misconduct cases. Insist on realistic evaluation of RM allegations rather than ‘head-in-the-sand’ approach.

When in doubt about process to follow, don’t go off the path you have defined your RM process. Ensure sufficient resources available to examine and process RM allegations (requires support at

local level). Ensure you have access to subject matter experts, access to technical support (we engage external computer forensics experts). Ensure that members of RM are given sufficient time and support to perform committee work.

Short-term savings can have truly long-term consequences. Figuring out a finance model that allows for urgent response to these matters is essential.

Ensure persons impacted by RM process are protected and safety maintained. Ensure confidentiality of RM allegation and proceedings. People looking at imposing sanctions want to know where complaint came from.

Stay informed and connected regarding status of RM cases once it’s reached inquiry and investigation stage.

Have confidence in your RIO. Support your RIO. Your support for RM process at highest institution level is essential.

Anne – walk through various stages of RM considerations. Step 1: Assessment:

Do you have a RM allegation? Time-consuming, money-consuming, emotion-laden process that you don’t want to use unless you need to.

Is there enough credible evidence to warrant an inquiry? We’ve had situations where we’ve had pressure to trigger the process quickly. But you need to step back and really assess whether it warrants RM allegation. You don’t want to put something in the process that doesn’t belong, but when it needs to be triggered, you need to be aware.

If you want to talk to respondent, you need to be prepared to sequester immediately. If it warrants a RM allegation, you move on to inquiry. I conduct inquiry with access to SMEs. We start out with RM office which starts with assessment and inquiry. Then it’s taken over by school or college which takes over investigation committee. I then help walk them through legal analysis. If you don’t get the right evidence at the beginning, you may not be able to follow through. We notify and sequester at the same time. At notification meeting, I’m presenting allegation. Rep. from deans office and department chair (as faculty representative that talks to respondent and can be there for them) are present. Notify, sequester, and give respondents time to respond in writing to written allegation.

Step 2: Inquiry:

Preliminary fact-finding to determine whether there’s substance to the allegation. Gray areas: There will not be clear answers all the time, but stick to your process no matter what.

Relocating people to the other labs, keep everything confidential. Hard to hide people going in to sequester. Sometimes respondent shares with their friends.

Witnesses: all in same lab, how do you protect them? Keep your door open and let them talk to you.

Committee members—they can’t put on their CV that they’re a member of a RM committee. No one knows what they’re doing. Many institutions have standing committee. Some have ad-hoc. Benefit to standing committee is that they can develop expertise.

Step 3: Investigation: Who conducts investigation: we go to school or college, and dean selects committee.

Some are independent; others need more handholding. Other institutions: inquiry committee becomes their investigation committee. Standard for RM finding: fabrication, falsification, and plagiarism. Also must be

significant departure from accepted practices of relevant research community. Must be done intentionally, knowingly, or recklessly. Must be proven by preponderance of evidence.

Do you look beyond the complaint, and if so, how far? It’s a rule of reason – you will probably look at other western blots. What if you find something else? We step back to inquiry stage, notify respondent and give them chance to respond in writing.

Dean of school or college appoints committee and has confidence in committee. Dean makes ultimate decision, meets with committee, and asks questions. VP of research are often the deciding officials. Institutional federal decisions are separate.

Potential sanctions at the end of process: At University of Washington, sanctions are not within ORI, they’re within school or

college. Correcting research record: We encourage investigation committee to say what needs to

be corrected. Respondent may refuse. Training requirements. Supervision requirements: someone making sure that research is clean in the future. Limits on duties/responsibilities: If there’s been a failure to supervise, don’t supervise for

a while. Institutional consequences: When we’re in the middle of a proceeding, there are decisions made

re: faculty’s relationship with institution made independently of investigation. Consider factors such as an assistant professor who’s up for mandatory promotion in middle of RM proceeding. Do you have mechanism to make this decision “pending upon results of investigation?” Same with awarding salary adjustments based on performance and merit – need to maintain confidentiality, but also want your performance evaluation to be fully informed.

Our office is located within employment considerations office – important to coordinate with offices that are in charge of promotions, tenure, raises.

At University of Washington, we tend to choose not to publicize. Respondent may initiative publicity. Collaborators or supporters may publicize – always be ready with response.

Comment from journal editor: The “can neither confirm nor deny” statement puts journals in awkward situation. If there really is allegation of misconduct, then journal has obligation to retract paper or request that institution retract paper. If we get something that’s plagiarized, it’s not even reviewed because we catch it with Authenticate, and we send detailed letter to authors with Authenticate report. Would also send out letters to deans of medical schools; never get response back. Can’t be policeman. Suggestion for editors: Complainants get notified when case

closes, so we encourage journal editors to look at policies; if you see acknowledgement, notify deans, notify us. In a case that’s pending, when you’re trying to maintain confidentiality, it’s very difficult to do anything but say there will be follow up. A respondent may early on acknowledge that there were problems, “errors” – when you get that, have a conversation and ask them if they’re prepared to alert journal. You can potentially encourage them to at least begin to correct. If there’s an error in publication, you need to contact journal, don’t have to tell them who did it.

Another challenge is a long case—if it goes to ORI, could take a while for them to get through us. We hold off until there’s a decision by ORI.

Regardless of determination of finding that could be years to come, at an early stage, we often know the data are not true. As institutions, we feel tied to “we can’t take action until we reach finding.” How can we establish best practice when we know the data is false – that often comes so early, we can’t allow something to sit there wrong for years. The best way to approach this is to look at issue: There’s a finding of falsification, fabrication, or plagiarism and a finding of RM. RM means that you can attribute the action to somebody. Figuring out that something is false is easy. The hard part is figuring out who did it and whether it was intentional, knowing, or reckless The correction of the literature can occur once you establish that there’s false data out in the community. Take action if your policies allow.

You have to know your processes at the institution. Confidentiality is there to maintain respondent’s due process rights and hinder complainant retaliation. It’s not there to protect the entire process, it’s there to inhibit people from knowing the players in the process. We may go overboard, but that’s what we do to protect everyone in the process. What facts do you have that some action should be taken?

Question: What percentage of faculty/PIs get their own legal counsel, and how active are they? Answer: About half have legal counsel. During interview process, they’re allowed to be present, but they don’t participate in the process. No active participating in legal counsel. Stress that it’s an administrative/academic process.

These records live forever; there’s a long retention schedule. These cases live on and on because of the number of public requests we get long after they’re closed. There are respondents themselves looking to gather information. Complainants who don’t believe that their concerns were adequately addressed. You absolutely have to keep files as clean as possible at the end.

Research is funded not only by federal agencies, but also philanthropic groups. When we have contract that’s not federal, we also may have obligations. Look at multiple funding sources.

Question Regarding counsel being present, do you allow them to coach and whisper to the respondent during the interview, or force them to leave the room? Answer: Allow it to an extent, then say “let the record reflect that prior to every question...” May terminate interview and figure out rules of engagement for the next round.

Thoughts on pros and cons of keeping full transcriptions versus summary? I like full transcription because they can reference exactly what witness said. A summary is not admissible, need transcript.

Counsel at CALTECH: We don’t allow witnesses to bring their counsel. We used to take summary approach but switched to court reporters.

In addition to falsification, fabrication, and plagiarism, there are a lot of improper acts, questionable research practices. Use lessons learned approach at end of the case: identifying RM issues and wherever else there were problems. Add that to training component.

Question: What about instances where character has severe psychiatric problem? Answer: Address the problem and behavior at hand. Attorney is there to represent client. Our job is to focus on problem at hand. We personally can’t make that kind of evaluation.

Question: How early in the process should we be thinking about legal consultation and senior leadership engagement? Answer: We start very early – we alert them when we receive

allegations. Depends on your organizational structure. Important that you get sense from leadership on how people work. I know what provost/president will need/want. Important to understand what particular needs of president/provost/board are; how early they want to know something, and from whom.

VP may have heard something elsewhere from someone in the university. It’s helpful for them to know that you’ve received a RM allegation to put that in the context of other RM happenings.

Question: What do you do when there’s a committee that’s gone off the rails and coming to wrong conclusion? What discussions do you have with deciding officials? Answer: I’m present when they interview witnesses and deliberations. I ask them for evidence. We have had divided committee because of different research community standards. I’ll ask the legal counsel to look at it and we’ll start asking questions. We’ve never reached conclusions that are unsupported. Get them to come to closure, impose deadlines on them, and make sure that these directives are coming from highest level. Bring in provost if you need.

Lauren Qualkenbush: Most problematic committees have been at inquiry level. There are times where Jay has gone back to ask committees to further review, provide guidance or additional clarification. It’s a balancing act as RIO to not influence, but get more info if needed.

 VIII.Small Group Discussion: Topics of Interest for Senior Institutional Leaders

A. Group 1: Handling Allegations of Research Misconduct

Possible incentives organizations and institutional leaders could bring to table. Is it possible to provide salary offset, professional development credit? Putting some money in discretionary fund for travel, computer equipment. Swapping time commitments. Read your culture and evaluate what faculty member wants to participate in this project. Complainant typically looking for indemnification. Sometimes protections needs to continue even after they’ve left.

Whistleblower protections. What other services do institutions have available through employee assistance programs? Matching up a mentor to advise a respondent in the process – someone who’s experienced in RM; been through the process and can explain. Incentives for the process itself – creating culture of protecting the scientific record. Meet with panel members face-to-face and stress importance.

General counsel. Some are on call for interviews. Some are there to answer ad hoc questions. Some there for document review. Some there for all meetings; some only there for charge meetings. Depends on where you have general counsel in the process. External counsel – how invasive do you want them to be in your process? Permitted for interviews? Half allowed them, other half didn’t.

Understanding the role of the RIO. If they’re expected to be at all panel meetings, institutional leaders need to know what service RIO is giving to panel during proceedings. Some people keep looking at data they find zero problems – know when to call it quits when you have enough to make a finding. When we hit 51%, we got it, let’s move on. Referrals to other orgs, get buy-in, referrals to fed agencies.

Specific actions for institutional officials – provide recognition for panel service at highest level. Provide small tokens for investigative panels; moderate gift cards. Support RIO and panel members in filling this role for institution. Stand behind policy and process. As we start a case, have to follow policy and process. Make yourself known – put a face to the process and make it real. Herd the cats – keep all the people who think they know better at bay so panel members and RIOs can get the job done.

B. Group 2: Fostering Research Integrity: Incentives and Monitoring

How can senior institutional leaders create culture of research integrity? How do you define culture of integrity? Ideally, it’s one where you create environment that encourages open

discussion of research issues, explaining that this is right thing to do, rewarding people for ethical behavior. Ongoing senior leadership messaging, tools to facilitate high quality research, not about punishment. Providing resources for high integrity research. Making sure institution values and supports RCR education, training and mentoring for everyone, not just those who are “mandated” – making sure that people know that’s valued.

Creating a system to protect whistleblowers. Provide opportunities in neutral locations to allow people to raise concerns (integrity cards for incoming grad students so they know who to go to). Ensure RM process is fair, objective, consistent; that everyone is treated same regardless of their position. Nothing kills credibility more than someone who has been treated differently (high flyers don’t get special passes).

What are the barriers for promoting RI and how can senior leaders address those barriers? Promotion and tenure requirements – most institutions are out of touch with current state of research and publishing, funding – stuck in 1980s model – need to realign and update and show we value and reward people who conduct ethical, high-quality research.

Important to address “not my issue” perspective that we hear from humanities, who don’t think it occurs in their fields (only in biomedical). Try not to only use biomedical cases as examples, use social, behavioral, engineering, other areas. Try to get universities away from constant, crisis-driven response, reduce administrative burdens. Talk to governmental relations people. Globalization of science – different people may have diverse cultural norms. Measure effectiveness of RCR programs. We need to continue educating RCR educators about new tools.

What recommendation for senior institutional officials can we offer? Don’t tolerate bad behavior regardless of person’s stature or position. Set up culture of zero tolerance; visibly honor and reward good behavior, people who provide good mentoring (“incident-free” award). Finding ways to measure and reward high-equality science. Good practices metrics: Clean IRB submissions. People who meet deadlines. Attend committee meetings. People who are great ambassadors for institutional cultural values.

How can this group work together to share recommendations? Share what you’ve learned up and down, broadly and widely. ORI: Continue providing training, educational resources. Tools you provide are what allow us to do our jobs. Elevating the importance of RM. Take it to AAU. Not taboo; talk about it. Create internships and fellowships; accreditation programs for RIOs. Certification programs.

C. Group 3: Misconduct Involving Human and Animal Research

How can senior institutional leaders ensure proper handling of RM in context of human and animal research? Policies to recognize animal/human RM in place. Appropriate training. RM training for IRBs, IACUC, IBC – could be helpful when they’re looking at misconduct issues. Certain kinds of cases – allegation of IRB and ethical research. Ensuring consistency in our application.

What are barriers to handling RM in human and animal research, and how can leaders address these barriers? Politics. Social rankings, intimidation, lack of autonomy, resources, lack of empowerment.

What specific recommendations for senior institutional leaders can this group offer? In my particular institution, I’m the first person to be RI director, so recommendations are those that I’ve instituted. RI advisory group – keep minimum amount of work. Look at best practices and minimum compliance. Go through checklist and come up with gaps. On best practices, chairs of committees responsible for best practices. Which of these will work here, which ones can we implement. Set up time with Chairs, periodically, meet with VP of research or institutional official to let them know: this is what we’ve done, these are our obstacles, these are our concerns, and here are our risk mitigation strategies.

How can this group work together to share those recommendations? Encouraging future participation at future workshops. Communicate this info with fellow researchers.

D. Group 4: Responsible Conduct of Research Within and Beyond the Institution

Policy revisions: Is your policy up to date – large versus small cases – takes long time to update policy, what can you do in interim – supplement? Do too many cross-references undermine people outside or inside institution?

Involving faculty earlier in process versus later in the process: when does your institution involve faculty, at the inquiry phase or very early in the process?

Standing committees: Most institutions did not have them. Look at Harvard as an example of standing committees.

Addressing barriers – what are some of the good things you’re doing for prevention? Interfacing with different areas in your universities – training of directors/ committee members within office. Web modules.

Commercialization/tech transfer driving RI? Is there a push toward commercialization that is somehow influencing RI? COIs with RIO – return to indirect costs. Who pays for RM investigation? Who pays for recoupment of funds?

Vocal complainants and vulnerable complainants. Limit to what you can do – reiterate confidentiality, reiterate your policy. Complainant trying to find traction – what do you do with that individual? Refer back to policy. Consider relationship between complainant and respondent. How much information do institutions feel like they can release to complainant? Some institutions keep complainant informed throughout process; some treat them as witnesses.

Policies may not align and there might be difficulty with decision makers. Do what makes sense; RM proceeding should take place where RM allegedly happened. Institution 1 may not tell Institution 2 about what’s going on – how do we communicate while maintaining confidentiality?

Public versus private – hospital versus institution. Differences in experienced/inexperienced programs. Difference between high impact journal versus low-impact journal. Not much difference – taking into account mitigating factors in a settlement. If respondent makes admission, those could be considered in settlements.

Visa barriers – moving someone out of department, have to re-file for their visa. Vulnerable populations that make complaints. Consider making RCR part of immigration training when they get to campus. Whose job is it to protect individuals? PIs might be uncomfortable taking the person. Supervisory structure – individual PI has supervisor where lab goes into receivership and lab/PI oversees lab and actions of lab. Might assist with lab leadership.

Recommendations: should there be mandatory training for faculty? What flags should an institution be looking for? Metrics to determine whether RCR is working. Recommendation that ORI continue these meetings.

IX. Concluding Remarks

Dave Hudson: Given the huge diversity of audiences, impressed by similarities in our approaches and things I’ve learned. We work from the same playbook. Depth in thoughtfulness – spectacular approach, flair, dedication. I have to come to meetings like this to be around other RIOs. Terrific input from federal partners. Need to meet them, know them, store their numbers. Tag team pairs – seeing how you work together was impressive. Good communication, good integration of activities – very important. The power that brings to your organization is impressive.

What would we all like to do next? Is this worthwhile, is it worth doing , and how should we do it in future?

Tremendously helpful meeting; learned a ton. Recommendations about sharing best practices is great for us. What information can we share about timelines? We all have allegations that sit

outside of PHS. Would be interesting to see metrics and numbers about how long the process takes. Working group trying to pull together those stats. It’s hard when you’re asked: What’s the national trend. We can look at ORI’s, but it’s only a piece of the puzzle. How should this be offered to other senior institutional officials who weren’t able to be here?

Carl Henry from UK: We have programmatic arrangements with CDC And NIH; this has been hugely informative. The biggest thing that it provides personally is that it’s more energizing and provides me with more confidence to make the case for compliance, integrity [back home].

Very important to describe diversity of problem. Start statistics through surveys, opinion – what hurdles we’re facing.

Reassurance that I’m not the only one who struggles with this stuff. Reassuring to discover that federal agencies are welcoming, supportive, helpful, engaged, willing to take our calls. Our collaborators – we have a shared mission, shared purpose. We’re not in competition with

ORI or NSF – we work collaboratively. Thanking ORI and John for putting this on. Remarkable thing about our community – willing

to share as much as we are that in another context would be embarrassing. Wide angle lens shot of some issues. Recommendation for moving forward—zooming in a little bit and having two-day sessions on specific slices. Narrowing the focus to concentrate a bit deeper.

These two days were a journey for me in trying to understand federal world and how things work. Refreshing to see how much you care. What I would suggest to have more people like me. Important for students and post-docs are good people who care. Have more outreach at bottom, not just top. Research ethics, metrics are important – getting metrics on how research ethics is taught. Institutional structures – would like to know why RCR programs are in graduate studies. What does it mean to be ORC on campus? Research-driven FOA – RFP coming out in January.

Leickness Chisamu Simbayi, South Africa: It was pleasure to hear first hand what these folks do. In charge of two RIOs, this meeting provided more clarity on their functions and responsibilities.

Junko Nishitani: come from very small institution with limited resources, staffing issues. Trying to meet all of the demands and help investigators do good research. Taking care of them and making sure that they conduct research ethically and produce good results. Will be taking back lot of information to share with leadership. Not just research integrity, but integrity for everything.

Constrained budgets and time – Indiana sequestration webinar – use as a model to extend the reach of these meetings, bring in more people with exposure.

Emily Schriver to address RCR issues at National Association of College and University Attorneys (NACUA) conference.

John Carfora: We need to be resources to reach other. We’re neighbors now.