UL and the UL logo are trademarks of UL LLC © 2013

Health Information Technology – Research Challenges

Michael E. WiklundGeneral Manager – Human Factors EngineeringUL–Wiklund (formerly Wiklund Research & Design)

The Research Project

NIST retained Wiklund R&D (now part of UL) to develop guidelines for EHR usability and to develop a framework for EHR usability assessment.

Project funded by the American Recovery and Reinvestment Act of 2009.

Key tasks:

•Survey, observe, and interview EHR users

•Conduct HFE inspection of sample EHRs

•Conduct usability test of sample EHRs

•Develop EHR user interface design guidelines

•Develop recommended process for EHR usability assessment

•Document results in a report

2

Progress

• Conducted large scale survey of EHR users

• 559 respondents: physicians, nurses, administrators

• Vanderbilt University Medical Center facilities

• 3 month process

• 18% response rate

• Facilitated high response rate by visiting departments to invited responses

• Evaluated user experiences with StarPanel and multiple “off-the-shelf” EHRs that are integrated into Vanderbilt’s EHR

• Many new insights pertaining to topics such as: information saturation, value of defaults, attention to updating baseline information, information integration

• Future reports will share distilled findings

3

Topic might not be riveting…

4

…but it is pertinent!

Photo credit: http://etc.usf.edu/clippix/pix/airplane-rivets-2_medium.jpg

UL-Wiklund’s Partners

• Vanderbilt University

• University of Wisconsin (initially)

• Design Science

5

Project Status

We have made slow progress.

There are government rules pertaining to:

•Paperwork reduction

•Protecting human subjects

•EHR vendors having equal opportunity to submit their EHRs for assessment

•Contract performance period

•Legal consent by the sponsoring agency

6

Photo credit: http://3.bp.blogspot.com/-MwaoVCK5aKA/TtpmP0o0dsI/AAAAAAAABms

/WyJh53uuBP4/s1600/T+and+H.jpg

Paperwork Reduction Act (44 U.S.C. 3501 et seq.)

Federal law enacted in 1980.

Excerpts:

•Minimize the paperwork burden for individuals, small businesses, educational and nonprofit institutions, Federal contractors, State, local and tribal governments, and other persons resulting from the collection of information by or for the Federal Government.

•Ensure the greatest possible public benefit from and maximize the utility of information created, collected, maintained, used, shared and disseminated by or for the Federal Government.

•Minimize the cost to the Federal Government of the creation, collection, maintenance, use, dissemination, and disposition of information.

7

Paperwork Reduction Act (44 U.S.C. 3501 et seq.)

Our research plan called for using the same questions (e.g., survey questions) to collect data from more than 10 people.

Example: We have already collected data from hundreds of EHR users (e.g., physicians, nurses, technicians) working for Vanderbilt University Medical Center.

Such requests:

•Describe the data (i.e., information) that you are collecting.

•Explain why the data is needed.

•Estimates the time and cost (for the public) to respond to the request.

Project impact:

NIST had to place an information collection request with Office of Management and Budget (OMB) to collect EHR-related data.

Approval process was lengthy.

8

Federal Wide Assurance (FWA)

An institution commits to Health and Human Services (HHS) that it will comply with the requirements in the HHS Protection of Human Subjects regulations at 45 CFR part 46.

HHS human subject protection regulations…were first issued in 1974.

In 1978, the National Commission or the Protection of Human Subjects of Biomedical and Behavioral Research published “Ethical Principles and Guidelines for the Protection of Human Subjects of Research,” also known as the Belmont Report, named after the Belmont Conference Center where the Commission met when drafting the report.

Source: http://www.hhs.gov/ohrp/humansubjects/index.html

9

Federal Wide Assurance (FWA)

Project impact:

We submitted a formal, written, and binding commitment to Office for Human Research Protections (OHRP) stating that UL-Wiklund would fully comply with human subjects protection guidelines.

Our submission included:

•Company’s identifying information, including the identify of human protections administrator.

•List of the ethical principles that we would follow; we cited Belmont Report.

•Designation of IRBs that would review our research (activities covered under the FWA).

•Binding letter of agreement with external IRB to properly oversee our research.

Approval process was lengthy.

10

Institutional Review Boards

NIST required that its own IRB review and approve all research plans. This may be the case with other federal agencies.

Research plans also had to be approved by research sites, such as Vanderbilt University, to conduct research in their hospitals.

Medical institutions are unlikely to abdicate responsibility to another organization – even a federal agency – for human subjects projection associated with research on their premises.

Approval process was lengthy.

11

Equal opportunity for EHR assessment

Our plan has been to assess a diverse set of EHRs by means of HFE inspection and usability testing.

Simple path would be to select several EHRs for assessment based on appropriate selection criteria (e.g., use environment, user interface features).

This path was problematic because it could give (or be perceived as giving) certain EHRs vendors an advantage or disadvantage.

Solution was to advertise in the Federal Register the opportunity for developers to submit their EHRs for assessment.

Important: We will evaluate the EHRs only for the purpose of identifying good design practices. Our published results will not cite specific EHRs.

This step was time consuming.

12

Announcement on 2-14-12

13

Announcement on 2-14-12

14

NIST is soliciting interest in supplying electronic health record (EHR) systems for use by NIST in research to develop a framework for assessing the usability of health information technology (HIT) systems, EHRs in particular, and performance-oriented user interface design guidelines for EHRs.

NIST is soliciting interest in supplying electronic health record (EHR) systems for use by NIST in research to develop a framework for assessing the usability of health information technology (HIT) systems, EHRs in particular, and performance-oriented user interface design guidelines for EHRs.

Legal Review

NIST’s legal team had to review all research plans before they could be submitted to the agency’s own IRB for further review.

15

Many hurdles…

16

Photo credit: http://msnbcmedia2.msn.com/j/MSNBC/Components/Slideshows/_production/_archive/2012-%20LONDON-OLYMPICS/ss-120727-emotional-moments-july27-aug4/ss-120804-oly-emotional-moments/ss-120804-emotional-moments-21.ss_full.jpg

…but necessary.

Lessons Learned

• Recognize that government-sponsored research introduces multiple responsibilities that might not otherwise come into play when conducting privately-funded research.

• Add considerable time into the research schedule to obtain FWA and PRA approvals.

• Plan to obtain multiple IRB approvals when conducting multi-site research, particularly when conducting it at healthcare institutions.

• Perfect your research plans early because changing them (presumably for the better) would require many re-approvals.

17

UL and the UL logo are trademarks of UL LLC © 2013

Questions?