-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
1/65
Driving QualityA Health IT AssessmentFramework or Measurement
A CONSENSUS REPORT
Health ITUtilization
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
2/65
The National Quality Forum (NQF) operates under a three-part mission to improve thequality o American healthcare by:
building consensus on national priorities and goals or perormance improvementand working in partnership to achieve them;
endorsing national consensus standards or measuring and publicly reporting onperormance; and
promoting the attainment o national goals through education and outreach programs.
This work was supported with unding by the U.S. Department o Health and Human Services.
Recommended Citation: National Quality Forum (NQF), Driving QualityA Health ITAssessment Framework or Measurement: A Consensus Report,Washington, DC:NQF; 2010.
2010. National Quality Forum
All rights reserved
ISBN 978-0-9828421-3-3
No part o this report may be reproduced, stored in a retrieval system, or transmitted, in
any orm or by any means electronic, mechanical, photocopying, recording, or otherwise,
without prior permission o the National Quality Forum. Requests or permission to reprint
or make copies should be directed to:
Permissions
National Quality Forum
601 13th Street NW
Suite 500 NorthWashington, DC 20005
Fax 202-783-3434
www.qualityforum.org
National Quality Forum
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
3/65
Driving QualityA Health IT Assessment Framework
or Measurement: A Consensus Report
i National Quality Forum
Foreword
THE U.S. HEALTHCARE SYSTEM is burdened by high costs and inecient deliverymodels. Shortcomings in quality inormation available to providers and consumerscontribute to these challenges. Health IT is considered a promising tool that can provide
inormation to healthcare stakeholders, allowing them to make care decisions based onindividual preerences and evidence-based practices, and in doing so, improve quality,saety, and eciency o the health system. As policy makers, providers, and payersincreasingly look to health IT to provide actionable inormation, it is critical to developan inrastructure that also enables the collection o comparable inormation on how andwhen health IT systems are used.
In 2008, the National Quality Forum (NQF) endorsed nine structural voluntary consensusstandards, developed by the Health IT Structural Measures Panel, to assess and encourageclinician adoption o health IT. NQFs endorsement o these standards was an initial startor health IT adoption to help improve quality o care. Next, NQF recognized the need to
advance understanding o the specic eatures and unctions o health IT that improve quality.
In January 2010, NQF convened the Health IT Utilization Expert Panel, which includedbroad representation rom the healthcare community, to develop the Health IT UtilizationAssessment Framework. The ramework, described in detail in this report, is designed tohelp dene, understand, and measure how providers use health IT systems. It also identiesimportant and common approaches or how we should build health IT systems to captureinormation about their meaningul use and lays a oundation or the development ohealth IT usage measures.
NQF thanks the Health IT Utilization Expert Panel members, the Expert Panels Chair,
Blackord Middleton, Vice Chair Eric Schneider, and NQF members or their contributionsto developing an inrastructure that, in the uture, will make inormation widely availableand signicantly improve the quality o care delivered in this country.
Janet M. Corrigan, PhD, MBAPresident and Chie Executive Ofcer
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
4/65
Driving QualityA Health IT Assessment Framework
or Measurement: A Consensus Report
ii National Quality Forum
Table of Contents
Executive Summary...................................................................................................v
Background and Introduction .................................................................................... 1
Expert Panel Analysis ............................................................................................... 4
Methodology and Project Approach ....................................................................... 4
Health IT Utilization Assessment Framework Development ......................................... 5
Health IT Utilization Assessment Framework Components .......................................... 6
Examples o Measuring Health IT Use ...................................................................... 10
Example 1: Clinical Decision Support ................................................................... 10
Example 2: e-Prescribing .................................................................................... 19
Example 3: Order Set ........................................................................................ 20
Example 4: Clinical Summary ............................................................................. 24
Recommendations and Future Work ......................................................................... 33Notes................................................................................................................... 38
Appendix A Health IT Utilization Expert Panel Members ........................................ A-1
Appendix B Potential Data Requirements or Future Framework Iterations ..................B -1
Appendix C Health IT Utilization Assessment Framework Actor Elements ..................C -1
Appendix D Health IT Utilization Assessment Framework Action Elementsand Subelements ............................................................................ D-1
Appendix E National Voluntary Consensus Standards or Health IT:Structural Measures 2008 .................................................................E -1
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
5/65
Driving QualityA Health IT Assessment Framework
or Measurement: A Consensus Report
iii National Quality Forum
Executive Summary
HEALTH INFORMATION TECHNOLOGY (HEALTH IT) oers great promise to improve health-care quality, saety, and aordability, and the health o the population. Passage o the recent HealthInormation Technology or Economic and Clinical Health Act (HITECH) in the American Recovery
and Reinvestment Act (ARRA) is expected to signicantly drive increased adoption o health ITsystems. As health IT system use becomes more widespread, it will be necessary to assess whetherthe tools are eectively and meaningully used. Although some evidence shows use o health ITsystems can decrease errors o omission and commission and reduce unnecessary, ineective, andharmul care, other evidence is less convincing. Assessing and tracking the perormance o health ITsystems requires measures o health IT systems, including measures o unctions and capabilities, aswell as when and howhealth IT systems are used.
This report is based on the work o the National Quality Forums (NQFs) Health InormationTechnology Utilization Expert Panel (Expert Panel). Specically, the report examines, denes, andorganizes the data needed to measure eective health IT use to better understand how health IT
tools can improve the eciency, quality, and saety o healthcare delivery. The report builds onthe work o NQFs Health Inormation Technology Expert Panel (HITEP), which was established toaccelerate eorts to ensure health IT will eectively support quality measurement. The Health ITUtilization Expert Panel expands on HITEPs Quality Data Set (QDS), a ramework developedto clearly dene concepts used in quality measures and clinical care to drive the use o qualitymeasurement based on inormation available rom an electronic health record (EHR).
The Expert Panel developed the Health IT Utilization Assessment Framework(ramework). Theramework is designed to dene a method or expressing data that can be captured by health ITsystems to understand and measure their eectiveness. Health IT use assessment can providevaluable inormation or most healthcare stakeholders, including the quality improvement community,
the health IT vendor community, providers, payers, purchasers, and policymakers. The ramework isexpected to support eective and meaningul EHR use assessment by:
enabling development o measures o eective health IT use;
understanding capabilities o the EHR and other health IT tools to meet meaningul userequirements;
assessing unintended consequences o health IT usage;
enabling inormation capture as a byproduct o clinical workfows;
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
6/65
National Quality Forum
iv National Quality Forum
enhancing collaboration between health
IT vendors, purchasers, implementers, andcertiying bodies by encouraging the use ocommon health IT assessment strategies;
enabling determination o high-priorityhealth IT usage that supports certication oreal-world implementations; and
encouraging clinical eectiveness researchregarding unintended consequences ohealth IT usage as well as research todetermine eective health IT utilization.
This ramework provides a unique approachto identiying and measuring: 1) the use ohealth IT applications; 2) whether the workfow(driven by the systems user interace) occurs asdesigned; and 3) that such use improves careprocesses, quality, and saety. The report pro-vides our examples: 1) measurement o clinicaldecision support (building on the work o theNQF Clinical Decision Support Expert Panel),2) e-prescribing, 3) order sets, and 4) clinicalsummaries. Each o the examples is modeled
based on metrics in the 2010 Final Rule or theEHR incentive program.
This report is only a rst step in establishinga standard methodology to measure the useo health IT systems to identiy their eectiveuse. Currently, standards exist only or EHRcertication, not or assessing eectivehealth IT use. The meaningul use EHR incentiveprogram and certication requirements orinteroperability (sending inormation rom one
system to another) provide the incentive, but astandard mechanism to determine appropriateutilization o EHRs does not yet exist. Standardsare needed to assess EHR use; without themconsistency and comparability are not guaran-teed. The ramework described in this reportwill signicantly inorm standardization eortswith respect to structural components o EHRs
and other clinical inormation systems. Such
standardization will allow or clearly denedmeasures with reliable, consistent, and achiev-able results while reducing the eort requiredto assess clinical system eectiveness by systemvendors, implementers, or users o such systems.Perormance measures o health IT systems usingthese standard user interactions and transactionswill enable clinical eectiveness research, helpdetermine unintended consequences o healthIT, and evaluate the real-world usability o
products and applications.
Health IT Utilization AssessmentFramework Refnement and EvolutionRecommendations1. NQF should incorporate the Health IT
Utilization Assessment Frameworkinto theQDS Model and continue to manage itsevolution with public consensus as existing
standards and quality measures evolve.Specically, the ramework should beincorporated as a new QDS category,listing each o the actions identied in thisreport in the action-actor-contenttriplet(Figure 1).
2. The Department o Health and HumanServices should consider the adoption ohealth IT utilization metrics as an EHR certi-cation requirement; specically, adopt thecapture, logging, and evaluation by EHRs
o each o the 20 triplets identied in thisreport as EHR certication requirements.
3. The health IT use data requirements shouldbe incorporated into standard rameworkssuch as the HL7 EHR Functional Model toencourage urther evaluation and eortsby the appropriate standard developmentorganizations (SDOs).
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
7/65
v National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
4. Standards development organizations, pro-
essional societies, workfow planners, andother key stakeholders and entities shouldcollaborate to standardize, harmonize, andidentiy denitions o and gaps in roles orall users o clinical applications and health ITsystems.
5. NQF should pursue a call or measures ohealth IT utilization.
As health IT use quality measures areconceptualized and developed, the HealthIT Utilization Assessment Frameworks datarequirements provide clear guidelines or the in-ormation necessary and potentially available.The work o the Health IT Utilization ExpertPanel advances the ocus rom adopting healthIT to appropriately and eectively using healthIT. This work signals to the healthcare commu-nity that adoption o a health IT system aloneis not sucient to promote greater ecienciesin care and improve health outcomes; rather,a commitment to eective health IT use and to
monitoring, measuring, and reporting this use isnecessary to achieve these goals.
ACTION ACTOR
CONTENT
ACTION ACTOR
Health IT Use Data
Figure 1: The Health IT UtilizationAssessment Framework
Determining usage o health IT requiresidentiying the action expected, the actorperorming the action, and the contenton
which the action is taken. All three componentsare required to more accurately and preciselyunderstand the use o health IT applications.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
8/65
Driving QualityA Health IT Assessment Framework
or Measurement: A Consensus Report
1 National Quality Forum
Background and Introduction
Background
HEALTH INFORMATION TECHNOLOGY (HEALTH IT) oers great promise to improvethe quality, saety, and aordability o healthcare, and the health o the population. TheOce o the National Coordinator or Health Inormation Technology (ONC) projectedthat electronic health record (EHR) adoption can reduce healthcare costs by 20 percentper year.1 Although some have questioned that goal, providers over the past several yearshave increasingly implemented health IT systems (e.g., EHRs, electronic prescribing) ininpatient and ambulatory settings. Passage o the recent Health Inormation Technology orEconomic and Clinical Health Act (HITECH) in the American Recovery and ReinvestmentAct (ARRA) is expected to signicantly drive adoption o health IT systems.2 HITECHwill provide more than $20 billion over the next ve years to help providers becomemeaningul users o health IT. For health IT systems to ulll their promise, however, theymustsupport patient care directlythrough clinical decision support and quality improve-mentand support multiple uses o health inormationthrough public reporting, publichealth surveillance, and clinical eectiveness research.
As health IT system use becomes more widespread, it will be necessary, through thedevelopment and use o metrics, to assess whether the tools are being eectively andmeaningully used. Although some evidence shows the use o health IT systems candecrease errors o omission and commission and reduce unnecessary, ineective, andharmul care, other evidence is less convincing.3-9 Identiying and developing measureso health IT activities and tracking perormance on these metrics will be critical to assess-
ing eective health IT use and driving more appropriate use where necessary. Measuringthe quality o health IT use also requires an understanding o the systems unctions and thecapabilities that track and monitor when and howit is used.
In January 2010, the National Quality Forum (NQF) convened the Health InormationTechnology Utilization Expert Panel (Expert Panel; see Appendix A or list o members).The goal o the Expert Panel was to examine, dene, and organize the inormation neededto measure eective health IT use to better understand how health IT tools are used andultimately to improve the eciency, quality, and saety o healthcare delivery.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
9/65
National Quality Forum
2 National Quality Forum
This Expert Panel builds on the work o
NQFs Health Inormation Technology ExpertPanel (HITEP). NQF convened HITEP, withsupport rom the Agency or HealthcareResearch and Quality (AHRQ), to accelerateongoing eorts dening how health IT canevolve to eectively support quality measure-ment. HITEPs output, the Quality Data Set(QDS), is a ramework that clearly denesconcepts used in quality measures and clinicalcare and is intended to enable quality mea-
surement based on the inormation availablerom an EHR.
In 2007, the rst HITEP (HITEP I) developedand released a ramework to acilitate thedevelopment, use, and reporting o qualitymeasures rom EHR systems. The report thatollowed, Recommended Common Data Typesand Prioritized Perormance Measures orElectronic Healthcare Inormation Systems,proposed 11 data categories and 39 data
types or a set o 84 high-priority quality mea-sures to enhance capabilities or the electroniccapture o data or quality measurement.In its second report, Health Inormation Tech-nology Automation o Quality Measurement:Quality Data Set and Data Flow, the secondHITEP (HITEP II) developed the QDS to enableautomated, patient-centric, longitudinal qualitymeasurement. The QDS is intended to serveas a centralized, maintained repository o data
requirements (concepts, data types, data ele -ments, and code lists) associated with qualitymeasures and data denitions that provideunambiguous meaning or each data elementin a quality measure.
HITEP I and II developed a structure orquality measure data types and data fow(i.e., the QDS). HITEP used data typeto dene
a concept (e.g., medication) and how it was
expected to be used (e.g., administered,ordered, etc.). HITEP urther dened howthat inormation is captured within a clinicalworkfow with data owattributes:
source(e.g., the originator o the inorma-tion, e.g., a clinician, patient, or device);
recorder(e.g., a clinician, patient, or deviceand possibly dierent rom the source);
setting (e.g., hospital, home, ambulatorysetting); and
health record feld(e.g., location in theEHR where the inormation should reside).
These data fow attributes dene inormationabout the data captured to enable a morespecic understanding o the clinical careprocess. They also enable clinical decisionsupport (CDS) workfows to more clearly speciyexpected data sources, recorders, and settings.The QDS Model exists as a dynamic modelthat will expand and undergo versioning to
support uture needs or measurement, CDS,and care delivery.
The QDS has been incorporated into theHealthcare Inormation Technology StandardsPanel (HITSP) updates to the Quality Interoper-ability Specication, a standard that encodeselectronic quality measure data, and theHITSP components to which it reers.10 TheONC created HITSP in 2005 to promoteinteroperability and the exchange o inorma-
tion between electronic health systems. HITSPspecically identied an electronic source anda standard code set or each data categoryand data type in the HITEP report. Leveragingthe Quality Interoperability Specication willallow the QDS to become part o healthinormation exchange standards used by thehealth IT community.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
10/65
3 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
Additional inormation about user interactions
with the EHR is necessary to determine themeaningul use o unctionalities, such aselectronic prescribing (e-prescribing) and CDS.Specically, this will allow heath IT system usersto use the tools in a manner that acilitatesadherence to evidence-based care, directlysupports better patient care, and improvesoutcomes.11 The QDS supports detailed qualitymeasure specication or use in EHRs, butcurrently there is no standard approach to
evaluating whether and how EHR unctions areused. The approach to monitoring EHR activitieshas been primarily ocused on transactionsthat can produce claims or payment, or thoseneeded to track the use o consumable (e.g.,medications) and durable use (e.g., IV pumps,ventilators). The Health Insurance Portability andAccountability Act (HIPAA) has encouragedauditing o activities within the EHR, althoughsuch unctions are mostly used to determine
access and management o personal healthinormation (PHI).12 Auditing is essential tomanaging privacy and security, but it is notsucient to determine the utilization o EHRunctions within the usual process o care.
Introduction
This report describes the Health IT UtilizationExpert Panels approach to developing a rame-
work to describe the inormation required tomeasure eective health IT utilization and pres-ents the Expert Panels nal output, the HealthIT Utilization Assessment Framework(rame-work) itsel. The ramework is designed to helpdene a method or expressing data that canbe captured by health IT systems to understandand measure their usage.
Better understanding o how and what
health IT eatures and unctions providersactually use is valuable inormation or mosthealthcare stakeholders, including the qualityimprovement community, the health IT vendorcommunity, payers, purchasers, providers, andpolicymakers alike.
For the quality improvement community, theramework can provide specic data elementsto inorm uture perormance measures andpractices, including those to identiy unintended
consequences o health IT usage. For the healthIT vendor community, the ramework encouragesinormation capture about the use o healthIT as a byproduct o clinical workfows. Bydening and identiying such inormation, theramework will also provide qualitative andquantitative inormation to support moreeective implementation and analysis o usageby vendors, certiying bodies, implementers,purchasers, and providers. The ramework
could assist providers in better understandinghow eectively they use an EHRs eaturesand unctions. In addition, it could oerproo and documentation or participationin incentive programs and other relevantinitiatives. An increase in data logged inclinical systems will also provide inormationto drive research and study o health IT systems.For the policy community, the ramework caninorm the understanding o what health IT
capabilities and high-priority unctionalitiesare necessary to suppor t meaningul userequirements and certication o EHRs.
The work o the Expert Panel advances13the ocus rom adopting health IT to appropri-ately and eectively using health IT. This worksignals to the healthcare community that adop-tion o a health IT system alone is not sucient
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
11/65
National Quality Forum
4 National Quality Forum
to promote greater eciencies in care and to
improve health outcomes; rather, a commitmentto eective health IT use and to monitoring,measuring, and reporting this use is necessaryto achieve these goals.
Expert Panel AnalysisMethodology and Project Approach
The Health IT Utilization Expert Panel conducted
seven virtual meetings between January andJuly 2010, and it implemented the ollowingproject approach (Figure 2) to identiy the dataelements necessary to measure health IT usage.The Expert Panel developed the Health ITUtilization Assessment Frameworkwith ullconsideration o ongoing national eor ts andactivities to increase health IT use and improveoverall healthcare quality.
The Expert Panel selected the NationalPriorities Partnership (NPP) Goalspatient
engagement, care coordination, end-o-lie
care, saety, population health, and overuseto inorm development o the ramework andensure its relevance to direct patient care.14Using NPP Goals to guide the rameworksdevelopment was intended to make certainthat the data requirements identied supportmeasurement and reporting o key health ITunctions or existing and uture measurementdirections.
Additionally, the Expert Panel reviewed
the Department o Health and Human Services(HHSs) Health IT Policy Committees Meaning-ul Use Matrixs Stage 1 Policy Objectives15(which were driven by the NPP Goals) to en-sure the ramework supported national eortsaround meaningul use and health IT adoption.Collectively, the emphasis on the NPP Goalsand meaningul use objectives ensured theramework would acilitate national eorts thatincentivize clinician health IT use by identiyingthe data required to measure the use o thoseunctions.
NPP High-Priority Goals
ClinicalObjectives
HITFunctionalites
Model of DataRequirements
Figure 2: Health IT Utilization Expert Panel project approach
The Expert Panel identied the data elements necessary to measure health IT usage in support onational clinical objectives and goals.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
12/65
5 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
Health IT Utilization AssessmentFramework DevelopmentIn developing the ramework, the Expert Panelrst conducted an analysis o the major healthIT unctions described in: 1) the Centers orMedicare & Medicaid Services (CMSs)Notice o Proposed Rule Making (NPRM) onmeaningul use o EHRs and 2) the CerticationCommission or Health Inormation Technology(CCHIT) specied health IT unctions neces-
sary to support meaningul use requirementsand certication.16 The Expert Panel thenidentied the data categories necessary tomeasure those unctions in a health IT system(see the next section, Framework Components).
The Expert Panel dened and agreed uponthe ollowing principles to steer development othe Health IT Utilization Assessment Framework:
The rst version o the ramework will ocuson data requirements to support process
measures (methods by which healthcareis delivered) and structural measures(characteristics o the environment in whichhealthcare is delivered).17 The Expert Panelexpects that the ramework will also enableimproved health outcomes, given thatadherence to evidence-based practicescan drive better health outcomes.
The rameworks data elements shouldallow or and enable the dierentiation ovarious levels o utilization. For example, to
determine health IT eectiveness in dierentlevels o health IT usage, data are requiredto capture multiple levels o utilization.
The ramework should be implementableand extensible (i.e., able to be modied)within health IT systems. Certication criteriathat meet meaningul use requirements in-ormed data element selection (see the nextsection, Framework Components).
The current ramework is not intended to
be an exhaustive list o data requirementsor measurement o health IT use. The datarequirements in the current ramework arebased on components o the January 2010CMS NPRM.
The relevant certication criteria andmeaningul use requirements inormed therameworks development; however, theramework does not dene specic datasources or requirements, which are denedat local implementation sites. With the rame-work as a oundation or potential inormationrequirements, uture health IT systems could bedesigned and implemented in local settings ac-cording to local practices to eed that inorma-tion into specic workfow.
EHR certication standards currently exist,although a standard mechanism to determineappropriate use o EHRs is not yet in place.Future work to develop a standard on eective
health IT use and certication requirementsmay build on the ramework, but standardsdevelopment, implementation, and regulationare outside the scope o work or the Health ITUtilization Expert Panel.
The Expert Panel specically identied anddeveloped several supporting examples todemonstrate how the ramework can enablehealth IT systems to measure eective use andsupport the meaningul use objectives: CDS,
e-prescribing, order sets, and clinical summa-ries (see the section Examples o MeasuringHealth IT Use, below). The examples are notNQF-endorsed measures, nor do they refectthe existence o health IT utilization measures.Measure development was not in the scopeo the Expert Panels activities or in NQFsdomain; however, the Expert Panel recommends
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
13/65
National Quality Forum
6 National Quality Forum
a call or health IT usage measures to improve
quality and saety o care.
Health IT Utilization AssessmentFramework Components
Measures o health IT use must capture dataabout how health IT usage impacts healthcaredelivery across several dimensions, includingcost and quality. Specically, a health IT usemeasure must be able to identiy:
Actor: a person or electronic system thatperorms actions required in a measure ohealth IT utilization,
Content: the concept on which an action istaken, and
Action: something a measure recommendsto a person or a computer programmed bya person.
The actor, content, and action are the essen-
tial components o a utilization data element,represented in Figure 3. A utilization dataelement requires logging o actions taken bywhom (actor) and about what (content). Thedata element is not directly visible to a clinicalapplication or health IT system user; rather, ameasure o health IT use would utilize thesecomponents to capture the necessary inormationas the user interacts with the systems unctions.
The Expert Panel determined that actors,
content, and actions should be mapped toexisting health IT terminology standards thatallow inormation to be shared across health-care settings unambiguously. Aligning theHealth IT Utilization Assessment Frameworkwith existing health IT standards can encourageinteroperability and integration across healthIT systems at a national level. The ramework is
Content(the concept on
which an actionis taken)
Actor(a person or
electronic systemthat perorms
actions required)
Action(somethinga person or
computer systemprogrammedby a personcan do)
Figure 3: Components of autilization data element
CONTENTCode List
(Taxonomy)
AttributesTime/date stampAntecedent eventExpected action
ACTIONACTOR
purposeully agnostic regarding standardsand their incorporation into certicationcriteria, and the Expert Panel recommendsthat the appropriate standards harmonizationbody should identiy and dene standardterminologies (see Recommendations and
Future Work, below).The ollowing proposed attributes were iden-
tied as common to all ramework elements:
Source: the originator o a data element,which may be an individual or a device
Recorder: the individual or device thatenters the data element in a health recordeld (may also be the source o the data)
Setting: the physical location where a dataelement is captured, dening the encounter
location where the data are expected tooriginate
Date/time: the precise time and daterecorded in an electronic health system
Method: The manner in which the dataelement is captured (e.g., data entry by auser, a system query rom one applicationto another, etc.)
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
14/65
7 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
Justication (or action or lack o action):
inormation about why an action is taken ornot taken, potentially taken rom a pick listor a ree-text entry
Content specic attributes: inormation (ormetadata) that provides additional detailabout an individual element (e.g., ormedications, the content-specic attributesare, or example, requency, duration, dose,and route).
The ramework does not dene specicactors or association with actions and content.The Expert Panel developed the rameworkwith the understanding that variation in localpractice, scope o practice laws, and availableresources will infuence implementation ohealth IT systems. Next, actors, content, andactions are described in more detail.
ActorAn actor can be a member o the healthcaredelivery team, a patient, a caregiver, or anelectronic system. This component o a utiliza-tion data element captures who perorms thespecic health IT action being measured. In thisversion o the ramework, actors are dened byroles (human or health IT system). To evaluateexisting standards suitability to represent rolesin the ramework, the Expert Panel conducteda preliminary mapping o actors to threehealth IT standards: the HITEP II Recorderattributes (actors), HL7 actor roles, and LOINC
Document Ontology Axis Values (roles).18This exercise identied some inconsistenciesamong standards or dening actor roles thatshould be considered or uture work (seeRecommendations and Future Work, below).As described previously, the ramework ispurposeully agnostic o standard terminologies.
The evaluation o vocabularies or content and
their incorporation into cer tication criteriashould be addressed by the appropriate stan-dards development and harmonization entities.
Reer to Appendix C or the actors and rolesthat represent actions in the ramework, aswell as other taxonomies that the Expert Panelconsidered.
ContentContent is the substance or subject matter onwhich actions are taken. This component o autilization data element captures the inorma-tion about which a health IT action is expected.In this version o the ramework, content elements(Box 1) were derived rom CCHIT requirementsand the CMS NPRM. Duplicate content itemsrom the two lists were removed. The NQFQDS Model Version 2.1 inormed the rame-work content. Aligning the Health IT UtilizationAssessment Frameworkwith the QDS provides
a solid oundation or both health IT usage andquality measurement. The list o content elementsis not intended to be exhaustive; content isexpected to be managed using the QDSModel as it evolves over time. Many o theseelements are managed in the QDS using thecategory system characteristics. The most cur-rent version o the QDS Model is maintainedon the NQF website.19
ActionAn action is an interaction with the health ITsystem that can be the product o human actionor a programmed activity o the health IT systemitsel. The action is the health IT unctionalitythat is measured or the health IT interventionthat is called or in a quality measure. The
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
15/65
National Quality Forum
8 National Quality Forum
Box 1: Health IT Utilization Assessment Framework Content Elements
1. alerts responded to by a user 23. medication order (prescription)
2. allergy list 24. medications (Beers Criteria)
3. care modifcations based on clinical decision 25. non-medication allergy on allergy listsupport rules 26. notifcations by patient preerence
4. claims submitted electronically to all payers 27. patient preerence or ollow-up care5. clinical summaries were provided 28. patient preerence or preventive care6. clinical summary 29. patient summary record rom other providers7. condition 30. patients at high risk or cardiac events on
8. demographic inormation aspirin prophylaxis9. diagnostic test results 31. patients with access to personal health inormation
10. diastolic blood pressure changes electronically
11. education provided 32. prescriptions (permissible) electronically
12. encounters where medication reconciliation is perormed 33. problem list
13. ormulary or preerred drug list 34. procedures perormed
14. health maintenance items perormed 35. quality measure results
15. height 36. reportable laboratory results to public health
16. high-profle order in order set 37. smoking status
17. immunization allergy to immunization registry 38. syndrome-based public health inormation
18. immunizations39. systolic blood pressure changes
19. insurance eligibility 40. transitions in care or which summary care recordis shared20. laboratory test result
41. vital signs21. medication allergy list42. weight changes22. medication list
action concept allows or Application ServiceManagement, a method o managing peror-mance and quality o service.20 A common
classication system was used to describe allactions based on unctions described in theMeaningul Use Proposed and Final Rules.Similar actions were classied into commonthemes. The Expert Panel updated the ac-tions with services that are required to supportgreater attention to uture measurement, includ-ing patient engagement in care and shared
decisionmaking. The Expert Panels analysisconrmed that the categories weresucient to meet utilization measurementneeds. Appendix D presents the list o actionsthat were identied by the Expert Panel. Thislist is not intended to be exhaustive.
The ramework includes 20 categories ointeractions with the health IT system, each owhich can be perormed by a human operatoror the system itsel. The ramework actioncategories are listed below (see Appendix D
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
16/65
9 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
or a listing o the individual elements
reviewed by the Expert Panel in developingthe ramework):
1. Access: The act o retrieving data or acomputer le.
2. Acknowledge: To ocially recognize,admit, or accept receipt o an object orinormation.
3. Alert: To make someone aware o apossible danger or diculty.
4. Calculate: To compute mathematically.
5. Create: To produce something, as in aprinted report or electronic copy.
6. Discontinue: To stop or end an activity thatis planned or is happening regularly; alsoto remove an element rom existing patientinormation, such as an allergy rom anallergy list.
7. Document: To create a record o acts,events, symptoms, or ndings.
8. Implement: To put into eect or action.9. Notiy: To inorm or warn ocially to make
something known.
10. Order: An instruction or request to bring,supply, perorm, or activate something.
11. Perorm: To carry out an action oraccomplish a task, especially one requiringcare or skill.
12. Receive: To receive or take somethingprovided.
13. Recommend: To suggest something asworthy o being accepted use or done.
14. Reconcile: To make two or more potentiallyconficting things consistent or compatiblesuch that inconsistencies are resolved or
explained. Reconciliation can be perormed
with a wide range o content elements;some examples include medication lists,problem lists, allergy lists, patient demo-graphics, and social history. Specically,medication reconciliation is identied asa meaningul use criterion: medicationreconciliation is the process o comparinga patients medication orders to all o themedications that the patient has beentaking. This reconciliation is done to avoidmedication errors such as omissions, dupli-cations, dosing errors, or drug interactions.It should be done at every transition o carein which new medications are ordered orexisting orders are rewritten. Transitions incare include changes in setting, service,practitioner, or level o care.21
15. Remind: To cause someone to rememberor think o something, such as to take aspecic action to maintain or improvehealth.
16. Report: To give detailed inormation about
results o aggregate research, analysis, orinvestigations.
17. Review: To examine something critically tomake sure it is adequate, accurate, andcorrect and to determine i new actionsshould be undertaken.
18. Stratiy: To divide or arrange into classes,castes, or social strata into a series ograded statuses.
19. Transmit: To communicate a message,
inormation, or news.20. Update: To provide someone or something
with the most recent inormation orwith more recent inormation than waspreviously available.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
17/65
National Quality Forum
10 National Quality Forum
Examples of MeasuringHealth IT UseExamples o clinical decision support (CDS),e-prescribing, order sets, and clinical summa-ries are provided below to demonstrate howthe Health IT Utilization Assessment Frame-workcan enable measurement o eectiveuse in EHRs and support the meaningul useobjectives. Each example depicts a sample o
utilization data elements but does not representall inormation required or the example. Theexamples are not NQF-endorsed measures,nor do they refect current measures o healthIT utilization in development. Measure devel-opment was outside the scope o the ExpertPanels activities and NQFs domain; however,the Expert Panel recommends the developmento eective health IT use measures to improvequality and saety o care (see Recommenda-
tions and Future Work, below).
Example #1: Clinical Decision Support
Alert overrides are evidence that, or someclinicians, CDS rules can be ineective andpotentially viewed as a nuisance rather thanas an enabler o improved care delivery.Overrides alone, however, do not tell the entirestory because providers may comply with CDS
guidelines ater a latent period.22 Thereore,the steps or measuring CDS usage andeectiveness must account or the act thataction resulting rom a clinical alert (e.g., anew medication order) may not occur untilsometime ater the alert is presented to the user(potential latency). Unless the alert is about an
issue immediately lie threatening to a patient,
clinicians may respond at the most appropriatetime in their own workfow, rather than at thetime the alert res.
In 2009 NQFs Clinical Decision SupportExpert Panel developed a classication ortaxonomy or CDS workfow, which waspublished in December 2010.23 Figure 4 dis-plays the taxonomys our components, each owhich is a basic step that must be programmedinto an EHR to initiate and ollow through with
a CDS rule: triggers, input data, interventions,and action steps. This gure shows initiationo a rule based on a trigger, access o inputdata, provision o an intervention, and recom-mendation o an action step. In many casesthe completion o one CDS rule can providethe trigger or the next rule in a chain o eventsto complete a complex process, as shown inthe gure.
Triggers (solicit, update, act, and time) arethose actions that initiate a CDS rule and caninclude any o the actions in the Health ITUtilization Assessment Framework. The inputdata are elements required by the rule todetermine what action to take; most can bedescribed with the existing QDS Model orexisting data. Interventions (log, display, andnotiy) are also actions in the ramework.Interventions are those things that the inorma-tion system can accomplish. The action steps
(collect inormation, request, acknowledge,communicate, and document) are actions thatare expected o the receiver o the inormationprovided by the rule. Any element o the CDStaxonomy can potentially be identied usingthe ramework.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
18/65
11 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
QDSInputData
Intervention
Trigger
ActionSteps
The CDS Taxonomys four functionalcategories:
1. The triggerinitiates a CDS rule.
2. The input data are represented by thecomponents o the QDS data types.
3. Interventions include the possible actionsthe inormation system can take to deliverinormation.
4. The action steps are actions a receiver othe inormation can perorm.
In a given cycle o a CDS rule, any input data,intervention, or action step may initiate a newtrigger and launch a new CDS rule.
Figure 4: NQF CDS Taxonomy
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
19/65
National Quality Forum
12 National Quality Forum
Trigger
Input Data
Intervention
Action Step
CDS TAXONOMY HEALTH IT UTILIZATION ASSESSMENT FRAMEWORK
Action Content Actor(a) Individual(b) System
Figure 5: Relationship between the NQF CDS Taxonomy and the NQFHealth IT Utilization Assessment Framework (template)
The our components o the CDS taxonomy are triggers, input data, interventions, and action steps. Theidentication o any CDS element requires an action, content, and actor, as described in the Health ITUtilization Assessment Framework. In the ollowing gures, the CDS taxonomy components and potentialassociated actions, content, and actors are displayed or two rules or opportunities or CDS in a clinicalworkfow: Rule A and Rule B.
Figure 5 displays a template that will demon-
strate or two CDS rules, or instances o CDSopportunities in a clinical workfow, how the
CDS taxonomy components include actors,
action, and content.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
20/65
13 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
Figures 6 and 7 represent the two CDS scenarios
or the CDS example: Rules A and B. Rule Aprovides guidance or modiying or discontinu-ing medication i a patients serum creatinine
rises above 0.5 mg/dL over a specied time
period, which suggests a change in renalunction (Figure 7). Rule B and its taxonomyare depicted in Figure 7.
Posting o serum creatinineresults with change over 3 dayso 0.5 mg/dL
Trigger
Medications accessed or renal-sensitive medicationsInput Data
Patient on aminoglycosidesand NSAIDsIntervention
Request order modifcationand/or documentationAction Step
CDS TAXONOMY HEALTH IT UTILIZATION ASSESSMENT FRAMEWORK
Action Content Actor
(a) Individual(b) System
Document Serum creatinine b) System(EHR)source is clinicallaboratory
Access Active medication listCheck or value set o renal
sensitive medications
b) System(EHR)source is activemedication list
Notiy Notiy Alert user to alteredpatient state
b) System(EHR)
Case 1: Request Communicate
Suggest Order modifcation Document Follow-up renal statusassessment
a) Provider
Figure 6: CDS Example Rule A
Alert the responsible provider i the serum creatinine rises above 0.5 mg/dL over 3 days when the patientis taking aminoglycosides and/or nonsteroidal anti-infammatory drugs (NSAIDS), and suggest an order to
modiy renal toxic medications and provide documentation.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
21/65
National Quality Forum
14 National Quality Forum
Rule Trigger:Rule A initiates with a trigger o creatinine elevation
o 0.5 mg/dL occurring over 3 days. As shown in the gure, thetrigger is storage o a new laboratory result and its comparison toprior results. In this case, the action is paired with the actorEHR asthe clinical laboratory (the source o the data) posts, or documentsa creatinine to the laboratory results section o the EHR. The contentrequired is the serum creatinine delta (change in value o 0.5 mg/dL). The ramework triplet is: document, actor(EHR), and content(serum creatinine delta).
Rule A Trigger DataAction: Document (Post)Actor: EHRContent: Laboratory result
Rule A Action Step DataAction: Order, document
Actor: Provider
Content: Medication order,Justifcationdocumentation
Rule A Input Data
Action: AccessActor: EHRContent: Active medication
Rule A Intervention DataAction: NotiyActor: EHRContent: Order, documentation
template
Each o the action steps may be completed asprovided, modied, or ignored. Because theactions may occur at uncertain latency aterthey are oered to the provider, a ollow-uprule exemplar is presented using a latentperiod as the trigger.
The CDS example progresses to Rule B,displayed in Figure 8, which is set to triggerbased on timing (elapsed time).
Rule Input Data: Rule A next searches or input data to determine
i the patient is on active renal-sensitive medications. The utilizationelement action is access, the actoris the EHR, and the contentis aset o medications.
Rule Intervention: Rule A intervention is the utilization action notiy,the actoris the EHR, and the contentis a documentation template toidentiy a need or NSAIDs or an order to lower aminoglycosidedosage and order to discontinue NSAIDs.
Rule Action Step: Rule A action steps are the items noted above aspart o the intervention. At this point in the process, the utilizationactions are either order or document, the actors are the treatingproviders, and the contentis the specic documentation or thespecic order.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
22/65
15 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
Time-generated trigger six hoursater interventionTrigger
Order modifcationDocumentation o reason or NSAIDsInput Data
No modifcation to orderor documentationIntervention
Request order modifcation
and/or documentationAction Step
CDS TAXONOMY HEALTH IT UTILIZATION ASSESSMENT FRAMEWORK
Action Content Actor(a) Individual(b) System
Time Intervention occurrence b) System (EHR)
Access Order modifcationDocumentation
b) System (EHR)Source: medica-tion orders,updates to activemedication list,documentation
Notiy NotiyProvider,Content=Reminders
b) System (EHR)
Request,Communicate
Suggest Order modiy
Documentation Follow-up renal statusassessment
a) Provider
Figure 7: CDS Example Rule B
The CDS example, with Rule B, presumes that the action steps in Rule A have not occurred; the rule useselapsed time as a trigger.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
23/65
National Quality Forum
16 National Quality Forum
Rule B Trigger DataAction: Elapsed timeActor: EHRContent: Order modifcation,
documentation,ollow-up renal statusassessment
Rule Trigger:Rule B initiates the trigger generated by the interven-
tion produced by Rule A, notiy. The latency period or this exampleis six hours, but it could be set to any latency window. As shown inthe gure, the trigger is an action that is identied based on elapsedtime. In this example, the action is paired with the actor(EHR) asthe intervention occurs, implying that the intervention action mustbe logged in such a way to provide data. The contentrequired arethe items identied in the action steps o Rule A (order modication,documentation, ollow-up renal status assessment), along withassociated inormation, metadata, indicating the individuals to whomthe notication was sent, the time, and the action steps provided in
the notication. The ramework triplet is: action (elapsed time),actor(EHR), and content(presence o action step rom Rule Aordermodication, documentation, ollow-up renal status assessment).
Rule B Action Step Data
Action: Order, Document
Actor: Provider
Content: Medication order,Justifcationdocumentation
Rule B Input DataAction: AccessActor: EHRContent: Order, Document
Rule B Intervention Data
Action: AccessActor: EHRContent: Order, Document
Rule Input Data: Rule B next searches or input data to determine ithe expected documentation, new order, or order modication hasoccurred. The action is access, the actoris the EHR, and the contentis one o a set o options (order, documentation).
Rule Intervention: Rule B intervention is the utilization action notiy,the actoris the EHR, and the contentis the same options provided
in Rule A (i.e., a documentation template to identiy a need orNSAIDs or an order to lower aminoglycoside dosage and order todiscontinue NSAIDs).
Rule Action Step: Rule B action steps are the same items noted inRule A, that is, those action steps provided as part o the intervention.At this point in the process, the utilization actions are either order ordocument, the actors are treating providers, and the contentis thespecic documentation or the specic medication order. As withRule A, each o the action steps may be completed as provided,
modied, or ignored.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
24/65
17 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
The rule output is an orderor aminoglycosides witha lower dosage, increasedrequency, or discontinuation.The actionis the order, theactoris the provider, and thecontentis an aminoglycosidemedication provided in thetaxonomy RxNorm. Specifcattributes o the medication
or this example will alsoinclude dose and requency.A time/date stamp isrequired.(Figure 8a) (Figure 8b)
The rule output is justifcationo continuation o medication(NSAID). The actionis thedocumentation, the actoris the provider, and thecontentis justifcation contextprovided in the taxonomySNOMED-CT. A time/datestamp is required.
RxNorm566693, 566649, 566692. 905149,
905144, 568521, 540728,573117...
(Aminoglycoside)
Figure 8a and 8b: Utilization data elements for determining effective CDS
Order
Medication Actor:Provider
AttributesTime/date stamp
DocumentNSAID Usage
Actor: Provider
AttributesTime/date stamp
SNOMED-CTTM
Xxxxx, xxxxxx, xxxxx. ...
(NSAID UsageJustifcation)
Measuring CDS EectivenessDetermining that CDS rules are eectiverequires careul analysis. Measurement oadherence to a rule oten requires creation o
careully constructed queries based on howeach clinical data repository is constructed. Theramework provides direction to standardizethis process by dening data context or action,actor, and content. In the examples above,the expected output is an entered order ordocumentation. Figures 8a and 8b indentiysample utilization data elements or determiningeective CDS.
Using these two new utilization data elements
(depicted in Figures 8a and 8b) in a measurecan result in the incorporation o existing QDSdata elements into the measures denominatorto dene the population and the incorporationo the utilization elements to determine thenumerator.
We can depict the relevant measure ohealth IT use o CDS as ollows.
As shown in Figure 9, a measure o CDSeectiveness can identiy the population o
patients in the denominator using existing QDSconcepts: 1) the Laboratory test result: serumcreatinine identied by a code list o LOINCcodes, with a result indicating a change 0.5mg/dL, and either or both o 2) Medicationactive: aminoglycosides identied by a codelist o RxNorm codes, or 3) Medication active:NSAID medications identied by a code listo RxNorm codes. Each o these denominatorelements uses the QDS Model as previously
described. The numerator contains new dataelements to indicate that either o the ollowinghas occurred: 1) an order action by a provideractoror aminoglycoside (using RxNorm)contentor 2) a documentation action by aprovider actoror reason and justicationcontent.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
25/65
National Quality Forum
18 National Quality Forum
AND
Aminoglycosides
ActiveMedication
RxNorm566693, 566649, 566692,905149, 905144, 568521,
540728, 573117...
(Aminoglycoside)
Serum Creatinine
Laboratory TestResult[Change 0.5]
LOINC82565, 82565, 2160-0. ...
(SerumCreatinine)
NSAID
ActiveMedication
RxNorm5577785, 566095, 567720,567719, 849437, 849727,
849736, 849479. ...
(NSAID)AND
AND/OR
QDS Data Element QDS Data Element QDS Data Element
The gure represents a sample o numerator and denominator elements present in the example measure oresponsiveness (order and/or documentation) to an alert about elevated serum creatinine in the setting orenally toxic medications (aminoglycosides or NSAIDS) = (count o aminoglycoside and/or NSAID orders)and/or (documentation)/patients with serum creatinine rising above 0.5 mg/dL over 3 days and takingeither aminoglycoside or NSAIDs.
Figure 9: An example measure of CDS effectiveness, with utilization data elements
RxNorm566693, 566649, 566692.905149, 905144, 568521,
540728, 573117...
(Aminoglycoside)
Order
Medication Actor:Provider
AttributesTime/date stamp
SNOMED-CTTM
Xxxxx, xxxxxx, xxxxx. ...
(NSAID UsageJustifcation)
DocumentNSAID Usage
Actor: Provider
AttributesTime/date stamp
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
26/65
19 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
Example #2: e-PrescribingThis example o measuring health IT use exam-ines generating and transmitting permissibleprescriptions electronically (e-prescribing).To measure that prescriptions are generatedand transmitted electronically requires anumber o data elements, or example, theprescription is generated, completed, and sent,and receipt is acknowledged. There are alsocontributing actors that can limit the scope oprescriptions that are available or electronicprescription. Such limiting actors includepatient preerence or a specic pharmacy,policies or rules regarding e-prescribing orcontrolled medications, and availability o
receiving systems or the prescription. Some o
the detailed data elements and limiting actorsare presented in Box 2.
This example is designed to highlight thedata elements that might be required orperormance reporting. Figure 10 identiesthe denominator as all patients who have hadencounters with the clinician to determine theappropriate clinician to whom to attribute theclinical care. Encounter: ambulatory is anexisting QDS data type, and the accompanyingcode list (value set) can constrain the encountersto those appropriate to the measure. In thisexample, the encounter concepts are codiedusing SNOMED-CT.
Box 2: e-Prescribing
Data elements for e-Prescribing:
The prescription was generatedThe prescription was transmitted/sent
The prescription was received (capability at the receiving end to handle the transmittal,e.g., a log showing the order was actually received)
Transmittal was acknowledged
System characteristics regarding policy Rules regarding controlled substances Availability o e-prescribing (due to state law, local statute, etc.)
Attributes of e-Prescribing required for data analysis:
1. All prescriptions written by the ambulatory practice (medication orders)
2. Method o prescribing (e.g., e -prescription, ax, written)
3. Type o prescription (e.g., drug classcontrolled substance)
4. Patient preerence or e-Prescription vs. other workfow
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
27/65
National Quality Forum
20 National Quality Forum
Also required in the denominator is anyorder (the action) by a clinician (the actor)
or a medication (the content). Thus the newutilization data element in the denominatoris the medication order action-actor-contenttriplet. The numerator is composed o expectedelements, that is, the prescription was transmit-ted, acknowledged by the pharmacy system,and acknowledged by the pharmacist. The
exclusion is shown using the existing QDS datatype o patient preerence. Table 1 lists the
utilization data types in this example and theirassociated actions, actors, and content.
Additional exclusions will likely be necessaryto handle system characteristics such as localpolicy regarding e-prescription or controlledmedications (exclusion = all controlled medica-tions) and other actors.
Transmit
Medication orderAc-tor: Physician
ATTRIBUTES1. Medication ID
RxNorm
ANY
(ANY)AND
QDS Data Element
QDS Data Element
Acknowledge
Medication orderActor: Pharmacy
SystemATTRIBUTES
1. Medication ID
RxNorm
ANY
(ANY)
Acknowledge
Medication orderAc-tor: Pharmacist
ATTRIBUTES1. Medication ID
RxNorm
ANY
(ANY)
Preference
Patient Preerence
ATTRIBUTES1. Inability to receive eRX
SNOMED-CTTM
264372000, 284748001...
(Pharmacy)AND
AND
NOT
Encounter
AmbulatoryActor: Clinician
SNOMED-CTTM
30872009, 185316007, 185346005,185317003, 390906997, 185318008,
185345009, 185347001, 185349003...
(Encounter)
Order
MedicationActor: Physician
RxNorm
ANY
(ANY)AND
Figure 10: An example measure of e-prescribing, with utilization data elements
The gure represents a sample o numerator and denominator elements present in the example measure ogenerating and transmitting permissible prescriptions electronically.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
28/65
21 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
Example #3: Order SetThis example o measuring health IT use ex-amines using evidence-based order sets andcomputerized provider order entry (CPOE). Tomeasure that an evidence-based order set isused within CPOE requires a more complexset o data elements than described in Example2. Order sets may be standardized withinorganizations. Even i they are derived rombest-practice order sets published by knowl-edge vendors or premier academic institutions,the order sets are requently modied locallyand updated. Thereore, a clear denition oan order set is required. This example will notaddress specic order set content. Rather, itwill ocus on the inormation required about anorder set (the metadata) that would assist withdeveloping a measure o utilization.
Order sets are oten composed o multiple
orders, some o which are essential to manag-ing the condition or process at hand. Otherorders within the set are based on convenience,or example, an inpatient order that includes,in addition to medications, tests, and treatmentsor a specic condition, as needed orders ormedication to assist with sleep, toileting unc-tions, and minor pain control. Use o an orderset implies that the set is accessed and entered
into the EHR or processing or any given pa-tient. Any one or many o the orders within a setcan be de-selected, and only one or a ew canbe selected or processing. Thereore, use o anorder set requires denition to enable measure-ment. Here, it is assumed that each essentialorder (evidenced-based orders that pertain tothe condition evaluated) is entered as is, modi-ed, or removed rom the set with documented
justication. Some o the detailed data elementsand limiting actors are presented in Box 3.
Table 1: Utilization data concepts and their associated actions, actors, and contentfor Example 2
Health IT UtilizationData Concepts Action Actor Content
Medication order Order Clinician Medication
Transmit medication order Transmit Clinician Medication order
Acknowledge medication order Acknowledge Pharmacy inormation system Medication order
Acknowledge medication order Acknowledge Pharmacist
Box 3: Order Sets
Data elements for order sets:
1. Essential orders in the order set can bespecically identied
2. Acceptable reasons or avoidance ospecic essential orders are provided
3. Alternatives to essential orders can beidentied
4. Patient characteristics requiring orderset are clearly identied
5. System characteristics6. Availability o order setsacceptable
sources
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
29/65
National Quality Forum
22 National Quality Forum
This example is designed to delineatethe data elements that might be required orperormance reporting. Figure 11 identiesan order set or patients with coronary arterydisease (CAD), diabetes, and moderate orsevere let ventricular systolic dysunction(LVSD), or heart ailure.
The denominator species each o the condi-tions that must be present in the same patient toinclude him or her. It also excludes all patientswho have had heart transplants. Each o thesedata elements can be identied using the QDSdata types with existing inormation in theEHR. The numerator elements, however, requiredata indicating that each element has beenprocessed and is tagged as an essential order
within the set. In this example, all essentialorders or acceptable exclusions are requiredto meet the numerator requirements. Elementsinclude an order or angiotensin convertingenzyme inhibitor (ACEI) or angiotensin receptorblocker (ARB) medication, or allergy to both,an order or a laboratory test (BNP), and an
order or diet education.This example also highlights a need or
new inormation about the data (metadata),specically an indication that each order is apart o the order set and that each element isan essential order within that set. The exampleis presented to review the use o the rameworkas a tool to construct measures.
OrderMedication ACEIActor: Physician
ATTRIBUTES
1. Order Set ID2. Essential Order
RxNorm204404, 311353, 311354,314076, 314077, 205325,
197884,...
(ACEI)OR Order
Medication ARBActor:PhysicianATTRIBUTES
1. Order Set ID2. Essential Order
RxNorm5577785, 153822, 577787,153823, 639439, 577776,
639543, 639537,...
(ARB)
AND
NOT
Medication AllergyACEIandARB
MedicationAllergy
1. Order Set ID2. Essential Order
RxNorm5577785, 153822, 577787,153823, 639439, 577776,
639543, 639537,...
(ARB)
QDS Data Element
AND Order Lab Test
BNP
Actor: PhysicianATTRIBUTES
1. Order Set ID2. Essential Order
LOINC30934-4,...
(BNP)
ORDER SET (only essential orders shown)
AND Order Diag. Test
Cardiac Echo
Actor: PhysicianATTRIBUTES
1. Order Set ID2. Essential Order
SNOMED-CTTM
51469006, 5216004,...
(CardiacEcho) A
ND
ANDOrder Intervention
Weight Monitoring
Actor: PhysicianATTRIBUTES
1. Order Set ID2. Essential Order
SNOMED-CTTM
307818003,...
(WeightMonitoring)
Order Communication(Diet Education)
Actor: PhysicianATTRIBUTES
1. Order Set ID2. Essential Order
SNOMED-CTTM
11816003, 226069004,...
(DietEducation)
ORDER MedicationActor: Physician
RxNormANY
(Any)
QDS Data Element
AND Diabetes
ConditionActive
SNOMED-CTTM236367002, 399144008,315051004, 73211009,11530004, 46635009...
(Diabetes)
QDS Data Element
AND Moderate or
Servere LVSDCondition
Active
SNOMED-CTTM46263006, 429589006,
395704004, 430396006,...
(Mod or SevereLVSD)
QDS Data Element
Heart Transplant
ProcedurePerormed
ICD-937.51, 33.6,...
(HeartTransplant)
QDS Data Element
Figure 11: An example measure of an order set for patients with CAD,diabetes, and moderate or severe LVSD, with utilization data elements
The gure represents a sample o numerator and denominator elements present in the example measure o
evidence-based order sets (medication, laboratory test, diagnostic test, intervention, and communication).
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
30/65
23 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
Table 2: Utilization data concepts and their associated actions, actors, and content forExample 3
Health ITUtilizationData Concepts Action Actor Content
Order Order Clinician Medicationangiotensin converting enzymeinhibitor (ACEI)
Order Order Clinician Medicationangiotensin receptor blocker (ARB)
Document Document Clinician Medication allergyACEI and ARB
Order Order Clinician Laboratory test (BNP)
Order Order Clinician Interventionweight monitoring
Order Order Clinician Interventiondiet education
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
31/65
National Quality Forum
24 National Quality Forum
Example #4: Clinical SummaryThere are dierent ways to consider measure-ment o sharing summary records with patientsand incorporating patient preerences in caredecisions. For this example a set o options ormeasurement o processes within the clinician-patient workfow is included. These optionsinclude: 1) transmission o clinical data,2) receipt o clinical data, 3) system acknowl-edgment o receipt o clinical data, 4) patientacknowledgment o receipt o clinical data,and 5) update o clinical data with patientpreerences. Thereore, this example includes
separate metrics to manage each o these ve
concepts.
Transmit Clinical DataThis metric is intended to identiy that a set odata is transmitted rom a clinical site (sender)to a patient (receiver). Some o the detaileddata elements are presented in Figure 12.Table 3 lists the utilization data concepts andtheir associated actions, actors, and content tomeasure transmitting clinical data.
A
NDTransmitActive Medication
Actor: ClinicianATTRIBUTES
1. Medication ID
RxNormANY
(Any)
TransmitProblems
Actor: ClinicianATTRIBUTES
1. Active2. Inactive3. Resolved
SNOMED-CTTM
ANY
(Any) A
ND
A
ND
A
ND
EncounterPerformedAmbulatoryencounter
SNOMED-CTTM30872009, 185316007, 185348005,185317003, 390906997, 185318008,
185345009, 185347001, 185349003...
(Encounter)
QDS Data Element
AND Individual
Characteristic
Patient ethnicity
CDCETHNICITY VALUE SET
(Ethnicity)
QDS Data Element
AND Individual
Characteristic
Patient race
CDCRACE VALUE SET
(Race)
QDS Data Element
AND Individual
Characteristic
Patient language
THE INTERNETSOCIETY LANGUAGE
VALUE SET(Language)
QDS Data Element
TransmitMedication Allergy
Actor: ClinicianATTRIBUTES
1. Medication ID
RxNormANY
(Any)
TransmitCare Goals
Actor: ClinicianATTRIBUTES
1. Source = Patient
SNOMED-CTTM
ANY
(Any)
TransmitFunctional StatusActor: Clinician
ATTRIBUTES
1. Source = Patient
SNOMED-CTTM
ANY
(Any)
TransmitPatient PreferenceInability to use eRx
Actor: PatientATTRIBUTES
1. Source = Patient
SNOMED-CTTM
264372000, 284748001,...
(Pharmacy)A
ND
Figure 12: An example measure of transmitting clinical data, withutilization data elements
The gure represents a sample o numerator and denominator elements present in the example measure.This example examines transmission o clinical data (medications, problems, medication allergies, caregoals, unctional status) based on patient preerence; the example is stratied by ethnicity, race, andlanguage preerence.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
32/65
25 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
Table 3: Utilization data concepts and their associated actions, actors, and content fortransmitting clinical data
Health ITUtilizationData Concepts Action Actor Content
Transmit Transmit Clinician Active medication list
Transmit Transmit Clinician Active problem list
Transmit Transmit Clinician Active allergy list
Transmit Transmit Clinician Care goals (source o data = patient)Transmit Transmit Clinician Functional status (source o data = patient)
Transmit Transmit Clinician Patient preerences (source o data = patient)
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
33/65
National Quality Forum
26 National Quality Forum
Receive Clinical Data
This metric is intended to identiy that the seto data rom a clinical site (sender) is receivedby a patient (receiver). Some o the detaileddata elements are presented in Figure 13. The
data elements or this example are: 1) clinical
report received and 2) acknowledgement oreceipt o clinical data transmitted. Table 4 liststhe utilization data concepts and their associ-ated actions, actors, and content to measurereceiving clinical data.
ANDReceive
Active MedicationActor: Patient
ATTRIBUTES1. Medication ID
RxNormANY
(Any)
ReceiveProblems
Actor: PatientATTRIBUTES1. Active2. Inactive3. Resolved
SNOMED-CTTM
ANY
(Any) AND
AND
AND
EncounterPerformedAmbulatoryencounter
SNOMED-CTTM30872009, 185316007, 185348005,185317003, 390906997, 185318008,
185345009, 185347001, 185349003...
(Encounter)
QDS Data Element
AND Individual
CharacteristicPatient ethnicity
CDCETHNICITY VALUE SET
(Ethnicity)
QDS Data Element
AND Individual
CharacteristicPatient race
CDCRACE VALUE SET
(Race)
QDS Data Element
AND Individual
CharacteristicPatient language
THE INTERNETSOCIETY LANGUAGE
VALUE SET(Language)
QDS Data Element
ReceiveMedication Allergy
Actor: PatientATTRIBUTES
1. Medication ID
RxNormANY
(Any)
ReceiveCare Goals
Actor: PatientATTRIBUTES
1. Source = Patient
SNOMED-CTTM
ANY
(Any)
ReceiveFunctional Status
Actor: PatientATTRIBUTES
1. Source = Patient
SNOMED-CTTM
ANY
(Any)
ReceivePatient PreferenceInability to use eRx
Actor: PatientATTRIBUTES
1. Source = Patient
SNOMED-CTTM
264372000, 284748001,...
(Pharmacy)AND
The gure represents a sample o numerator and denominator elements present in the example mea-sure. This example examines receipt o clinical data (medications, problems, medication allergies, caregoals, unctional status) based on patient preerence. The example is stratied by ethnicity, race, andlanguage preerence.
Figure 13: An example measure of receiving clinical data, with utilizationdata elements
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
34/65
27 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
Table 4: Utilization data concepts and their associated actions, actors, and content forreceiving clinical data
Health ITUtilizationData Concepts Action Actor Content
Receive Receive Clinician Active medication list
Receive Receive Clinician Active problem list
Receive Receive Clinician Active allergy list
Receive Receive Clinician Care goals (source o data = patient)Receive Receive Clinician Functional status (source o data = patient)
Receive Receive Clinician Patient preerences (source o data = patient)
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
35/65
National Quality Forum
28 National Quality Forum
Acknowledge Receipt o Clinical Data (System)This metric is intended to identiy that thesystem receiving the clinical set o data hasacknowledged receipt. This type o acknowl-edgement is similar to a ax report o successultransmission. It does not indicate the patienthas seen or reviewed the summary. Some othe detailed data elements are presented inFigure 14. The data elements or acknowledging
receipt o clinical data (by the system) are:
1) clinical report received and 2) acknowledge-ment o receipt o clinical data transmitted (bythe system). Table 5 lists the utilization dataconcepts and their associated actions, actors,and content to measure acknowledgement oreceipt o clinical data (by the system).
ANDAcknowledge
Active MedicationActor: EHRATTRIBUTES
1. Medication ID
RxNormANY
(Any)
AcknowledgeProblemsActor: EHRATTRIBUTES
1. Active2. Inactive3. Resolved
SNOMED-CTTM
ANY
(Any) AND
AND
AND
EncounterPerformedAmbulatoryencounter
SNOMED-CTTM30872009, 185316007, 185348005,185317003, 390906997, 185318008,
185345009, 185347001, 185349003...
(Encounter)
QDS Data Element
AND Individual
CharacteristicPatient ethnicity
CDCETHNICITY VALUE SET
(Ethnicity)
QDS Data Element
AND Individual
CharacteristicPatient race
CDCRACE VALUE SET
(Race)
QDS Data Element
AND Individual
CharacteristicPatient language
THE INTERNETSOCIETY LANGUAGE
VALUE SET(Language)
QDS Data Element
AcknowledgeMedication Allergy
Actor: EHRATTRIBUTES
1. Medication ID
RxNormANY
(Any)
AcknowledgeCare GoalsActor: EHRATTRIBUTES
1. Source = Patient
SNOMED-CTTM
ANY
(Any)
AcknowledgeFunctional Status
Actor: EHRATTRIBUTES
1. Source = Patient
SNOMED-CTTM
ANY
(Any)
AcknowledgePatient PreferenceInability to use eRx
Actor: EHRATTRIBUTES
1. Source = Patient
SNOMED-CTTM
264372000, 284748001,...
(Pharmacy)AND
Figure 14: An example measure of acknowledging receipt of clinical data(by the system), with utilization data elements
The gure represents a sample o numerator and denominator elements present in the example measure.This example examines acknowledging receipt (system) o clinical data (medications, problems, medication allergies, care goals, unctional status) based on patient preerence. The example is stratied by ethnicity,race, and language preerence.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
36/65
29 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
Table 5: Utilization data concepts and their associated actions, actors, and content forthe example measure of acknowledging receipt of clinical data (by the system)
Health ITUtilizationData Concepts Action Actor Content
Acknowledge Acknowledge EHR System Active medication list
Acknowledge Acknowledge EHR System Active problem list
Acknowledge Acknowledge Clinician Active allergy list
Acknowledge Acknowledge EHR System Care goals (source o data = patient)Acknowledge Acknowledge EHR System Functional status (source o data = patient)
Acknowledge Acknowledge EHR System Patient preerences (source o data = patient)
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
37/65
National Quality Forum
30 National Quality Forum
Acknowledge Receipt o Clinical Data (Patient)This metric is intended to identiy that thepatient receiving the clinical set o data hasseen and acknowledged receipt. This type oacknowledgement requires human interven-tionopening and agreeing that the summaryhas been received. Some o the detailed dataelements are presented in Figure 15. The dataelements or the example measure o acknowl-
edging receipt o clinical data (by the patient)
are: 1) clinical report reviewed and 2) acknowl-edgement o receipt o clinical data transmitted(by the patient). Table 6 lists the utilization dataconcepts and their associated actions, actors,and content or measuring acknowledgemento receipt o clinical data (by the patient).
ANDAcknowledge
Active MedicationActor: Patient
ATTRIBUTES1. Medication ID
RxNormANY
(Any)
AcknowledgeProblems
Actor: PatientATTRIBUTES
1. Active2. Inactive3. Resolved
SNOMED-CTTM
ANY
(Any) AND
AND
AND
EncounterPerformedAmbulatoryencounter
SNOMED-CTTM30872009, 185316007, 185348005,185317003, 390906997, 185318008,
185345009, 185347001, 185349003...
(Encounter)
QDS Data Element
AND Individual
CharacteristicPatient ethnicity
CDCETHNICITY VALUE SET
(Ethnicity)
QDS Data Element
AND Individual
CharacteristicPatient race
CDCRACE VALUE SET
(Race)
QDS Data Element
AND Individual
CharacteristicPatient language
THE INTERNETSOCIETY LANGUAGE
VALUE SET(Language)
QDS Data Element
AcknowledgeMedication Allergy
Actor: PatientATTRIBUTES
1. Medication ID
RxNormANY
(Any)
AcknowledgeCare Goals
Actor: PatientATTRIBUTES
1. Source = Patient
SNOMED-CTTM
ANY
(Any)
AcknowledgeFunctional Status
Actor: PatientATTRIBUTES
1. Source = Patient
SNOMED-CTTM
ANY
(Any)
AcknowledgePatient PreferenceInability to use eRx
Actor: PatientATTRIBUTES
1. Source = Patient
SNOMED-CTTM
264372000, 284748001,...
(Pharmacy)AND
Figure 15: An example measure of acknowledging receipt of clinical data(by the patient), with utilization data elements
The gure represents a sample o numerator and denominator elements present in the example measure.This example examines acknowledging receipt (patient) o clinical data (medications, problems, medicationallergies, care goals, unctional status) based on patient preerence. It is stratied by ethnicity, race, andlanguage preerence.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
38/65
31 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
Table 6: Utilization data concepts and their associated actions, actors, and content forthe example measure of acknowledging receipt of clinical data (by the patient)
Health ITUtilizationData Concepts Action Actor Content
Acknowledge Acknowledge Patient Active medication list
Acknowledge Acknowledge Patient Active problem list
Acknowledge Acknowledge Patient Active allergy list
Acknowledge Acknowledge Patient Care goals (source o data = patient)Acknowledge Acknowledge Patient Functional status (source o data = patient)
Acknowledge Acknowledge Patient Patient preerences (source o data = patient)
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
39/65
National Quality Forum
32 National Quality Forum
Update Clinical DataThis metric is intended to identiy that thepatient receiving the clinical set o data hassubmitted updates. Some o the detailed dataelements are presented in Figure 16. The dataelements or the example measure o updating
clinical data are: 1) clinical report reviewed
and 2) update o clinical data transmitted(patient). Table 7 lists the utilization dataconcepts and their associated actions, actors,and content to measure the update o clinicaldata.
ANDUpdate
Active MedicationActor: Patient
ATTRIBUTES1. Medication ID
RxNormANY
(Any)
UpdateProblems
Actor: PatientATTRIBUTES
1. Active2. Inactive3. Resolved
SNOMED-CTTM
ANY
(Any) AND
AND
AND
EncounterPerformedAmbulatoryencounter
SNOMED-CTTM30872009, 185316007, 185348005,185317003, 390906997, 185318008,
185345009, 185347001, 185349003...
(Encounter)
QDS Data Element
AND Individual
CharacteristicPatient ethnicity
CDCETHNICITY VALUE SET
(Ethnicity)
QDS Data Element
AND Individual
CharacteristicPatient race
CDCRACE VALUE SET
(Race)
QDS Data Element
AND Individual
CharacteristicPatient language
THE INTERNETSOCIETY LANGUAGE
VALUE SET(Language)
QDS Data Element
UpdateMedication Allergy
Actor: PatientATTRIBUTES
1. Medication ID
RxNormANY
(Any)
UpdateCare Goals
Actor: PatientATTRIBUTES
1. Source = Patient
SNOMED-CTTM
ANY
(Any)
UpdateFunctional Status
Actor: PatientATTRIBUTES
1. Source = Patient
SNOMED-CTTM
ANY
(Any)
UpdatePatient PreferenceInability to use eRx
Actor: PatientATTRIBUTES
1. Source = Patient
SNOMED-CTTM
264372000, 284748001,...
(Pharmacy)AND
Figure 16: An example measure of updating clinical data, with utilizationdata elements
The gure represents a sample o numerator and denominator elements present in the example measure.This example examines updating clinical data (medications, problems, medication allergies, care goals,unctional status) based on patient preerence. It is stratied by ethnicity, race, and language preerence.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
40/65
33 National Quality Forum
Driving QualityA Health IT Assessment Framework or Measurement
Table 7: Utilization data concepts and their associated actions, actors, and content forthe example measure of update clinical data
Health ITUtilizationData Concepts Action Actor Content
Update Update Patient Active medication list
Active problem list
Active allergy list
Care goals (source o data = patient)
Functional status (source o data = patient)
Patient preerences (source o data = patient)
Similar to the previous examples, newinormation about the data (metadata) isrequired, specically the source (originator) oeach data element, to impart more meaningto the inormation. This example is presentedto review the use o the ramework as a tool toconstruct measures.
Recommendationsand Future WorkThe Health IT Utilization Assessment Frame-workis expected to evaluate eective andmeaningul EHR use and help avoid unintendedconsequences by:
enabling development o measures oeective health IT use;
understanding capabilities o the EHR andother health IT tools to meet meaningul userequirements;
assessing unintended consequences ohealth IT use;
enabling inormation capture as abyproduct o clinical workfows;
enhancing collaboration among health ITvendors, purchasers, implementers, andcertiying bodies by encouraging the use ocommon health IT assessment strategies;
enabling determination o high-priority
health IT use that supports certication oreal-world implementations; and
encouraging clinical eectiveness researchregarding unintended consequences ohealth IT use as well as research todetermine eective health IT utilization.
The ramework put orth in this report provides aunique approach to identiying and measuring:1) the use o health IT applications, 2) whetherthe workfow (driven by the systems user inter-
ace) occurs as designed, and 3) that such useimproves care processes, quality, and saety.This report, however, presents only a rst stepin establishing a standard methodology oridentiying and measuring the eective use ohealth IT.
-
8/8/2019 Driving Quality - A Health IT Assessment Framework for Measurement - NQF
41/65
National Quality Forum
34 National Quality Forum
Currently, standards exist only or EHR
certicationnot or eective health IT use.Certication requirements speciy the activitiesthat each EHR must accomplish and the termi-nology and categorization that should be usedor interoperability (sending inormation romone system to another).24 In the Fi