toward a guideline authoring workbench session #1 may 6, 1999 barry g. silverman, phd
TRANSCRIPT
Penn People
Lab for Informatics &
Intelligent Systems Tech
(LIIST)• Barry Silverman, PhD• Chen• Tony Yuan
Penn Database Group• Peter Bunneman, PhD• Val Tannen, PhD• Arnaud Sahuguet
Realtime Systems
Performance Lab• Insup Lee, PhD• Oleg Sikolski, PhD
Outline
• Purpose & Challenges
• Authoring Process1. Protocol Eligibility
2. Wellness/Prevention
3. Chronic (Ambulatory Care) Guidelines
4. Acute Care Guidelines
5. Critical Pathways
6. Education & Decision Support Materials
• Maintenance Requirements
• Summary
Purpose & Challenges
• Purpose of Today’s Session– Hypothetical “Tour” of finished CADSE
• Challenges to be Resolved Today
Challenges for Today
• Expose the Dream • Work Toward a Unified DSS View• Define Guideline Workbench Components
– Breadth
• Analyze Alternative Implementations (Depth)– Make vs. Buy– Usefulness, Scalability, & Usability
• Synthesize a number of Workbench Requirements
Figure 1 - A Proposal to Merge Knowledge Life Cycle Management, Open Standards, and Integrated Environment
Authoring Test & Eval Maintaining
ElicitationWizards
KB Models,Skeletons
Knowl.Grapher
Parser
HL7 RIM,XML/Kona
CommonMedical TermServer
DebuggingAgents
ClarityChecks
WorkabilityTesters
PerformanceEnhancers
CoherenceVerifiers
Correspon-denceValidators
Guideline ProjectMgt Support Sys.
RqmtsMapper
IRM
ChangeControl
Module History,Documentation
Milestone &WBS Viewer
Open-SW Canonical Form Guideline Repository
Rules for Workflow Templates for Notes Alerts/Remdrs for Orders
Data HandlineSemantic Messaging(Publish, Notify, Remind) Presentation
ExecutionEnvironment
zeus.seas.gwu.edu hera.seas.gwu.edu
poseidon.seas.gwu.edu
prometheus.seas.gwu.edu
INTERNET
Representation Repository
Guidelines RepositoryGuidelines Representation Toolkit Server
and Intelligent Surveys Generator
Client (provider/consumer station)
Guidelines KnowledgeProcessors Server
Installed Software:a.- Windows NT Workstation ver 3.1b.- Netscape Fast Track Server ver. 2.1(hostingautomated guidelines before publishing)c.- Sun Servlet Development Kit and RuntimeEnvironment (Servlet Proxy)d.-Guidelines Representation Toolkit andGuidelines Servletse.-Intelligent Surveys Generator Servlets (usingJESS to process questionnaires branching rules)f.- Guidelines Knowledge Processors Client class(connecting to Guidelines Knowledge ProcessorsServer)g- Sandia Labs.' JESS classesh.- Netscape Navigator ver 3.1
128.164.12.2 128.164.12.6
Installed Software:a.- Windows NT Server ver 3.1(running FTP service)b.- Netscape Enterprise Server ver.??(hosting Guidelines Repository andSurveys)c.- Bulletproof Jagg (ODBC/JDBCgateway)d.-Microsoft SQLServer ver 2.51(hosting Representation Repository,Guidelines Directory, & SurveysRepositories)e.- Netscape Navigator ver 3.1 (asenvironment's Client )
Surveys Repository
Installed Software:a.- Windows NT Workstation ver 4.0b.- Guidelines Knowledge ProcessorsServer (listening in port 18000 andreplaying using ports 1800x x=1...n)
Installed Software:a.- Windows NT Workstation ver 4.0b.- Netscape Communicatior 4.x asWWW browser
Outside word
Installed Software:Any browser compatible withJavaScript ver 3.x
Deployment Map
HFHS Vendor Components
Kn. Mgt. WorkbenchEnterprise Guideline Repository
Clinic-SpecificGuidelineRepos.
Division-SpecificGuidelineRepos.
Enter-prise
Local
Clinic-SpecificDivision-Wide
Kn. Administrators
Clinic-Access toKM Toolkit,Guideline Changes,& OLTP System
CADSE Ingredients (ATP Diagram)
G uide line Type(R e-use)
E lig ilityC rite ria
S ke le ta lP lans
R iskA ssessm ent
G LIFS em antics
E xtens ionsto G L IF
O rderingG u ide lines
W ork flowR u les
A rden- & H L7-C om p lian tK now ledgeM ed ia to rs
F eedbackG u ide le ts
SOAPCharting &
Paths
Visual Programming w /Controlled M edical Terminology
Groupware for Distributed Authoring, Checking, Debugging, Adapting, M aintaining
Decision Support Repository
Human Authors, Testers, M aintainers
C ross-A u thor C on flic t R eso lu tion In te r-G u ide line C on trad ic tion C heck ing
KNOWLEDGE MANAGEMENT GRAND CHALLENGE
National repository to capture/share “practice decision rules” and electronic guidelines (open, reusable, machine form)
JIT Filtered Push, Not Pull
NOTE1: Need intuitive editing yet guaranteed reliability for knowledge-ware.
NOTE2: Local user toggles/adapts desired widgets,
NOTE3: The Infrastructure Challenge (NCIS is assumed)
RQ - Guidelines-in-Use (tackling the authoring component)
• Can a terminology-enabled, visual editor/wizard be created that:– Is easy for medical professionals to use (cognitive fit)?– Supports guideline knowledge representations (e.g, eligibilities,
algorithms, rules, decision tables, etc.)– Causes guidelines to be better written (coherence, clarity,
completeness, workability, etc.)– Permits local revising/adapting to prevailing practice– Helps authors produce guidelines that “make the right thing the
easiest thing”
Knowledge Engineering Paradox
• Expert clinicians know what they don’t know, but don’t know what they know.
• Expertise is situationally triggered– Its hard to do a data dump a priori
• We need to spend this time together to “discover” your knowledge of guideline authoring– there ain’t no shortcut -- clinician time is essential to
design CADSE right
Work Toward a Unified DSS View(requires clinicians to specify it)
•What are you trying to put in front of the clinician-author ???????
•What should they put in front of clinician-practitioners ??????
•AGAIN: KE theory says you aren’t able to elaborate this fully a priori (so lets walk through some situations together)
The CADSE “Dream”
A Tour of what it could be.
(with apologies to Asymetrix Multimedia Toolbook Instructor)
Guidelines Representation Toolkit - Prototype Implementation ver. 1.0 alpha
14
Knowledge Management Workbench - Prototype Implementation
ver. 1.0 alpha (CADSE)
Scenario
Tom, the head of Guideline Development for Best HMO, purchased CADSE in order to develop a suite of guidelines including but not limited to:
1. Protocol Eligibility2. Wellness/Prevention3. Chronic (Ambulatory Care) Guidelines4. Acute Care Guidelines5. Clinical Pathways6. Education & Decision Support Materials
Tom found that CADSE ate up these tasks like a knife through butter.
One thing he particularly liked was CADSE’s breadth as a toolbox:
This interface allowed Tom to quickly complete his range of authoring needs
• Using Templates of Guideline Types
• Selecting Objects From Library (add new ones)
• Setting Objects’ Properties
• Filling Out Object Content
• Adding Behavior (via Script)
• Getting Wizard Help (where needed)
Working with templates
Tom began his guideline authoring using
one of the many templates. A template
is a prebuilt “book” that you customize
by adding your own content. Templates
save time and lend a consistent look and
feel to your guidelines.
In addition to using one of the templates
provided with CADSE, you can create
your own, custom template to distribute
to other guideline authors.
TOP DOWN
GUIDANCE PRINCIPLE
Author your Guideline
After Tom chose a template, he then
added his content. CADSE’s easy-to-
use authoring and development tools
make this task simple.
And CADSE is powerful enough to
handle a wide variety of technical
tasks—from creating and managing
new guideline objects to adapting those
in a reuse library.
PRINCIPLEOF
REUSE
Select objects from CADSE’s Library
He added features to his guideline by
selecting from the wide variety of
objects available in the CADSE
Library.
Library objects have built-in behavior
to handle such tasks as eligibility,
decision tables, workflow and logic
charts, and clinical paths, among
others.
ECOLOGICALARTIFACTS PRINCIPLE
ELICITATION TOOLS CONTAINER(Word processor, WWW browser)
Terminology-Enabled, Visual Guideline Editing Component
GenericTasks Assembly Wizard
Sentence Logic Constructors
Syntax structures
HL7 RIMInterface
GLIF SemanticPrimitives
ARDEN Syntax
Reusable parts libraries interface
Data & knowledge representation & access parts (OO and XML)
Datapath forms
Generic Tasks Parts
•Guidln Qualif•Risk Assmnt•Severity Lev.•Diagnosis•Treatment•Education
Clinician Input Parts
•Parameter tables•Decision tables•Rule structures/MLMs•Intention-based relations•Intermed Guidelines Syntax Structures
Consumer Input Parts
•Patient profile forms•Consumer self-assessment
User Communication
Parts
•email generators•Paging facility•XML generators
GLIF-Electronic Guideline Objects (EGO)and XML/DTDs•ARDEN Curly Braces/Term Server
•HL7 EDI
Semantic & Vocabulary
Servers Interface
Setting object properties
Tom customized objects in his
guideline by setting their properties.
Each object, such as an eligibility
assessment or clinical path, has
properties that define its appearance
and behavior.
Tom could easily set an object’s
properties using the Properties
dialog box or by responding to a
wizard.
POLYMORPHISMPRINCIPLE
(Cmd Keys, DMI or Agents)
Customize guideline behavior without limit using JavaScript (ECMA)
With CADSE’s easy-to-learn
JavaScript, Tom further customized
guidelines to meet unique needs. He
created new objects, customize existing
objects, and programmed guidelines to
exhibit unique behaviors or to simulate
special processes.
If he doesn’t want to write script the
wizard copies his actions and produces
script for him
ADAPTIVITY PRINCIPLE
How to Implement Tom’s CADSE “Dream”
• Analyze Alternative Implementations? – Make vs. Buy(& integrate)– Usefulness, Scalability, & Usability
• Make Option: Case Study– Protégé = dozens of person-years of effort
(scalability & usability still issues!!)– One could spend all the budget and not be done!
The Buy (& integrate) Option
• Identify Best of Breed DSS Authoring Tools– Scalable, usable, generic, open-standards based
• Identify Next Release Change Requirements– Lobby effort needed, evolvable API concept
• Assure Workbench is an Integration Platform– Interface Definitions– Common Presentation Layer Issues– Fill In Missing Components
Goals of a Knowledge Management Workbench
• Top Down Support– Templates
– Reusables
• Sharing Existing Objects– Libraries
• Bottom Up Support– Free-form Creations
– Custom Designs
– Local Adapting
• Capture of New Objects– Learning
– Indexing
NOTE: Guideline Checking is an Extensive Topic for the Workshop
• Group Development, Version Tracking, & Administration
Power vs. Learning Tradeoff Curve
Power of Expression
LearnabilityShallow Workbench(visual, with wizards)
Powerful Workbench(programming lang’s)
Hybrid
How should a workbench support guideline creation efforts?
Pros Cons
One System Desirable,powerful
Never finished,very costly
Many LowLevel Tools
Flexible, righttool for job
Feature explosion,non-integrated?
WizardPrompting
Fast, effective,helpful
Guide to Guidelinesdoesn’t fully exist
ANSWER: A mix of approaches is required (HYBRID)
11Answer/Define
Information Gathering Questions
10View/Identify
Data Collection Needs
9Study/Draw
Temporal Dependency
Flows
3See/Fill In
Textual Annotations & Explanations
4See/Provide Encounter Materials
(Multimedia Links)
8See/Author
Pseudo Rules
6See/Fill out Procedure Triggers
7View/Create
Decision Tables
2Show/Draw Algorithm Diagram
(Adapt reusables)
5View/Add
Consultation Tables
Exchange Protocol Definition Models
Vocabulary Server
Semantics, Syntax, Structure
Models
Critics•Clarity•Completeness•Logic•Lexicon
CanonicalGuidelinesRepository
(CGR)
• Knowledge Mediators
• Info Agents• SNOMED RT• LOINC• CPT 4• NDC
• ARDEN syntax• HL7 RIM • GLIF & EGO
Parsers & Compilers• Syntactic/semantic checkup• Data & knowledge binding• Executable generation
VGR Administration
• Ontology• CBR /Search• Version
Test/Execution Environment
•Workability•Benckmarking/Quality Ass.•Integration (R2Do2,CDR, Careweb)
Guidelines DevelopmentCycle
Task Completion Wizards/Help
Interface Files
1Find/View Existing
Guidelines
Syntactic and Semantic Tools(Servers)
DeploymentTools
(HOLON)
GuidelinesDevelopmentTools (COPE)
TASK LAYER
TOOLS LAYER
Level 1Registration
12Make Guidelines
Computer Operational
Level 2Registration
Level 2
Level 3 Registration
Figure 1 - Overview of the Three Layers of the Knowledge Management: Tasks Ring, Tools Ring, and Repository Core
Outline
• Purpose & Challenges
• Authoring Process1. Protocol Eligibility
2. Wellness/Prevention
3. Chronic (Ambulatory Care) Guidelines
4. Acute Care Guidelines
5. Clinical Pathways
6. Education & Decision Support Materials
• Maintenance Requirements
• Summary
Starting a New Guideline• Pick a Type• Automatically Create a “Container”• Wizard Asks Author for Boilerplate*:
– Authors, Date, etc.– Target population, needs, & cost implications– Rate of change & Practice variation– Evidence (degree of vs. opinion)– Difficulty of development, maintenance, implementation
* - KPNW’s “CPG Policy & Procedures” Paper (7/98), p. 8
The Knowledge Model
• IoM’s 5 Guideline Types (others?)– screening & prevention, – diagnosis & pre-diagnosis mgt of patients, – indications for use of surgical procedures– approp. use of techs/tests for clinical care– guidelines for care of clinical conditions
The Knowledge Model (cont’d)
For each Generic Type of Guideline, Need to Model:
• Skeletal Plans/Chapters & Sections
• Sorted by Workflow Element (Person-type, Sequential Step, Timing Points, etc.)
• Locally Adaptable
Importance of a Layout/eForms Manager
• Adapt all eforms (container GUI, eligibility objects, reusable components)
• Use to Create Many Items– Progress Note Forms– Clinical Path Tables– Educational Materials– Other
• Use to Create Other Tools (e.g., PocketCards)
REQUIRED: Highly Scalable/Usable/Adaptable Forms Layout Manager
Design electronic forms in a Visual Forms Designer & fill them out using:•Web-based Visual Forms Builder •Visual Forms Application Builder •Visual Forms Webpage Plug-in•Workflow for eForm Artifact
Support for other Form Designer Tools and interchange standards
e.g., Visual Forms Designer can also read FormFlow, OmniForm and JetForm forms after they are converted via the Visual Converter.
Container Layout Manager/”Visual Forms Designer”
Sample Visual Designer Screens
Examine some of the screens to assess role of clinician-authorsWww.mmacorp.comvfftw.htm
Make vs. Buy - Layout Manager
Decision CommentJetform orcomparable
Buy Assure open standardsare followed.Need to buildreusables library
Integrationw/Workbench
Make Seamless interfaceneededNon-uniformpresentation layer
Authoring Process:1. Protocol Eligibility
• At 1st Blush Eligibility Seems Straightforward– Inclusion Criteria– Exclusion Criteria (Mutable, Immutable)
• BUT, ….
There seem to be numerous “eligibility” type problems
• Standard inclusion/exclusion lists (“simple” branching)
• Level of care/risk stratification (multiple entry points, multi-workflows, more probabilistic?)
• Caremap progression checking (temporal issues)
• Health risk assessments (consumer-run pre-eligibility checks + lifestyle/self-care advice)
• Tools may differ somewhat for each
What kind of authoring will clinicians do?
• Decision tree? (e.g., Lumina/Spreadsheet)– has a well-grounded semantics– probability handled readily– fuzzy membership
• Visual Object Questionnaire “Programming” (e.g., Toolbook CBT)– may be the ECOLOGICAL artifact– convert to HTML/XML– what about script?
Case Study: “Uncomplicated Cystitis” Protocol Definition
• Eligibility Criteria– Female at or >18 and at or <65 years of age– History previous uncomplicated UTIs only– Classic Sx of <3 days: dysuria, freq. &/or nocturia
• Exclusion Criteria– Concurrent Coumadin Therapy– Pregnant (confirmed or suspicious) by history– Diabetes Mellitus, Renal insufficiency– UTI within last 6 months, or freq. post-coital UTI– Post-partum, less than 2 weeks …..
Advantages and Disadvantageswith Decision Tree (I)
• Advantages– Easy to create (reusable objects & wizards)– Table type input by user is efficient– Probability rollback procedure is a
GROUNDED SEMANTICS (extends GLIF)– Values can be queried from Patient Record – Can be implemented in certain ES shells
Advantages and Disadvantageswith Decision Tree (I)
• Disadvantages– Might not work with missing values– Might not interview at screen for missing values– Visual GUI is feature-rich (FEATURE
EXPLOSION) -- but less tedious than GLIF– How to parse Decision Tree output into GLIF (ES
shell rules are easier to parse)?– Often no visual developer GUI if in an ES shell
Same Case Study - Questionnaire Approach
• Asymetrix Multimedia Toolbook II – Used for Computer Based Training & Kiosks– Competitors - MacroMedia, HyperCard
• Output - HTML and Java
Eligibility Tree - Make vs. Buy Alternatives
PROs CONsGLIFapproach
Merger of DT & QuestionnaireStandards-basedOther vendor usage
Expensive, time & costDuplicating other toolsWill never be as goodHard to maintain
DecisionTreePack
Usable GUIGrnded SemanticsXslates to Sprdsht
Can vendors use it?
QuestionnaireTool
Ecologic ArtifactXslates to HTML
What script runs it?What engine needed?
Authoring Process:2a. Wellness
• What’s involved here? Any more than:– Handout materials (education & DSS)– Show/Discuss Counseling pages– Provide Links to support & chat groups
• What examples?
• If the above process is correct, do authors need website wrapping tools? (eg W4F later)
Authoring Process: 2b. Prevention
• This is like the Government’s “Put Prevention Into Practice” Kits????– Lots of schedules (physicals, regular exams)– Immunization calls/timelines/reminding– Workflow and timelines
• call center
• case manager
• appointment clerk
• nurse
• doc
While this may seem “intuitive”,
• HFHS Complains– Not intuitive– Too hard for their clinicians (WHY? Need to investigate)
• Possibility: No “model” of clinical world to map to– Outpatient clinic metaphor missing (=ecologic artifact)
• Work Performers = unmanaged spaghetti code
Alternative Timeline Management Metaphors
• Alternative 1 - Find a visual metaphor that cognitively fits each type of situation (DMI)
• Alternative 2 - Drive the interview for workflow info via a “knowledge model” of what needs to be collected (Agent I/F)
• A few ideas from R2Do2
The R2Do2 Agents from a Workflow View
Users, Patient,Doctor,Nurse, etc. Plan(ToDo Items)
UnPlan(Items)
Parser& BrokerRemoteRecords
Guideline Agents
(Short-lived,multi-thread)
ReminderAgents
(Long-lived,wake/sleep)
GuidelineKB Repository
Triggers,EventServer
Ongoing Alerts &Reminders
User AddsNotificationPreferences
Email, beeper, ITV, etc.
Age
7 years
DTaP (preferable) or DTP
DT
NoYes
Td
Contraindication to Pertusis
Case Study: The R2Do2 Immunization Agents’ Knowledge Base -- DTP Vaccine
Figure 3.1 Dependency Diagram of Logic for DTP Immunization
Contrain-dication
Vacc.Type
Recom.Schedule
Interval_from_previous_dose
age (in year, in month, and in week)
the number of prior doses
Vaccine_type of previous dose
Anaphylactic_
reaction
Encephalopathy
Table 3.2 Schedule for DTP vaccinationNumber of
prior vaccination0 1 2 3 4 Adult Booster
at birth First - over Dose
1 month Await
2 months First Second over Dose
3 months Await
4 months Second Third - over Dose
5 months Await
6 months Third First-Bo Over
Await Dose
15 months First -
Booster Second-
18 months Booster
First- Await
4 years First Second Third Booster Second
5 years available available Available available Booster
6 years
7 years Adult- Booster
Await Every 10 years
Metaphor that usesgraphics, 2-D seq,table format, andcolor to capturetemporalities(still need to addpersonnel resp.)
Table 3.1 Routine DTP vaccination Schedule (Source: MMWRa)
Dose Age Customary age / interval
Primary 1 2 months Age > 6 weeksPrimary 2 4 months 4-8 weeks after first dosePrimary 3 6 months 4-8 weeks after second doseFirst Booster 15-18 months 6-12 months after third doseSecond Booster 4-6 years No necessary if first booster is
administered after fourth birthday.Additional booster Every 10 years after last dose
Preceding Picture Conveys Much More Info Than a Table Format
Figure 3.4 Determine the recommended schedule of DTP vaccination.
if interval_of_separation >= minimum_wait thendue := “ now. ”;conclude true;
elsedue_date := time of (minimum_wait - interval_of_separation );due := “ due on ” || due_date || “ .”;conclude true;
endif;
prior_no sequence_no minimum_wait interval_of_separation
0 “first” 6 weeks age_in_week1 “second” 4 weeks interval_in_week2 “third” 4 weeks interval_in_week3 “first booster” 6 months interval_in_month4 “second booster” 4 years age_in_year
done “adult booster” 10 years interval_in_year
The semantics are of a temporal logic
Consumer Immunization
Record
Receive
Birth date
SSN
Mother’sFirst Name
Mother’s Last
Name
Mother’s Maiden Name
Mother’s SSN
Father’s First Name
Father’s Last Name
Father’s SSN
Provider ID
Provider
Last Name
First Name
Practice Address
Practice City
Practice State
ZIP
Give SubID Counter
AdministrationSubIDCounter
Date and TimeStart
Administration
Date and TimeEnd
Administration AdministeredCode
AdministeredUnits
Administered
Amount
AdministeredDosage Form
AdministeredDosage Form
AdministrationNotes
Administeredat Location
AdministratingProvider ID
Administeredper
Time/UnitAdministered
Strength
SubstanceLot Number
AdministeredStrength Units
SubstanceExpiration
Date
SubstanceManufacturer
Name
SubstanceRefusal Reason
Indication
CompletionStatus
ActionCode
System EntryDate and Time
ObservationIdentifier
Units
ObservationValue
ObservationSubID
ReferencesRange
AbnormalFlags Nature of
Abnormal Test
ObservationMethod
Probability
Observ ResultStatus
Date LastObs Normal Values
User Defined Access Checks
ResponsibleObserver
Date and Time of the
Observation
ClinicalObservation
Person
A kind of
A kind of
Email address
HCO Group No
HCO User ID
Password
To Do Item
Remind
Description
Due Date
Lead unit
Interval
Interval unit
Next reminder
Repeat Count
Repeat unit
Lead time
Repeat interval
Reminder No Repeat until date
Subject
To Do IDState
Questionnaire
Possible Answer
Question
Questionnaire ID
Question ID
Fill out
Interface Object ID
Interface Object Type
Parent Question ID
Vaccine Manufacturer
Produce
Name
ID
Name
Vaccine Code
Description
Include
ALT 2 - Agent Uses A Model to Drive the Interview: CDC’s HL7 Spec for Immun.
Data
AlertRemind
IndicationContraindicationPrecaution
translated to
activate
refer
1..n
1..n
refered
referee
trigger
1..n
Evoke criteria
produce
originate
Action
UrgencyExecution methodDescriptor
execute_action
refer to
1..n
use
1..n
based on
1..n
contains
1..n
Response message
Message kindContainer
infer about
contains
Attribute driven logic
Prediagnostic stageDiagnostic stagePost Diagnostic
Temporal logic
Clinical Logic Segment
Requisite attributes set
Attributes set
retrieve(object attribute ,query conditions)
Operational representation
Representation schemaLast updatedContainer idRepresentation container
call
refers
Knowledge Element
TypeElement IDPriority
Maintenance Element
TitleFilenameVersionInstitutionAuthorSpecialistDateValidationElement ID
Library Element
PurposeExplanationKeywordsCitationsElement ID
Guideline
Guideline ID
Reference Information Model
PersonPatientPharmacy_Service_AdministrationIndividual_Healthcare_ProviderPatient_AllergiesClinical_Observation
Vocabulary/Lexicon
ICD10NDRICD9SNOMEDVocabulary/Lexicon elementVocabulary/LexiconVocabulary/Lexicon Server
Given a Generic Model, a Wizard Should Work for New
Vaccines
• Wizard will interview for all the temporal semantical elements, and whatever else…
• Can a generic model be extracted for all vaccines? (one works for all 7 current ones)
• How many other generic wizards must be created? - a research task (cost-plus, not FFP).
vCalendar The Electronic Calendaring and Scheduling Exchange Format
• Consortium Members:– IBM, Lucent, Lotus, Microsoft, Novell,
Netscape, Oracle, etc.
• Open grammar for personal organizers, professional schedulers, calendar software
• GUIs for this grammar have already sold millions (HIGH USABILITY)
R2Do2 Reminding & Calendar GUIhttp://hera.seas.gwu.edu/prevention/prevention.html
http://bartok.seas.upenn.edu:8080/
Kill
Recur ComputeNext (IAction)
Launcher Launch(PAS)
Monitor(k)
Sleep Monitor
Awaken Check
Set Alarm ComputeNext(ISleep)
Remind ConsumerProvider
AFTER(Now,ISleep), Monitor
OODBConsumer Data
- DONE(Pk)
User Input Collect Scheme
Update 2Do
FiArchive
DONE(Pk) &- NULL(ISeparation)
-HOLDS(Pk) or [DONE(Pk) &NULL(ISeparation)]
DONE(PNk)
Mk
IAction
ISleep
ISleep
UserPreferences
Markov Chain Oriented
Make vs. Buy - Maintenance Tools
PROs CONsBuy VisualWorkflow
Already existsHelp/Trainingprovided
HFHS ComplaintsOpen Workflow Stds are Limited
Build NewMetaphor
Cognitive fitHigh Usability Pot.
Research neededEffort & time
Add Cal.Grammarto GLIF
Enhances GLIFReuses R2Do2 code
Will GLIF acccept?Time & Cost
Authoring Process:3. Chronic (Ambul. Care) Guidelines
• What is the “Chronic Guideline” Generic Structure?
• Show an example Guideline Form – (eg guideline-form widget from Protégé)
• Walkthrough Jetform Demo for Layout Mgt
• Walkthrough all Parts/Layouts– Textual instructions sheets– Visual Annotated Algorithm (GLIF Pallette, Risk-Costs-Benefits Table)– Drill Down to Charting Forms– Other
Typical Annotated Algorithm & Table
•Visual flowchart usefule for elicitation •(wizard assisted, terminology-enabled, GLIF semantics)
•Visual flowchart vital for display to users•(save as a GIF or JPEG?)
•Annotations include extensive supplemental material•clinician advice pages•patient educational materials (tables, graphs, etc.)•etc.
•Parse flowchart into machine executable (behind scenes)
Role of GLIF
• GLIF is an object oriented proposal for the interchange of machine executable guidelines
• Guideline Interchange Format
Intended Benefits of GLIF
• Store guidelines in repositories and eliminate duplication of effort between institutions
• Ease of dissemination (and maintenance)
• Deployment as executables
Strengths of GLIF
• Focuses on Types of Steps (bottom LHS)– Well-Formed Semantics of Step Constructs
• Very Good for Sequential Algorithms
R2Do2 Criterion Logic as a Model: Can Authors work with this?
http://hera.seas.gwu.edu/knowledge_factory/login.html
Limitations of GLIF (I)
• Drawbacks mentioned by the authors– Representation of medical concepts in absence
of a standard clinical vocabulary– Lack of a formal syntax for the representation
of conditional expressions– Need to express temporal information in
clinical guidelines– Recognize and handle uncertainties about
patient data
Limitations of GLIF (II)
• Class hierarchy is not clear– All the objects are inherited from Guideline
Model, but they share very few similarities
• Categorization of different steps– Easier to model simple algorithms than
annotations and complex guideline models– Many guidelines are loosely organized and lack
obvious causal effects between parts
Potential Improvements for GLIF(I)
• Revise the class structure and add more relationship representation between objects– basically GLIF can only model steps and their
links (fares poorly on many other relations)– guideline must offer a containers to hold all the
relevant information, definitions, etc. (EGO)
Potential Improvements for GLIF(II)
• Increase data structures– Table Structures (Clinical Paths, C/B Risk)– Equations, Continuous Graphs– Graphics
But all of this is presumes a NARROW INTERPRETATION
of Guideline SemanticsWhat is a BROADER
INTERPRETATION?????
Well for one thing, look at what CANNOT be supported by GLIF
semantics
CADSE requires research on high level clusters of GLIF semantics
Grounded Needs Grnding
Guideline 5 Types of GL
Skeletal Os Dec.Tree,Markov Chain
Ecol.Artifacts:(Clin.Path, Progr.Note, etc)
Steps &Stmts
Branch, Crit.,Cond,
Time,
Logic Terms, vCal,Booleans
Criterion logic,
What is the “Chronic Disease Guideline” Generic Structure?
• SOAP(IE) too individualized
• E&M Coding too high level
• Allan Khoury’s hospital “progress note”– diseases, problems, complaint– HPI– Review of Systems– Physical Exam– Impression/Plan– Orders
• Other? (eg., the above ignores workflow)
Insert Charting Objects (Using an eForm Layout Manager)
PATIENT CHART TEMPLATE
Patient Name _______________________________ Acct. Number _____________________ Date _____ /_____ /_____
PATIENT HISTORY
ALLERGIES ______________________________________________________________________________________
CC _____________________________________________________________________________________________
HPI _____________________________________________________________________________________________
________________________________________________________________________________________________
(location/duration/quality/context/severity/timing/modifying factors/associated signs and symptoms) or (three chronic orinactive conditions)
PFSH: Past__________________________________________________________________________________________
Family______________________________________________________________________________________________
Social______________________________________________________________________________________________
REVIEW OF SYSTEMS: (+, -, na ) NOTES
_____Const _____Musc/skel _____Eyes _____Skin _____ENT _____Neuro _____Cardio _____Psych _____Resp _____Endocrine _____GI _____Hem/Lymph _____GU _____Allergy/Immuno
_____ See additional notes
COMPLEXITY OF MEDICAL DECISION MAKING
CODABLE 1 _____________________________________ 2 _________________________________
Diagnoses/
symptoms: 3 _____________________________________ 4 ______________________________________
RULE OUT
Diagnosis 1 _____________________________________ 2______________________________________
Management Plan/Orders/Medications:
Tests ordered (reference codeable DX for each), performed, reviewed and old records reviewed:
Instructions/Counseling/Coordination of Care:
Time: in _________ out _________ with pt _________ Physician’s Signature____________________________________
Assessment of pain intensity and character:(QUESTIONNAIRE)
1.Onset and temporal pattern
When did your pain start? How often does it occur? Has its intensity changed?
2.Location
Where is your pain? Is there more than one site?
3.Description
What does your pain feel like?
What words would you use to describe your pain?
4.Intensity
(see next page)
5.Aggravating and relieving factors
What makes your pain better? What makes your pain worse?
6.Previous treatment
What types of treatments have you tried to relieve your pain?
Were they and are they effective?
7.Effect
How does the pain affect physical and social function?
Make vs. Buy - Annotated Algorithm
PROs CONsGLIF as-is Free
De facto standardMust build an EGO toimplement
GLIF adds(cond.logic& Tables)
Added power &usefulnessCan have good GUI
Intermed acceptability?
ProgressNotes
Highly flexible(use Layout tool)
We must buildAssure Open-std - XML
Authoring Process:4. Acute Care Guidelines
• Triage in the ER
• Indications for Use of Surgical Procedures
• Appropriate Use of Specific Technologies and Tests as Part of Clinical Care
• Outcomes capture and analysis
Building a wrapper for theNational Guideline Clearinghouse
• We present how to use the our Wrapper Factory (W4F) to write a wrapper for the National Guideline Clearinghouse.
• Our wrappers consist of – a retrieval layer (to fetch the remote Web document)– an extraction layer to extract information from the HTML source– a mapping layer to export the data to higher-level applications
• As a concrete illustration of extraction, we want to build an XML document that briefly describes the guideline.
Retrieval Rules
Extraction Rules
Mapping Rules
NSL
NSL
NSL
INPUT HTML treeHTML document
title
date
availability
Parser
Web
XML documen
t
Step 1: grab a guideline sample• Identify a Web page that describes a guideline
• Guidelines should all have (more or less) the same structure
The extraction wizard• The extraction wizard returns the Web document with some invisible
annotations.
• The display is identical to the original document.
• Text areas get highlighted when the user puts the mouse over them
– it helps the user identify information boundaries
– it tells the user how to reach this piece of information using the hierarchy of the document
Step 3: Define some extraction rules• Using the information provided by the extraction wizard, one can
now write the wrapper.
EXTRACTION_RULES{ all = html ( ->a[i]->pcdata[2].txt # ->a[j]->pcdata[2].txt # ->a[k]->pcdata[2].txt # ->p[l] ( .txt # ->a[1].getAttr(href) # ->p[m].txt ) ) where html->a[i].getAttr(name) = "'title'" and html->a[j].getAttr(name) = "'references'" and html->a[k].getAttr(name) = "'release_date'" and html->p[l].b[0].txt = "GUIDELINE AVAILABILITY:" and html->p[l]->p[m].txt =~ "Print copies";
}
The output XML document and its DTD<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE W4F_DOC [
<!ELEMENT W4F_DOC (GUIDELINE)>
<!ELEMENT GUIDELINE (REFERENCES,RELEASE_DATE,AVAILABILITY)>
<!ATTLIST GUIDELINE TITLE CDATA #IMPLIED>
<!ELEMENT REFERENCES (#PCDATA)>
<!ELEMENT RELEASE_DATE (#PCDATA)>
<!ELEMENT AVAILABILITY (TEXT,URL,COPY)>
<!ELEMENT TEXT (#PCDATA)>
<!ELEMENT URL (#PCDATA)>
<!ELEMENT COPY (#PCDATA)>
]><W4F_DOC> <GUIDELINE TITLE="Prevention of plague: recommendations of the Advisory Committee on Immunization Practices (ACIP)."> <REFERENCES>MMWR Morb Mortal Wkly Rep 1996 Dec 13;45(RR-14):1-15 [37 references]</REFERENCES> <RELEASE_DATE>1996 Dec 13</RELEASE_DATE> <AVAILABILITY> <TEXT>GUIDELINE AVAILABILITY:Electronic copies: Available from the Centers for Disease Control and Prevention (CDC).</TEXT> <URL>http://…..</URL> <COPY>Print copies: Available from the Centers for Disease Control and Prevention, MMWR, Atlanta, GA 30333. Additional copies can be purchased from the Superintendent of Documents, U.S. Government Printing Office, Washington, DC 20402-9325; (202) 783-3238.</COPY> </AVAILABILITY> </GUIDELINE></W4F_DOC>
Work to be completed on Penn’s W4F
• Classic Power vs. Usability Tradeoff– Working and Scalable (wrapper is for programs)– No GUI yet available (not usable by clinicians)
• Other Research Issues ...
Authoring Process:5. Clinical Pathways
• Reusable Tables and Other Objects
• Several Options to Alter Objects:– Wizard interview to change them– Control Preference Screens to change them– Direct manipulation
• Objects self-parse into class hierarchies, XML sheets, precedence relations, & workflows
Typical TableBuilder Screenshot
Day 1Admit
Day 2Surgery
Day 3ICU/Rm
Day 4Xfer
ConsultsTestsActivityTreatmentsMedsDietTeaching
Educational Materials Generation
• Education Materials Text/Table Construction– Clinician DSS Pages (eg, evidence tables)– Patient Education Handouts
• Useful Toolset– Type and format– Cut & Paste from other sources– Webpage authoring (e.g., Netscape Composer)
• Also Publishing the Concept Map/Website Semantic Net (RDF) is helpful for maintenance
Outline
• Purpose & Challenges
• Authoring Process1. Protocol Eligibility
2. Wellness/Prevention
3. Chronic (Ambulatory Care) Guidelines
4. Acute Care Guidelines
5. Clinical Pathways
6. Education & Decision Support Materials
• Maintenance Requirements
• Summary
Maintenance/Admin Requirements
• Guideline Keeper: KPNW P&P pages 13, 14, & 23– consulting expert on that guideline– exception & outcome tracking– literature surveying/detect evidence shifts– recommends guideline updating
• Want tools to help him/her
Maintenance/Admin Tools• General literature tracking tools
• User/Exception/Outcome reports and fixes tracking
• Administrative Tools– set access privileges/maintain security keys
– track usage/adapting of guideline objects/versions
– manage library (checkin, checkout) of diverse containers, components, reusable objects, BLOBs (unlike Ontyx’s homog. term graph)
– inter- and intra-guideline error checking
Traditional Maintenance Paradox
• Project Leaders and Organizations pressure project staff to complete software tasks with minimal budget and delay
• This savings in the development cycle is the very thing that leads to unmaintainable code downstream
• Maintenance is 50-90% of most SW project budgets
What are the Maintenance Needs and How can We Predict Them?
• Examine Case Studies
• Study literature
• Find Principles/Methods (e.g., SEI Capability Maturity Model)
Case Study: DEC’s XCON Expert System
• XCON = VAX config expert system (1983-present)
• One of the world’s 1st expert systems (largest)
• 12,000 rules, 69 locations, released 3x/year
• Vastly successful (configuration quality and sales)
• All production operations=70 mission-critical ESs
• 10,000 users, 250 staff, $200 million/year savings
• kept DEC afloat for many years
XCON Maintenance Nightmares
• Domain misunderstood - VAX change rate overlooked, clusters unhandlable, rules not reusable for other than VAX 11/780s
• Poor Make-Buy Decision - OPS Shell was inexpensive. They allowed it to drive design
• Employee turnover -- no one knows entire KB, can’t remove another’s rule easily, little documentation (as always)
• Irony - unreadable, high level KB
XCON’s Maintenance Solution: Basic Reading & ‘Riting (2Rs)
CLARITY ENFORCING EDITORMaster Structural TemplateOne Partition, One Problem
Domain-Specific Problem Solving MethodsOne Function Per Rule
Legal Syntax for Each Rule
OPS5 Knowledge BaseAuthors
Maintenance Case Study 2: The COPE Kn. Management System
• Automatically help Army to generate, critique, index, and maintain ~600 new guideline type documents/year
• Use of DDN (classified Internet, pre-Web)• Started 1989= just ahead of C++ and Windows• Fielded 1991 (1992 IAAI awardee)• Major code in C++ & Windows Precursors• Code Maintenance Issue: How to convert seamlessly
to C++ and Windows
How Did COPE Handle Maintenance?
• 5,000 Chunk “Static KB” = basic guidelines on document critiquing & document gen. Wizards– Published index/concept map (automated)– On-line updating tutorials– On-line updating editors/critics
• Direct manipulation GUI at XCON’s R2 level
– Color and textual cuing (critiquing, feedback)
How Did COPE Handle Maintenance (continued)?
• Dynamic KB of 600 new documents/year parsed and stored as OO annotated decision algorithms
• World Model concept map & editor (like CMT server and tools)
• Automated indexing (down to reusable sentence structures) via Case Based Reasoning (CBR) algorithm
• Password protected signature chain for KB version updating
• Use of CBR to push reusables to users
Make vs. Buy - Maintenance Tools
Decision CommentOntyx VersionController
Buy Will it work on heterog.Object library?
XCON-R2Type Editor
Make/Buy
This should be CADSEitself
Wizard Editor Buy?
CBR Tool Buy May have to embellishand integrate
Guideline Debug & Check
Outline
• Purpose & Challenges
• Authoring Process1. Protocol Eligibility
2. Wellness/Prevention
3. Chronic (Ambulatory Care) Guidelines
4. Acute Care Guidelines
5. Clinical Pathways
6. Education & Decision Support Materials
• Maintenance Requirements
• Summary
What we covered today• Workbench challenges
– usefulness, scalability, usability– power vs. learnability– make vs. buy
• Dream CADSE scenario
• Workbench Requirements:– What are you trying to put in front of the clinician-author – What should they put in front of clinician-practitioners to make
right thing easiest thing
What we covered today (continued)
• Common, Ecological Artifacts Need to be Authored for a Range of Guideline Types (chronic, acute, prevention, wellness, educ, etc.)– eForms, GUIs
– workflows and calendar/temporal metaphors
– algorithms & their annotations
– eligibility/assessment trees & questionnaires
– table structures (ClinPath, Risk, Dosage, etc.)
– educational materials
What we covered today (continued)
• 15 Tools of the workbench– Breadth -- which ones to include– Depth -- how to ground &
implement them
Make vs. Buy - Summary
Parts to Buy Parts to MakeWorkbench JDK, Oracle8 Envmt & I/FsType Contaner JDK eForm,ClassesLayout Mgr JetForm? reusablesElig.Tree Tool TBD? manyWorkflow OpenWkfl Std
Cal.GrammarMetaphors
Annotated Alg GLIF GLIF AddsWrapper Tool K2, Ariadne GUI, otherClinPath Tool Wizard Genrtr GUI/ReusblesMaintenance CBR, Ontyx World model
Hardest Research Challenges Identified Today
• PRAGMATICS: “Discovering” ecological artifacts– How to best help authors create guidelines materials– Clinician epistemology (don’t know what they know)– In lieu of 1st princ, at regular intervals need: interviewing time,
situational experiments, feedback discussions
• SEMANTICS: Enhancing GLIF “Layers”– Guidelines, Skeletal Parts, Steps, Logic
• SYNTAX: Creating OO implementations (tools) with direct manipulation GUIs and wizards– useful, scalable, usable
What Did we Skip Today
• Integrating the Term Picker*• Groupware Editor*• Guideline Checking (lengthy topic)*• Guideline Executables (EGO Class Library)*• Order Feedback Rules -- Author, Make/Buy*• Knowledge Mediation Layer (lengthy topic)*• Update on Make vs Buy Investigations*• Other?
* - Topics to be covered at length in the 2-day Phila. workshop