Selecting, Planning, and Surviving the Implementation

of anOnline Compliance System

Carmi Anderson & Yvonne RumseyCarmi Anderson & Yvonne Rumsey

Allina Hospitals & ClinicsAllina Hospitals & Clinics

Overview: Selection & PlanningOverview: Selection & Planning

•• Background: Allina Research AdministrationBackground: Allina Research Administration

•• Researching VendorsResearching Vendors

•• Narrowing the FieldNarrowing the Field

•• Product EvaluationProduct Evaluation

–– Engaging StakeholdersEngaging Stakeholders

•• Creating/Using Selection CriteriaCreating/Using Selection Criteria

•• Final DecisionFinal Decision

•• Planning the ImplementationPlanning the Implementation

BackgroundBackground

Allina Research AdministrationAllina Research Administration

–– Institutional Review Boards (Institutional Review Boards (IRBsIRBs))

••Human Subjects ProtectionHuman Subjects Protection

–– Sponsored Projects Administration (SPA)Sponsored Projects Administration (SPA)

••Research Billing ComplianceResearch Billing Compliance

Goal: Single submission system for bothGoal: Single submission system for both

Researching VendorsResearching Vendors

•• MethodsMethods–– Online Search ToolsOnline Search Tools

•• Big PlayersBig Players

–– Research Community: University, Other Hospitals, Research Community: University, Other Hospitals, Research OrganizationsResearch Organizations•• Make ContactMake Contact

•• Interview those with experience in EInterview those with experience in E--SubmissionSubmission

•• Initial FindingsInitial Findings–– IRB: Many vendors/systems on marketIRB: Many vendors/systems on market

–– SPA: No marketed billing compliance systemsSPA: No marketed billing compliance systems•• Needed capability to incorporate SPANeeded capability to incorporate SPA

Narrowing the Field Narrowing the Field (Product Comparison Tool)(Product Comparison Tool)

IRB/SPA Common Submission Tool Comparison Worksheet

REQUIREMENTS System A System B System C System D System E

BASIC FEATURES

Web/Internet based Yes Yes No Yes Yes

Online Submission Yes Yes No Yes Yes

Create your own forms Yes Yes Yes Yes Yes

Custom rule creation for data entry Yes Yes Yes

Automated Agendas & Minutes Yes Yes Yes Yes

Online Correspondence: Stipulations, Reminders, etc. Yes No Yes Yes

Accomodates Manual Review Yes Yes Yes Yes

Narrowing the Field cont.Narrowing the Field cont.

••Major ConsiderationsMajor Considerations

–– Functionality: Online SubmissionFunctionality: Online Submission

–– Ability to incorporate SPAAbility to incorporate SPA

••Ability to share common informationAbility to share common information

•• Separate routing & approval processesSeparate routing & approval processes

–– CostCost

Narrowing the Field cont.Narrowing the Field cont.

•• Billing/Usage OptionsBilling/Usage Options

–– License the SoftwareLicense the Software

•• Large oneLarge one--time cost (Capital)time cost (Capital)

••Needed internal IS SupportNeeded internal IS Support

–– Lease the SoftwareLease the Software

••Monthly feeMonthly fee

••Technical Support IncludedTechnical Support Included

Product EvaluationProduct Evaluation

•• Product DemosProduct Demos

–– Get to know the productGet to know the product

•• Customer InterviewsCustomer Interviews

•• Stakeholder Review & FeedbackStakeholder Review & Feedback

•• Vendor DiscussionsVendor Discussions

Customer InterviewsCustomer Interviews

•• Sought customers from vendor directly Sought customers from vendor directly and indirectly from IRB Forumand indirectly from IRB Forum

•• Gather Questions from TeamGather Questions from Team

–– Was training adequate?Was training adequate?

–– How is technical support/customer service?How is technical support/customer service?

–– What is user feedback?What is user feedback?

–– What would you do differently?What would you do differently?

Engaging Stakeholders in Selection Engaging Stakeholders in Selection ProcessProcess

•• Builds support for the decisionBuilds support for the decision

–– Shares ownershipShares ownership

•• Better understand the network of effectsBetter understand the network of effects

–– Surface concernsSurface concerns

–– OmissionsOmissions

–– RisksRisks

•• Aids in transitionAids in transition

Engaging Stakeholders in Selection Engaging Stakeholders in Selection Process cont.Process cont.

•• Stakeholders Stakeholders –– those affected directly or those affected directly or indirectly by the changeindirectly by the change

•• Key Stakeholders Key Stakeholders –– representative group of representative group of those directly affectedthose directly affected–– Principal InvestigatorsPrincipal Investigators–– Research CoordinatorsResearch Coordinators–– IRB MembersIRB Members–– Sponsored Projects Administration ContactsSponsored Projects Administration Contacts

•• Requires: Commitment & EngagementRequires: Commitment & Engagement

Stakeholder Evaluation WorksheetStakeholder Evaluation WorksheetAgenda/Topics of Discussion Pros Cons

STUDY DESIGN/CREATION FOR SUBMISSION TO IRB (RESEARCHERS)

-System Sign On/Security

-Creation of study application

-Information Sharing

-Submission of multiple forms

-Other

IRB MANAGEMENT (IRB STAFF)

-Receipt of study

-Communication with research personnel

Stakeholder RecommendationStakeholder Recommendation

••What to ExpectWhat to Expect

–– Diverse OpinionsDiverse Opinions

–– Concerns (take back to vendor)Concerns (take back to vendor)

•• How to EvaluateHow to Evaluate

–– Set CriteriaSet Criteria

••What is most important to the decision?What is most important to the decision?

Vendor DiscussionsVendor Discussions

•• Express all concernsExpress all concerns

•• Take excellent notesTake excellent notes

•• If critical, get it in writingIf critical, get it in writing

•• Respect their expertise/adviceRespect their expertise/advice

•• Be honestBe honest

Creating Selection CriteriaCreating Selection Criteria

•• Create Clear ObjectiveCreate Clear Objective

–– Procure an IRB/SPA common submission tool Procure an IRB/SPA common submission tool that will result in the greatest increase in that will result in the greatest increase in efficiency for IRB, SPA, and Researchers efficiency for IRB, SPA, and Researchers while meeting all regulatory and compliance while meeting all regulatory and compliance obligations of researchobligations of research

AllinaAllina’’ss Product Selection CriteriaProduct Selection Criteria

•• Supports Complete and Accurate Study Supports Complete and Accurate Study

SubmissionSubmission

•• Ease of NavigationEase of Navigation

•• Excellent Technical SupportExcellent Technical Support

•• Information is Accessible & SecureInformation is Accessible & Secure

•• Provides FlexibilityProvides Flexibility

•• Reduces Duplication of EffortReduces Duplication of Effort

Product Evaluation Criteria* Criteria supports research staff, IRB members and administrative staff by:

Supports Complete and Accurate

Study Submission

� Providing guidance

� Allowing for easy return of inaccurate submissions

� Reducing potential for errors in submissions to IRB or SPA

� Reducing stipulations

� Preventing returned submissions

� Tracks progress of submission

Ease of Navigation � System is logical and intuitive

� Movement to any area of the system is simple

� Mistakes are easy to correct

Excellent Technical Support � Support is responsive and quick

� Feedback from customers is positive

� System is stable

� Reliance on vendor for daily operations is minimal

Information is Accessible &

Secure

� Providing transactions and project documentation where needed

� Allows for ongoing design of necessary reports

� Data is searchable & extractable

� Help menus available

� Provides links to outside information where needed

� Privatizes data

Provides Flexibility � Responsive to regulatory changes, researcher needs and IRB

administrative needs

� Customizable to SPA and IRB processes

Reduces Duplication of Effort � Minimizes data entry

� Provides single administrative system for SPA & IRB

� Allows online tracking of projects for all users

� Supports project communication and collaboration

Completing The EvaluationCompleting The EvaluationCompare and contrast using the criteriaCompare and contrast using the criteria

Example: Ease of NavigationExample: Ease of Navigation

System A: Here you would write how navigation is System A: Here you would write how navigation is handled throughout System A. What was the handled throughout System A. What was the stakeholder experience? Staff experience? Customer stakeholder experience? Staff experience? Customer feedback?feedback?

System B: Navigation described in System B. Stakeholder System B: Navigation described in System B. Stakeholder experience. Staff experience. Customer feedback.experience. Staff experience. Customer feedback.

Analysis: Describe differences. What are positives and Analysis: Describe differences. What are positives and negatives to each? Do some outweigh others? negatives to each? Do some outweigh others?

Final DecisionFinal Decision

•• Group DecisionGroup Decision

•• Use Evaluation ReportUse Evaluation Report

–– Begin with clear objectiveBegin with clear objective

–– Discuss big differencesDiscuss big differences

–– Obtain consensusObtain consensus

••Can everyone support the final decision?Can everyone support the final decision?

What I LearnedWhat I Learned

•• Be honest with what you know and what Be honest with what you know and what

you donyou don’’tt

•• Ask the questionsAsk the questions

•• Everyone needs to participateEveryone needs to participate

•• Do not tie yourself to the outcomeDo not tie yourself to the outcome

•• Get it in writingGet it in writing

Planning for ImplementationPlanning for Implementation

•• ScopeScope

–– What is affected?What is affected?

–– Who is affected (Stakeholders)?Who is affected (Stakeholders)?

–– What/Who is not affected?What/Who is not affected?

•• Assign Roles & ResponsibilitiesAssign Roles & Responsibilities

•• Project TimelineProject Timeline

•• RisksRisks

–– Planning for unknownsPlanning for unknowns

AllinaAllina’’ss Project Lead Roles AssignedProject Lead Roles Assigned

•• Project LeadProject Lead–– IRBIRB

–– SPASPA

•• CommunicationsCommunications

•• TrainingTraining

•• Vendor Liaison Vendor Liaison

•• Data MigrationData Migration

•• TechnologyTechnology

•• System Start UpSystem Start Up

Project TimelineProject Timeline

•• Best estimate of work planBest estimate of work plan

•• Outline Major MilestonesOutline Major Milestones

–– DeliverablesDeliverables

–– HandoffsHandoffs

•• Agree upon with vendorAgree upon with vendor

••Must be kept current throughout projectMust be kept current throughout project

Questions?Questions?

After the SelectionAfter the Selection

Communicating with Stakeholders

Configuring the System:Configuring the System:WhoWho——StaffStaff

•• Designate one person to manage Designate one person to manage configuration of the system.configuration of the system.

•• Look for someone who . . .Look for someone who . . .–– Enjoys challenges and trialEnjoys challenges and trial--andand--error learning.error learning.

–– Is comfortable working with technology.Is comfortable working with technology.

–– Is detail oriented and focused on customer service.Is detail oriented and focused on customer service.

–– Is not easily frustrated.Is not easily frustrated.

–– Has strong communication skills.Has strong communication skills.

–– Possesses a thorough knowledge of processes.Possesses a thorough knowledge of processes.

Configuring the System:Configuring the System:WhoWho——StakeholdersStakeholders

They gain... They provide... You gain...

Configuring the System:Configuring the System:WhatWhat

•• Every system requires customization, how Every system requires customization, how much depends on the system, your much depends on the system, your requirements, and your budget.requirements, and your budget.

•• Typical areas requiring changes includeTypical areas requiring changes include……

–– Forms.Forms.

–– Reports.Reports.

–– Communications (e.g., notices to researchers).Communications (e.g., notices to researchers).

–– Work processes and system roles.Work processes and system roles.

Configuring the System:Configuring the System:Related Decisions to MakeRelated Decisions to Make

•• To continue to maintain paper filesTo continue to maintain paper files

•• To migrate data from an existing To migrate data from an existing electronic systemelectronic system

•• To use electronic or handwritten To use electronic or handwritten signaturessignatures

Configuring the System:Configuring the System:When (1)When (1)

•• Establish a reasonable timeline for system Establish a reasonable timeline for system completion.completion.

–– Staff workloadStaff workload

–– Complexity of proposed changesComplexity of proposed changes

•• Expect delaysExpect delays——deliberately schedule deliberately schedule flexibility into your timeline.flexibility into your timeline.

•• As much as possible, meet your scheduled As much as possible, meet your scheduled milestones even when the vendor or IT milestones even when the vendor or IT

may not.may not.

Configuring the System:Configuring the System:When (2)When (2)

•• For consistency, work from a single tool For consistency, work from a single tool for tracking deadlines.for tracking deadlines.

–– DatabaseDatabase

–– SpreadsheetSpreadsheet

–– Shared Outlook CalendarShared Outlook Calendar

•• Determine in advance whether project or Determine in advance whether project or daily work will take priority when daily work will take priority when deadlines conflict.deadlines conflict.

Configuring the System:Configuring the System:HowHow

•• Avoid the temptation to replicate an Avoid the temptation to replicate an existing system.existing system.

•• Decide in advance how your team will Decide in advance how your team will make decisions.make decisions.

•• Evaluate your current work processes to Evaluate your current work processes to determine what to retain and what to determine what to retain and what to improve.improve.

••Maintain thorough documentation of all Maintain thorough documentation of all changes.changes.

Configuring the System:Configuring the System:Interacting with the VendorInteracting with the Vendor

•• Schedule regular (weekly) meetings with Schedule regular (weekly) meetings with the vendor to clarify progress.the vendor to clarify progress.

•• Conference calls: Restate key points to Conference calls: Restate key points to ensure agreement.ensure agreement.

•• Clarify the definitions of jargon to be sure Clarify the definitions of jargon to be sure both sides are referring to the same thing.both sides are referring to the same thing.

•• Take notes of meetings, noting decisions Take notes of meetings, noting decisions made and next steps to take.made and next steps to take.

Testing the SystemTesting the System

TrainingTraining

•• Provide a variety of training venues:Provide a variety of training venues:

–– In personIn person

–– SelfSelf--pacedpaced

–– System Help/Instructional Text in the SystemSystem Help/Instructional Text in the System

–– WebWeb--based (based (LiveMeetingLiveMeeting, Go, Go--To Meeting)To Meeting)

••Tailor the training to user roles.Tailor the training to user roles.

••Conduct training as close to Conduct training as close to ““go livego live””as possible.as possible.

Implementing the System:Implementing the System:Some Tips (1)Some Tips (1)

•• Avoid publicizing the specific Avoid publicizing the specific ““go livego live””date until the project is nearing date until the project is nearing completion.completion.

•• Choose the implementation plan that best Choose the implementation plan that best suits your situation: phased or all at once.suits your situation: phased or all at once.

•• Consider freezing submissions for a short Consider freezing submissions for a short period of time.period of time.

Implementing the System:Implementing the System:Some Tips (2)Some Tips (2)

•• Expect an initial drop in productivity (e.g., Expect an initial drop in productivity (e.g., longer turnlonger turn--around times) when the around times) when the system is initially introduced. system is initially introduced.

•• Remember that glitches can occur despite Remember that glitches can occur despite the best planning. Exercise patience.the best planning. Exercise patience.

••Maintain communication with the vendor Maintain communication with the vendor during the initial weeks after system during the initial weeks after system implementation.implementation.

Supporting the UsersSupporting the Users

•• Expect questions, and train staff to triage Expect questions, and train staff to triage and respond to them.and respond to them.

•• Prepare others to anticipate a learning Prepare others to anticipate a learning curve (3curve (3--6 months).6 months).

•• Follow up with additional training as Follow up with additional training as needed.needed.

•• Remember that the system will continue Remember that the system will continue to evolve over timeto evolve over time——system support system support

needs to evolve as well.needs to evolve as well.

Communicating During the ProjectCommunicating During the Project

Communicating During the ProjectCommunicating During the Project

When communicating to future usersWhen communicating to future users……

•• Focus on the benefits of the new system.Focus on the benefits of the new system.

•• Anticipate their needs and help users Anticipate their needs and help users understand how the changes will impact understand how the changes will impact them.them.

•• Provide more details as the Provide more details as the implementation date approaches.implementation date approaches.

Questions?Questions?