evaluation of the virtual naval hospitalcrmd0002710.a2/final december 2000 evaluation of the virtual...

CRMD0002710.A2/FinalDecember 2000

Evaluation of the Virtual NavalHospital

Peter H. Stoloff

Center for Naval Analyses4825 Mark Center Drive • Alexandria, Virginia 22311-1850

Copyright CNA Corporation/Scanned October 2002

Approved for distribution: December 2000

LaurieJ. May, Director UMedical TeamResource Analysis Division

This document represents the best opinion of CNA at the time of issue.it does not necessarily represent the opinion of the Department of the Navy.

APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITEDFor copies of this document, call CNA Production Services (703) 824-2122 or (703) 824-2123

Copyright © 2000 The CNA Corporation

Contents

Executive Summary........................................................................................................... iiiBackground..........................................................................................................................!

Purpose. ....................................................................................................................... ....2Issues............................................................................................................................... 2

Methods and Data...... .................................................................................................... ......5Survey Design.................................................................................................................5

Medical Professionals.............................................................................................^Non-Medical Personnel ...........................................................................................6

Cost Data... ..................................................................................................................... .7

Results...... ........................................................................................................................... .9Internet Access.............................................................................................................. 10Reasons for VNH Use....... ............................................................................................ 12

Training Support.................................................................................................... 14Medical Planning Process. ................................................................................. ....15Improved Care.... ................................................................................................... 16Provider Confidence........ ...................................................................................... 17Saving Patient Sick Days. ...................................................................................... 17Information Resources...... ..................................................................................... 18Alternate Internet Sites .......................................................................................... 19Preferred Modality....................................... .................................................... ......20Suggested Enhancements to the VNH... ............................................................... .20User Evaluation. .............................................................................................. ......20

Use of the VNH by Non-Medical Navy Personnel.... .............................................. .....21Business Case Analysis.................................................................................................23

Costs .....................................................................................................................24Benefits. ............................................................................................................. ....24

Appendix 1. Hours and Reasons for VNH Use .............................................................. 1-1

Appendix 2. Training Support ......................................................................................... 2-1

Appendix 3. Suggested Improvements and Enhancements ............................................ 3-1

Appendix 4. Other Reasons for VNH Use by Patients ................................................... 4-1Appendix 5. Comments .................................................................................................. 5-1Distribution List.............................................................................................................. D-l

THIS PAGE INTENTIONALLY LEFT BLANK

Executive Summary

BackgroundThe Virtual Naval Hospital (VNH) is a digital medical library administered over the

Internet by the Electric Differential Multimedia Laboratory, University of Iowa College ofMedicine in collaboration with the U.S. Navy Bureau of Medicine and Surgery (BUMED). ACD-ROM version of the VNH is also distributed to Navy health care providers. Its purpose isto deliver authoritative medical information to point-of-care medical providers to help takebetter care of patients.

Evaluations of the VNH to date have focused on information needs of medical providersand readership of the World Wide Web (WWW) site. No analysis of VNH utilization patterns,derived benefits, or media preferences has been done. The goal of this evaluation is to providean analysis of the VNH that can be used to document lessons learned, and planning for futureservices that might be offered.Issues

Among the issues we considered are the following:

• What is the impact of the VNH on clinical training for primary care providers?• What is the impact of the VNH on the medical planning process for combat

operations and operations other than war?• Does access to the VNH result in improved patient care?• What are the alternative technologies for delivering similar kinds of medical

information?• Does the VNH lead to cost savings?• What materials should be added to the digital library to fill gaps in medical

education?• What are users' attitudes toward using the VNH?• What are the monetary costs and benefits associated with VNH use by the Navy

medical community?• How is the VNH used by non-medical Navy personnel?

ApproachWe developed survey questionnaires administered over the Internet and by email to elicit

feedback on the issues from two target populations—military health care providers and othermilitary personnel seeking health care information. To motivate a high response rate fromdeployed health care military providers, they were contacted by email or Naval message andurged to view the Website and participate in the survey. Participation in the survey by otherpopulations was urged by "attention getters" placed on the VNH. The survey wasadministered by CNA through a link provided directly on the VNH.

in

ResultsThere were 852 responses to the survey for health care providers, mostly to the Web-based

version. About half of the surveys were authenticated as coming from military health careproviders and were retained for analysis. The response rate from the non-medical populationwas poor—only 177 respondents.

Overall, for military medical professionals:

• The greatest general use for the VNH is for patient care information.• Use of the VNH for training is mainly for Professional Military Education (PME)

and teaching preventive measures to ships' crew.• Use of the VNH resulted in improved patient care for 70 percent of respondents.• Use of the VNH improved provider confidence in making diagnoses and

formulating treatment plans for 58 and 50 percent of respondents, respectively.• Use of the VNH resulted in avoiding SIQ days, mostly because of improved

primary prevention practices (57 percent of respondents), and MEDEVACs fordiagnostic purposes.

• Use of the VNH facilitated planning of Naval Operations (Operations Other ThanWar—85 percent; Combat Operations—58 percent of respondents).

• The VNH is cost effective, producing an estimated annual net savings of $258K.• Users are mostly satisfied with the VNH (94 percent). Dissatisfaction focused

mainly on the CD-ROM, which is only updated semi-annually, resulting incomplaints that the information was not current.

RecommendationsAlthough satisfaction with the VNH was high, there were some complaints about the

currency and difficulty of locating information on the CD-ROM. Therefore, for ease ofuse, include a search engine on the CD-ROM version of the VNH.

The existence of the VNH is not as widely known among Navy health careprofessionals as it could be. Therefore, BUMED should develop a process for spreadingthe word of the existence and uses of the VNH for both Navy medical professionals andother Sailors.

IV

Background

No set of medical providers is more isolated than the GMO and IDC at sea and in thefield. They are tasked with promoting wellness in the warfighter and attending to themedical diseases and injuries of the warfighter. To deliver high-quality medical care,naval health care providers and their patients need to be armed with convenient access tocurrent, authoritative medical information.

The Virtual Naval Hospital (VNH) is a digital medical library available to the generalpublic over the Internet. It is administered over the Internet by the Electric DifferentialMultimedia Laboratory, University of Iowa College of Medicine, in collaboration withthe U.S. Navy Bureau of Medicine and Surgery (BUMED).

The Navy approached digital library researchers at the University of Iowa, who had ahistory since 1992 of providing digital library services to isolated rural primary careproviders and their patients via the Virtual Hospital digital library.1 The Navy wished toleverage Iowa's expertise and lessons learned to deliver digital library services to isolatedNavy primary care providers and patients at sea.

The Virtual Naval Hospital lowers barriers to accessing information for diseasediagnosis, decision support, treatment, and follow-up as well as for patient education. Bydelivering medical information to the point of care, the Virtual Naval Hospital can helpimprove clinical outcomes. The Virtual Naval Hospital provides a platform for tele-education for the U.S. Navy that can be completely integrated with the U.S. Navy's andDepartment of Defense's current and future telemedicine programs. By moving expertmedical information and not people, the Virtual Naval Hospital uses technology as aforce multiplier to help provide care in remote and isolated areas, thus eliminating timeand distance as obstacles to the delivery of care and helping to take medical care back tothe deckplates. By eliminating providers' and patients' information isolation, bettermedical care can be delivered to Sailors and Marines, thus helping to maximizereadiness, allowing the Navy to fulfill its missions (from the VNH website).

The purpose of the VNH is to deliver authoritative medical information to point-of-care medical providers to help take better care of patients. It is used by Navy medicalpersonnel to provide health-related diagnostic and treatment protocol information. It isalso intended as a source for both provider and patient education and health promotion, aswell as a reference library of BUMED instructions. Because Internet access is sometimesproblematical at sea, the VNH is also available to Navy Medical personnel on CD-ROM.

1 M.P. D'Alessandro et al. "The Virtual Hospital: The Digital Library Moves from Dream to Reality.'Academic Radiology 1999; 6:78-80.

PurposeThe VNH became operational in 1997 and has experienced tremendous growth since,

averaging about 70,000 visitors in the first 3 months of 2000. Evaluations of the VNH todate have focused on information needs of medical providers and readership of the WorldWide Web (WWW) site.2 The goal of this evaluation is to provide an analysis of theVNH that can be used to document lessons learned, and planning for future serviceswhich might be offered.

Of additional concern to the Navy is the cost-effectiveness of the VNH. A goal ofthis study is to provide an economic analysis and develop a business model to identifycosts and savings associated with development and maintenance of the VNH.

A cost-effectiveness analysis typically compares the relative costs of alternate meansof providing some product. In this case, the product is knowledge. The immediateeffects or benefits of gaining knowledge are not always immediately apparent. This isparticularly true for knowledge acquired in the process of obtaining "professionaleducation," such as a health care provider keeping up with the medical literature. When aprovider uses the VNH to help make a diagnosis, the independent contribution ofinformation obtained from the VNH may be difficult to isolate. That is because a healthcare provider typically uses multiple resources, such as consultations with colleagues, toaid in making a diagnosis. Thus, if a correct diagnosis led to some savings, how much ofthat savings should be attributed to the VNH?

Another aspect of a cost-effectiveness analysis is the substitution of alternatetechnologies for the VNH. Keeping in mind that the VNH is a reference tool, what arethe alternatives for providing the same information? We will focus on alternate media.

An obvious alternative is printed materials. In many cases, however, it is impracticalto provide paper copies of the many books, journals, and other materials contained theInternet and CD-ROM versions of the VNH because of space limitations aboard ship. Noother practical alternatives come to mind.

IssuesAmong the issues considered in this report are the following:

• What is the impact of the VNH on clinical training for primary care providers?• What is the impact of the VNH on the medical planning process for combat

operations and operations other than war?• Does access to the VNH result in improved patient care?• What is the preferred medium for the VNH (Internet or CD-ROM)?• What are the alternative technologies for delivering similar kinds of medical

information?

2 D.M. D'Alessandro et al. "Information Needs of Naval Primary Care Providers and Patients at Sea."Military Medicine 1999; 164(2):127-131.

What materials should be added to the digital library to fill gaps in medical educa-tion?What are users' attitudes toward using the VNH?How is the VNH used by non-medical Navy personnel?What are the monetary costs and benefits associated with VNH use by the Navymedical community?

Methods and Data

Many users of the VNH are in isolated locations (i.e., aboard ships, in remote duty sta-tions). These users are hard to contact by postal mail and telephone—the traditional methodsfor collecting survey data. The Internet was a natural vehicle for survey administration for thedata collection. Most users of the VNH do so over the Internet, and could be contacted andmotivated to complete a survey form integral with the VNH website. Therefore, we devel-oped a survey questionnaire that could be administered over the Internet to elicit feedback onthe issues. Two populations were targeted for the survey:

• Navy health care providers and administrators.• Other military personnel seeking health care information.The intended sample of Navy health care providers and administrators comprised Navy

physicians, dentists, Independent Duty Corpsmen, nurses, Medical Service Corps, and variousmedical technicians. To motivate a high response rate from deployed health care military pro-viders, they were contacted by email, Naval message, and by notices in the Surgeon General'scolumn in the Navy Medical Information Center (NMIMC) Newsletter. They were urged toview the VNH website and participate in the survey. Alternatively, they could request a com-puter readable copy of the survey that could be emailed back to CNA after completion. Par-ticipation in the survey by other military personnel was by urged by "attention getters" placedon the VNH. The Internet version of the survey was administered by CNA through a linkprovided directly on the VNH.

Survey DesignTwo surveys were constructed. One was for Military Medical Professionals (MMP);

a second for other military personnel seeking health care information (i.e., "patients").When accessing the survey over the Internet, respondents were greeted with a screen in-troducing the survey, allowing respondents to select the appropriate form. Because thesurvey was available to the public, those choosing the MMP survey were asked to pro-vide an identification code used to authenticate respondents as members of the Navymedical community. The codes were distributed to Naval medical commands during theannouncement process. Because there are two versions of the VNH (Internet and on CD-ROM), survey questions for medical professionals elicited information on the use of bothversions.

Medical ProfessionalsThe MMP survey form consisted of 11 sections and an introductory portion. The sec-

tions included:1. Personal Information (respondent's specialty, nationality, branch of service,

duty station, and age).

2. Internet Access.3. Impact on Clinical Training (hours/week VNH use for various training as-

pects).5

4. Medical Planning Process (focused on VNH use to plan medical support forcombat and operations other than war).

5. "Just in Time" medical information (mainly for health care providers).6. Provider Confidence (evaluation of VNH role in boosting provider confidence

in making diagnoses and treatment/follow-up plans).7. Saving Patient Sick Days (degree to which VNH information allowed patients

to get back to work sooner by avoiding sick-in-quarter (SIQ) days, orreturning to full or limited duty status).

8. Use of Alternative Medical Information Resources (relative frequency of useof different sources of medical information to answer medical questions;preference for Internet vs. CD-ROM versions of the VNH).

9. Suggestions for improving the VNH.10. Evaluation of the VNH (with respect to breadth of topics, level of detail,

accuracy of information, and overall satisfaction).11. Comments.The email version of the survey questionnaire for MMP paralled that of the Web-

based survey. In response to a request for the survey by email, a floppy disk imagewas sent by return email. Respondents also received instructions for completing thesurvey and returning the resulting data electronically. In some instances where emailcould not be used, a floppy disk with instructions was sent via postal mail.

Response to the survey was mostly voluntary. However, an initially poorvoluntary response rate prompted BUMED and the Navy Medical Information Center(NMIMC) to request that local commanders urge participation by medicalprofessionals. There was no attempt to publicize the VNH outside the Navy medicalcommunity.

Non-Medical PersonnelThe survey form for other military personnel (i.e., patients) was also accessable

through a link on the VNH internet site. It consisted of 7 sections and an introductoryportion. The sections included:

1. Personal Information (respondent's specialty, nationality, branch of service,duty station and age).

2. Internet Access.3. Source of medical information.4. Reasons for using the VNH.5. Benefits of using the VNH.6. Evaluation.7. Comments.

The data collection period for both surveys was from February through August2000.

Cost DataCost data were obtained from the University of Iowa College of Medicine Electric

Differential Multimedia Laboratory, the VNH's administrators. The money buys theNavy access to the University of Iowa's Virtual Hospital infrastructure, by piggybackingthe VNH on top of the Virtual Hospital's personnel, content creation workstations, Webservers and bandwidth. The Navy is not charged for the use of Virtual Hospital contentcreation workstations, Web servers, or bandwidth.

In 1997 the VNH was budgeted for $197,000; and in 1998- September 2000 it was budg-eted for $250,000 each year. Actual expenditures were about $ 182,000 / year on the VNHtimes 3.75 years, or $682,000.

Results

There were 852 responses to the survey for health care providers, mostly to the Web-based version. Respondents were authenticated on the basis of their entering the assigned"survey code," or identification of the IP address of the host submitting the survey as amilitary facility. Applying this screen resulted in a sample size of 462 respondents.3 Allbut 69 of the survey responses were via the WWW site. About 50 percent of the emailedresponses came from those deployed on ships. In addition, about 20 percent of the 462respondents had never previously used the VNH and were excluded from those parts ofthe analysis requiring familiarity with the VNH.4

The sample was post-stratified on the basis of medical specialty. The stratificationallowed us to determine how representative the sample was (in terms of the relativedistribution of the specialties of the sample and population. A secondary purpose of thestratification was based of expected differences in utilization patterns of informationavailable from the VNH.

The strata consisted of five groups, as described below:1. Administrative (Medical Service Corps (MSC) - Administrative).

2. Clinical/technicians (nurse, laboratory technicians, Hospital Corpsmen (HM)).3. Dental (dentists and dental-assistants (DN)).4. Provider (physicians, physician-assistants, nurse-providers, midwives,

psychologists, and Independent Duty Corpsmen (IDC)).5. Other (respondents who did not identify their specialty).

Table 1 shows the proportions of survey respondents and the actual population sizesmaking up the stratification. The data suggest that the sample distribution is notrepresentative of the Navy MMP population as a whole. This will limit our ability toproject the extent of VNH benefits Navywide.

Respondents were from a wide range of duty stations. Table 2 shows the distributionof duty stations by medical specialty.

Eighty surveys submitted by non-USN medical personnel were excluded.4 Some Navy medical personnel were directed to view the VNH website and respond to the survey.

Those without Internet access or the CD-ROM did not have the opportunity to use the VNH.

Table 1. Sample Distribution of Navy Medical Specialties

Specialty

Administrative

Tech

Dental

Provider

Sample

14%37%

11%

38%

Population

62%

12%

11%

15%

(Frequency) (462*) (37,280)

* The distribution of percentages of respondents by specialtywho had used the VNH was similar. Excludes 24 respon-dents in the "Other" category.

Table 2. Respondents by Specialty and Duty Station (percentage of sample)

Table 3

SpecialtyDutystationClinicFMFHQHospitalNEPMUResearchSeabeesShipOther3

Total

Admin2.60.40.93.11.30.70.03.31.113.3

Tech8.97.82.27.60.00.20.71.7

6.135.3

Dental5.20.90.40.70.00.40.21.31.1

10.2

Other0.70.00.21.70.90.00.00.21.35

Provider8.54.81.38.32

0.21.34.15.7

36.2

Total25.913.9

521.44.11.52.210.715.3100

a Mainly Air Squadrons and Reserve Units.

Internet AccessBandwidth is at a premium, and often rationed, aboard ship. This could limit access

to the VNH via the Internet for deployed medical professionals. The data shown in table3 suggest that 83 percent of survey respondents have Internet access at their duty sta-tions.5 This is not to say that all medical professionals enjoy that level of access. Notethat most respondents (85 percent) did so via the WWW. This presents a bias toward

5 All military medical professionals should have access to email. While the survey did not specificallydefine "Internet access," we assume this term to mean access to the WWW.

10

those with access to the WWW. The percentages of email and WWW respondents whoreported having Internet access were 10 percent and 93 percent, respectively.6

Table 3. Internet Access and Duty Station

Duty station

Clinic

FMF

HQHospital

NEPMU

Other

Research

Seabees

Ship

% with access

92

91

87

85

79

80

71

90

49

Total* 83

* Includes respondents who did not use the VNH.

Likewise, only half of those who are on ships reported access to the Internet at theirduty stations.

For those who do have access to the Internet at their duty stations, most have high-speed access with a Local Area Network (LAN). As shown in table 4, there is some dif-ferentiation in Internet access by specialty.

Table 4. Internet Access at Duty Station by Specialty

Specialty

Admin

Tech

Dental

Other

Provider

% with access

74

85

68

96

85

LAN access

98

84

87

80

82

Total 82 85

While most respondents seem to have a high speed Internet connection at their dutystations, do they have sufficient access for job-related activities? In table 5, we show

6 Note that some individuals who responded to the survey by the WWW may have done so from loca-tions other than their duty stations. This was evident by the non-military IP addresses associated with thesource of the submissions.

11

self-reported actual and required, or "needed," job-related Internet use by medical spe-cialty. The column labeled "shortfall" is the difference between the requirement and ac-tual use. On average, the shortfall is less that one hour per week. This suggests that thosewho do have Internet access at their duty stations have sufficient bandwidth to meet mostof their requirements.

The column labeled "supplement" shows the average number of hours/week thatmedical professionals supplement their job requirements while away from their duty sta-tion (i.e., at home). Note that the supplemental hours well exceeds the shortfall. Thismay reflect the tendency of many to spend more time than anticipated to browse theInternet, either because of the difficulty of locating information or because of one's in-quisitive nature.

Table 5. Job-related Internet Use at Duty Station (average hours/week)

Specialty

Admin

Tech

Dental

Other

Provider

Actual

7.9

7.4

4.1

10.1

6.9

Required

8.0

7.9

5.0

11.2

7.8

Shortfall

0.2

0.5

0.9

1.1

0.9

Supplement

3.5

5.0

2.5

6.6

5.3

Total 7.0 7.7 0.6 4.7

Reasons for VNH UseIn this section we focus on how Navy medical professionals use the VNH and the

amount of time they use it for various activities.The VNH is used for a variety of information-seeking activities. Those in the medi-

cal specialties represented have different information needs. This is reflected in table 6,which shows the percentage of VNH use (either Internet or CD-ROM) for the activitiessupported by the VNH. There was a relatively high level of use observed for each activ-ity. The percentages suggest that information about patient care is the most sought-after—regardless of specialty.

12

Table 6. VNH Use for Various Activities (percentage within specialty)

Activity

Specialty

Admin

Tech

Dental

Other

Provider

Patient care

52

67

83

83

65

Healthpromotion

52

54

77

75

56

Prof. Ed.

52

63

74

75

58

Train others

51

57

72

71

55

Admin

57

54

72

67

52

Other

30

15

36

21

11

Total 67 58 61 58 56 18

How many different kinds of information do users seek from the VNH? The data intable 7 suggest that most respondents use the VNH to support multiple activities.

Table 7. Number of Activities of VNH Use (percentage of users)

Number of Activities of Use

Specialty

Admin

Tech

Dental

Other

Provider

Total

1

15

9

5

5

10

9

2

3

5

5

5

2

4

3

0

8

0

5

10

6

4

13

13

5

5

11

11

5

26

45

44

55

54

46

6

44

20

41

25

14

24

Table 8 shows the average weekly use (hours) of the VNH for the various activities,and media (Internet and CD-ROM) preferences for each. (A breakdown by medical spe-cialty is shown in appendix 1.) The time spent using the VNH on the Internet was aboutthe same for all but Health Promotion (and "Other"). The time spent using the CD-ROMversion for the various activities was about half of that spent on the Internet.

13

Table 8. VNH Use for Various Activities (hours/week; percentage preferences by media)

Activity

Patient care

Health promotion

Prof education

Train others

Administration

Other

Total

Mean

Internet

1.0

0.5

1.2

1.0

1.1

0.2

5.0

hours/wk

CD-ROM

0.6

0.4

0.8

0.7

0.7

0.0

3.2

Preference (% sample)

Neither

32

12

33

38

39

66

N/A

Internet

55

43

50

48

47

28

N/A

CD-ROM

13

45

17

14

15

6

N/A

Most users prefer the Internet version of the VNH for their medical informationneeds. An exception was for those seeking health promotion information. In this case,preferences were about the same for Internet and CD-ROM access. A relatively largepercentage said they had no media preference for most activities, with the exception ofhealth promotion. When using the VNH for health promotion, respondents had near equalpreferences for the Internet and CD-ROM versions.

Training SupportThe VNH was intended to be a training support tool—both for the professional

education of the military medical user and for the training of others. The survey used forcollecting feedback from VNH users requested vignettes on how the VNH was used forthis purpose. Appendix 2 contains excerpts of their responses. The most common usesfor the VNH as a training support tool7 are the following:

• Preparing lectures for professional education of medical staff

• Feedback on current guidelines for medical practice

• MANMED training

• Preventive medicine crew training (smoking prevention, first aid)

• Advancement training self-study guide

• Pharmacology training

• Sick call screening

• Staying current within specialty

• Training HM reservists, EMTs

• CBR training

7 The items listed were perceived by survey respondents as being training-related.

14

• Train laboratory technicians

• Patient feedback

• Pre-deployment training (port specific)

• HIV education

• Make LTGs for crew training

• Research clinical guidelines

• Source for information on administrative discharges for teaching Administra-tive Psychiatry course

• Source of graphics for training lectures

• Hotlinks in presentations

• Source for MEDLINE searches for use in developing training materials

• Safety lectures for flight crews.Thus, the VNH is seen as a resource for training materials. Professional Military

Education (PME), for both self-training to keep current and for advancement, was a ma-jor focus. The VNH is widely used as source materials for preventive medicine training(e.g., first aid, smoking prevention) of ship's crew.

Medical Planning ProcessUnlike digital medical libraries meant to serve the general public, the VNH is a

unique source of materials for planning medical aspects of military operations. As a mili-tary organization, the Navy must be prepared for combat operations. Another role thathas emerged in recent years requiring the participation of Navy medical professionals isOperations Other Than War (OTW), such as disaster relief. Tables 9 and 10 show thepercentage of Navy medical professional use of the VNH for both kinds of medical op-erations planning, and their evaluation of VNH in facilitating8 the planning process. Theresults suggest that the VNH is more widely used for planning operations OTW. When itwas used for this purpose, it mostly facilitated the process. The Internet version tended toprovide a greater degree of facilitation than the CD-ROM for both operational planninguses. This could be the result of the Internet version being more current and the scarcityof information on the CD-ROM.

8 In this context, facilitation refers to being able to locate information for structuring medical inputs,determining data-gathering requirements, and completing planning documents, and was generally helpful inthe process.

15

Table 9. Use of the VNH for Medical Planning for Combat Operations (percentage ofresponses)

SpecialtyAdminTechDentalOtherProviderTotal

Internet401735222024

UsedCD-ROM

382131121823

FacilitatedEither1

402135252326


CD-ROM1173291005049

Used either the Internet or CD-ROM versions of the VNH.

Table 10. Use of the VNH for Medical Planning for Operations Other Than War (percentageof responses)



UsedCD-ROM

324225172732

FacilitatedEither1

454729294242

Internet5498561008785

CD-ROM1496501008078

1 Used either the Internet or CD-ROM versions of the VNH.

Improved CareA primary purpose of the VNH is to provide information to military point-of-care

providers. A measure of effectiveness for this role is improved health care. Table 11shows VNH user's evaluation of the extent to which having the VNH available improvedpatient care.

Table 11. Degree of improved Care from VNH Use (percentage of users)

InternetSpecialty Great dealAdminTechDentalOtherProviderTotal

103117312023

Some374127546047

CD-ROMNone Great deal

532857152030

10199

252318

Some193614503431

None714577254350

The majority (70 percent) of those using the Internet version of the VNH to providecare felt that the available information resulted in some degree of improved care. (Themodal response was "some improvement.") While those in all specialties responded to

16

the question of improved care, perhaps the responses of providers should receive themost attention. Among providers, 80 percent of those using the VNH over the Internetfelt that its use resulted in improved care.

Provider ConfidenceImproved care can be the result of making the correct diagnosis and prescribing

proper treatment and follow-up. The audience for the VNH is often a health careprovider, such as a Corpsman, with limited medical training. The VNH can provide "justin time" medical information to aid in diagnosis and treatment prescription, in theabsence of other resources or colleagues with whom to consult. Table 12 showsestimates of the extent of provider confidence attributable to the VNH.

Table 12. Provider Confidence Attributed to the VNH1 (percentage with improvedconfidence)


DiagInternet

136529647058

nosesCD-ROM

105423405444

TreatmentInternet

156027606050

CD-ROM102725502124

The modal response was "some confidence."

When we again focus on providers, the majority said that the VNH did boost theirconfidence. The data suggest that the VNH plays a stronger role in boosting providerconfidence in making diagnoses (70 percent), than for making treatment decisions(60 percent).

Saving Patient Sick DaysHaving the correct medical information on hand can increase local capability in

diagnosing and treating and can result in savings to the Navy. An alternative to aprovider making a "just in time" diagnosis and prescribing the correct treatment is to seeka consult. Often, the patient must be sent away from the duty station for the consult.This results in a loss of productivity for both the patient and someone from the medicalstaff who may need to escort the patient. Table 13 shows responses to a series ofquestions seeking to measure the degree to which information provided on the VNH hadresulted in getting a patient back to work sooner by avoiding sick-in-quarter days orreturning the patient to full or limited duty status.

17

Table 13. Ways VNH Avoided Sick-ln-Quarter Days (percentage reporting SIQ avoidance)

Avoid MEDEVAC



CD-ROM

163132302426

Primaryprevention


CD-ROM

215929605849

Accuracy ofdiagnosis


CD-ROM

115624305041

Reducecomplications


CD-ROM

114429304336

Decrease needfor consult


CD-ROM

163824305037

Focusing on providers, the greatest savings arise from improved primary preventionpractices. Note that the other areas in which medical information can get a patient back towork more quickly are related to promptness and accuracy of a diagnosis. For example,with "just in time" medical information on hand to make a rapid and correct diagnosis,time is saved by avoiding the need for a consult, and perhaps a MEDEVAC for thatpurpose. This chain of events should return the patient to work sooner than if a diagnosisand subsequent treatment were delayed.

Information ResourcesMilitary medical professionals were asked to estimate the likelihood of using each of

a series of information resources to answer questions arising when seeing a patient. Theywere asked to report their time distribution for the resources listed (i.e., percentage of anarbitrary 2-hour period spent using each information resource). As an example: "Supposeyou had spent 2 hours researching a question: one hour reading journals (50% of timespent), 30 minutes on the Internet with the VNH (25%), and 30 minutes with an onboardcolleague (also 25%)..." The total should be 100 percent.

Table 14 shows the results separately for those with different medical specialties. Theuse of books and onboard colleagues dominate as the time spent using informationresources. Providers and dental specialists tended to favor other Internet resources overthe VNH. These results are somewhat ambiguous. They do not address the relativeefficiency of a given resource in conveying information per unit time. For example,providers reported spending an average of 7 minutes on the Internet with the VNH and 10minutes with some other Internet information resource. Does this speak to the relativeefficiency of using these resources? That is, does it take 7 minutes to locate informationon the VNH and 10 minutes on an alternate site to locate and read the same information?Or do providers fail to find what they are looking for and then use a different resource?The structure of the survey question does not allow us to answer these questions with anycertainty.

18

Table 14. Likelihood of Using Alternate Information Resources (percentage of time/patientvisit typically used1)

Resource

SpecialtyTechDentalProvider

Books352541

Journals71011

Onboardcolleagues

204215

Onshoreconsults

41211

VNHInternet

1517

Other VNH CD- Otherinternet

7610

ROM602

computer512

Other331

Based on 109 respondents completing the question.

Alternate Internet SitesThe VNH is one of many Internet-based information sites used by Navy medical

professionals. When asked to list alternate sites, respondents identified hundreds ofInternet-based resources. Table 15 lists the most frequently identified sites.

Table 15. Frequently Mentioned Alternate Internet-based Medical Information Resources

Internet ResourceAAFPAMAAmerican Dental Assn (ADA.ORG)BUMED homepageSUPERSCenters for Disease Control (www.cdc.gov)Grateful MedMDCONSULTMEDLINEMEDSCAPENational Library of Medicine (www.nlm.nih.gov)Naval Operations Medical Institute (NOMI)Navy Environmental Health center (www-nehc.med.navy.mil)OVIDPub Med Med Online NEHC web siteVirtual Hospital Iowa (VH.ORG)WEBMDWHO.ORG

19

Preferred ModalityAccess to the Internet for medical personnel at sea is often limited. However, if

Internet access were not a problem, more than half (52 percent) of respondents wouldprefer to access the VNH over the Internet rather than from the CD-ROM.9

Reasons for an Internet preference were mostly related to currency and breadth ofinformation. Reasons for an CD-ROM preference, were mainly related to an unreliable,slow, or non-existent Internet connection, and ease of use and portability the CD-ROM.

Suggested Enhancements to the VNHManagers of the VNH are constantly striving to improve their product(s). In an effort

to get feedback on how this should be done, users were surveyed on ways to improve andenhance the VNH. Respondents were asked to provide comments for both the Internetand CD-ROM versions. Four areas of VNH functionality were scrutinized:

Most helpful VNH resource. The MANMED (NAVMED P-l 17) along with itsadministrative instructions, was often cited as the most helpful resource for each of thespecialty groups. Providers used reference materials associated with their medicalspecialty (e.g., orthopedics, GMO Manual). No explicit suggestions for enhancements ofthe "most helpful resource" were mentioned.

Problems with use. Most problems cited with using the VNH were related toInternet access. Otherwise, currency of information on the CD-ROM, lack of detail, andneed for an improved search engine were often mentioned as problems.

Information for providers that should be added. Links to other Websites, moredental information, and HM-level information were cited as useful additions.

Information for patients that should be added. Links to other Websites, patienthandouts, and self-care "algorithms" were indicated as areas that would enhance VNHuse by patients.

Appendix 3 shows respondents' comments.

User EvaluationRespondents were asked to evaluate the VNH along four dimensions:

• Breadth of topics.

• Level of detail.

• Accuracy of information.

• Overall satisfaction.Tables 16 and 17 summarize their ratings for the Internet and CD-ROM versions,respectively. Overall, the VNH gets high marks. The Internet version had a higheroverall satisfaction rating (94 percent) than the CD-ROM version (90 percent). The

9 The Internet was the most frequently cited media preference for each specialty.

20

area receiving the lowest rating is "Level of Detail," particularly among providers(82 percent satisfactory).

Table 16. Ratings of Internet Version of VNH (percentage satisfactory)

Dimension


Breadth10096100929295

Detail959690928290

Accuracy100100100929598

Overallsatisfaction

1009995868994

Table 17. Ratings of CD-ROM Version of VNH (percentage satisfactory)

Dimension


Breadth1009786888389

Detail1009783757685

Accuracy100100100888592

Overallsatisfaction

1009786888590

Use of the VNH by Non-Medical Navy PersonnelThe VNH provides information for both the medical professional and patients on

health promotion/disease prevention and first aid, consumer health informationtextbooks, consumer health organizations, and aids to searching the Internet. There were177 responses to the Web-based survey for non-medical professionals by thoseidentifying themselves as military personnel. Sixty-one percent identified themselves asUSN personnel. Three percent of respondents were deployed on ships.

Given the poor response rate, it will not be possible to generalize the results. Thedata shown in table 18 are merely suggestive of the impressions of a self-selected sampleof non-medical military users of the Internet version of the VNH.

21

Table 18. Reasons for VNH Use by Military Patients (percentage of respondents using forstated reason)

Reason Percentage use

Self-diagnosis 21

Clarify diagnosis 22

Research medical problems of others 3 2

Research use of medications 33

Healthy living advice 31

Just curious 37

Other 36

Military "patients" use the VNH for multiple reasons. Curiosity and "other" were themost frequently cited reasons. (See appendix 4 for a listing of the "Other" reasons.)

The VNH is only one source of medical information for respondents. Table 19 showsthe percentage of respondents who use a variety of other sources.

Surprisingly, the most frequently cited source of medical information was printedmaterial, such as books. Medical providers, either at the duty station or elsewhere, wereindicated as a source of medical information by 81 percent of respondents.

Table 19. Sources of Medical Information for Military Patients (percentage using source)

Source Percentage using source

Duty station medical provider 53

Other medical provider 51

TV programs 27

Computer software 32

Printed material, e.g., books 62

Other Web sites 23

Family and friends 29

Radio 12

Newspapers 25

"Patients" were asked to indicate the benefits they felt they derived from using theVNH. Most people reported multiple benefits (table 20). Forty-three percent of Navypersonnel reported one or more benefits. The VNH was cited as having solved some sort

22

of medical problem using information found there. In some instances (12 percent), theVNH was perceived as being responsible for being able to "get back to work sooner."

Table 20. Benefits of VNH Use by Military Patients (percentage reporting benefit)

Benefit Percentage reporting

Solved a problem 30

Relieved anxiety 13

Able to get back to work quicker 12

Any 43

The level of satisfaction with the VNH was high for the military patients respondingto the survey (table 21). All respondents reported satisfaction with the accuracy ofinformation. It is unclear how these non-medical personnel evaluated "accuracy."

Table 21. Evaluation of VNH by Military Patients (percentage satisfactory)

Criterion Percentage satisfactory

Breadth of topics 95

Level of detail 93

Accuracy of information 100

Overall satisfaction 97

Again, there is no evidence of how well these data represent military patients in-general.

Business Case AnalysisThe Navy has made a financial investment in developing and supporting the VNH.

As such, it expects a return on its investment. It is necessary to quantify the costs andbenefits to demonstrate a return on investment. The return on investment can becalculated as either:

1. The cost of the VNH relative to an alternative information resource producingthe same benefit, or

2. The difference in the cost of the VNH and its resulting monetary benefit orsavings.

23

There are two reasonable alternatives for a library of medical reference materialscomparable to that provided by the Internet version of VNH. One is the CD-ROMversion of the VNH; the other is paper copies of these same materials.

Using the first calculation method, we would compare the cost of the Internet-basedVNH against the alternative of providing the Navy medical community with just the CD-ROM, and with providing paper copies of all the materials on the VNH.

CosfsThe determination of costs is straight-forward. The Navy spends $182,000 per year

for both versions of the VNH.10 This buys the Navy access to the University of Iowa'sVirtual Hospital infrastructure, by piggybacking the VNH on top of the Virtual Hospital'spersonnel, content creation workstations, Web servers, and bandwidth.

All costs are common to the WWW and CD-ROM versions of the VNH, with theexception of the $10,000 for CD-ROM production and distribution, which is CD-ROMspecific. Therefore, considering a CD-ROM-only version of the VNH as an alternative to theInternet version would produce little savings.1'

Providing paper copies of all of the materials on the VNH is impractical for thefollowing reasons:

• Space limitations aboard many Navy vessels.

• Excessive distribution costs.

• Difficulty of maintaining currency of information.For these reasons, we do not view a paper-copy version of the VNH a viable

alternative.

BenefitsAttributing monetary benefits to the VNH is a difficult matter. The results of a CNA

study12 of the costs and benefits of shipboard telemedicine was used to set the bounds forour estimates.

The results of the survey suggest that the VNH can directly save money by:

• Reducing lost productivity of Sailors by returning them to limited or full dutystatus sooner through more accurate diagnoses and treatment plans.

• Avoiding the need for MEDEVACs and on-shore consults by providinginformation to point-of-care doctors and Corpsmen, facilitating onboarddiagnosis and formulation of treatment plans.

10 The Navy has spent S682K, from March 1997, through September 2000, on the VNH, for an averageof $182K per year.

" The marginal cost of providing a CD-ROM copy of the VNH is about $2 per unit.12 F. E. Garcia et al. A Cost-Benefit Analysis of Shipboard Telemedicine, Center for Naval Analyses

Research Memorandum, Sept 1997.

24

• Avoiding SIQ days through health prevention measures motivated byinformation obtained from the VNH.

It is necessary to make some assumptions to determine the actual benefit arisingfrom the above. The assumptions fall into three categories:

1. Apportioning benefit between VNH and other contributors.

2. Estimating a multiplier, or number of cases per respondent.

3. Estimating the extent of benefit per incident (i.e., number of sick days).Apportioning. One assumption involves the relative contribution of information

gathered from the VNH. It is likely that other information contributes to thesebenefits. The analysis showed that medical providers use various information sources,such as consultation with onboard colleagues, other medical Websites, and journals,to make medical decisions. Respondents (providers) indicated that about 7 percent ofthe time researching a "typical" medical problem was associated with the VNH. Thisdoes not necessarily mean that 7 percent of the benefit should be attributed to theVNH. Suppose that the VNH provided the only information responsible for thedecision leading to the benefit. Under these circumstances, all of the benefit shouldbe attributed to the VNH. Alternately, if an onboard colleague initially provided thecritical information, and the VNH was used to confirm the diagnosis, it is not clearhow the benefit should be distributed. For purposes of our analysis, we will assumethat 100 percent of the benefit can be attributed to the VNH.

Incidents multiplier. Respondents were only required to report, in general, ifuse of the VNH resulted in avoiding MEDEVACs and/or reducing SIQ days. Thenumber of MEDEVACs avoided and the number of patients having reduced SIQ dayswas not elicited by the survey.13 Because we have no basis for determining thenumber of medical personnel represented by each respondent, we assume that therespondent only represents him/herself, resulting in an "incidents multiplier" of one.This will tend to produce a conservative estimate of savings. (There are likely to beother users of the VNH whose savings are the not captured with this approach.)

Extent. The data collected in the survey identified that use of the VNH resultedin reduced SIQ days. However, the data did not identify the number of incidents, orthe actual number of SIQ days saved. The situation was similar for avoidance ofMEDEVACs.

MEDEVACs. If the respondent reported "a great deal," this equated to oneMEDEVAC avoided. If the respondent reported "somewhat," this equatedto one-half MEDEVAC avoided.

SIQ days. We assume that each report that use of the VNH resulted in thepatient being returned to duty saved 2 SIQ days. This value incorporatestime saved by avoiding consultations, communication delays in contactingconsultants not at the duty station, and the like.

13 It was felt that respondents might not be able to accurately recall that level of detail and might nottake the time to gather the data.

25

Savings per incident. Costs avoided per MEDEVAC are estimated to be $4,400.This estimate is based on the CNA study cited earlier. The cost avoided by returning aSailor to duty more quickly is a measure of recaptured productivity. At a minimum,this can be represented by his/her pay and benefits. The average Sailor is paid$17/hour, based on an E-5 with 8 to 10 years of service (the median of the paygradedistribution.) The resulting savings is $136/day.14

The net savings for avoided MEDEVACs is calculated as:1. Number of incidents x cost avoided per incident, where

2. Number of incidents = (number of reports) x (report weight).

The net savings for avoided SIQ days is calculated as:1. Number of incidents x number of days x cost avoided per day, where2. Number of incidents = (Number of reports) x (report weight)

Based on these calculations, VNH use would result an annual estimated savings of$211,200 + $229,296 = $440,496. The results are summarized in table 22.

Table 22. Estimated Savings from VNH Use

VNH contribution to savingsGreat deal Somewhat Combined

MEDVACsNumber reportsWeightNumber incidentsCost/incident'Savings

151

15

660.533 48

$ 4,400$ 211,200

SIONumber reportsWeightNumber incidentsCost/incident2

Days/incidentNet savingsTotal saved

175 4931 0.5

421.5$ 136

2 2S 229,296$ 440,496

r$4,400/MEDEVAC2 $17/hour, times 8 hours, or $136/day

14 This estimate of the monetary value of a Sailor's work does not include other benefits, such as thevalue of medical care after retirement. Including such additional benefits would raise the hourly estimate to$30, or $240/day.

26

Subtracting the cost of $182,000 from the estimated savings shown in Table 22,results in a annual net positive saving of $258,000. This estimate is based on theassumption that all of the benefit is attributable to the use of the VNH.

From these data it is also possible to calculate the percentage of the benefit thatmust be attributed to the use of the VNH for net positive savings. That is, for savingsto be at least equal to the $182,000 annual cost. This is simply equal to the ratio ofcosts to benefits, or about 41 percent.15

ConclusionsA sample of convenience responded to a survey questionnaire aimed at eliciting

feedback on their use of the VNH. The survey was administered over the Internet and byemail. The majority of responses were to the Internet version, and may therefore reflectthe biases of those with an Internet connection at their duty stations.

Overall, for military medical professionals, the results suggest:

• The greatest use for the VNH is for patient care information.• Use of the VNH (Internet) resulted in improved patient care for 70 percent of

respondents.• Use of the VNH improved provider confidence in making diagnoses and

formulating treatment plans for 58 and 50 percent of respondents, respectively.• Use of the VNH resulted in avoiding SIQ days, mostly due to improved primary

prevention practices (57 percent of respondents).• Use of the VNH facilitated planning of Naval Operations (Operations Other Than

War—85 percent; Combat Operations—58 percent of respondents).• Use of the VNH for training is mainly for Professional Military Education (PME)

and teaching preventive measures to ships' crew.• The VNH is cost effective, producing an estimated annual net savings of

$258,000.• Users are mostly satisfied with the VNH (94 percent). Dissatisfaction was mainly

focused on the CD-ROM, which is only updated annually, resulting in complaintsthat the information was not current.

• The mostly widely used sources of medical information are books and periodicals.

The general impression obtained from VNH user comments was that Internetconnectivity afloat was poor (i.e., slow due to restricted bandwidth). However, as thecontent of the VNH is mainly text (HTML), even low bandwidth connections should beadequate.

The majority of health care professionals reporting data by email were selected toparticipate by higher authority. About half of those submitting comments said they never

182,000/440,496 = 0.413

27

heard of the VNH. This suggests that the CD-ROM is not widely available to somemedical personnel, despite the fact that over 5,000 are distributed annually.

RecommendationsBased on the data elicited from survey respondents, we recommend the following:

• Include a search engine on the CD-ROM version of the VNH.• Develop a process for spreading the word of the existence and uses of the VNH

for both Navy medical professionals and other Sailors (patients).• Update the CD-ROM more frequently; perhaps annually.• Improve site navigation.• Add more content on dental information and human anatomy.

28

Appendix 1. Hours and Reasons for VNH Use

The tablesor CD-ROM),

Specialty

Admin

Tech

Dental

Other

Provider

Total

below show the average hours of VNH use and media preference (Internetfor the different uses of information offered on VNH.

Table 1-1. VNH Use for Patient Care (hours/week)

Mean

Internet

0.7

1.1

0.4

1.7

1.2

1.0

hours/week

CD-ROM

0.0

1.1

0.1

0.6

0.5

0.6

Neither

72

25

54

40

19

32


Internet

28

59

38

50

66

55

CD-ROM

0

16

8

10

15

13

Table 1-2. VNH Use for Health Care Promotion (hours/week)

Specialty

Admin

Tech

Dental

Other

Provider

Total

Mean

Internet

0.7

0.5

0.1

2.2

0.5

0.5

hours/week

CD-ROM

0.1

0.6

0.4

0.2

0.2

0.4

Neither

0

18

11

1111

12


Internet

81

33

61

39

34

43

CD-ROM

19

49

28

50

55

45

Table 1-3. VNH Use for Own Prof Education (hours/week)

Specialty

Admin

Tech

Dental

Other

Provider

Total

Mean

Internet

0.5

1.6

0.2

3.3

1.0

1.2

hours/week

CD-ROM

0.1

1.4

0.6

2.0

0.5

0.8

Neither

78

19

51

44

23

33


Internet

19

54

34

39

64

50

CD-ROM

3

26

14

17

14

17

1-1

Table 1-4. VNH Use for Clinical Training of Others (hours/week)

Specialty

Admin

Tech

Dental

Other

Provider

Total

Mean

Internet

0.3

1.5

0.2

2.8

0.7

1.0

hours/week

CD-ROM

0.2

1.4

0.4

0.6

0.3

0.7


Neither

74

25

56

53

29

38

Internet

26

52

32

35

60

48

CD-ROM

0

24

12

12

11

14

Table 1-5. VNH Use for Administration (hours/week)

Specialty

Admin

Tech

Dental

Other

Provider

Total

Mean

Internet

2.6

1.0

0.5

1.0

0.7

1.1

hours/week

CD-ROM

0.2

1.3

0.0

2.2

0.3

0.7


Neither

63

26

53

56

33

39

Internet

34

50

35

38

55

47

CD-ROM

3

24

12

6

13

15

1-2

Appendix 2. Training Support

The descriptions of how military medical professionals reported using the VNH as atraining support tool are tabulated below. The responses are broken down by medical spe-cialty.

Table 2-1. Training Support

Specialty

Admin

Dental

Use

ReferenceBuild training curriculumTrain ship's crewGuidelines for PE

Train HMs to treat combat casualties

Patient feedback on Occupational MedicineResearch readiness standardsAdvancement trainingMANMED training of students

Inservice staff training

Reserve unit trainingDevelop sick-call screening courseCBR casualty training

Personal advancement

Inservice training

Other

Provider

Train AD laboratory techniciansDeployment training

UseasNAVMEDP-117Make LTGs for crew training

2-1

Specialty UseReference to supplement materials on-handUse manuals and interactive areas including x-ray and trauma pagesResearch clinical guidelines for training

Point out VNH as source of information for operational medicalofficersChemical weapons defense treatment training material sourceReference materials for medical staffPatient education at reserve center

BUMED articles to teach Admin discharges in Admin Psychiatry

GMO trainingUse VNH graphics for training presentationsHotlinks in presentationsHealth promotion; first aid trainingAnswer Corpsmen questions

Teach CE classes for EMTs and IDCsSick call screenSource for handoutsProfessional development of junior HM staffMEDLINE searchesMedical surveilance information for port visits

Safety lecture material sourceSeabee Operational Medical Guide training material source

2-2

Duty station

Ship

Ship

Ship

Ship

Specialty

IDC

HM

Other

Physician RT

Comments

This is a great tool for a IDC at sea. Also great source for training.

thanks

Please make these and any updates available to MSC Commands.

I would like to have as much of the website info placed on CD-ROM.I m not aware of how much data the CD has but if the website hasmore than 650MB; I m sure there are providers out there like me whowouldn't mind copying the site onto a hard drive from m

Ship Nurse I really appreciate this useful reference source being available to me atno cost.

5-7

Appendix 3. Suggested Improvements and Enhance-ments

Improvements and enhancements for the VNH suggested by military medical surveyrespondents are tabulated here. The tables describe:

1. Most helpful information resource.2. Greatest problem using the VNH.

3. Information for health care providers that should be added.4. Information for patients that should be added.

(Note that the suggestions were extracted from respondents'comments and are uned-ited.)

Specialty

Table 3-1. Most Helpful VNH Resource

ResourceAdmin MANMED P117 is available.Tech access to training manuals and publicationsTech BUMED administrationTech CORRESPONDENCE COURSESTech Diagnostic trainingTech HM 3&2; l&C; Merck ManualTech Immunizations information; current medical directives not found on Navy web sites; andTech MANMEDTech Manuals and InstructionsTech Operational Ob/GynTech picturesTech Reference MaterialTech SURFLANT 6000Tech The use of NAVMED P-117Tech Used to find updates and up to date findings.Dental DiagnosisDental policy references and manualsOther Flight surgeon manualOther Manual of Naval Preventive Medicine: NAVMED P-5010; U.S. NavyProvider admin on pregnant womenProvider Administrative infoProvider Administrative references; health promotionProvider Clinical infoProvider Diagnosis and treatmentProvider GMO manual; women's health 98Provider I am fairly new to the site; so hard to comment at presentProvider InstructionsProvider LibraryProvider no comment

3-1

Specialty ResourceProvider not used muchProvider OrthopedicsProvider pharm dataProvider pharmacyProvider The breakdown of topicsProvider The Links Page

Table 3-2. Greatest Problem Using the VNH

Specialty

Admin

Clinical

Clinical

Clinical

Clinical

Clinical

Clinical

Clinical

Dental

Dental

Dental

Other

Internet

slow downloads

Navigation

trying to locate information quickly

SPEED TO SLOW

Unable to access

getting used to find things quicker

Can t use at work

Downloading files while underway.

I HAD NEVER HEARD OF THEUNTIL TODAY.

Lack of detail

no dental information

Access time via internet.

CD-ROM

broken links

Navigation

same

MANMED needs to be updated; also youneed a better way to view and print articleson it.

CDROM Damage

getting used to find things quicker

Wish more of the website update made to theCD

Files being current.

VNHTHE DENTAL DEPT WAS NEVER SENTA COPY OF THE CD-ROM.

Did not use

no dental information

- Everybody wanting to borrow it. - Naviga-

Provider

Provider

Provider

Provider

Provider

Provider

Provider

tion through all of information is best ac-complished through Outline mode in myopinion.

Good info for MO s and IDC s but you Sameleave the basic Corpsmen without info atthat level.

Server down; Slow links; disconnects.

access to internet

same as above

Finding Something specific quickly

loss of internet access

Out of date; Limited selection

updating have old version

search engine was very limited

same

not having most recent CD

search is difficult to to get phrasing to same as internet problemmatch engine s

3-2

Specialty Internet CD-ROM

Provider

Provider

Provider

Provider

Provider

Provider

Provider

Provider

Too difficult to search and the material is Too troublesomeoften not up to date

Ship does not support internet access. Out of date material; old referencesHave to utilize VNH at home.

Couldn't get on the web

unfamiliarity with the site.

INCORRECT SPELLING!

Did not know it existed

Out of date

no comment

Need to use it more to be able to accessthings on it faster.

To slow to load up

Same

Out of date

Getting access via antiquated phone lines No problems

Table 3-3. Information for Health Care Providers to Add

Specialty TopicAdmin links to other sites for each category of informationTech Anything that relates to Preventive Medicine; EnvironmentalTech I honestly don't know. I really don't use it for patient careTech instructionsTech Send out messages with new information that has been addedDental develop the dental information for dental providers; IDCs andDental More involved treatment processesOther Egans Fundamentals of Respiratory Care Seventh Edition -Other I think it's about right. You don't want it to be too inclusive.Provider A resource to easily download current publications; directivesProvider Basic HM level info; field medicineProvider How about photos of real cases! An interactive site for dx &Provider Links between medical and administrative disposition ofProvider more text book access and linksProvider NAVMED P-5066 Nursing procedures manual; CNSL/CNSPProvider Orthopaedic Examination illustrationsProvider surgical emergencies

Specialty

Table 3-4. Information for Patients to Add

TopicAdminTechTech

links to other sites for each category of informationI honestly don't know. I really don't use it for patient careMore PT handouts for simple injuries and immunizations.

3-3

Specialty TopicTech Self CareTech stretching exercisesDental Links to more informationDental oral health maintenance and prevention infoOther about right as well, continued improvement is always welcome.Provider 100% compliance on all treatments given! Do your part & we'll do oursProvider Format like Griffith's explaining medical and administrative dispositionProvider More low impact physical therapy type topics; medication information sheetsProvider more patient handouts in printable format would be nice.Provider Take care of your self algorithms

3-4

Appendix 4. Other Reasons for VNH Use by Patients

Reasons non-medical survey responses gave for using the VNH Website are listed.

Table 4-1. Other Reasons for VNH Use by Military Patients

ReasonAccess to administrative manualsAdminBecause no military facility available in jrb forth worth texas area for retirees; and dependents.Belong to the VVACme/ceuCombat stressCombat stress/psychological issues-researchCommand forced me toEditor for base newspaperEnvironmental health & safetyFind requirements for commissioning physicals on nrotc students. Surface warfare. Aviation; nuclear; etc.For my jobHelp with establishing programs in our facilityI am a civilian working for the air force rote. My husband is a marine with the navy ROTC. I put togetherpower point slides for cadet instruction; CPR and 1st aid being 2!I am a safety & occ him specialist and am constantly researching items; trying to keep up to date oninformation that is job related.I'm the MID officer hereInfo on bone marrow donor proceduresInstructionsInstructorship in various physical activitiesJob related; OSH professionalKeep updatedLook at safety related itemsLooking for instructionsLooking for NAVMED instructionManual of medical departmentMedications availabilityMy grandfatherOccupational safetyOver-seas program developmentOverall medical informationP-l 17; other manuals not available at commandPersonnel administrationPrep. For visits to the doctorPrk eye surgeryProvide reference informationRecruit / applicant screening

4-1

ReasonReferred by other userReporting instructor for FS and AMSO at school of aviation safetyResearchResearchResearch for PA programResearch other itemsResearch pub for reservistsResearch pubs for civilian jobResearch separation physicalsResearching biological warfareResearching BUMED/Navy policyReviewing solicitationSearching for pharmacies available to retireesSearching for training courses that can be done at home; on-line.Shipboard training aidsStandards for restricted and unrestricted line officersSupport Brazilian navy aviation squadrons which don't have a flight surgeonTeaching and researchTo develop course materialTrainingTricare prime remote (active duty)Trying to figure out how to complete the overseas screening process from a remote duty station. Is 5 hoursfrom an MTF. I have a civilian provider; but how do we complete the forms?UndergroundWant to be a doctorWar college researchXO major shore site and saw the info in the CO call kit

4-2

Appendix 5. Comments

Comments volunteered by military medical respondents are tabulated below.

Table 5-1. Comments

Duty station Specialty Comments

Clinic

Clinic

Clinic

Clinic

Clinic

Clinic

Physician_RT Do not consider information posted there as reputable and trustworthy.

HM We need more sites like this for the navy

MSC_clinical I use it as a resource for PI 17. I do not like all the spelling errors ortyping errors that I find. That really puts me off; for both the CD-ROM and Internet versions.

Other How do I get the CD-ROM version?

IDC I liked using the CD-ROM on my ship. I only hope some day myclinic gets a system where we can easily access VNH.

Dentist I noticed a great lack of detail. The VNH idea is wonderful -especially for IDC s and other GMO s on their first operational tour.My concerns for them include the lack of treatment and diagnosisdetail and the possibility of having no internet connect.

Clinic

Clinic

Clinic

Clinic

Clinic

Clinic

Clinic

Clinic

IDC EXCELLENT PRODUCT. PLEASE KEEP IT UP...

Physician_RT Please forward a copy of the VNH-CD.

IDC Although no one reference can be the only reference; VNH comesclose.

IDC I rarely use the VNH resources. When I have used the CD-ROM itwas very informative because it had everything I was looking for atthe time. I found it lacking in researching the full breadth of myposition as an IDC.

Nurse I think it's great and I wish I knew about it sooner.

Physician_other I think this is a good resource that should continue and expand.

HM Presently; I am very pleased with the information you provide and willcontinue to use it.

HM The VNH website is a great resource and training tool. The CD-ROMis a great help also when the internet happens to be down in my area. Ithoroughly enjoy using the website and CD-ROM for my needs.

Clinic Dentist The oral cavity and related structures are part of the human systems.If nothing more than to support the IDC and the independent dentalofficer afloat; the VNH is an outstanding format for this.

Clinic HM VNH has always been an outstanding consolidated source ofinformation.

5-1


Clinic

Clinic

Clinic

FMF

HM I would like to see something related to Navy instructions. A ROMwith such info would be the greatest thing for staff training purposessince Picketed Herring.

Nurse As noted advertise more.

Physician_other VNH is very good. Hope it gets better. Why not have access to teleli-brary through VNH?

HM The Iowa Family Practice information was the key thing that attractedme to the VNH originally; I was disappointed that it was not on theCD. It is a very useful guide for sick call reference.

FMF

FMF

FMF

HM Outstanding resource for all to use.

Physician_other I would prefer a link to the Navy Library with Stat Ref access andperhaps some full text journal capability

HM Phenominal product. I have actually given copies to other Corpsmenfor use in their units. I am a Reservist and I find the VNH to be theperfect starting point in solving many of the question that arise on aday to day basis. Bravo Zulu; Semper Fidelas

FMF

FMF

FMF

FMF

FMF

FMF

FMF

Physician_RT The CD room without internet is not useful for professional medicalofficers.

HM Navigation can be improved quite a bit

HM I think it s a great tool; but by providing the required medical pubs andBUMED Instructions; would make it indispensable.

HM You guys and gals did a great job on the development of this web siteand CD. Bravo Zulu.

HM The door has finally opened up for my fellow Corpsman to accessvaluable medical information to support the Marine Corps in remoteplaces on I-I duty. Bravo - Zulu

Physician_other It is a great resource; which I have used many times. Unfortunately Ihave a 1998 CD-ROM which doesn't have nearly the information thatthe web site does. Both are necessary though as often Internet accessis not available

HM Think it s a great program. If you could put the BUMED Pubs inPDF. That would be great. Also a few of the other BUMED instruc-tions would be nice to.. The NATO BIO/CHEM for reference wouldbe good.. Great program. Great overall paper saver.. Keep

FMF IDC This is a great tool for operational medicine and I often use it for re-search and recommend this site to other providers and even patients.Thank you; this site can only get better.

5-2


FMF HM I was a little disappointed when I received the CD Rom and discov-ered it was not as complete as the web site. BUT; after thinking aboutit; it would be impossible to include all the information on the web siteon a single CD Rom. Also; multiple CD ROMs

FMF

FMF

Physician_other Out of date too quickly.

HM It would be nice to see an interactive program for Corpsmen to in-crease their knowledge of basic and advanced Medical procedures.Junior Corpsmen could have a program for the basics and SeniorCorpsmen could have programs on various specialty topics.

Hospital

Hospital

HM

Other

I have been using your site and CD ROM a lot lately and have referrednumerous of my co-workers to it who are looking for info dealingwith navy medicine

This is a wonderful way to disseminate a huge quantity of informationin one area. Medical personnel are now able to quickly gather infowithout having to peruse through piles of reference books.

Hospital

Hospital

Hospital

Hospital

MSC_Scientist The evaluation section of this survey should have at least a third op-tion. The VNH is not well organized for my specialty and I require avery broad range of information; much of which is not available on theVNH.

MSC_clinical this is a good site. Keep up the good work!

Nurse This resource was not known to me until this month. Perhaps morepublicity.

Physician_RT I have used the VNH very little. At its inception and with slow con-nections it was not useful during clinic times; easier resources wereavailable after hours. I have now been re-introduced to the site and itappears it will be much more helpful in now

Hospital

Hospital

Hospital

Hospital

Hospital

MSC_clinical VNH has been an excellent source of information for me personally.It would have been very helpful if this was available at the beginningof my career 26 years ago.

HM love to see some biomedical related topics

Physician_RT Internet medical access is critical. Military should provide links intoon-line library such as MD Consult Pub-Med texts etc.

Physician_RT Haven't used the CD ROM version

Physician_RT By your questions; you appear to be targeting operational medicalofficers. I am a sub-specialist at a major teaching hospital; which youmay want to take into consideration when reviewing my responses.

5-3


Hospital

Hospital

Hospital

DT

Nurse

Nurse

I don't have a current VNH CDROM and as I have access to the netdo not need one. Making one available to all operational and overseasproviders was helpful to me in the past. I think many providers are notfamiliar with all the VNH has to offer.

You are on the right track. Please continue to look ahead and integratethe new resources that come available. Making it as efficient as possi-ble will ultimately save money by saving on the long manhourswasted searching for the info needed.

I have been very satisfied with both the internet and CD versions ofVNH. I have been extremely satisfied with the technical support andassistance received when needed. Both resources are outstanding andneeded. Thanks for the effort.

Hospital

Hospital

Hospital

Hospital

Hospital

Hospital

Hospital

HQ

Nurse Overall and excellent job. I only wish that the Canadian Forces Medi-cal Service would be able to provide such an excellent service to us;up here in Canada. Cheers.

Physician_other Make the service faster

PhysicianJRT Visited VNH on several occasions over several years, concept is agood one and should be developed further. However; if I had a choiceI would elect sites such as MDConsult over NVH due to the widevariety of resources available on the site.

Physician_RT A Great resource! Keep improving it.

MSC_Admin. A great tool for healthcare professionals!

Nurse This method of healthcare is future focused and would like to see itcontinued to meet the needs of healthcare providers.

Nurse This resource was not know to me until this month. Perhaps morepublicity.

Physician_other It is a great help to have access to information that is not easily avail-able in my own (small)navy. This goes especially for all kind of hand-books (FS; GP; Corpsmen etc.)

HQ

HQ

HQ

MSC_Admin. Please keep it up! We need these resources in the field and withoutthe VNH we are back where we were 5 years ago!

Dentist Quite extensive looks good

IDC The internet is so readily available, so I do not go the VNH often.However it is nice having specific Navy medical info available.

HQ HM Thank goodness for this website. It has been a source of great helpsince I found it last year. Thank you and update; update; update.

NEPMU MSC_Scientist Just before taking this survey I took another look at the site and be-cause I was looking everywhere; not just for something specific; Ifound all kinds of useful stuff for preventive medicine. But the sitemarkets itself.

5-4


Other

Other

Other

Other

Other

Physician_RT This is a valuable resource and should be continued.

Physician_RT Potentially very useful -if current and easy to use.

DT The survey was too strongly oriented to provision of health care formy answers to be of great value.

Other It has become a very useful tool. I applaud whoever had the vision aswell as time and money to put it all together. I hope it will continue todevelop into an increasingly useful tool for the physicians out in thefield/off-shore.

Other I work in a Marine Corps Safety and Environmental shop. I mainlyuse the program to ensure that we are following proper procedure forour safety and env programs. It makes it easier to find most of theinformation I need in one place.

Other

Other

Other

Other

Other

Other

Other

Other

Physician_other I think VNH CD would be good but I must have missed where topick it up.-1 was aware of its existence but no one on our carrier usedit for our six month deployment. A distribution plan to major com-mands with a small presentation might be helpful

IDC I am very satisfied with VNH. I primarily use it presently as a refer-ence tool more than patient diagnosis and treatment protocols. GreatSource.

IDC I believe it is a great tool that can help out quite a bit in bolstering thequality of care and training that we can provide to our patients andcrewmembers.

HM I appreciate all your hard work!!!

MSC_clinical I would like to test the CDROM version when I get the chance; find-ing the site was a stoke of luck; before that I didn't have knowledge ofit. It is very useful and a constant learning facility

Other Thank you for opening this site to us. You have made a significantcontribution to the standardization of patient care. This is an amazingaccomplishment. Thank you.

HM This is the best resource that the Navy has created so far.

HM Thanks for the modality! Please keep improving. An icon to get di-rectly to instructions (BUMED/NAVMEDCOM) would be nice.

Other

Other

Other

HM Very nice tool. It adds greatly to the overall effectiveness of activeduty and retired Medical Department Personnel.

DT There should be inservice training at command levels for all levels ofprofessionals and ranks from E-l and up. The more people knowabout this and how to use it the better we will be to provide for ourpatients and customers.

MSC_Scientist Thank You for the VNH website! I have found it a useful tool andhave used it on numerous occasions.

5-5


Other

Other

Other

Other

Other

Research_Facility

HM

IDC

HM

If you want corpsmen who are not IDC trained to make better use ofVNH; then start by training them in its uses in corps school and outfitus with laptops and your CD-ROM

This is an outstanding program. I recommend it to everyone I know.

I don t know how I worked without it. I depend on VNH now.

MSC_Admin. Thanks for having it!

HM Provide information in PDF format as well.

HM Although I am not performing in the capacity of a clinical corpsman.Having the CD-ROM for VNH will help me continue with brushingup on the essential items required of my rate. I am looking forward toany new information that will be available on the site.

Seabees Physician_other

Seabees

Seabees

Physician_RT

HM

When I initially started using the web site it was fairly informative;especially when looking up instructions and regs. However; it lackedsignificant detail and often did not provide any new information.Once I found some key civilian sites (Fpnotebook)

Valuable tool in conjunction with more operationally oriented sources;and good for field troop training in medical topics...

How do I get VNH on CD-ROM? What is the possibility on havingthis link up to related research papers and Medical Journals?

Ship

Ship

Ship

Ship

HM As stated earlier; all Naval Ships Medical Departments should beplaced on the automatic mailing list to receive the CD-ROM and anyup-dates. Thank you very much for providing me with a copy of theCD-ROM and use of your information provided on both the CD andInternet.

IDC I'm very greatful for having the CD-ROM.

Physician_other like the CD; would like to receive an updated version

Physician_RT The Internet gives us relatively current info while the CD supplies uswith information if the Internet is not accessible especially smallerships.( Is the response address really [email protected] or

Ship

Ship

Ship

Ship

IDC I would like a new version of the CD. I would like all references re-quired by the BUMED 6280.1 series to be added. These instruc-tions/pubs are required to be on all ships.

Physician_RT VNH - It s a good idea

IDC I would like to see an easier way to find current instructions etc. forthose of us that are not really a computer guru.

Physician_other Great tool; would like the updated CD-Rom

5-6

evaluation of the virtual naval hospitalcrmd0002710.a2/final december 2000 evaluation of the virtual...

Documents