ACCURACY OF DIAGNOSES DELIVERED BY AN AUTOMATEDHAND-HELD NERVE CONDUCTION DEVICE IN COMPARISONTO STANDARD ELECTROPHYSIOLOGICAL TESTING IN PATIENTSWITH UNILATERAL LEG SYMPTOMSKEVIN SCHMIDT, MD, MBA,1 NATALY MONTES CHINEA, BS,2 ERIC J. SORENSON, MD,3

JEFFREY A. STROMMEN, MD,1 and ANDREA J. BOON, MD1

1Department of Physical Medicine and Rehabilitation, Mayo Clinic and Foundation, 200 First Street SW,Rochester, Minnesota 55902, USA

2University of Puerto Rico Medical School, San Juan, Puerto Rico3Department of Neurology, Mayo Clinic and Foundation, Rochester, Minnesota, USA

Accepted 5 September 2010

ABSTRACT: Introduction: Automated hand-held nerve con-duction study (NCS) devices are being marketed for use in thediagnosis of lumbosacral radiculopathy (LSR). In this study wecompared the specificity and sensitivity of a hand-held NCS de-vice for the detection of LSR with standard electrodiagnosticstudy (EDX). Methods: Fifty patients referred to a tertiary refer-ral electromyography (EMG) laboratory for testing of predomi-nantly unilateral leg symptoms (weakness, sensory complaints,and/or pain) were included in the investigation. Twenty-five nor-mal ‘‘control’’ subjects were later recruited to calculate the speci-ficity of the automated protocol. All patients underwent standardEDX and automated testing. Results: Raw NCS data were com-parable for both techniques; however, computer-generatedinterpretations delivered by the automated device showed highsensitivity with low specificity (i.e., many false positives) in bothsymptomatic patients and normal controls. Conclusions: Theautomated device accurately recorded raw data, but the inter-pretations provided were overly sensitive and lacked the speci-ficity necessary for a screening or diagnostic examination.

Muscle Nerve 43: 9–13, 2011

Several hand-held automated nerve conductionstudy (NCS) devices have been developed over thepast decade.1 Such systems utilize disposable ana-tomically configured biosensors that purportedlyallow practitioners who are not formally trained inelectrophysiological techniques to perform andinterpret accurate nerve conduction studies. Theresults are analyzed by a computer software systemthat automatically generates real-time reports. Thereports include nerve conduction data originallydisplayed by the monitor, response waveforms,comparison to reference data, and summary data,along with an ‘‘interpretation’’ of normal vs. abnor-mal. Such systems are designed to allow physiciansand practitioners who are not formally trained inelectrodiagnosis (EDX) to diagnose nerve lesions,such as diabetic neuropathy, carpal tunnel syn-drome, and lumbosacral radiculopathy (LSR).

There is significant concern within the electro-diagnostic physician community regarding these

devices for a number of reasons, including: (1)technical issues that cannot be controlled and canaffect NCS results; (2) performance of NCS in theabsence of needle electrode examination (NEE;considered an integral component of the standardEDX examination in order to rule in or out manydiagnoses)2; (3) lack of individualized test designand interpretation based on real-time acquisitionof clinical and electrophysiological data; and (4)performance of studies by personnel with no train-ing in electrodiagnosis and limited expertise inneuromuscular disease.

The NC-Stat device (Neurometrics, Inc., Wal-tham, Massachusetts) is currently one of the mostprevalent of the automated devices; it utilizes sin-gle-use, nerve-specific, anatomically configured bio-sensors, which are designed to ensure correctplacement and accurate NCS results. The monitorcustomizes and calibrates the test for individualpatients; corrects for skin temperature; and dis-plays the latency, duration, amplitude, and conduc-tion velocity of the nerve being studied. Data col-lected are transmitted via a ‘‘docking station’’ tothe company’s central database. The data are thencompared with reference values and analyzed, anda report is generated automatically. This report isthen transmitted to the practitioner via facsimileor e-mail and includes the nerve conduction dataoriginally displayed on the monitor, comparison toreference data, and summary data. The nerve con-duction data are archived by the manufacturer fortrending and sequential testing.

According to the manufacturer, the NC-Statmachine is able to reliably diagnose a number ofneurologic conditions, including lumbosacral ra-diculopathy (LSR; see www.neurometrix.com). Di-agnosis of radiculopathy is accomplished by meas-uring multiple F-wave responses, which arecompared with the asymptomatic limb and cor-rected for demographic covariates; however, evenif the recorded responses are shown as valid andreliable waveforms, the value of utilizing F-wavesfor the diagnosis of LSR remains controversial.First reported by Eisen and colleagues,3 the

Abbreviations: CTS, carpal tunnel syndrome; EDX, electrodiagnosis;EMG, electromyography; INR, international normalized ratio; LSR, lumbo-sacral radiculopathy; NCS, nerve conduction studies; NEE, needle elec-trode examination

Correspondence to: A. J. Boon; e-mail: andrea@mayo.edu

VC 2010 Wiley Periodicals, Inc.Published online 24 November 2010 in Wiley Online Library(wileyonlinelibrary.com). DOI 10.1002/mus.21911

Key words: automated nerve conduction studies, electromyography (EMG),radiculopathy, lumbosacral

Hand-held NCS Device Accuracy MUSCLE & NERVE January 2011 9

sensitivity of F-waves in the detection of LSR hasbeen shown to range from 18% to 90%.4 Someinvestigators have asserted that this variance is dueto methodological differences among the studiesand that, with correction for demographic covari-ance (i.e., patient age and height) and use of mul-tiple measurements in different nerves with com-parison to the asymptomatic limb, NCS alone mayprovide sensitivities that rival NEE.4,5 The NC-Statmachine works under this assumption, although ithas yet to be tested rigorously by non–industry-sponsored research.

Assuming that the NC-Stat is able to detect LSRwith a reasonable degree of sensitivity, a secondquestion is whether the results yield sufficient clini-cal specificity to allow non–EMG-trained physiciansto accurately diagnose and treat unilateral legsymptoms. The objective of this study was to com-pare the sensitivity and specificity of the NC-Statmachine with the ‘‘gold standard’’ of traditionalEDX in a population of patients referred for uni-lateral lower limb symptoms. We hypothesized thatthe sensitivity of this automated nerve conductiondevice in the diagnosis of L4, L5, and S1 radicu-lopathy would be inferior to the gold standard offormal electrodiagnostic testing and would yield ahigher false-positive rate than standard electrodiag-nostic testing in normal controls.

METHODS

The study population consisted of two groups. Thepatient cohort included 50 subjects, aged 18–75years, who were referred to the Mayo Clinic elec-tromyography (EMG) laboratory with complaintsof predominantly unilateral leg pain, numbness, orweakness for 20 days or more. Patients with knownconfounding diagnoses (i.e., already identifiedprior to their standard EDX study) were excluded(known peripheral neuropathy, myopathy, lowerextremity mononeuropathies, plexopathy, ordefects of neuromuscular transmission). Patientswith recent anticoagulant use and internationalnormalized ratio (INR) >3.0, history of bleedingdiathesis, or history of diabetes mellitus were alsoexcluded. All subjects underwent focused historyand physical examination as well as standard EDXtesting (NCS/NEE) at the discretion of the testingphysician. Once eligibility was determined subjectswere invited to participate in the study and offered$50 remuneration for their time. Informed con-sent was obtained according to the guidelines ofthe Mayo Clinic institutional review board.

Additional standard NCS were performed onthe contralateral limb such that each patientreceived bilateral fibular and tibial motor NCSwith F waves (minimum F-wave latency recorded).

Patients were then tested with the NC-Stat deviceprogrammed for general use, utilizing the ‘‘SciaticaProtocol.’’ This protocol included testing of bilat-eral tibial and fibular nerves and one sural nerve.Of the 50 symptomatic patients recruited to thepatient cohort, 22 had findings consistent with L4,L5, or S1 radiculopathy on standard EDX. Thiswas defined as: (1) old or residual radiculopathyby the presence of neurogenic motor unit poten-tial changes without polyphasia, varying motor unitpotentials, or increased insertional activity in atleast two muscles with the same root innervationbut different peripheral nerve innervation; or (2)active radiculopathy characterized by reducedrecruitment, long duration, complex and/or vary-ing motor unit potentials, and fibrillation poten-tials in at least two muscles with the same nerveroot innervation but different peripheral nerveinnervation. The remaining 28 symptomatic sub-jects had either a normal EDX exam or evidenceof another distinct neuromuscular diagnosis basedon their standard EDX study.

Initial data analysis, performed during the datacollection period, revealed a significant discrep-ancy between the results of standard EDX andthose of automated testing. For this reason a pro-tocol amendment was approved by the institutionalreview board to recruit an additional 25 controlsubjects. The control cohort consisted of 25 volun-teers, aged 18–75 years (14 of whom were over 50years), with no prior history of LSR. These subjectswere recruited from patients referred to the MayoEMG laboratory for upper limb symptoms (cervicalradiculopathy, carpal tunnel syndrome, ulnar neu-ropathy), who denied history of low back pain orlower limb symptoms, history of a neuromusculardisorder involving the lower limbs, or history of di-abetes. Clinic employees who met these criteriawere also recruited as subjects in the asymptomaticgroup. Testing included standard and automatedNCS (bilateral fibular and tibial motor NCS andsural sensory NCS of one limb) and limited NEE(including the tibialis anterior, peroneus longus,and medial gastrocnemius muscles) on one lowerlimb, performed by a staff physician who wasblinded to the subject’s symptoms.

All automated NCS were performed by experi-enced EMG technicians who were blinded to thefindings of the standard EDX. These technicianshad been trained by a Neurometrix company rep-resentative in appropriate placement of the biosen-sors and use of the device. Data were de-identifiedprior to transmission; the subject’s study number(1–75), month and year of birth, weight, andheight were provided as required by the manufac-turer in order to calculate normal values for thatindividual. Data acquired by the NC-Stat machine

10 Hand-held NCS Device Accuracy MUSCLE & NERVE January 2011

were transmitted to the manufacturer electroni-cally and were analyzed based on methods proprie-tary to the manufacturer. According to the manu-facturer the nerve conduction information wascompared with a large database of previouslyrecorded data, which were adjusted for the ageand height of the patient and subsequently deter-mined to be normal or abnormal. Transmitteddata were secured according to the manufacturer’sinformation privacy and security policy, which maybe accessed at http://www.neurometrix.com/privacy.htm.

Statistical Analysis. Each of the 50 subjects in thepatient cohort were classified by their standardEDX testing into an L4 radiculopathy, L5 radicu-lopathy, S1 radiculopathy, or no radiculopathy. Assome subjects had more than one level involvedon one or both sides, they may have been includedin more than one of the groups listed. For exam-ple, a subject with both an L5 and an S1 radiculop-athy on standard testing was included as 1 case ofL5 radiculopathy and 1 case of S1 radiculopathy.These results were compared with the results ofthe automated device interpretation in a similarfashion. In some subjects, the automated interpre-tation reported more than one radiculopathy.These subjects were again included in each rootlevel reported. For example, a subject with bilateralL5 radiculopathy and left S1 radiculopathy in theautomated interpretation report was included as 2cases of L5 radiculopathy and 1 case of S1radiculopathy.

For each radiculopathy level (L4, L5, and S1),a four-by-four table was created for the patientcohort, and sensitivity and specificity were calcu-lated. In addition, a four-by-four table was createdto include the identification of a radiculopathy atany level by either technique. From this table sensi-tivity and specificity were calculated for the detec-tion of at least one radiculopathy at any levelamong the symptomatic patient cohort. To testspecificity further, in the control cohort, specificitywas calculated as 1 � (number of false-positive ra-diculopathies identified by NC-stat device/numberof asymptomatic controls). The asymptomatic con-trol cohort had no evidence clinically or electro-physiologically to suggest LSR.

Raw data were compared by completing a sin-gle linear regression for each of the distal laten-cies, response amplitudes, and F-wave latencies. Acorrelation coefficient was obtained for each, andthey were reported comparing the raw databetween the two techniques.

RESULTS

Demographic data are presented in Table 1. In thecontrol cohort, subjects were stratified by age and

gender. In the symptomatic patient cohort, sub-jects were predominantly older, with 25 of 29females and 18 of 21 males >50 years of age.

In the symptomatic patient cohort the resultsof automated testing were analyzed and comparedusing standard EDX as the gold standard for thediagnosis of LSR. If standard NCS and EMG wereperformed bilaterally (at the electromyographer’sdiscretion), both limbs were included; therefore,although there were only 50 subjects in thiscohort, a total of 80 limbs were studied. The sensi-tivity and specificity of the NC-Stat device inpatients who presented with symptoms suggestiveof LSR are presented in Table 2. The device wasfound to be quite sensitive in detecting LSR,although it was poorly specific. Among the patientcohort, the sensitivity of the device was 0.00 in L4radiculopathy, 0.69 in L5 radiculopathy, and 0.64in S1 radiculopathy, with an overall specificity of0.14. On standard EDX testing, 28 of the 50 symp-tomatic patients were either found to have normalEDX findings or a neuromuscular diagnosis otherthan LSR. The automated interpretation corrobo-rated these findings in only 4 cases (all cases inwhich both the standard and automated testingwere interpreted as normal).

Diagnoses other than LSR on standard EDX test-ing included 3 cases of peripheral neuropathy, 1thoracic radiculopathy, 1 cauda equina syndrome, 2lumbosacral plexopathies, 1 motor neuron disease,and 1 L3 radiculopathy. The NC-Stat device did notidentify any of these diagnoses (the automated diag-nosis generated in all cases was LSR). Diagnosesother than LSR reported by the NC-Stat included 2peripheral neuropathies, 1 sensory peripheral neu-ropathy, 1 mild polyneuropathy, 1 sensory polyneu-ropathy, and 1 fibular neuropathy; however, in all ofthese cases, standard EDX was normal.

In the control cohort of 25 asymptomatic nor-mal subjects, all standard EDX studies were nor-mal, but with automated testing 18 of 25 had find-ings consistent with either LSR (17) or sensorypolyneuropathy (1), rendering a specificity of 0.32for automated testing. Specificities for the differentroot levels are presented in Table 3.

Table 1. Gender and mean age of enrolled subjects.

Group 1* Group 2† Group 3‡

Mean age (range) 59 (34–89) y 70 (48–85) y 51 (20–73) yFemales (<50 y) 3 1 5Females (>50 y) 13 12 6Males (<50 y) 2 1 5Males (>50 y) 10 8 9

*Symptomatic patients with no evidence of radiculopathy on NCS/EMG(n ¼ 28).†Symptomatic patients with radiculopathy on NCS/EMG (n ¼ 22).‡Asymptomatic controls (n ¼ 25).

Hand-held NCS Device Accuracy MUSCLE & NERVE January 2011 11

In addition to the data regarding presence ofradiculopathy we compared the raw nerve conduc-tion data obtained by the NC-Stat with conven-tional methods. The raw minimum F-wave valuesobtained with standard EDX testing were com-pared with the mean F-wave values obtained withthe automated device. A strong correlation wasfound between the standard and automated F-waves (Table 4). We also found a strong correla-tion between the amplitudes and distal latenciescollected with the two methods. Overall, we foundthat the NC-Stat device was able to accurately col-lect raw nerve conduction data.

DISCUSSION

An inexpensive, non-invasive, and automated tech-nique that accurately identifies neuromuscular con-ditions of the lower limb would be valuable to prac-titioners in a number of specialties. Indeed,automated NCS technology is increasingly beingembraced in a variety of medical settings. There areseveral devices that have been marketed as such, butnone have been studied rigorously to determine ifthey are a reliable substitute for standard EDX test-ing. At this time, there is no class I evidence to sup-port the use of such devices. All supportive studieshave been industry-sponsored, thus there is a con-cern regarding bias and validity. The goal of ourstudy was to determine whether such devices yieldsufficient specificity to allow non–EMG-trained physi-cians to diagnose unilateral leg symptoms with clini-cally relevant accuracy.

The NC-Stat is an easy-to-use hand-held devicethat uses anatomically designed biosensors for stimu-lation and recording of nerve conduction data.

These data are then uploaded to the manufacturerand analyzed using proprietary algorithms basedlargely on F-wave parameters. We found that theNC-Stat is able to accurately record raw F-wave datain agreement with other studies.6 Given theguarded, proprietary nature of the company’s ‘‘nor-mal’’ data set, however, the machine is subject tocriticism regarding interpretation errors and false-positive results. The general consensus is that F-waves alone are too non-specific to be clinically use-ful in diagnosing LSR in isolation. F-waves assess thefastest conducting motor fibers and, because mostradiculopathies result in partial axonal loss ratherthan demyelination, F-waves are often normal incases of LSR. This brings into question the validityof the algorithm used by the NC-Stat, which identi-fied radiculopathy in 37 of our 75 subjects who werefound to be normal on standard testing (including17 of 25 asymptomatic normal controls).

Our study has demonstrated a significant dis-crepancy between the results of standard EDX andthose of automated testing, for symptomatic andasymptomatic subjects alike. Even in cases wheresensitivity of the automated testing was relativelyhigh, specificity was unacceptably low. This was par-ticularly evident in the control group, in which spec-ificity was only 0.32. Automated and standard testingcorrelated perfectly in just 14 of the 75 subjects stud-ied, 11 of whom had normal exams with both test-ing methods. The automated device also incorrectlysuggested LSR in 8 cases where other neuromuscu-lar disease was detected on standard EDX, includingpathology such as motor neuron disease or lumbo-sacral plexopathy. If treatment decisions are madebased on inaccurate diagnoses, this can lead to

Table 2. Sensitivity and specificity for the NC-Stat device for 50 patients presenting with symptoms of lower limb pain,numbness, or weakness.*

NC-Stat Sensitivity 95% CI Specificity 95% CI

L4 radiculopathy 0/7 (0.00) 0.00–0.31 80/80† (1.00) 0.96–1.00L5 radiculopathy 18/26 (0.69) 0.51–0.84 39/61 (0.64) 0.52–0.75S1 radiculopathy 9/14 (0.64) 0.40–0.85 36/73 (0.49) 0.38–0.61Peripheral neuropathy 0/3 (0.00) 0.00–0.60 67/72 (0.93) 0.87–0.99Any radiculopathy 22/22 (1.0) 0.88–1.00 4/28 (0.14) 0.05–0.30

*Using EDX as ‘‘gold standard.’’†Number greater than 50 as some patients were diagnosed with multiple radiculopathies on standard EDX; for instance, left S1, bilateral L5 radiculopathy.

Table 3. Specificity of NC-Stat automated diagnoses inasymptomatic control subjects.

Level of LSR L4 L5 S1 Any

Subjects withoutLSR on standard EDX

25 25 25 25

NC-stat agrees without LSR* 25 17 13 8Specificity of NC-stat 1 0.68 0.52 0.32

*Cases in which both conventional and automated testing found no evi-dence of radiculopathy at the level noted.

Table 4. Correlation of F-wave values obtained with standardand automated testing.

Nerve Observations Pearson’s r P-value

Right fibular 29 0.92 <0.0001Left fibular 47 0.90 <0.0001Right tibial 29 0.82 <0.0001Left tibial 47 0.94 <0.0001

Standard testing recorded minimum F-wave latency; automated testingrecorded mean F-wave latency.

12 Hand-held NCS Device Accuracy MUSCLE & NERVE January 2011

unnecessary surgery or lack of appropriate treatmentfor serious disease processes.

Our findings are consistent with the only othernon–industry-sponsored study to date to evaluatethis technology. Katz evaluated the potential roleof an automated NCS device in screening indus-trial workers for carpal tunnel syndrome (CTS).7

He assessed 1695 primarily male asymptomaticworkers who were screened for CTS with the NC-Stat automated machine.7 Median motor distallatencies obtained with the machine were plottedon a frequency histogram, and the 95th percentilewas used as the cut-off for abnormality. The datafell within a normal distribution, but of the 587subjects who were identified as either borderline(221) or definitely abnormal (366) on the auto-mated report interpretation, only 81 actually hadmotor distal latencies greater than the 95th per-centile. Based on these findings, the investigatorsconcluded that the machine significantly over-diag-noses CTS in an asymptomatic population ofindustrial workers. This raises concern about thevalidity of ‘‘positive’’ studies in the occupationalmedicine clinic, hand surgery office, or industrialsetting in which such machines are being used asscreening or diagnostic tools.

Weaknesses of this study include the relativelysmall number of patients studied. Ideally, a largerpopulation of both symptomatic and asymptomaticsubjects would have been studied, but the cost ofthe biosensors for the automated device was pro-hibitive. Furthermore, based on the low levels ofspecificity of the automated device in both patientsand normal controls, it is unlikely that larger studynumbers would have increased specificity to ac-ceptable levels of a clinically useful test, given the95% confidence levels for the current data (Table2). Also, the study group was older and predomi-nantly Caucasian, reflecting the local population.

One further limitation that the device manufac-turer would likely point out is that, within thesymptomatic patient group, there were severalpatients with L4 radiculopathy diagnosed on stand-ard EDX who were not detected with automatedtesting. The manufacturer does not purport todiagnose L4 radiculopathy specifically but doesmarket this product for use in detecting ‘‘lumbo-sacral radiculopathy’’ and ‘‘diabetic polyneurop-athy’’ in patients who present with lower limbsymptoms. Therefore, it seems reasonable toassume that when this device is used in a primarycare setting to screen for LSR or polyneuropathy, anegative result could lead the testing physician toassume that LSR has been ruled out, when in factan L4 or more rostral radiculopathy is present.

In conclusion, the results of this study suggestthat the NC-Stat device significantly over-diagnosesLSR in both symptomatic and asymptomatic sub-jects, which may lead to unnecessary interventionor repeat testing. Our findings do not support theclinical application of automated testing in the di-agnosis of LSR.

REFERENCES

1. Lesser EA, Starr J, Kong X, Megerian JT, Gozani SN. Point-of-servicenerve conduction studies: an example of industry-driven disruptiveinnovation in health care. Perspect Biol Med 2007;50:40–53.

2. American Association of Neuromuscular and Electrodiagnostic Medi-cine. Proper performance and interpretation of electrodiagnosticstudies. Muscle Nerve 2006;33:436–439.

3. Eisen A, Schomer D, Melmed C. An electrophysiological method forexamining lumbosacral root compression. Can J Neurol Sci 1977;4:117–123.

4. Gozani SN, Kong X, Fisher MA. Factors influencing F-wave latencydetection of lumbosacral root lesions using a detection theory basedmodel. Clin Neurophysiol 2006;117:1449–1457.

5. Wells MD, Meyer AP, Emley M, Kong X, Sanchez R, Gozani SN.Detection of lumbosacral nerve root compression with a novel com-posite nerve conduction measurement. Spine 2002;27:2811–2819.

6. Leffler CT, Gozani SN, Cros D. Median neuropathy at the wrist: diag-nostic utility of clinical findings and an automated electrodiagnosticdevice. J Occup Environ Med 2000;42:398–409.

7. Katz RT. NC-stat as a screening tool for carpal tunnel syndrome inindustrial workers. J Occup Environ Med 2006;48:414–418.

Hand-held NCS Device Accuracy MUSCLE & NERVE January 2011 13