A POPULATION-BASED COMPARISON OF CIREN AND NASS CASES USING SIMILARITY SCORING Joel D. Stitzel1,2, Patrick Kilgo3, Brian Schmotzer3, H. Clay Gabler2, J. Wayne Meredith1

1 Wake Forest University School of Medicine, Medical Center Blvd, Winston-Salem, NC 27157 2 Virginia Tech – Wake Forest University Center for Injury Biomechanics, Medical Center Blvd, Winston-Salem, NC 27157 3 Emory University, Atlanta, GA 30322 ABSTRACT

The Crash Injury Research and Engineering Network (CIREN) provides significant details on injuries, and data on patient outcomes that is unavailable in the National Automotive Sampling System (NASS). However, CIREN cases are selected from specific Level I trauma centers with different inclusion criteria than those used for NASS, and the assertion that a given case is similar to the population of NASS cases is often made qualitatively. A robust, quantitative method is needed to compare CIREN to weighted NASS populations. This would greatly improve the usefulness and applicability of research conducted with data from the CIREN database. Our objective is to outline and demonstrate the utility of such a system to compare CIREN and NASS cases.

This study applies the Mahalanobis distance metric methodology to determine similarity between CIREN and NASS/CDS cases. The Mahalanobis distance method is a multivariate technique for population comparison. Independent variables considered were total delta V, age, weight, height, maximum AIS, ISS, model year, gender, maximum intrusion, number of lower and upper extremity injuries, and number of head and chest injuries. The technique provides a unit-independent quantitative score which can be used to identify similarity of CIREN and NASS cases.

Weighted NASS data and CIREN data were obtained for the years 2001-2005. NASS cases with Maximum AIS 3 resulted in a subset of 1,869 NASS cases, and 2,819 CIREN cases.

Results of the analysis demonstrate the utility of the distance technique to identify similarity of CIREN cases with the average NASS case. All NASS means were within 10% of CIREN and higher except Total Delta V was 9% higher for CIREN, CIREN cases were 50/50 male:female, and mortality of CIREN cases was 38% lower than for NASS/CDS. Results demonstrate that on average the

CIREN cases analyzed had a greater proportion clustered about the mean distance for NASS cases than did the NASS cases, with a very similar average distance (similarity score) of 3.75. Maximum distance (worst similarity score) was 14.75. The mean peak in probability density for CIREN (0.37) is slightly higher than for NASS (0.34). The distribution in the main body for both datasets is unimodal and nearly symmetric, and the overall distribution is slightly skewed right. For body region specific injuries, similiarity increases then decreases gradually with increasing number of injuries for distance scores between 1 and 2. Maximum AIS similarity increases and then begins to decrease with a minimum distance of about 1.5. Distance is very high (9) for very low age (<1 year), and similarity improves to 3-4 only around 12-13 years of age, representing the dissimilarity between children and adults in the CIREN and NASS populations. Model year distribution indicates hat older and more recent model years are not more associated with a lower similarity score in and of themselves.

This study has important implications for CIREN research studies, for which a similarity score could be assigned to each CIREN case based on overall, crash, anthropometric, or injury severity to the NASS population or another population of interest. The result is a tool that can be used with CIREN data to make stronger conclusions about the biomechanics and outcome of injuries, by quantitatively demonstrating real-world relevance. INTRODUCTION

Since its inception in 1998, the Crash Injury Research and Engineering Network (CIREN) has been involved in research into the causation of injury in automobile crashes (Runge 1996; Scally, McCullough et al. 1999; NHTSA 2001; Wang 2001). A unique feature of CIREN cases is that all case occupants receive care at a Level I trauma center and receive a full case review with medical personnel. Case occupants in the CIREN database also receive followup interviews, providing valuable data on outcomes of persons involved in car crashes.

However, CIREN case inclusion criteria are very different from National Automotive Sampling System (NASS) inclusion criteria, in which cases are sampled on a regional geographic basis, using two zone centers and 27 primary sampling units (PSUs) spread throughout the United States. NASS cases use a statistical weighting technique designed to make the sample population based, i.e. the NASS database is intended to be a national sample which represents accurately the spectrum of types of crashes in the United States.

CIREN is a powerful tool for collecting detailed injury data which relates to clinical outcomes. CIREN also collects the full set of National Automotive Sampling System (NASS) data (crash data), but

also collects other medical data. These data include the International Classification of Diseases, 9th edition (ICD-9) codes, Orthopedic Trauma Association (OTA) codes, and other more common clinical injury data, as well as long-term 6 and 12 month outcomes through the 36-item Short Form Health Survey (SF-36) and the Pediatric Quality of Life Inventory (PedsQL) reports, radiology reports and operative notes, a detailed hospital course for case occupants, and the case occupant’s radiology studies.

In the past CIREN has been used to analyze crash and vehicle characteristics and their relationship to injury patterns. Early CIREN work focused on factors influencing injury pattern and outcomes in car vs. car impacts versus other vehicle categories (Siegel, Loo et al. 2001). Leg and lower extremity injuries quickly surfaced as important, as those involved in CIREN research realized the outcomes from these injuries were of great importance, and some of the first CIREN biomechanics studies were undertaken (Assal, Huber et al. 2002; Tencer, Kaufman et al. 2002). One of the earliest CIREN studies focused also on aortic injury and the relationship of outcome to the associated injuries as well as the crash characteristics involved (Siegel, Smith et al. 2002).

Subsequently, several researchers have attempted to combine CIREN and NASS analyses into one study, drawing on CIREN for information about biomechanical causation of injury while also performing a side-by-side analysis of the same injury using NASS data. These studies have focused on biomechanics of neck injuries, specifically C2 Dens and Odontoid fractures (Yoganandan, Pintar et al. 2004; Yoganandan and Pintar 2005) the proper use of automatic crash notification systems to identify patients with serious injuries (Augenstein, Perdeck et al. 2003), and pelvic and thoracic injuries (Tencer, Kaufman et al. 2005), the last of which is notable for including an analysis of NCAP data as well.

The detailed injury data available in CIREN also lends itself to prediction of injury using computational models. These types of studies have been undertaken for the pelvis (Tencer, Kaufman et al. 2007), thoracic aorta (Siegel, Yang et al. 2006) and head (Moran, Key et al. 2004).

A number of general studies outline characteristics of cases in CIREN and may be organized by occupant type, crash type, or body region. Pediatric-specific CIREN studies have been undertaken (Brown, Jing et al. 2006), as well as studies focusing on the protective role of subcutaneous fat against injuries in the abdominal region (Wang, Bednarski et al. 2003). One crash type study focused on near-side impact and the effect of door characteristics (Tencer, Kaufman et al. 2005), and another on vehicle mismatch (specifically light truck and passenger vehicle) (Acierno, Kaufman et al. 2004). Body-region specific studies form a greater proportion of the studies

undertaken utilizing CIREN data. Maxillofacial and orbital injury studies focus on the role of airbags in orbital blowouts (Francis, Kaufman et al. 2006) and in maxillofacial injuries (Brookes 2004) and also highlight the role of the a-pillar in maxillofacial injuries (Brookes, Wang et al. 2003). Pelvic injuries are of frequent interest in CIREN studies and tend to focus on biomechanics and long term consequences (Stein, O'Connor et al. 2006). Upper extremity studies (Conroy, Schwartz et al. 2006), and lower extremity studies also exist, a few of the lower extremity studies focusing on calcaneal fractures (Benson, Conroy et al. 2007), lower extremity injuries and poor long term outcome (Read, Kufera et al. 2004), and femur fractures in low speed crashes and the effect of muscle forces (Tencer, Kaufman et al. 2002). Spinal cord (Smith, Siegel et al. 2005) and thoracic aortic injury studies exist (Siegel, Smith et al. 2004). Finally, mild traumatic brain injury (MTBI) studies (Dischinger, Read et al. 2003) which focus on long term outcomes, and head injury (Nirula, Mock et al. 2003) studies which focus on vehicle contact points are of course included.

A unifying theme in all of these studies has been their use of detailed injury data: injury data much more detailed than can be obtained from the NASS database, and their increased focus on long-term outcomes of patients. Many of these studies are outlined as investigations into mechanism of injury and injury outcomes, and many are presented as case studies with guarded assertions about their relationship to injury prevalence in the total population of motor vehicle crashes. In many of the studies very detailed information is available about the injury, yet the full vehicle investigation (NASS data) is still included, facilitating the understanding of vehicle crash performance and safety system performance to the injury sustained.

A potential limitation to extending the conclusions of many of these studies to the full population of cases in the United States has been the use of CIREN data itself. Because the CIREN dataset is a small dataset that does not represent a population-based sample, the conclusions about individual cases or a group of cases within CIREN carry some weight but are more difficult to generalize to the general population of crashes in the field. It is difficult to determine whether a given CIREN case represents a ‘one in a million’ crash or injury scenario that is not expected to be representative of a typical real-world crash.

There is a key similarity between NASS and CIREN, which enables one to make an important comparison between CIREN and NASS. Because the CIREN data system uses the NASS crash data structure as a backbone for all of its crash data, all of the entries are the same and this part of the database is virtually identical between NASS and CIREN. That means that one can use the fields entered in

NASS for a CIREN case to make a comparison between case similarity for the two databases.

Fortunately, analytical and statistical techniques allow easy comparison between datasets and there are statistical techniques to determine if a given case or set of cases is similar to another population of cases. Thus, the objective of this study is to present a method to quantify the similarity between a given CIREN case, or a subset of CIREN cases of interest, or the entire CIREN database, and the population of NASS cases. In so doing, it would be possible to say more definitively that measures from CIREN are highly applicable to real world crashes, based on their similarity to NASS crashes. The end goal is a ‘similarity score’ that can be assigned to a CIREN case to compare the datasets. Hypothetical questions that might be answered using this information include: Given a subset of NASS cases of interest,

1) How do we identify CIREN cases that ‘match’ it, or are very similar?

2) What is the best way to make a comparison? 3) What variables in NASS or CIREN are important in describing

the differences? All these questions are not answered but are proposed as a framework for future research.

Figure 1 is a pictorial representation of the question. The population of NASS cases can be described as a large circle, and a population of CIREN cases as a smaller circle. A ‘similarity score’ can be used to determine if CIREN cases are a subset of NASS/CDS, what the nature and size of this overlap is, and whether CIREN contains crashes that are very dissimilar, or too dissimilar to be considered to be within the universe of NASS crashes.

NASS/CDSCIREN

NASS/CDS

CIREN

Is CIREN a subset of NASS/CDS?

Does CIREN containcrash scenarios not in NASS/CDS or

vice - versa?

What is the nature and size of the

overlap?

CIREN

NASS/CDS

NASS/CDSCIREN

NASS/CDSCIREN

NASS/CDS

CIREN

NASS/CDS

CIREN

Is CIREN a subset of NASS/CDS?

Does CIREN containcrash scenarios not in NASS/CDS or

vice - versa?

What is the nature and size of the

overlap?

CIREN

NASS/CDS

CIREN

NASS/CDSNASS/CDS

Figure 1. Schematic of ways to think about NASS/CDS and CIREN databases.

Depending on the nature of a given study, conclusions from

the CIREN database may be criticized because of the sampling technique. It is easy to argue that a given CIREN crash may or may not be representative of the population in a qualitative way. However, few of these arguments are made quantitatively.

The objective of this study is to present a quantitative method by which a given CIREN case or a population of CIREN cases may

be compared quantitatively to the population of NASS cases, so that more substantive conclusions can be made using CIREN data. METHODS The general approach is to seek to compute a ‘similarity score’ between CIREN cases and the average NASS case. This similarity can be computed using any subset of variables contained in both databases, resulting in a k-dimensional distance, where k is the number of variables common between the data sources that are being compared.

The Mahalanobis distance computation can be used to make this comparison, and the resulting Mahalanobis Distance (DIST) can be thought of as a similarity score to compare CIREN cases to the ‘average’ NASS case.(Mahalanobis 1936) Mahalanobis distance is a multivariate measure of distance. All variables are compared on the same scale - DIST standardizes them, so individual units for each measurement don’t matter. The method takes into account the correlation between the k variables as well. So, for instance, if high Delta-V crashes would normally result in large intrusions, it is low delta V crashes with high intrusions or high delta V crashes with low intrusions that would stand out, not necessarily a very high delta V crash. These correlations statistically are made utilizing every entry considered in the analysis.

The distance score is based on correlations between variables. Mahalanobis distance differs from Euclidean distance as it takes into account the correlations of the data set and is invariant to scale– i.e. it is not dependent on the scale of measurements, or the units of the individual variables.

The Mahalanobis distance is computed using a group of k variables with mean μ:

( ) Tkμμμμμ ,...,,, 321=

And a covariance matrix ∑ for the vector x ( )T

kxxxxx ,...,,, 321= Mahalanobis Distance (DIST) is defined as:

( ) ( ) ( )μμ −−= ∑ − xxxDIST T 1 A higher DIST score means that a particular sample is less

similar to a population, and a lower DIST score means that the sample is more similar to the population. A DIST score of 0 would indicate a case with the means exactly the same as the population (x-μ=0, a scenario which becomes less probable with smaller datasets and greater numbers of variables being compared.

An example of this is to take the height and weight of two men (Figure 2). In general, weight is higher for people who are taller, and height is lower for people who weigh less. Male #1 is 6 inches

above average height and 100 lb above average weight. Male #2 is 6 inches above average height, and 100 lb below average weight. Using Euclidean distance, these two cases are equidistant from the average (centroid). Each person is equidistant in the x-y direction from the average. However, by Mahalanobis distance, Male #2 is much further from the average. The DIST technique identifies the Male who is outside the average for the population (Male #2) and assigns a larger distance to him than for Male #1.

Height

Wei

ght

Male #2

Male #1

1σ

AverageMale

Height

Wei

ght

Male #2

Male #1

1σ

AverageMale

Figure 2. Simple example illustrating Mahalanobis distance. Male 1 and 2 are

equidistant from the average in Cartesian coordinate space, but Male 2 is much further from the average and would have a higher DIST or less similarity.

Mahalanobis distance is therefore an effective multivariate

measure for how far points are apart in k-space in the context of the correlations between them. DIST can be used as a similarity score, and this is often how it is used.

Since DIST is a multivariate measure and the score is computed in k-space, k can be any number of variables, and could include every point in the dataset. For the current study, the variables in Table 1 were included. They are separated into Crash, Injury, and Anthropometric Variables.

Table 1. Variables included in analysis of NASS and CIREN databases. Crash Variables Injury Variables

Total Delta V Vehicle Model Year

Maximum Abbreviated Injury Score (MaxAIS)

Maximum Intrusion Injury Severity Score (ISS)Anthropometric

Variables Number of Lower Extremity Injuries

Occupant Age Occupant Gender

Number of Upper Extremity Injuries

Occupant Height Number of Head Injuries Occupant Weight Number of Chest Injuries

Included in the analysis were, for NASS, all cases with

Maximum Abbreviated Injury Score (MAIS) ≥ 3, from years 2001 – 2005. This resulted in a subset of 1869 NASS cases. For CIREN, all CIREN cases from 2001 to August 2006 were chosen, resulting in a subset of 2,819 cases.

Because the DIST score requires a mean for each variable included, and there are data missing from both the CIREN and NASS databases, missingness must be addressed. Three common possibilities are: 1. impute with averages, 2. estimate the covariance structure with multiple imputation methods, or 3. delete the observations. The selection of an imputation method taking into account the covariance structure would be subject to some debate and is probably deserving of a study in itself. Deleting the observations would result in a much reduced and incomplete dataset, and therefore was not performed. For this study, missing variables were imputed with column-wise averages.

The weighting system in NASS is part of what makes NASS the population-base sample that it is. In this analysis, NASS sampling weight coefficients were used to weight the covariance matrix. This approach is basically as if each row of data (each case occupant) was repeated k times, where k is the NASS weight for that row. This has the effect of expanding the population of NASS cases greatly, but that is what happens in all NASS analyses. RESULTS

PART I: GENERAL COMPARISON OF NASS AND CIREN DATABASES: Comparing the NASS and CIREN databases for one year (2005) highlights an important difference between the databases. Because the primary NASS/CDS selection criteria is that the case involve a ‘tow-away’ crash, NASS contains more than half (57%) MAIS 0 crashes, and 35% MAIS 1 crashes, and 6% MAIS 2 crashes. These are largely low injury crashes with very low threat to life. CIREN contains stipulations for including low MAIS (usually MAIS 2) but contains mostly MAIS 3, 4, and 5 cases (Figure 3).

57%

35%

6% 12%

47%

20%

13%

0.1%

0.2%

0.5%1.6% 4%4%

0%0%

10%

20%

30%

40%

50%

60%

0 1 2 3 4 5 6Maximum Abbreviated Injury Score (Max AIS)

% C

ases

NASS/CDS (weighted)CIREN

Figure 3. Distribution of cases by Maximum AIS – NASS/CDS 2005 versus

CIREN. NASS is composed of lower severity cases.

Selecting NASS/CDS and CIREN for AIS 3+ cases only in the year 2005, weighted and unweighted NASS/CDS and CIREN are compared (Figure 4). NASS/CDS and CIREN are very similar in terms of distribution of Maximum AIS, with lower Max AIS slightly underrepresented and higher Max AIS slightly overrepresented in CIREN. This effect seems to be enhanced by the weights in NASS, as unweighted numbers are closer to CIREN numbers.

67%

21%

9%

56%

22%

16%

6%

56%

23%

16%

3%

5%0%

10%

20%

30%

40%

50%

60%

70%

3 4 5 6Maximum Abbreviated Injury Score (Max AIS)

% C

ases

NASS/CDS (weighted)NASS/CDS (unweighted)CIREN

Figure 4. Distribution of MAIS3+ cases – NASS/CDS 2005 versus CIREN.

NASS/CDS and CIREN are roughly comparable. Weighted NASS and CIREN means for some of the variables

included in the analysis are shown in Table 2.

Table 2. Weighted NASS means and CIREN means for study dataset.

Variable Weighted NASS Mean

CIREN Mean

Total Delta V (kph) 60.4 65.8 Age (years) 38.0 36.7 Weight (kg) 75.7 72.8

Height (m) 1.69 1.64 Maximum AIS 3.70 3.39 ISS 23.6 21.6 Mortality 0.24 0.15 Vehicle Model Year 1994 1997 Male:Female 56:44 50:50

By normalizing CIREN means to NASS means (NASS

Mean/NASS Mean=1, CIREN Mean/NASS Mean=X) an easier comparison can be made. All CIREN variables compared are within 10% of their NASS means (vary from 0.9 to 1.1) except for mortality, which is about 63% of the NASS mean for mortality, and gender, which is 89% of the NASS mean. In the CIREN sample the ratio of males to females is more equal (50:50), and the mortality of CIREN patients is on average much lower than that of NASS patients. All CIREN patients go to a Level I trauma center, a potential partial explanation for differential mortality which was not tested in this study. A more likely explanation is that fatalities are not avoided in NASS, but are generally avoided in CIREN unless they occur after the person has been admitted to the trauma center. This finding, in fact, underscores the need for a metric to compare NASS to CIREN, as mortality variation would tend to change the average DIST for a NASS case versus a CIREN case. This is likely a variation clearly influenced by inclusion criteria, which must be dealt with in analysis by selecting cases for comparison based on the criteria, or coming up with a comparison metric like DIST.

0.6

0.7

0.8

0.9

1

1.1

Tota

l Del

ta V

Age

Wei

ght (

kg)

Hei

ght (

m)

Max

AIS ISS

Mor

talit

y

Mod

el Y

ear

Mal

e

Nor

mal

ized

Mea

n

Normalized NASS means

Normalized CIREN means

Figure 5. Normalized CIREN to NASS Means

Using the DIST score one of the most informative

comparisons to make is to compare the NASS and CIREN populations directly by looking at probability density of the similarity scores. In the calculations, the similarity score for a CIREN case is calculated using the covariance matrix for the NASS

population, and the NASS population similarity scores are also calculated and plotted for comparison.

The distance distributions are roughly the same. In fact there are a greater proportion of cases clustered about the mean distance for CIREN data than there are for NASS data. The mean peak for CIREN is slightly higher (0.37) than for NASS (0.34). The distribution in the main body for both datasets is unimodal and nearly symmetric, and the overall distribution is skewed toward lower similarity cases. Near the lowest distances there is a small collection of CIREN data that is very near the average which may represent imputed data. One might infer from this graphic, particularly from the greater proportion of CIREN data clustered about the mean distance of 2.5 to 4.0, that CIREN is more like the average NASS case than NASS, but NASS is the standard to which CIREN is being compared and though the distributions are similar they are not identical.

0

0.1

0.2

0.3

0.4

0 2 4 6 8 10 12 14Mahalanobis Distance

Prob

abili

ty D

ensi

ty

CIRENNASSCIREN

Figure 6. Probability density versus Mahalanobis distance.

The NASS and CIREN means are similarly distributed, with a greater proportion of CIREN cases clustered about the mean than for NASS cases.

Unfortunately the DIST score, being a calculation in k-space

and a multivariate measure, does not lend itself to simple visualization of the populations being compared. However, simplifications allow more direct comparison between the databases. One potential method is to perform a principal components (PC) analysis, whereby linear combinations of the variables best describing the variability within the datasets are calculated. Taking the first 2 principal components allows one to graphically depict the maximum amount of variation in the datasets while also allowing visualization in a two-dimensional (x and y axis) plot. This allows one to look at the data in a figure something like Figure 2, and compare population overlap. By plotting the first two principal components for each entry in the database and separating CIREN

from NASS one can visualize the overlap between the two datasets. The result is Figure 7.

There is a remarkable amount of overlap between the two datasets. The two principal components here together explain 39.0% of the point variability and so may be interpreted to give a pretty good picture of the overall variability in the dataset. They are not as comprehensive as the DIST score, but explain much of the variability. Some differences are apparent. Looking at the variable coefficients from the principal components, some of the differences (the CIREN data skewed to the right on the PC1 axis) are largely due to some low MAIS included in the analyses of CIREN that were not included in NASS.

One could also draw a smaller concentric ellipse centered on the CIREN data mean (PC1=PC2=0) represented by the thick black line but stopping at the maximum extent of the NASS population. In so doing one would minimize the DIST score as well. This has important implications, as in a given analysis the NASS or CIREN data of interest can be chosen at will, and by minimizing the DIST score or controlling the PC, one can choose the data from the other source with greater similarity to the mean. This effect is demonstrated visually using the PC plots, but in practice one could use one score – DIST – to accomplish this task.

-10 -5 0 5

-4

-2

0

2

4

6- NASS- CIREN

Principal Component 1

Prin

cipa

l Com

pone

nt 2

-10 -5 0 5

-4

-2

0

2

4

6- NASS- CIREN

Principal Component 1

Prin

cipa

l Com

pone

nt 2

-10 -5 0 5

-4

-2

0

2

4

6- NASS- CIREN- NASS- CIREN

Principal Component 1

Prin

cipa

l Com

pone

nt 2

Figure 7. Principal Component plots for NASS and CIREN data showing dataset overlap. The first 2 principal components explain 39.0% of the variability in the datasets. A subset (dark black line) of CIREN data has been chosen which lies

within the area containing NASS data.

The next few figures illustrate DIST versus some of the region-specific injury variables included in the analysis. For number of head, chest, upper extremity, and lower extremity injuries the minimum DIST increases gradually with increasing number of

injuries. With increasing number of injuries the minimum DIST increases very quickly and most quickly for lower extremity injuries.

02468

10121416

0 5 10 15# Head Injuries

Mah

alan

obis

Dis

tanc

e

02468

10121416

0 5 10# Chest Injuries

Mah

alan

obis

Dis

tanc

e

15

02468

10121416

0 5 10 15# Upper Extremity Injuries

Mah

alan

obis

Dis

tanc

e

02468

10121416

0 5 10 15 20 25# Lower Extremity Injuries

Mah

alan

obis

Dis

tanc

e

Figure 8. Mahalanobis Distance versus Number of Injuries to the Head, Chest,

Upper, and Lower Extremities Figure 9 shows DIST versus crash and vehicle characteristics

for Total Delta V, Maximum Intrusion, and Model Year. More of the data is clustered below the mean (Minimum DIST) than for injury variables. Model year alone has a relatively weak relationship to DIST.

02468

10121416

0 20 40 60 80 100 120 140 160Total Delta V (kph)

Mah

alan

obis

Dis

tanc

e

02468

10121416

0 30 60 90 120 15Maximum Intrusion (cm)

Mah

alan

obis

Dis

tanc

e

0

02468

10121416

1980 1985 1990 1995 2000 2005 2010Model Year

Mah

alan

obis

Dis

tanc

e

Figure 9. Mahalanobis distance versus Total Delta V, Maximum Intrusion, and

Model Year

For injuries, Maximum AIS drops between 3 and 4 for minimum distance (about 1.5), and for very low or very high Max

AIS, DIST increases quickly to 3 and higher, even for AIS 2 or 5. The ISS distribution is distinctly different from many of the graphics, because ISS has a collection of possible values which are not equally spaced, there is a less discernable V shape to the minimum DIST in the distribution and DIST does not seem to be very strongly influenced by ISS, but has a lower minimum at ISS of around 21.

02468

10121416

0 2 4 6 8Maximum AIS

Mah

alan

obis

Dis

tanc

e

02468

10121416

0 10 20 30 40 50 60 70 80Injury Severity Score

Mah

alan

obis

Dis

tanc

e

Figure 10. Mahalanobis distance versus Maximum AIS and ISS

Figure 11 shows anthropometric variables. For age, DIST is

very high for very low age representing the dissimilarity between children and adults in the CIREN population. Children are not as well represented in the CIREN or NASS databases as adults and so their anthropometric characteristics tend to create high DIST scores.

02468

10121416

0 10 20 30 40 50 60 70 80 90 100Age

Mah

alan

obis

Dis

tanc

e

02468

10121416

0 50 100 150 200Weight, kg

Mah

alan

obis

Dis

tanc

e

02468

10121416

40 60 80 100 120 140 160 180 200 220Height, cm

Mah

alan

obis

Dis

tanc

e

Figure 11. Mahalanobis Distance versus Age, Weight, and Height

DISCUSSION

Examining the DIST versus independent variable distributions does not give a complete picture of the relationship between that variable and DIST, as DIST is a multivariate measure, but does infer the effect of altering inclusion criteria in the CIREN database. It can hint at how to alter the independent variable

distributions chosen for inclusion in a given analysis, to minimize overall distance.

For instance, one could simply decide to take cases for an analysis for which the overall similarity score is no higher than 5. Using the current approach, that dataset would include 62.5% of the CIREN data. One could account for much of the 37.5% of the data excluded from such an analysis by using the thresholds in Table 3. These values are obtained by examining the minimum distance at each of the levels of the independent variable, and establishing that value as a threshold above which any value of the independent variable is likely to result in a DIST score of 5 or more.

The values in Table 3 represent a hypothetical scenario created to suggest how the inclusion criteria in a given analysis might be changed in order to influence the similarity score, without restricting the similarity score directly. A possible use of this information is to utilize similarity scoring to restrict future inclusion criteria in CIREN database in order to prospectively change inclusion criteria for the database to contain cases more similar to NASS cases. CIREN inclusion criteria are subject to variation over time but these criteria are usually changed based on expert discussion and qualitative reasoning. The similarity score gives a way potentially to quantify this reasoning.

However, there are limitations to this approach. Holding the independent variable within these ranges will eliminate high DIST scores certain to occur outside the range of interest. However limiting DIST by limiting a single independent variable is not sufficient to eliminate all low similarity cases within the ranges chosen. For that, a similarity score threshold would be needed which would have the effect of limiting all independent variables.

Table 3. Hypothetical CIREN minimums and maximums for inclusion criteria

to help attain a similarity score (DIST) of 5 or less

Variable CIREN Minimum

CIREN Mean

CIREN Maximum

Total Delta V (kph) 0 65.8 94 Age (years) 0 36.7 94 Weight (kg) 7 72.8 159 Height (cm) 97 164 N/A Maximum AIS 1 3.39 6 ISS 0 21.6 75 # Head Injuries 0 1.131 9 # Chest Injuries 0 1.236 8 # Upper Ex Injuries 0 1.542 9 # Lower Ex Injuries 0 2.72 11 Max Intrusion (cm) 0 30.8 101 Vehicle Model Year 1994 1997 N/A

Another possible use for the similarity score is for quality

control of the data. CIREN and NASS, like all databases, are subject to misentry of data or incorrect data. The Mahalanobis distance technique identified a number of cases in CIREN and NASS where there appeared to be errors in entry. An additional benefit of utilizing a similarity score would be that obvious errors in entry (entries off by an order of magnitude because of an extra digit typed, etc) would artificially increase the distance score and would draw further scrutiny during the QC process. For example, a 5 kph delta V crash with a maximum intrusion of 60 cm is highly unlikely and might indicate that 5 kph was entered when 50 kph was intended, or 60 cm was entered when 6 cm was intended. However it is notable that there are only 4 or 5 of these cases in CIREN, and similarly small percentage in NASS, which indicates a high level of quality control in these databases. The 6-10 highest DIST scores in CIREN are all children, with large delta v’s and low injuries, or high ISS with survival. This points out the uniqueness of the pediatric population. Nevertheless, the DIST score could be used as one form of quality control for the database.

One important point relates to missingness in the CIREN and NASS datasets. Since the dataset chosen in this study had small amounts of missing data column-wise averages were imputed rather than taking a more sophisticated approach like multiple imputation (Schafer 1999). For datasets with moderate to large amounts of missing data the use of these methods could have some utility and might confer some estimation advantages above and beyond simply imputing with sample means. This issue should be investigated further within the context of database comparisons using Mahalanobis Distance.

A final point regarding this method is as it relates to the pediatric population. Examining Figure 11 it is difficult to avoid the inference that age-related variables particularly in younger age ranges seem to be strongly influencing the DIST score. In terms of the implications for analysis of pediatric cases, the analysis suggests that aspects of pediatric anthropometry tend to change the DIST score substantially, but this is based on an analysis using relatively few variables. In practice, it may be prudent to select crash, demographic, or injury criteria of interest and then use the DIST metric to select a population of cases from NASS and CIREN that are similar. It would be up to the person(s) doing the analysis to decide which variables should be limited to select cases for comparison, and which should then be included in that comparison, or whether all variables should be included in that comparison. This analysis suggests that when two to three of 13 variables are highly variable for the pediatric population (height, weight, and age), those

variables tend to create large variation in DIST scores. This may be related to the fact that while other variables of interest may be expected to vary normally, height and weight tend to change exponentially with age. Future analysis using this method may need to account for variables which have an exponential dependence on other variables included in the analysis.

RECOMMENDATIONS FOR IMPLEMENTATION: One of the most useful methods to implement this approach may be to assign a similarity score to each CIREN case retrospectively and prospectively. One could create 4 additional derived variables in the CIREN database: overall similarity, anthropometric similarity, crash similarity, and injury similarity.

The overall similarity score could include all of the variables in the NASS and CIREN databases common between both databases, or a comprehensive subset determined by a panel of experts. The anthropometric similarity score would consist of height, weight, gender, etc. The crash similarity score would consist of PDOF, Delta V, Crush, Collision Deformation Classification, etc. The injury similarity would consist of MAIS, ISS, body region specific variables, outcome, etc. Each of these criteria should probably be determined by a group of experts familiar with CIREN and NASS and should be implemented into the CIREN database as a standard. Doing so would allow for the selection of CIREN cases for a given study.

The similarity scoring approach for selecting CIREN cases would not always need to be NASS mean-specific either. For instance if one were interested in doing a study of injuries in the elderly in CIREN, one could use an age criteria to select cases in NASS, and then calculate a similarity score for CIREN cases to identify that population in CIREN. Doing so would allow for greater control over the population selected, a quantitative measure of that populations’ or case’s similarity, and the conclusions drawn from such an analysis could be made in a quantitatively stronger way.

The similarity scoring approach for selecting CIREN cases would also not always need to be NASS specific. For instance if one were interested in doing a more exhaustive study of outcomes, the National Trauma Databank (NTDB), a collection of over 2 million cases with detailed injury and outcome data (containing the fact that an individual was involved in a car crash but not detailed data about the crash) could be used. One could include some of the anthropometric and all of the injury variables in CIREN to determine an injury similarity to the NTDB, and utilize the two databases to draw conclusions about injury and outcome in motor vehicle crashes.

Finally, the similarity score could be used as a sort of merge variable, to allow the strengths of NASS (population based) and

CIREN (detailed injury data) to be utilized concurrently. One could perform NASS analyses and infer injury and outcome information from similar cases in CIREN. Conversely, one could put a set of CIREN injuries or outcomes into context by presenting a set of similar cases from NASS and inferring real world incidence using population-based data. CONCLUSION

For the variables chosen, CIREN crashes are similar to NASS crashes for serious to fatal injuries (MAIS 3+),and taking into account all of the assumptions made in this study. This is a valuable approach to improve understanding and use of CIREN data. The method could be refined to be able to assign an overall NASS similarity score to each CIREN crash.

More importantly, one might envision a methodology to describe crash similarity, anthropometric similarity, or injury similarity alone. Any of these could be used to create a comparison. However, one could extend this to include all variables common between the datasets, and eliminate some of the criticism of the analysis method pertaining to the use of a non-population based sample. Lastly, a corollary to the study, is that Mahalanobis distance represents a good method to quality control data in CIREN and NASS by pointing out entries that are far from the ordinary.

This is the first study to quantitatively show the similarity between CIREN and NASS/CDS. The similarity score distributions are alike, showing CIREN cases are similar to NASS cases with serious injuries. A NASS similarity score assigned to each CIREN case could be used to select similar cases based on crash, anthropometric or injury characteristics. This has important implications for the usefulness of CIREN in rulemaking, for which NASS is currently the gold standard. An approach to incorporate the technique has been outlined. Given the detailed data on biomechanics of injuries and outcomes that is available from CIREN, this technique will allow CIREN data to have an increased level of relevance to organizations making decisions about how to improve crash safety. ACKNOWLEDGEMENTS

Work was performed for the Crash Injury Research and Engineering Network (CIREN) Project at Wake Forest University School of Medicine in cooperation with the United States Department of Transportation/National Highway Traffic Safety Administration (USDOT/NHTSA). Funding has been provided by Toyota Motor North America Inc. under Cooperative Agreement Number DTNH22-05-H-61001. Views expressed are those of the

authors and do not represent the views of any of the sponsors or NHTSA. REFERENCES Acierno, S., R. Kaufman, et al. (2004). "Vehicle mismatch: injury

patterns and severity." Accid Anal Prev 36(5): 761-72. Assal, M., P. Huber, et al. (2002). "Are drivers more likely to injure

their right or left foot in a frontal car crash: a crash and biomechanical investigation." Annu Proc Assoc Adv Automot Med 46: 273-88.

Augenstein, J., E. Perdeck, et al. (2003). "Characteristics of crashes that increase the risk of serious injuries." Annu Proc Assoc Adv Automot Med 47: 561-76.

Benson, E., C. Conroy, et al. (2007). "Calcaneal fractures in occupants involved in severe frontal motor vehicle crashes." Accid Anal Prev.

Brookes, C., S. Wang, et al. (2003). "Maxillofacial injuries in North American vehicle crashes." Eur J Emerg Med 10(1): 30-4.

Brookes, C. N. (2004). "Maxillofacial and ocular injuries in motor vehicle crashes." Ann R Coll Surg Engl 86(3): 149-55.

Brown, J. K., Y. Jing, et al. (2006). "Patterns of severe injury in pediatric car crash victims: Crash Injury Research Engineering Network database." J Pediatr Surg 41(2): 362-7.

Conroy, C., A. Schwartz, et al. (2006). "Upper extremity fracture patterns following motor vehicle crashes differ for drivers and passengers." Injury.

Dischinger, P., K. Read, et al. (2003). "Causes and outcomes of mild traumatic brain injury: an analysis of CIREN data." Annu Proc Assoc Adv Automot Med 47: 577-89.

Francis, D. O., R. Kaufman, et al. (2006). "Air bag-induced orbital blow-out fractures." Laryngoscope 116(11): 1966-72.

Mahalanobis, P. C. (1936). "On generalised distance in statistics." Proceedings of the National Institute of Sciences of India 12: 49-55.

Moran, S. G., J. S. Key, et al. (2004). "The applicability of a computer model for predicting head injury incurred during actual motor vehicle collisions." J Trauma 57(1): 99-103.

NHTSA (2001). "Crash Injury Research and Engineering Network (CIREN): an update." Ann Emerg Med 38(2): 181-2.

Nirula, R., C. Mock, et al. (2003). "Correlation of head injury to vehicle contact points using crash injury research and engineering network data." Accid Anal Prev 35(2): 201-10.

Read, K. M., J. A. Kufera, et al. (2004). "Life-altering outcomes after lower extremity injury sustained in motor vehicle crashes." J Trauma 57(4): 815-23.

Runge, J. W. (1996). "NHTSA Crash-Injury Research and Engineering Network." Ann Emerg Med 28(4): 450-1; discussion 451-2.

Scally, J. T., C. A. McCullough, et al. (1999). "Development of the Crash Injury Research and Engineering Network." Int J Trauma Nurs 5(4): 136-8.

Schafer, J. L. (1999). "Multiple imputation: a primer." Stat Methods Med Res 8(1): 3-15.

Siegel, J. H., G. Loo, et al. (2001). "Factors influencing the patterns of injuries and outcomes in car versus car crashes compared to sport utility, van, or pick-up truck versus car crashes: Crash Injury Research Engineering Network Study." J Trauma 51(5): 975-90.

Siegel, J. H., J. A. Smith, et al. (2004). "Change in velocity and energy dissipation on impact in motor vehicle crashes as a function of the direction of crash: key factors in the production of thoracic aortic injuries, their pattern of associated injuries and patient survival. A Crash Injury Research Engineering Network (CIREN) study." J Trauma 57(4): 760-77; discussion 777-8.

Siegel, J. H., J. A. Smith, et al. (2002). "Deceleration energy and change in velocity on impact: key factors in fatal versus potentially survivable motor vehicle crash (mvc) aortic injuries (AI): the role of associated injuries as determinants of outcome." Annu Proc Assoc Adv Automot Med 46: 315-38.

Siegel, J. H., K. H. Yang, et al. (2006). "Computer simulation and validation of the Archimedes Lever hypothesis as a mechanism for aortic isthmus disruption in a case of lateral impact motor vehicle crash: a Crash Injury Research Engineering Network (CIREN) study." J Trauma 60(5): 1072-82.

Smith, J. A., J. H. Siegel, et al. (2005). "Spine and spinal cord injury in motor vehicle crashes: a function of change in velocity and energy dissipation on impact with respect to the direction of crash." J Trauma 59(1): 117-31.

Stein, D. M., J. V. O'Connor, et al. (2006). "Risk factors associated with pelvic fractures sustained in motor vehicle collisions involving newer vehicles." J Trauma 61(1): 21-30; discussion 30-1.

Tencer, A. F., R. Kaufman, et al. (2005). "The role of door orientation on occupant injury in a nearside impact: a CIREN, MADYMO modeling and experimental study." Traffic Inj Prev 6(4): 372-8.

Tencer, A. F., R. Kaufman, et al. (2007). "Reducing primary and secondary impact loads on the pelvis during side impact." Traffic Inj Prev 8(1): 101-6.

Tencer, A. F., R. Kaufman, et al. (2005). "Factors affecting pelvic and thoracic forces in near-side impact crashes: a study of US-NCAP, NASS, and CIREN data." Accid Anal Prev 37(2): 287-93.

Tencer, A. F., R. Kaufman, et al. (2002). "Femur fractures in relatively low speed frontal crashes: the possible role of muscle forces." Accid Anal Prev 34(1): 1-11.

Wang, S. C. (2001). "Today's CIREN: news you can use." Ann Emerg Med 38(2): 182-3.

Wang, S. C., B. Bednarski, et al. (2003). "Increased depth of subcutaneous fat is protective against abdominal injuries in motor vehicle collisions." Annu Proc Assoc Adv Automot Med 47: 545-59.

Yoganandan, N., F. Pintar, et al. (2004). "Injury biomechanics of C2 dens fractures." Annu Proc Assoc Adv Automot Med 48: 323-37.

Yoganandan, N. and F. A. Pintar (2005). "Odontoid fracture in motor vehicle environments." Accid Anal Prev 37(3): 505-14.