upperclassmonroe.blogs.wm.eduupperclassmonroe.blogs.wm.edu/.../08/monroe-article.docx · web viewat...

Running Head: HOW SURVEY METHOD AFFECTS DIRECT-SPENDING REPORTS 0

How Survey Method Affects Direct-Spending Reports

Anthony J. Finch

HOW SURVEY METHOD AFFECTS DIRECT-SPENDING REPORTS 1

Abstract

This study examined the influence of survey method on direct spending reports at 3 Virginia

sporting events. Respondents were allocated to 3 survey groups: (1) online, (2) in-person, and

(3) online plus in-person. Statistical analysis found that survey method introduced bias to

respondents’ spending estimates, but this bias did not fully account for all differences observed

between survey methods. Consequently, an undefined demographic difference also existed

between the online and in-person groups. The data suggests that future economic impact studies

of sporting events should use both survey methods to minimize bias.


Introduction

From mega-events such as the Olympics and FIFA World Cup to smaller-scaled local events

like marathons, event-planners frequently encounter questions regarding the return that local

economies can expect for hosting sporting events. These events frequently incur very large

investments, which are sometimes difficult to quantify and track (Li & Blake, 2009). Since local

taxpayers sometimes assume the financial burdens of hosting the event (such as construction of

new facilities, grants awarded to the hosting organization, etc.), local governments frequently

require studies to be performed to verify that there is a positive return-on-investment. Of

particular concern in such analysis is the amount of ‘new’ money that can be brought into the

local economy by such events, where new money is defined as money brought to the local

economy by non-local tourists that would not have visited if not for the event in question.

To effectively measure these effects, economic impact studies typically collect survey data

from visitors attending the event. These data usually include estimated spending across several

different categories, distances travelled to and from the host location, and whether the event in

question was the primary reason for the visit, along with demographic questions (Crompton, Lee,

& Shuster, 2001; Mondello & Rishe, 2004). These figures are then used to infer the direct

spending of all new visitors the event brought to the economy, which in turn are applied to

models to predict the total economic impact of the event on the economy. These models have to

account for a number of complexities, since this direct spending induces more local spending by

local businesses, which do so again, causing a propagation effect that can significantly increase

the economic impact of such an event (Crompton, Lee, & Shuster, 2001). Some have questioned

this process, especially in large events, citing evidence that induced spending is distributed over

a larger region than that which is making the direct investment in supporting the event (Mills &


Rosentraub, 2013). If so, such regional substitution could substantially impact the returns

observed by the region actually hosting the event.

Regardless of any substitution effects, however, economic impact studies for these events

require surveys that can estimate event-induced spending with a certain degree of accuracy.

Typically, survey data of this sort is collected in-person on pen-and-paper surveys, either at the

event or, for mega-events, as tourists leave the area from airports or other public long-distance

transportation nodes; however, researchers have recently begun using online surveys more and

more (Case & Yang, 2009; Case, Dey, Lu, & Schwanz, 2013; Dolnicar, Laesser, and Matus,

2009). Online surveys have been shown to have several advantages, including significantly

reduced administration costs, but concerns have arisen about whether or not online surveys

generate results that are comparable to those produced by other survey methods (Ward, Clark, &

Zabriskie, 2012; Dolnicar, Laesser, & Matus, 2008).

Several recent studies have examined the advantages of online survey methods, and their

research has revealed a plethora of benefits of using online over in-person surveys, including

reduced costs and time investments (Dolnicar, Laesser, & Mattus, 2008; Ward, Clark, Zabriskie,

& Morris, 2012). However, despite these advantages, concerns over the consistency of data

between survey methods have rightly been drawn into question, as a number of studies have

yielded very different results across survey methods (Case, Dey, Lu, Phang, & Schwanz, 2013;

Case & Yang, 2009; Dolnicar, Laesser, & Matus, 2008) .

Review of Literature

Large-scale events like the Olympics and FIFA World Cup are often studied for their impact

on the local economies, and these studies are often dependent upon in-person survey

information. For example, Lee and Taylor used in-person surveys for their estimates of spending


at the 2002 World Cup (2005). In other instances, economic impact studies of large events use

national or local tourism information gathered by governmental bodies to estimate spending by

out-of-town tourists, such as in the case of Blake, who used statistics from the United Kingdom’s

Office for National Statistics (2005). Otherwise, as noted by Blake, these large-scale economic

impact studies simply fail to “provide the visitor spending estimates that the results are based

on.” Of course, this makes it difficult to comment on the quality of input data for such studies,

but in the case of Office for National Statistics, it may be noted that all data is, and always has

been, collected from using in-person, paper-and-pencil surveys (2005; 2013). In short, it can be

confidently stated that a significant body of economic impact research is dependent upon in-

person surveys of travel expenditures.

Reliance of such studies upon surveys has spawned a large body of research into the

dependability of these survey methods, some of which has supported the hypothesis that, in the

case of economic impact studies, survey method may not affect results. Olberding and Cobb

(2007) didn’t observe statistically significant differences in responses to online versus telephone

surveys, and went so far as to state that online survey methods were reliable “when there is

strong evidence that the population of interest utilizes email” (p. 29). Ward, Clark, Zabriskie,

and Morris (2012) found that college students were more willing to truthfully answer certain

highly personal survey questions in the online format than in the paper-and-pencil format, but

observed that the actual differences produced in most data would be “minimal at best in actual

interpretation of the data” (p. 522). Since it’s unlikely that direct spending estimates would fall

into the category of questions in which they observed these differences, it’s unlikely that there is

any important effect stemming from perceived anonymity in online surveys.


Further results, however, seem to indicate that significant differences may stem from survey

methods. Case, Dey, Lu, Phang, and Schwanz (2013) observed significantly higher direct

spending estimated collected from in-person surveys than online ones, although Case and Yang

(2009) observed precisely the opposite effect (although in both cases these effects were

statistically significant). Dolnicar, Laesser, and Matus (2008) found that the populations that

preferred mail surveys and that preferred online surveys were nonhomogeneous, although they

were equally unrepresentative of the target population (according to census data); however, they

also found that online surveys yielded superior data (lower dropout rate and fewer omitted

questions). They cautioned, though, that their study was taken from a highly technologically-

inclined population, and therefore may be unrepresentative of the population as a whole. In

addition, it did not test consistency of direct spending estimates between the two survey methods.

An unpublished, preliminary study by Case, Dey, Finch, Springman, and Sekhar in 2013 of

13 sports events conducted between 2008 and 2011, all involving in-person and online portions,

resulted with convincing evidence that the two methods would not produce similar results. Of

the 13 events, all of the observed online spending report averages were lower than their in-person

counterparts. A paired T-test on these data was performed (omitting a single point because it

was significantly affecting variance), yielding a p-value of less than 0.005; when a Wilcoxon

Signed-Rank test was performed to include the one excluded point, the resulting p-value was less

than 0.001. Furthermore, the only demographic difference between the populations was age,

although this difference was modest at most, with online survey methods showing slightly higher

proportions of individuals under 29 years of age and in-person survey methods showing slightly

higher proportions of individuals over 60 years of age. This study, along with evidence cited


above, provided evidence that survey method would significantly affect the results of economic

impact studies.

In the past, studies have attempted to correct bias in survey data by weighting along certain

parameters that may be more representative of the actual target population, such as in the case of

Leeworthy, Wiley, English, and Kriesel (2001), who note that “Constructing weighted averages

to account for differential sampling across strata in survey research is a common practice” (p.

96). However, since preliminary results seemed to imply that there are not significant

demographic differences between sample populations, weighting along certain population

parameters may be unwarranted and unhelpful.

In lieu of that methodology, this study has been constructed to investigate whether survey

respondents, on average, will respond according to a consistent pattern of relative over- or

underestimation. If so, then it may be possible to construct a heuristic to translate between the

two survey methodologies, and therefore to compensate for the relative bias between survey

methods.

Purpose of This Study

This study was designed to allow for the examination of the relative bias in direct spending

reports between survey methods. By splitting potential respondents into three groups (those

responding to in-person, online, and both in-person and online survey methods), we can observe

the effects of survey method independent of variance introduced by differing respondents and

possibly create a heuristic that would allow for hypothetical conversions between the two

methodologies.


Data Cleaning

Data entered in surveys, especially online surveys, often contains errors that would render it

unusable unless cleaned properly. Such errors include data that is entered as words instead of

numbers, numerical entries that are entered as ranges instead of solid estimates, or responses

such as “could not remember.” For this study, unusable data was cleaned to allow its use. Any

individuals that did not answer direct spending questions or that indicated that they lived in the

area where the event was held were not used for this analysis, as they would inevitably skew the

data. Data that was entered as a range instead of a single number was averaged (for example, an

entry of “150-200” was considered to be “175”), and data entered in the form of words was

directly translated to a numerical entry. In several cases, individuals took the same survey

multiple times. Often, these data were identical, but there were slight variations in several

instances. In such cases, whichever data set was more complete was used; if both were

complete, but used slightly differing numbers, these were averaged.

In one case, an individual responding to the online survey for the Crawlin’ Crab event entered

spending on retail shopping to be $300,125 but did not respond to the next question. This data

was not in line with the rest of the subject’s spending estimates, so it was assumed that the

respondent intended to indicate that spending on retail shopping was $300 and spending on

transportation was $125. In addition, one online response to the Lacrosse tournament data to the

Transportation category was entered as 11000; it was assumed, based on the respondent’s other

entries (which all had been entered with a “.00” at the end), that the respondent intended to enter

110.


Results

XTERRA 2014 East Coast Championships

The XTERRA 2014 East Coast Championship series of races was held in Richmond, VA, in

June 14th and 15th, 2014. 93 individuals responded to the in-person survey; 106 individuals

responded to the online survey. Of the 93 individuals that responded to the in-person survey, 58

provided email addresses and agreed to participate in the follow-up study that was performed;

however, only 22 individuals were able to be identified and their results paired. This was, in

part, due to a large number of individuals that filled out the follow-up survey, but did not provide

the same email address as they did in the original (in-person) survey.

Among respondents to both surveys, “Lodging” was the only individual category that had a

significant p-value for a difference in spending (with a p-value of 0.0096), but Total spending

also demonstrated a significant difference (with a p-value of 0.0123). In addition, variance ratio

tests resulted in significant p-values for the Food, Retail, Transportation, Tourism, and

Entertainment categories. A full table of results is available in Appendix A: Results from the

2014 XTERRA East Coast Championship.

Data from respondents that only responded to one survey method resulted in only one

significant p-value on t-tests between online and in-person means (Tourism), and one p-value

approaching significance (Entertainment, with a p-value of 0.0988). Variance ratio tests were

significant in all categories except the Lodging, Other and Total categories, although the Total

category had a p-value that was very close to significance (with a level of 0.0581).

Shapiro-Wilk tests for Normally-distributed residuals resulted in all significant p-values.

Breusch-Pagan test for heteroskedasticity were significant in the Retail and Tourism categories,

although the Food category was very close to significance (at a value of 0.0680). Influential


points were observed in the Food, Retail, Transportation, Entertainment, Other, and Total

categories. Both the Entertainment and Other categories had 2 influential points. Notably, one

respondent accounted for 5 out of the 8 total influential points. We have omitted this

individual’s responses in the forthcoming analysis to prevent contamination of more normal data.

This, combined with the Shapiro-Wilk tests and a theoretical reasoning for consistent

heteroskedasticity, also prompted us to use robust standard errors for the remainder of our

analysis.

Once the single respondent with consistently influential responses was removed, paired t-tests

resulted in reasonably similar results; however, variance ratio tests resulted in only 2 significant

values (Food and Tourism, with Other approaching significance), as opposed to 5 previous to the

respondent’s removal.

A series of linear regressions was created, forcibly omitting a constant, and their coefficients

were tested against the constant 1 to test for a 1-to-1 correspondence between results. Of these,

only the Lodging, Other, and Total categories resulted in significant p-values. None of the other

p-values would have been significant at even the 0.1 significance level. Additionally, normal

(unforced) robust linear regressions were created for each category. Significant p-values were

observed on ANOVA tests for the models for the Food, Lodging, Transportation, and Total

categories, although there was one influential point in the Transportation category that

dominated the model; when this point was omitted, the ANOVA resulted in a non-significant p-

value.

A comparison between the surveys of individuals that only responded to the online survey

and online portion of individuals that responded to both survey methods revealed strong

consistency between the two groups. None of the T-tests on the means had a significant p-


value, and only one variance ratio test had a significant p-value. On the other hand, there were

strong dissimilarities between survey results for individuals that responded only to the in-person

survey and results for those that agreed to participate in the follow-up survey online. Comparing

the two groups with T-tests yielded 4 significant p-values (Retail, Transportation, Tourism, and

Total) and 2 near-significant p-values (Food and Entertainment, with p-values of 0.0530 and

0.0707 respectively). Similarly, all variance ratio tests except that for the Other category had

significant p-values.

When the in-person group was divided between those individuals that agreed to participate in

the follow-up survey and those that refused to do so (regardless of whether or not they did so),

only one p-value for T-tests is significant at the 0.05 level (Lodging); however, 3 others are

significant at the 0.1 level (Retail, Tourism, and Total). Variance ratio tests between the same

groups result in significant p-values for all except the Transportation and Entertainment

categories.

T-tests performed on the in-person surveys of those who agreed to participate in the follow-up

survey and did so compared to those who agreed to do so and did not revealed only one

significant p-value (Transportation), and only one other p-value nearing significance

(Entertainment, with a p-value of 0.0915). Variance ratio tests, however, were significant for all

categories except for Lodging.

Beach Bash Lacrosse Tournament 2014

The Beach Bash Lacrosse Tournament was held June 22nd and 23rd, 2014, in Virginia Beach,

Virginia. 176 individuals responded to the in-person portion of the survey, with 44 providing

email addresses and agreeing to participate in the follow-up online survey; 18 responded to the

online follow-up, although one individual did not provide a matching email address, forcing that


data to be discarded. Unfortunately, the event organizer neglected to send out the online survey

for those that didn’t take the in-person survey; for this reason, the forthcoming analysis does not

have a general ‘online-only survey respondent’ group. In addition, for the Tourism,

Entertainment, and Other categories, there were not enough nonzero responses to perform a

proper analysis of the data provided by subjects that responded to both survey methods.

Paired T-Tests between in-person and online spending reports of individuals that responded to

both surveys yielded only one significant p-value, in the Entertainment category. No other

categories would have been significant at the 0.1 significance level. Additionally, no variance

ratio tests yielded significant p-values. Breusch-Pagan tests for heteroskedasticity yielded two

significant p-values, in Lodging and Retail, although Shapiro-Wilk tests for Normal residuals

yielded no significant p-values. A full summary of test results is available in Appendix B:

Results from the 2014 Beach Bash Lacrosse Tournament.

There were 2 influential observations for the Lodging category, 3 for Retail, 2 for

Transportation, and 2 for the Total category. None of these points was consistently influential.

In addition, many were influential solely because they constituted one of the few nonzero

responses received in a certain category.

A series of correlations was created with a forced constant of 0, and then tested against the

null hypothesis that the correlation coefficient was 1. None of these tests produced a significant

p-value, or a p-value approaching significance. Additionally, unforced linear regressions with

robust standard errors were created to analyze the relationships between each of the groups.

ANOVA tests on these models produced only 2 significant p-values, in the Lodging and Total

categories.


In order to verify that the group of respondents that took both surveys was representative of

the entire population, T-Tests and variance ratio tests were performed on each category between

these two groups. Ultimately, 4 categories yielded significant p-values on their T-Tests (Retail,

Transportation, Other and Total), and 3 categories yielded significant p-values on their variance

ratio tests (Retail, Transportation, and Other). An analysis of those that provided an email

(thereby agreeing to take the follow-up, regardless of whether the individual proceeded to do so

or not) compared to those that did not yielded 2 significant p-values on T-Tests (Entertainment

and Total), although 2 others (Food and Transportation) would have been significant at the 0.1

significance level. Variance ratio tests between these two groups yielded 4 significant p-values

(Transportation, Entertainment, Other, and Total), and another that was nearing significance

(Lodging).

A similar analysis of those that agreed to participate in the follow-up survey and those that

actually did produced only one significant p-value among the T-Tests (Retail), although the

Retail, Tourism, Entertainment, and Other categories all yielded significant p-values for variance

ratio tests.

Crawlin’ Crab Half-Marathon 2012

The online survey distributed by the event staff did not include an Other spending category;

however, the post-event survey exclusively for individuals that agreed to participate in the

follow-up survey did include this category, as did all of the in-person surveys. In addition, the

response rate of individuals that agreed to participate in the follow-up that actually did so was

very high. Therefore, there was not a large enough sample of individuals that agreed to

participate, but neglected to do so, to warrant an investigation comparing that group to those that

did participate in the follow-up or to individuals that didn’t provide any email address at all.


Paired T-Tests on each spending category between the data provided by individuals that

responded to both survey methods yielded 3 significant p-values. These were in Lodging,

Transportation, and Total spending. None of the other categories had p-values approaching

significance, and all three had higher averages reported online than in-person. Variance ratio

tests only yielded two significant values, in the Transportation and Total categories, which both

demonstrated higher variance in online spending reports than in-person ones.

Breusch-Pagan Tests for heteroskedasticity yielded significant values in all categories except

for the Lodging and Other categories; in addition, Shapiro-Wilk tests for Normal Residuals

yielded p-values that were all significant at the 0.001 level. There was 1 influential observation

for the Food, Tourism, Entertainment, Other, and Lodging categories, and 2 for the Total

category. Only one point was consistently influential. This point was not removed for further

analysis because the size of the sample was much larger than data gathered at the previously

mentioned events.

Tests of Regression Coefficients from regressions with forced 0 constants against the constant

1 resulted in only 2 significant p-values; these were in the Entertainment and Other categories,

which were both significant at the 0.01 significance level. ANOVA tests performed on unforced

linear regressions using robust standard errors for all categories resulted in only 1 non-significant

p-value, in the Entertainment category. This regression had a p-value of 0.1748. All regressions

and associated ANOVA tests are available in Appendix C: Results from the 2012 Crawlin’ Crab

Half-Marathon.

T-Tests of in-person responses between individuals that responded to both survey methods

and those that only responded in-person yielded only 1 significant p-value, in Entertainment.

Transportation was the only other category approaching significance, with a p-value of 0.0922.


Variance ratio tests on the same categories were all significant except for the Lodging category,

which had a p-value of 0.0605.

As mentioned above, there was no ‘Other’ spending category included in the online survey; it

is therefore omitted from the ‘Total’ category in the forthcoming analysis. Among online

respondents, those who responded to both survey methods exhibited significantly differentiated

spending from those that responded only to the online survey in only the Tourism category. The

Entertainment category was approaching significance, with a p-value of 0.0729. Variance ratio

tests were significant in every category except for Lodging, which had a p-value of 0.2861.

For those individuals that responded to only one survey method or the other, significant

differences between the means of online and in-person respondents were observed in every

category except Retail and Transportation, and the Retail category was approaching significance

with a p-value of 0.0774. Variance ratio tests also were significant in all categories except for

‘Food.’

Tests on All Respondents to Both Surveys

Wilcoxon Signed-Rank tests performed on each category yielded no significant p-values.

Regressions were performed on each category of spending using dummy variables to represent

the different events and dummy variables multiplied by the in-person result to find the effects of

each event separate from the others. These regressions resulted in significant event-specific

regression coefficients for the Food, Lodging, Tourism, and Entertainment categories.

The “Food” category exhibited significant p-values for all coefficients except the coefficient

on in-person reports of spending on food at the Crawlin’ Crab event; however, this coefficient

was approaching significance, with a p-value of 0.081. The Lodging category exhibited a single

significant p-value among the dummy variables; this was on spending at the XTERRA event.


The regression on the Tourism category yielded a significant p-value in both dummy variables

involving the Crawlin’ Crab event (the constant and spending at the Crawlin’ Crab event). The

entertainment regression exhibited a single p-value, on the Crawlin’ Crab constant.

Discussion

Several complications arose in the analysis of the results, which were addressed before any

interpretation was attempted. In the case of the XTERRA event, one influential point was

excluded from a large portion of the analysis. This was necessary because of the relatively small

sample size involved; simply using robust standard errors was insufficient to deal with this.

Comparatively, a number of influential points were left alone in the Crawlin’ Crab event because

the sample size was much larger. In addition, a certain amount of caution should be used when

viewing spending reports in the Tourism, Entertainment, Retail, and Other categories, since there

were very high proportions of 0 responses to these categories. These responses likely affected

the results observed, and also reduced the effective sample sizes considerably. Also, as

mentioned above, the event staff from the Beach Bash event neglected to send the online survey

to the event’s attendees. Fortunately, online surveys were sent to in-person respondents that

agreed to participate in the follow-up survey, allowing the forthcoming analysis to include at

least that portion of data collected from the event.

Among respondents to both surveys, paired T-tests yielded six significant p-values. The

Lodging and Total categories were both differentiated in the XTERRA and Crawlin’ Crab

events. In addition, the Entertainment category was differentiated in the Beach Bash event and

the Transportation category was differentiated in the Crawlin’ Crab event. Tellingly, both of the

events in which Lodging was significantly differentiated were also significantly differentiated

along the Total category in the same direction (i.e. that, if Lodging estimates were higher in-


person, then Total estimated spending was also higher in-person). In addition, Lodging estimates

given during the online survey for the Beach Bash event were also greater than their in-person

counterparts, although these numbers were not significantly differentiated. It should also be

noted that, in both cases, online estimates for spending on Lodging were greater than their in-

person counterparts.

Although this data is by no means conclusive, it does suggest that the Lodging category may

play a significant role in the differentiation between total spending reports given in-person and

online. Especially given the average size of respondents’ reported spending on Lodging

compared to other spending categories, this relationship seems as though it could be a key

differentiator between the two methodologies. Unfortunately, this is not corroborated by data

from individuals that responded to only one survey or the other. While the Crawlin’ Crab’s

online respondents reported significantly higher spending on lodging than their in-person

counterparts, the XTERRA event yielded precisely the opposite results (although these numbers

were not significantly differentiated, with a p-value of about 0.17). In addition, this would not

explain why preliminary research showed in-person respondents consistently reporting higher

spending than online respondents.

Furthermore, a series of Wilcoxon Signed-Rank tests on paired spending reports for each

category yielded no significant p-values. In fact, none of these tests had a p-value below 0.5.

This evidence gives relatively strong indication that a consistent trend does not exist for

individuals to give higher estimates for one survey method when compared to the other.

Additionally, regressions with forced constants of 0 resulted in only two regression coefficients

that were significantly differentiated from 1, and these were both from the Crawlin’ Crab half-


marathon. With all of these results combined, it is difficult to say with any certainty that the

survey method is truly inducing different responses when respondents are kept constant.

Because of this, it is very difficult to even begin creating a heuristic that may allow for the

conversion between online and in-person responses. Even if it were possible to do so

consistently within a single event, such a heuristic would be unlikely to be globally applicable,

given the inconsistency of results between the different events and the number of dummy

regression coefficients that were significantly differentiated from 0 in the large regressions on all

respondents to both surveys.

In looking at the possibility that individual heuristics may be used for specific events, this

study looked at a number of regressions on spending estimates between the two survey methods

while holding respondents constant. Conventional linear regressions using robust standard errors

on each of the categories in each of the events resulted in 9 significant p-values on the regression

coefficients (10, if the XTERRA event’s “Transportation” regression is counted, despite its

influential point). This gives a reasonably strong indication that the two surveys are correlated.

Still, there is not enough evidence to say that these regressions could be used as heuristics that

would enable conversion between online and in-person surveys, especially since there may be

other factors at work (as discussed below).

Since there was no online-only group available from the Beach Bash event, there were only 2

events to compare the results of online-only and in-person-only respondents. These results

conflicted, with the two being significantly differentiated along 5 of 7 categories of spending in

the Crawlin’ Crab event and being significantly differentiated along only 1 of the 8 categories of

spending. Coupled with preliminary data mentioned above, this suggests that the survey

methods are not equivalent in a significant number of cases. Thus, there is a certain discrepancy


present in the data. While respondents to both surveys exhibited some limited differentiation

between survey methods, these differences are not as significant as differences between general

online-only and in-person-only data would suggest. This suggests that there may be another

element at work, in addition to the survey method directly affecting individual respondents’

spending reports.

In order to explain this apparent discrepancy, a number of additional tests were performed to

examine whether or not the survey respondents that agreed to participate in both parts of the

survey were truly representative of both of the populations. Surprisingly, there were a large

number of significant p-values in tests between in-person responses of those individuals that

participated in the follow-up study and responses from those that did not do so. In all, there were

9 significant p-values among these tests, although only 1 of these came from the Crawlin’ Crab

event. Since only 24 of these tests were performed, this is a large enough proportion to warrant

attention, and it seems clear that there is some kind of correlation at work. Interestingly, these

results were not mirrored in the online responses, which had no significant p-values for T-tests

between online responses from respondents that only took the online survey and those that took

both surveys. This should be tempered by the fact that there was no online-only group for the

Beach Bash event, but it is still an important finding.

These findings were, however, somewhat difficult to explain. Preliminary studies indicated

that there should not be significant differences in the demographics of the in-person and online

populations, yet the present results suggest that some kind of difference likely separates those

that agree to participate in the follow-up survey from those that don’t, but no such difference

exists between the former and the online group as a whole. Although online literacy seems a

likely candidate at first, this would not make sense, since all participants in the events had email


addresses that they provided to the event coordinators when they signed up for the events (which

was how online surveys were distributed after the event). In short, it seems as though there is

some demographic difference between the groups for which preliminary research did not

account, but this unforeseen element cannot be identified without further investigation.

Variance ratio tests were performed in almost every instance that T-tests were. The vast

majority of these tests were significant, indicating that the distributions arising from these

various categories and survey groups may not have been similar, even in cases where their means

were alike. Although an in-depth examination of the distributions arising from responses to

various survey methods is difficult to construct based solely on averages and variance-ratio tests,

these tests suggest that even survey groups that seem relatively similar based upon their averages

and T-tests may, in fact, be produced by very different populations with very different

distributions.

Conclusion

This study yielded a number of significant findings, despite its failure to find a consistent

heuristic that could translate between online and in-person responses to direct spending reports at

sporting events. Unpublished preliminary results suggesting that in-person responses should be

consistently higher than their in-person counterparts were supported, but seem unlikely to be due

solely to respondents’ propensity to answer survey methods differently. It appears as though the

groups of individuals that respond to each of the survey methods may not be entirely consistent.

This conclusion contradicts preliminary research that suggested that the two groups should be

demographically consistent. It remains unclear how the two groups may be differentiated.

A heuristic translator between the two survey methods may still be viable, but would require

multiple parts. If this demographic factor can be identified, weighted averages can be used to


account for these differences in the same manner as Leeworthy, Wiley, English, and Kriesel in

2001. Once such a method is applied, it may or may not be necessary to use a regression to

account for bias introduced by the survey methods to individual spending reports. If so, present

results suggest that such heuristics will likely be applicable only to the event from which they are

generated. It will be difficult to determine which survey method more accurately represents the

target population (all event-goers), but identifying what truly separates the groups could suggest

what each of the survey methods may be misrepresenting. This would be invaluable in future

researchers’ analyses, allowing them to more accurately portray spending induced by sports

events.

Regardless of the reasons, the results suggest that online and in-person survey methodologies

are not equivalent. This is an important finding, since previous research has yielded conflicting

results. Therefore, future researchers conducting direct spending surveys should gather both

online and in-person data and analyze these data separately. This would eliminate the relative

biases of each method, for insignificant additional cost when compared to the costs of

conducting only an in-person survey. Some surveyors conducting online-only surveys may elect

not to conduct both surveys, however, because of the relatively large additional costs associated

with the in-person data collection. Given the apparent propensity for online spending report

averages to be lower than in-person averages, this seems to be more prudent, at least, than using

only the in-person surveys since results will likely be lower (and thus less prone to

overestimation, at least relative to in-person surveys).

In sum, this study supports the claim that online and in-person survey methods yield

significantly differentiated results and suggests that further research into the demographics of

respondents to these two survey methods is necessary. Until the nature of these differences


becomes apparent, results from this study indicate that future research using direct spending

reports should acquire such reports from both survey methods and keep them separate during

analysis. Since most studies of direct spending reports are conducted to facilitate cost-benefit

analyses, it seems prudent for researchers that cannot use both methods to use online spending

reports because of their propensity to underestimate spending when compared to in-person

reports. The present results suggest that this would lead to more conservative spending

estimates.


References

Blake, A. (2005). The economic impact of the London 2012 Olympics. Christel DeHaan

Tourism and Travel Research Institute. Retrieved May 19, 2014, from

http://epress.lib.uts.edu.au/research/bitstream/handle/2100/994/Impact 2005_5.pdf

Case, R., & Yang, W. (2009). A study to examine differences between in person and online

survey methodologies. Sport Management International Journal, 5(2), 5-10.

Case, R., Dey, T., Lu, J., Phang, J., & Schwanz, A. (2013). Participant spending at sporting

events: An examination of survey methodologies. Journal of Convention and Event

Tourism, 14, 1-21. doi: 10.1080/15470148.2012.755607

Crompton, J. L., Lee, S., & Shuster, T. J. (2001). A guide for undertaking economic impact

studies: The springfest example. Journal of Travel Research, 40(79), 79-87. doi:

10.1177/004728750104000110

Dolnicar, S., Laesser, C., & Matus, K. (2009). Online versus paper: Format effects in tourism

surveys. Journal of Travel Research, 47(3), 295-316. doi: 10.1177/0047287508326506

Lee, C., & Taylor, T. (2005). Critical Reflections On The Economic Impact Assessment Of A

Mega-event: The Case Of 2002 FIFA World Cup. Tourism Management, 595-603.

Leeworthy, V., Wiley, P., English, D., & Kriesel, W. (2001). Correcting response bias in tourist

spending surveys. Annals of Tourism Research, 83-97.

Li, S., & Blake, A. (2009). Estimating Olympic-related investment and

expenditure. International Journal of Tourism Research, 337-356.

Mills, B., & Rosentraub, M. (2013). Hosting mega-events: A guide to the evaluation of

development effects in integrated metropolitan regions. Tourism Management, 238-246.


Mondello, M. J., & Rishe, P. (2004). Comparative economic analyses: Across cities, events, and

demographics. Economic Development Quarterly, 18(331), 331-342. doi:

10.1177/0891242404269505

O'Brien, M., Savage, D., Osborn, A., & Lovegrove, J. (2005). Travel trends 2004: A report on

the International Passenger Survey. Retrieved May 19, 2014, from

http://www.ons.gov.uk/ons/rel/ott/travel-trends/index.html

Olberding, D., & Cobb, S. (2007). On-line and telephone surveys: The impact of survey mode on

spending estimates by participants in a major urban marathon. ICHPER-SD, 2(1), 27-32.

Travel Trends, 2013. (2014). Retrieved May 19, 2014, from

http://www.ons.gov.uk/ons/rel/ott/travel-trends/2013/index.html

Ward, P., Clark, T., & Zabriskie, R. (2012). Paper/pencil versus online data collection: An

exploratory study. Journal of Leisure Research, 44(4), 507-530.


Appendix A: Results from the 2014 XTERRA East Coast Championship

Comparisons of Paired Observations from Respondents to Both Surveys

Paired T-Tests

Category Online Mean IP Mean P-ValueFood 135.91 104.32 0.0924Lodging 183.41 155.59 0.0096Retail 35 14.55 0.2133Transportation 77.3 43.64 0.0783Tourism 0.68 1.82 0.4879Entertainment 11.86 10 0.74Other 26.67 28.81 0.9092Total 469.62 357.41 0.0123

F-Tests for Variance

Category Online Std. Dev IP Std. Dev P-ValueFood 121.26 76.41 0.0398Lodging 143.39 122.80 0.4836Retail 89.92 35.01 0.0001Transportation 108.21 66.01 0.0281Tourism 3.20 6.64 0.0015Entertainment 30.91 18.77 0.0268Other 77.03 74.77 0.8911Total 326.90 241.00 0.1706

Breusch-Pagan Tests

Category P-ValueFood 0.0680Lodging 0.1482Retail 0.0000Transportation 0.1996Tourism 0.3562Entertainment 0.0000Other 0.8895Total 0.1820


Shapiro-Wilk tests for Normal Residuals


Tests (as above) with Consistently Influential Point Removed

Paired T-Tests

Category Online Mean IP Mean P-ValueFood 128.09 95 0.0924Lodging 192.14 163 0.0094Retail 17.62 10.48 0.4462Transportation 61.93 40.95 0.1405Tourism 0.71 1.9 0.4885Entertainment 7.67 3.69 0.93Other 15.14 10.18 0.8853Total 425.31 333.95 0.0226




Tests of Regression Coefficients (No Constant) Against 1

Category P-ValueFood 0.124Lodging 0.009Retail 0.179Transportation 0.23; 0.825 (one more influential point

removed)Tourism *****Entertainment 0.437Other 0.000Total 0.097***** There were only 2 nonzero responses in-person and 1 nonzero response online, invalidating this test

Linear Regressions (Outlier Removed)

_cons 3.846753 30.07765 0.13 0.900 -59.10649 66.8 FoodIP 1.307879 .3670719 3.56 0.002 .5395884 2.076169 FoodOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust

Root MSE = 85.672 R-squared = 0.5030 Prob > F = 0.0021 F( 1, 19) = 12.69Linear regression Number of obs = 21

_cons 11.78148 9.541432 1.23 0.232 -8.188967 31.75193 LodgingIP 1.106512 .0602083 18.38 0.000 .9804941 1.232529 LodgingOnl~e Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust



_cons 13.9 8.553199 1.63 0.121 -4.002052 31.80205 RetailIP .355 .3791883 0.94 0.361 -.4386503 1.14865 RetailOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


_cons 27.92096 15.62839 1.79 0.090 -4.789643 60.63157TransportationIP .8305346 .1374133 6.04 0.000 .5429253 1.118144 Transportation~e Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


_cons 36.6254 19.68623 1.86 0.079 -4.733836 77.98463TransportationIP .4787359 .5050194 0.95 0.356 -.5822706 1.539742 Transportation~e Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


*****One more influential point removed


_cons .7731959 .7913468 0.98 0.341 -.883112 2.429504 TourismIP -.0309278 .0323158 -0.96 0.351 -.0985656 .0367099 TourismOnl~e Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


_cons 3.711314 3.443636 1.08 0.295 -3.4963 10.91893EntertainmentIP .4886023 .5504011 0.89 0.386 -.6634004 1.640605 Entertainment~e Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


_cons 18.51052 16.34918 1.13 0.272 -15.7087 52.72974 OtherIP -.0941672 .1138435 -0.83 0.418 -.3324444 .14411 OtherOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust



_cons 126.4043 64.84746 1.95 0.066 -9.322979 262.1316 TotalIP .8950676 .1564672 5.72 0.000 .5675781 1.222557 TotalOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


Scatter Plots

010

020

030

040

050

0

0 50 100 150 200 250FoodIP

Fitted values FoodOnline


010

020

030

040

0

0 100 200 300 400LodgingIP

Fitted values LodgingOnline


050

100

150

0 20 40 60 80 100RetailIP

Fitted values RetailOnline

050

100

150

0 20 40 60 80 100RetailIP



010

020

030

0

0 100 200 300TransportationIP

Fitted values TransportationOnline

050

100

150

200

0 20 40 60 80 100TransportationIP


*One more influential point removed


05

1015

0 10 20 30TourismIP

Fitted values TourismOnline

020

4060

8010

0

0 10 20 30 40 50EntertainmentIP

Fitted values EntertainmentOnline


010

020

030

0

0 50 100 150 200OtherIP

Fitted values OtherOnline

020

040

060

080

010

00

0 200 400 600 800TotalIP

Fitted values TotalOnline


Residual Plots

-100

010

020

030

0R

esid

uals

0 50 100 150 200 250FoodIP

-100

-50

050

100

150

Res

idua

ls

0 100 200 300 400LodgingIP


-50

050

100

150

Res

idua

ls

0 20 40 60 80 100RetailIP

-100

010

020

0R

esid

uals

0 100 200 300TransportationIP


05

1015

Res

idua

ls

0 10 20 30TourismIP

-20

020

4060

80R

esid

uals



010

020

030

0R

esid

uals

0 50 100 150 200OtherIP

-400

-200

020

040

0R

esid

uals

0 200 400 600 800TotalIP


Comparisons of Online-Only and Online Portion of Respondents to Both Surveys

T-Tests

Category Online (Both) Online (Only) P-ValueFood 135.91 131.54 0.8814Lodging 183.41 186.14 0.9413Retail 35 35.01 0.9995Transportation 77.3 77.22 0.9974Tourism 0.68 1.76 0.3496Entertainment 11.86 11.69 0.9819Other 25.45 11.51 0.4370Total 469.62 454.87 0.8642


Category Online (Both) Std. Dev

Online (Only) Std. Dev

P-Value

Food 121.26 120.60 0.9202Lodging 183.41 186.14 0.1579Retail 89.92 77.40 0.3392Transportation 108.21 96.22 0.4479Tourism 3.20 8.39 0.0000Entertainment 30.91 36.89 0.3620Other 75.39 67.75 0.4872Total 326.90 455.37 0.0870

Comparisons of In-Person-Only and In-Person Portion of Respondents to Both Surveys

T-Tests

Category IP(Both) IP (Only P-ValueFood 104.32 152.67 0.0530Lodging 155.59 200.49 0.1924Retail 14.55 62.85 0.0166Transportation 43.64 105.14 0.0100Tourism 1.82 9.36 0.0182Entertainment 10 29.36 0.0707Other 27.5 17.71 0.5995Total 357.41 577.58 0.0113



Category IP (Both) Std. Dev IP (Only) Std. Dev P-ValueFood 76.41 153.99 0.0007Lodging 122.80 181.48 0.0475Retail 35.01 153.13 0.0000Transportation 66.01 155.52 0.0001Tourism 1.42 2.80 0.0000Entertainment 4.00 9.79 0.0000Other 74.77 78.10 0.8564Total 240.10 566.47 0.0001

Comparisons of In-Person Email-Providers and Non-Email-Providers

T-Tests

Category Email No Email P-ValueFood 126.23 165.34 0.2497Lodging 148.37 257.14 0.0056Retail 28.46 88.49 0.0917Transportation 88.04 94.34 0.8274Tourism 4.04 13.29 0.0689Entertainment 27.11 20.86 0.6868Other 17.11 24.86 0.6892Total 439.34 664.31 0.0774


Category No Email Std. Dev Email Std. Dev P-ValueFood 179.15 109.68 0.0011Lodging 194.90 138.54 0.0230Retail 198.05 69.58 0.0000Transportation 120.58 154.03 0.1283Tourism 26.81 15.42 0.0002Entertainment 70.52 74.04 0.7731Other 106.17 52.65 0.0000Total 679.64 362.78 0.0000


Comparisons of In-Person Email Providers that Did and Did Not Participate in the Follow-Up Survey

T-Tests

Category Did Follow-Up Didn’t Do Follow-Up P-ValueFood 104.32 140 0.1873Lodging 155.59 143.83 0.7475Retail 14.55 37.21 0.1629Transportation 43.64 115.94 0.0406Tourism 1.82 5.43 0.3082Entertainment 10 37.86 0.0915Other 27.5 10.57 0.3233Total 357.41 490.85 0.1318


Category No Email Std. Dev Email Std. Dev P-ValueFood 76.41 125.30 0.0196Lodging 122.80 149.14 0.3512Retail 35.01 83.74 0.0001Transportation 66.01 185.22 0.0000Tourism 6.64 18.96 0.0000Entertainment 18.77 92.21 0.0000Other 74.77 31.62 0.0000Total 240.10 416.92 0.0102

Comparisons of Online-Only and In-Person-Only Respondents

T-Tests

Category Online Mean IP Mean P-ValueFood 131.54 152.67 0.3531Lodging 186.14 200.49 0.1708Retail 35.01 62.85 0.1708Transportation 77.22 105.14 0.1942Tourism 1.76 9.36 0.0116Entertainment 11.69 29.36 0.0988Other 11.51 17.71 0.6038Total 454.87 577.58 0.1472



Appendix B: Results from the 2014 Beach Bash Lacrosse Tournament

Comparisons of Paired Observations from Respondents to Both Surveys

Paired T-Tests

Category Online Mean IP Mean P-ValueFood 179.53 185 0.8524Lodging 249.29 235.29 0.4848Retail 34.33 36.67 0.8665Transportation 92.35 70.88 0.2220Tourism 0 9.41 0.1773Entertainment 0 17.53 0.0528Other 5.88 0 0.3322Total 557.35 553.41 0.9403


Category Online Std. Dev IP Std. Dev P-ValueFood 88.68 85.81 0.8967Lodging 157.27 172.26 0.7201Retail 43.95 43.57 0.9644Transportation 65.24 58.32 0.6588Tourism 0 27.49Entertainment 0 34.55Other 24.25 0Total 261.77 281.52 0.7746

Breusch-Pagan Tests

Category P-ValueFood 0.8802Lodging 0.0372Retail 0.0351Transportation 0.7667TourismEntertainmentOtherTotal 0.1353




Tests of Regression Coefficients (No Constants) Against 1


Linear Regressions

_cons 167.1149 51.50369 3.24 0.005 57.33741 276.8925 FoodIP .0671053 .2446414 0.27 0.788 -.4543355 .588546 FoodOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust



_cons 59.42104 43.53538 1.36 0.192 -33.37244 152.2145 LodgingIP .8069606 .1255554 6.43 0.000 .5393455 1.074576 LodgingOnl~e Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


_cons 23.58511 10.6676 2.21 0.046 .539168 46.63106 RetailIP .2931333 .3337766 0.88 0.396 -.4279471 1.014214 RetailOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


_cons 63.12068 22.57647 2.80 0.014 15.00007 111.2413TransportationIP .4124054 .2592784 1.59 0.133 -.1402334 .9650442 Transportation~e Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust



_cons 200.5934 131.4421 1.53 0.148 -79.56887 480.7556 TotalIP .6446549 .1843487 3.50 0.003 .2517249 1.037585 TotalOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


Scatter Plots

010

020

030

0

0 100 200 300 400FoodIP

FoodOnline Fitted values


020

040

060

0

0 200 400 600LodgingIP

LodgingOnline Fitted values

050

100

150

0 50 100 150RetailIP

RetailOnline Fitted values


050

100

150

200

250

0 50 100 150 200TransportationIP

TransportationOnline Fitted values

020

040

060

080

010

00

200 400 600 800 1000TotalIP

TotalOnline Fitted values


Residual Plots

-200

-100

010

0R

esid

uals

0 100 200 300 400FoodIP

-200

-100

010

020

0R

esid

uals

0 200 400 600LodgingIP


-100

-50

050

100

Res

idua

ls

0 50 100 150RetailIP

-100

-50

050

100

Res

idua

ls



-400

-200

020

040

0R

esid

uals

200 400 600 800 1000TotalIP

Comparisons of In-Person-Only and In-Person Portion of Respondents to Both Surveys

T-Tests

Category IP Portion of Both Mean

IP only Mean P-Value

Food 211.29 185 0.2628Lodging 289.16 235.29 0.2355Retail 78.25 35.29 0.0019Transportation 115.89 70.88 0.0324Tourism 13.40 9.41 0.5926Entertainment 15.94 17.53 0.8592Other 2.61 0 0.0163Total 726.53 553.41 0.0296



Category IP Portion of Both Std. Dev

IP only Std. Dev P-Value

Food 120.88 85.81 0.1150Lodging 174.48 172.26 0.9739Retail 89.25 43.57 0.0020Transportation 188.55 58.32 0.0000Tourism 38.86 27.49 0.1118Entertainment 34.74 34.55 0.9428Other 13.56 0 0.0000Total 378.66 281.52 0.1716

Comparisons of In-Person Email Providers that Did and Did Not Participate in the Follow-Up Survey

T-Tests

Category Follow-Up IP Mean No Follow-Up Mean P-ValueFood 185 180.56 0.8816Lodging 235.29 285.81 0.3009Retail 35.29 77.81 0.0392Transportation 70.88 91.30 0.2691Tourism 9.41 12.96 0.7666Entertainment 17.53 2.96 0.1107Other 0 1.11 0.3265Total 553.41 652.52 0.2809


Category Follow-Up IP Mean No Follow-Up Mean P-ValueFood 85.81 109.76 0.3071Lodging 172.26 121.38 0.1090Retail 43.57 87.90 0.0051Transportation 58.32 59.30 0.9700Tourism 27.49 51.1269 0.0124Entertainment 34.55 12.03 0.0000Other 0 5.77 0.0000Total 281.52 309.06 0.7105


In-Person Email Providers vs Non-Providers

T-Tests

Category Non-Providers Mean Providers Mean P-ValueFood 217.58 182.27 0.0593Lodging 289.84 266.30 0.3836Retail 78.33 61.39 0.2263Transportation 120.92 83.41 0.0617Tourism 13.48 11.59 0.7941Entertainment 18.60 8.59 0.0423Other 2.92 0.68 0.1240Total 741.67 614.23 0.0264


Category Non-Providers Std. Dev.

Providers Std. Dev. P-Value

Food 122.47 100.14 0.1296Lodging 183.83 143.42 0.0625Retail 89.85 76.27 0.2167Transportation 205.01 59.10 0.0000Tourism 36.09 43.18 0.1276Entertainment 37.22 24.15 0.0016Other 14.65 4.52 0.0000Total 390.67 299.38 0.0462


Appendix C: Results from the 2012 Crawlin’ Crab Half-Marathon

Comparisons of Paired Responses from Respondents to Both Surveys

Paired T-Tests

Category Online Mean IP Mean P-ValueFood 100.58 93.92 0.4794Lodging 103.66 83.7 0.0430Retail 52.7 47.4 0.5362Transportation 73.62 48.3 0.0062Tourism 8.02 9.8 0.4312Entertainment 9.2 12.1 0.4958Other 18.5 16.3 0.7491Total 366.28 311.52 0.0361



Breusch-Pagan Tests





Tests of Regression Coefficients Against 1


Linear Regressions

_cons 39.21588 14.61645 2.68 0.010 9.827533 68.60423 FoodIP .6533658 .182121 3.59 0.001 .2871871 1.019545 FoodOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust



_cons 20.22936 12.94724 1.56 0.125 -5.802813 46.26153 LodgingIP .9967819 .0886286 11.25 0.000 .818582 1.174982 LodgingOnl~e Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


_cons 24.85877 9.47997 2.62 0.012 5.798016 43.91953 RetailIP .5873676 .1955039 3.00 0.004 .1942808 .9804545 RetailOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


_cons 47.30089 17.02815 2.78 0.008 13.06349 81.53828TransportationIP .5449092 .2115611 2.58 0.013 .1195371 .9702813 Transportation~e Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust



_cons -1.865162 1.187427 -1.57 0.123 -4.252644 .52232 TourismIP 1.00869 .2882939 3.50 0.001 .4290363 1.588344 TourismOnl~e Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


_cons 6.470376 3.008157 2.15 0.037 .4220702 12.51868EntertainmentIP .2255888 .1637717 1.38 0.175 -.1036964 .5548739 Entertainment~e Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


_cons 6.286265 6.588198 0.95 0.345 -6.960195 19.53272 OtherIP .7493089 .0659064 11.37 0.000 .6167951 .8818227 OtherOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust



_cons 44.16726 30.45181 1.45 0.153 -17.06021 105.3947 TotalIP 1.034003 .130371 7.93 0.000 .771875 1.296132 TotalOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


Scatter Plots

010

020

030

040

0

0 100 200 300 400FoodIP

Fitted values FoodOnline


010

020

030

040

050

0

0 100 200 300 400LodgingIP

Fitted values LodgingOnline

010

020

030

0

0 50 100 150 200RetailIP



010

020

030

0



050

100

150

200

0 50 100 150TourismIP

Fitted values TourismOnline


050

100

150


Fitted values EntertainmentOnline

010

020

030

040

0

0 100 200 300 400 500OtherIP

Fitted values OtherOnline


050

010

0015

00

0 200 400 600 800TotalIP

Fitted values TotalOnline

Residual Plots

-100

010

020

030

0R

esid

uals

0 100 200 300 400FoodIP


-100

010

020

030

040

0R

esid

uals

0 100 200 300 400LodgingIP

-100

010

020

0R

esid

uals

0 50 100 150 200RetailIP


-100

010

020

030

0R

esid

uals


-100

-50

050

100

Res

idua

ls

0 50 100 150TourismIP


-50

050

100

150

Res

idua

ls


-100

010

020

030

0R

esid

uals

0 100 200 300 400 500OtherIP


-200

020

040

060

0R

esid

uals

0 200 400 600 800TotalIP

Comparisons of In-Person Only and the In-Person Portion of Respondents to Both Surveys

T-Tests

Category Both Mean Only Mean P-ValueFood 100.58 109.87 0.4514Lodging 103.66 118.27 0.4171Retail 52.7 64.64 0.2577Transportation 73.62 56.42 0.0922Tourism 8.02 13.90 0.2219Entertainment 9.2 19.05 0.0247Other 18.5 5.68 0.1993Total 366.28 387.83 0.6247



Category Both Std. Dev Only Std. Dev P-ValueFood 76.89 104.32 0.0106Lodging 114.06 142.28 0.0605Retail 63.92 97.26 0.0006Transportation 63.85 82.41 0.0310Tourism 29.83 41.22 0.0069Entertainment 24.46 47.68 0.0000Other 68.29 36.47 0.0000Total 277.08 361.82 0.0244

Comparisons of Online-Only and the Online Portion of Respondents to Both Surveys

T-Tests

Category Both Mean Only Mean P-ValueFood 93.92 87.25 0.6080Lodging 83.7 71.19 0.4189Retail 47.4 50.09 0.7800Transportation 48.3 51.08 0.7323Tourism 9.8 2.06 0.0378Entertainment 12.1 5.59 0.0729Other*Total** 295.22 266.65 0.4100*Omitted in online-only survey**Omits Other spending for “Both”


Category Both Std. Dev Only Std. Dev P-ValueFood 71.02 104.2 0.0024Lodging 91.93 104.83 0.2861Retail 51.65 79.07 0.0008Transportation 42.19 69.75 0.0001Tourism 25.05 10.24 0.0000Entertainment 23.17 18.12 0.0245Other*Total** 192.05 271.03 0.0060


Comparisons of Online-Only and In-Person-Only Respondents

T-Tests

Category Online Mean IP Mean P-ValueFood 109.87 87.25 0.0243Lodging 118.27 71.19 0.0000Retail 64.64 50.09 0.0774Transportation 56.42 51.08 0.4529Tourism 13.90 2.06 0.0000Entertainment 19.05 5.59 0.0000Other*Total** 382.15 266.65 0.0001


Category Online Std. Dev IP Std. Dev P-ValueFood 104.32 104.20 0.9990Lodging 142.28 104.831 0.0000Retail 97.26 79.07 0.0034Transportation 82.41 69.75 0.0171Tourism 41.22 10.24 0.0000Entertainment 47.68 18.12 0.0000Other*Total** 357.59 271.03 0.0001


Appendix D: Results from Tests on All Respondents to Both Surveys

Wilcoxon Signed-Rank Tests

Prob > |z| = 0.8288 z = -0.216Ho: FoodOnline = FoodIP

adjusted variance 58930.63 adjustment for zeros -717.50adjustment for ties -93.13unadjusted variance 59741.25

all 89 4005 4005 zero 20 210 210 negative 40 1950 1897.5 positive 29 1845 1897.5 sign obs sum ranks expected

Wilcoxon signed-rank test

Prob > |z| = 0.9672 z = -0.041Ho: LodgingOnline = LodgingIP


all 89 4005 4005 zero 42 903 903 negative 24 1560.5 1551 positive 23 1541.5 1551 sign obs sum ranks expected



Prob > |z| = 0.5151 z = 0.651Ho: RetailOnline = RetailIP


all 87 3828 3828 zero 39 780 780 negative 21 1377.5 1524 positive 27 1670.5 1524 sign obs sum ranks expected


Prob > |z| = 0.5821 z = -0.550Ho: TransportationOnline = TransportationIP





Prob > |z| = 0.9761 z = -0.030Ho: TourismOnline = TourismIP


all 89 4005 4005 zero 79 3160 3160 negative 5 426.5 422.5 positive 5 418.5 422.5 sign obs sum ranks expected


Prob > |z| = 0.7057 z = -0.378Ho: EntertainmentOnline = EntertainmentIP


all 89 4005 4005 zero 63 2016 2016 negative 14 1068.5 994.5 positive 12 920.5 994.5 sign obs sum ranks expected



Prob > |z| = 0.9944 z = 0.007Ho: OtherOnline = OtherIP


all 88 3916 3916 zero 76 2926 2926 negative 6 494 495 positive 6 496 495 sign obs sum ranks expected


Prob > |z| = 0.9788 z = -0.027Ho: TotalOnline = TotalIP





Regressions

_cons 167.1149 50.09744 3.34 0.001 67.47315 266.7567 CRABFOODIP .4902417 .277209 1.77 0.081 -.0611158 1.041599XTERRAFOODIP 1.087231 .3563871 3.05 0.003 .378391 1.79607 Crab -129.2529 51.36262 -2.52 0.014 -231.4111 -27.09471 XTERRA -151.6241 55.69946 -2.72 0.008 -262.4081 -40.84008 FoodIP .0671053 .2379617 0.28 0.779 -.406191 .5404015 FoodOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


_cons 59.42104 42.34669 1.40 0.164 -24.80483 143.6469 CRABLODGINGIP -.1594842 .2069311 -0.77 0.443 -.5710618 .2520934XTERRALODGINGIP .3053392 .1348434 2.26 0.026 .0371411 .5735372 Crab -42.83844 43.84797 -0.98 0.331 -130.0503 44.37341 XTERRA -49.07568 43.13651 -1.14 0.259 -134.8725 36.72111 LodgingIP .8069606 .1221273 6.61 0.000 .5640544 1.049867 LodgingOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust



_cons 20.03747 11.81312 1.70 0.094 -3.466935 43.54187 CRABLODGINGIP .0997575 .0857788 1.16 0.248 -.0709154 .2704305XTERRALODGINGIP -.196258 .1664317 -1.18 0.242 -.5274048 .1348889 Crab -3.525394 13.53121 -0.26 0.795 -30.44826 23.39747 XTERRA 39.8274 44.96576 0.89 0.378 -49.64037 129.2952 RetailIP .3898873 .1924197 2.03 0.046 .0070325 .7727421 RetailOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


_cons 63.12068 21.96004 2.87 0.005 19.44304 106.7983CRABTRANSPORTA~P -.1744513 .3007134 -0.58 0.563 -.7725581 .4236555XTERRATRANSPOR~P .5961501 .3440198 1.73 0.087 -.0880914 1.280392 Crab -32.33885 23.9377 -1.35 0.180 -79.94998 15.27227 XTERRA -29.83037 27.34728 -1.09 0.279 -84.223 24.56226TransportationIP .4124054 .252199 1.64 0.106 -.0892083 .9140191 Transportation~e Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust



_cons -4.44e-14 . . . . . CRABTOURISMIP .7112766 .0790209 9.00 0.000 .5541072 .868446XTERRATOURISMIP -.0294118 .0303132 -0.97 0.335 -.0897036 .0308801 Crab 4.095562 1.979984 2.07 0.042 .1574533 8.033671 XTERRA .7352941 .7422587 0.99 0.325 -.7410286 2.211617 TourismIP 7.07e-15 4.73e-09 0.00 1.000 -9.40e-09 9.40e-09 TourismOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


_cons 5.68e-14 . . . . .CRABENTERTAINM~P .2024013 .118845 1.70 0.092 -.0339766 .4387792XTERRAENTERTAI~P .877027 .5476212 1.60 0.113 -.2121696 1.966224 Crab 10.23791 3.524716 2.90 0.005 3.227391 17.24843 XTERRA 3.093366 3.496119 0.88 0.379 -3.860274 10.04701 EntertainmentIP -1.10e-15 . . . . . EntertainmentO~e Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust

Root MSE = 21.809 R-squared = 0.1837 Prob > F = . F( 4, 83) = .Linear regression Number of obs = 89


_cons 5.882353 5.876096 1.00 0.320 -5.804966 17.56967 CRABOTHERIP .3599774 .4630217 0.78 0.439 -.560954 1.280909XTERRAOTHERIP 0 (omitted) Crab -3.4402 6.263394 -0.55 0.584 -15.89784 9.017438 XTERRA 9.574662 17.67222 0.54 0.589 -25.57467 44.72399 OtherIP .3890954 .2275532 1.71 0.091 -.0634989 .8416896 OtherOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


_cons 200.5934 127.8532 1.57 0.120 -53.70154 454.8883 CRABTOTALIP -.0833973 .2001038 -0.42 0.678 -.4813957 .3146012XTERRATOTALIP .4575169 .2750723 1.66 0.100 -.0895909 1.004625 Crab -94.65079 131.063 -0.72 0.472 -355.3298 166.0282 XTERRA -124.9014 146.7315 -0.85 0.397 -416.7445 166.9417 TotalIP .6446549 .1793153 3.60 0.001 .288004 1.001306 TotalOnline Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust


upperclassmonroe.blogs.wm.eduupperclassmonroe.blogs.wm.edu/.../08/monroe-article.docx · web viewat...

Documents