chapter two the subset questionnaire · the entire questionnaire, you are able to address this...

FDTL DATA Project: Quantitative Research Methods

Peter Skehan 2003 1

Chapter Two Working with the BIELT Datasets

Analysing the Subset Questionnaire

Aims of the Chapter The general purpose of the present chapter is to introduce you to working with actual data, using SPSS. More specifically, though, the aims are: • to work with a dataset • to learn how to analyse data to answer specific questions • to learn to use a range of graphical techniques:

• pie diagrams, bar charts, clustered bar charts • to learn to use a number of descriptive statistical techniques:

• frequencies, descriptives, crosstabs, and means Relevant reading for this chapter are still Howell (1997), Chapter 2, and Miller et al (2002), Ch.2.

Using Graphs Simple Bar Charts The small version of the BIELT questionnaire is based on only 100 cases, and is slightly easier to work with at the outset. (Note also that not all the questions from the original questionnaire are included in the actual datasets. Some questions have been dropped because they don’t convert easily to numeric format. In the real world, the information in responses to these questions would need to be analysed by different means.) Later you will work with the entire dataset, with more than 1000 cases. For now, the focus will be on using basic analytic techniques, both visual and numerical, to get a first impression of the data.

Task 2.1 A First Graph: Open SPSS, if you haven’t already done so, and then open the Small BIELT dataset. Your screen should fill with numbers, and, if you scroll your way around this matrix of numbers, you will see that even the small version of the BIELT questionnaire is pretty big. Notice, in passing, that although this is a different dataset, there are resemblances to the lateralisation dataset from Chapter One. The dataset contains numerically coded responses to questionnaire-elicited data. In addition, it is organised to bring out the structure of the data, e.g. first with columns for responses to organising variables, such as gender, age and then responses to the attitudinal sections, e.g. to the formation of BIELT. Go to Graphs from the drop-down menus, and then select Bar. This will bring up a new screen in which you have to choose between different types of Bar Graphs. Choose Simple, and then click Define. At the next screen all you have to do is choose a variable name from the list on the left. In this case, choose Salary and then press the


Peter Skehan 2003 2

arrow button beside Category Axis to select salary. Click the radio button for Number of Cases if it is not already selected and then click O.K. You should get the graph shown below, in the Feedback section.

Feedback on Task 2.1 The output from each SPSS procedure is sent to an Output file. This opens automatically when you generate some output, since SPSS assumes that you will want to look at it! But when you are done, don’t close this window - simply click on the tab at the bottom of the screen to return to the data view. If you do this, SPSS will write output cumulatively to your emerging output file. In this way, you can scroll back through the output file if you want to look at output you generated earlier in your analysis session. This is often very helpful to do. Note also that if you do try to close the output window (rather than simply move to another window) SPSS asks you if you want to save the output. It also asks this question when you finish the entire SPSS session. If you want to save the file, simply give it a name, and save it to a useful location on your floppy or hard discs. In the bar chart below, the horizontal axis shows the different salary levels, corresponding to the numerical codes in the data of 1 to 6, although labelled in the bar chart by the value labels they have been given. The vertical axis shows the number of people who earn each salary coding. Notice that SPSS has worked out a good scale to represent this.

Small BIELT Questionnaire: Salary

Salary

£36+K£31-35K£26-30£21-25£16-20k£10-15kMissing

Cou

nt

30

20

10

0

This interesting bar chart gives a snapshot of the salary picture amongst the people from the small BIELT questionnaire. There is clearly a distinctive shape. What one first looks for with a chart of this sort is a normal distribution, i.e. a symmetrical distribution, with one peak, and with the number of cases falling away on either side of the peak. This is probably the


Peter Skehan 2003 3

classical distribution shape in statistics. What is clear is that we don’t have such a distribution here! If one ignores the first bar, which refers to missing values, the remaining chart shows a trend downwards, with fewer and fewer people as we reach higher salaries. But there are exceptions to this. The lowest salary (£10-15k) is earned by the largest number of people, but then the next salary grouping up (£16-20k) is earned by appreciably fewer people, while the next grouping after that (£21-25k) is earned by more people than the immediately previous category. Then we have two reductions in the height of the bars followed by an increase in the highest salary level of all.

Task 2.2 A picture is worth a thousand words, or so the saying goes. And this is certainly an intriguing picture! It is worth thinking a little to try to understand what is happening. Do you think: a) the particular shape is the result of chance factors? or b) there are reasons to account for the values which are shown?

Feedback on Task 2.2 A picture is a suggestive starting point here. We can speculate, but we cannot confirm: that would need independent evidence. But two factors do come to mind: • the distribution of this chart is consistent, first of all, with a casualised

profession, i.e. in which there are lots of part-time workers. Such an explanation would be consistent with the high number of people in the lowest category, and then a “blip” in the £16-20k range, as one moves from the part-timers (£10-15) to low paid full-timers (£21-25). In other words, the £16-20 range is taken up with particularly low paid full-timers or part-timers with a lot of teaching hours.

• we then need to explain the increase in the highest salary category of all. Here, two independent reasons come to mind:

- our profession is becoming more managerial, and the bar for the highest salary level reflects this.

- there is systematic bias in the responses to the questionnaire. Inevitably distributed questionnaires are more likely to be received, in the first instance, by managers, as the text accompanying the questionnaire makes clear, and this group then forms a disproportionate number of questionnaire respondents, relative to the situation in the profession as a whole.


Peter Skehan 2003 4

Reflection

The procedure that you have just learned, to produce simple bar charts, is very powerful, and you should use it whenever appropriate to help develop familiarity with a dataset, and give you a “feel” for how it is distributed and structured. There is no substitute for this, and such a stage has to precede any computation of fancier statistics. So reflect on the procedure a little. Remember, all you need to do is: • choose Bar from the Graphs option in the initial drop-down menus • choose Simple as the type of graph you want (you will extend this later) • Define the graph, allowing you to choose the variable you want to see • Explore the results

Task 2.3 To help you develop the routine of using Bar Charts, here are a series of questions which you can answer by exploring a set of variables: Question to Answer Variable to use 1. Did more men or women return this questionnaire? Gender 2. What is the most common age amongst respondents? Age 3. Roughly, how much do Employed respondents Employed vs. Freelance outnumber the Freelancers? 4. How representative of the international range of ELT Country or World location is the data from this version of the questionnaire? 5. What is the (very predictable) shape of the chart Fee (Annual individual showing views on the membership fee? membership fee)

Feedback on Task 2.3 To answer the set of questions in the exercise, you needed to generate five bar charts, and then interpret each one. (If you are using SPSS 10 or higher, you will need to press Reset before you choose a new variable, or alternatively, simply over-type.) I would offer the following answers: 1. Women do outnumber men as questionnaire respondents. (Although I

have to admit I expected the disparity to be greater, on the basis of my experience within the profession. Again, one has to wonder about bias in the distribution and the returns.)

2. Again, a little suspiciously, it is between 46-50, a little older than one might expect, and, therefore, suggestive of questionnaire distribution and response patterns.


Peter Skehan 2003 5

3. My guess here would be that the values are around 75 to 20, so around almost four-to-one.

4. Clearly, the answer to this one is “Not very much”. The responses are dominated by people from the U.K. and the only other areas to be represented in this data (and not much at that) are the Rest of Europe and the Middle East. To make statements about groups other than the U.K., therefore, needs different, or at least, additional data. (When you look at the entire questionnaire, you are able to address this more).

5. Very clearly and predictably, the tendencies that the higher the projected membership fee, the fewer people there are who recommend it! (Note the slight ‘blip’ with £50.)

Clustered Bar Charts So far, the bar charts we have looked at have been very straightforward. Each has taken a particular measure (salary, say) and then looked at it in isolation, simply using the categories that define the scale. It is possible, though, to produce slightly more complicated bar charts. When you have chosen Chart and Bar, you have always (I assume) made the Simple choice. But there are two others: Clustered and Stacked. We will not look at Stacked here: its format is fairly obvious, and you can explore it for yourself. But we will look briefly at Clustered since this does enable you to create some immediately useful visual representations of the data.

Task 2.4 To explore Clustered charts, make the following choices: Graphs leading to: Bar leading to: Clustered leading to: Define This brings up a new screen. As before, choose Salary as your main variable, for the Category Axis, but there is now space to move your selection for the clustering variable also, the Define Clusters by box. Choose Gender for this, and then click O.K. Have a look at the graph which you produce, and think about what it means.


Peter Skehan 2003 6

Feedback on Task 2.4

Salary plotted to show gender

Salary

£36+K£31-35K

£26-30£21-25

£16-20k£10-15k

Missing

Cou

nt20

10

0

Gender

Missing

Male

Female

In SPSS, on screen, you should also have glowing colour, which makes for a better effect here. But in any case, (and ignoring the missing values), it is interesting to see two patterns here, one for women and one for men. The bars for women, plotted as a function of salary, show that as salary rises, fewer women are involved. But the pattern for men is not the same. There are two places where, as salary increases (from £16-20k to £21-25k, and from £31-£35k to £36+k) the number of men shown also increases. This seems to corroborate but extend the pattern from looking at salary without clustering. It would appear that if part-time status is a factor in the profession, then this influences male and female salaries quite differently. It would seem that men are much less represented in the part-time sector of the ELT profession. Unsurprising perhaps, but discouraging, nonetheless.

Reflection The broader point for you to consider here, though, is that it was the Clustered form of the bar chart which brought this out quite neatly. Gender lends itself to this quite well, since there are only two categories to deal with here, and then the bar chart which is produced is not too detailed: it is possible to see the two co-existing patterns without getting lost in the detail. Possibly a three-category variable in place of gender would also work, but probably beyond that, things would not be so clear. In any case, bear in mind the Cluster facility: it is a powerful visual tool, provided it is appropriately used.


Peter Skehan 2003 7

Task 2.5 Use the Cluster option in the bar chart to explore what the relationship is between gender and age. (Hint: Choose Age as your category axis.)

Feedback for Task 2.5 You should get the following bar chart: Age plotted to show gender

Age

56-6551-55

46-5041-45

36-4031-35

26-30Missing

Cou

nt

20

10

0

Gender

Missing

Male

Female

Here the situation seems to be one where the typical (most commonly occurring) male age is 41-45, with a reasonably symmetrical distribution around this area. In contrast, the distribution for women shows a higher typical value (46-50), with this peak also being more pronounced. This is an interesting pattern. One might have expected much less variation here, if ELT were a stable career. It appears that men tend to leave the profession post-45, while women enter it.


Peter Skehan 2003 8

Pie Charts The two chart types we have looked at (Simple and Clustered) will take you a long way, and indeed cover most situations you will encounter. There is one other type, though, which is worth a quick look, more for its visual than its practical appeal. This is the Pie Chart, a type of display which can be effective when there are only a small number of categories for a variable or measure.

Task 2.6 To obtain a Pie Chart, follow these steps: Graphs Pie Summaries for groups of cases (radio button) Define Define slices by (choose your variable here: on

this occasion, Country or Location) Once again, look at the output, and (a) think about how you would describe the chart, and (b) how you would interpret it.


Peter Skehan 2003 9

Feedback for Task 2.6 The Pie Chart should look as follows:

Geographical Distribution of Questionnaire Respondents

Middle East

Rest of Europe

U.K

Missing

This output confirms the finding from an earlier bar chart: the responses are dominated by people who are based in the U.K. The Rest of Europe group have a clear slice of this pie, even though it is not that big, while the Middle East group account for only a sliver. In terms of interpretation, this U.K. domination is probably based on a combination of the density of ELT work in the U.K., together with the greater ease of distribution of the questionnaire in this location.

Reflection Of course, you could also produce pie charts for questionnaire items such as age, salary, gender, or employed vs. freelance. To my mind, these are not so useful. With the former two, age and salary, there is a scale implied, ranging from low values to high values (e.g. 20-25 year olds, up to 56-65 year olds). A pie chart does not really convey this structure, since the pie contains simply categories which account for the area of the entire circle. The gender and employed vs. freelance items do not contain scales (which is good), but it doesn’t convey very much to have a pie which could only ever have two slices!

Task 2.7 Create a pie-chart to show the age structure of the respondents to the small version of the BIELT questionnaire.


Peter Skehan 2003 10

Feedback for Task 2.7

The sequence you should have used is: Graphs Pie Chart (then the radio button for Summaries for groups of cases should be

clicked) Define (Reset) then the Age variable should be selected and moved

across to the Define Slices by box O.K. and you should get the output shown below:

Pie Chart of Age Structure

56-65

51-55

46-50

41-45

36-40

31-35

26-30

Missing

Nothing terribly exciting, but the chart does bring out clearly enough that 46-50 is the age range which accounts for the largest number of respondents.

Task 2.8 You might try, (with the pie chart open in the SPSS Output Navigator), seeing if you can find out how to “explode” one of the slices. This technique is used when one of the categories from a pie chart is particularly important, and needs to stand out. Use the SPSS help facility to do this. Use Help and then Topics and type in Pie Chart. Then choose Explode slice, and How to. There is no feedback to this task.



Reflection The moral to all this is: think about your data, and the most meaningful way to represent it visually. The fact that some shiny option exists in SPSS doesn’t mean that it’s right to use it. It’s appropriateness and communicating effectively that count.

Using Numerical Techniques to Describe Data So far, we have used charts to try to convey what is happening with the data in the British Institute Questionnaire, and have seen that they are very useful in this regard. But we can also describe this data in a more numerical fashion. There are four basic procedures to master in this regard. The sequence of choices is given below, with each accompanied by a brief account. Note that each of these sequences refers to the SPSS drop-down menus, and the further choices that become available for selection from such menus. Note also that all these numerically-oriented techniques start with Analyze. Analyze, Descriptive Statistics, Frequencies • This is the simplest of the possible

techniques. It simply tabulates the number of times any particular category is used.

This option can be used in conjunction with graphical output or not, as preferred.

Analyze, Descriptive Statistics, Descriptives • This procedure allows the selection of individual or multiple variables, and then allows standard statistics to be produced, such as mean, standard deviation, and so on.

Analyze, Descriptive Statistics, Crosstabs • A procedure which allows cross-

tabulation of categorical variables (e.g. gender, salary), generating matrices of cells in which a simple frequency count of cases is given.

Analyze, Compare Means, Means • The most powerful of the descriptive

techniques, this allows means, standard deviations etc (as with the Descriptives procedure), but enables powerful control of output, enabling sub-categories to be easily generated, in such a way that patterns in the data can be made more salient.



Frequencies The simplest of the techniques is Frequencies, since all this does is provide simple counts, of the sort you could do by hand (although once a dataset gets large, it very nice to have the computer undertake the drudgery of doing the counting). We can illustrate the use of this procedure with what might be considered the basic question of the entire questionnaire: Q1: A new body to represent the British ELT profession is necessary where responses run from 1 (Strongly disagree) to 6 (Strongly agree).

Task 2.9 The way to use Frequencies here is: Analyse Descriptive Statistics Frequencies At this point you need to select the variable (or variables) which you want data on. For this first trial with Frequencies, simply choose q1nw_bod, and use the arrow box to move this variable from the left-hand box to the box headed Variable. You will note at this point that the screen, although simple-looking, contains buttons in the lower half, with the labels Statistics, Charts, and Formats. First of all, choose Statistics and you will open yet another screen, this time offering you a bewildering array of choices. For now, click to make the four choices of Mean, Standard Deviation, Minimum and Maximum. Then, click Continue (SPSS 9: OK) and this should return you to the previous box. Next, choose Charts (and to peek ahead, you will be ignoring Formats). And there, available once again, is Bar Chart. What the heck! Choose it! In this way, you will get both numerical and chart output, and you will be able to use either or both of these. Make sure, under Chart Values, that Frequencies is chosen, not Percentages. Once again, click Continue (SPSS 9: OK) and this should return you to the previous screen, where now you are ready to click the O.K. button to actually generate the output itself. (First exposure to all this may seem contorted, but you soon get used to it, and soon will be able to flash through these screens pretty quickly.)



Feedback for Task 2.9: Part One You should get the output shown, in three separate parts, below. Note that this output will appear on your screen all together. It has been separated here into three sections to enable effective discussion of it to take place. Output: Part One

Frequency data for the "New Body" Item Statistics

Q1NW_BOD981

4.91841.2491

1.006.00

ValidMissing

N

MeanStd. DeviationMinimumMaximum

This basic table shows you the name of the variable, the number of cases, both valid and missing, and the mean, standard deviation, minimum and maximum. The table shows that there is only one missing value (i.e. a person who didn’t complete this questionnaire item) and so we can confidently proceed. It is worth routinely checking the number of missing value cases, since if it turns out to be elevated for some reason, then you need to think further (a) as to why, and (b) how justifiable it is to proceed. So a quick look is merited, even if it mainly results in nothing other than reassurance. Following these “data checking” lines, we jump next to the minimum and maximum values which are shown (we’ll come back to the mean and standard deviation). The minimum value shown here is ‘1’ and the maximum is ‘6’. These values may not set the pulse racing. The point, though, is that the computer will happily tell you what the minimum and maximum values are for each variable. Bear in mind that when you input a large amount of data, there is every chance that, at the data input stage, you may make keystroke mistakes, and, in the enormous task of getting lots and lots of data into the computer, you will not notice. So it’s good practice to check minimum and maximum values routinely, since if you have made a big coding mistake, this is where you may well see a clue. Checking in this way won’t help you if you have keyed in a ‘4’ instead of a ‘2’, but if you’ve keyed in ‘22’ instead of ‘2’, it will be detectable, just as would be a 7 or 8. (Why not a ‘9’?) So exploit this mechanical thoroughness of the computer and check for mistakes. Finally we have the key statistics of the mean (which reflects centrality or typicality or central tendency), and the standard deviation (which tells us about the amount of variation about the mean). These values, of 4.92 and 1.25 respectively indicate that the average rating was well up on the six step scale of agreement for this item, and that the amount of dispersion was not



small, but wasn’t particularly large, either. In other words, most of the people who responded gave clear approval to the notion of having a new body to represent British ELT. The coverage of the standard deviation here is not satisfactory! At one level, it is not difficult to understand that it reflects the amount of variation or dispersion in a set of scores. So a standard deviation of 0.85 (for example), would indicate less variation (or more bunched scores) than the value of 1.25 that is found here. Imagine comparing scores on some measure for two classes, one of which is streamed or selected, and the other of which is mixed ability with no streaming. It is likely that the streamed class, i.e. perhaps selected to have a certain ability range, would have a smaller standard deviation, reflecting the greater closeness in scores that is the result of selectional policy. If you can bear it, this rather superficial explanation is enough for now. If you want to read more, try Howell (1997, pp 46-48). Howell makes the point that, as a rule of thumb, the standard deviation is usually about one-sixth of the range (i.e. minimum to maximum), and that the further you are from this value, the more you should check your work. But this rule-of-thumb also gives you a handle on deciding whether the variation you are dealing with with a particular set of scores is unusually great, or the reverse. We will return to the standard deviation at various points in the course. Output: Part Two Q1: A new body to represent the British ELT profession is necessary Valid Frequency Percent Valid

Percent Cumulative Percent

1.00 2.00 3.00 4.00 5.00 6.00 Total Missing Total

33614333998199

3.03.06.114.133.339.499.01.0

100.0

3.1 3.1 6.1

14.3 33.7 39.8

100.0

3.16.1

12.226.560.2

100.0

This table, the one that gives the procedure Frequencies the name Frequencies simply tallies the number of cases for each numerical category of the variable chosen. In addition to the “raw” number, you also get the percentage associated with that category when the total number of cases is the basis for the calculation; the percentage only for the non-missing values; and the cumulative percentage, so that you can tell, for example, that 26.5% of the cases had ratings of 4 or below. (Note also that the Valid Percent category is pretty much the same as the Percent figures in this case, but that if there were a substantial number of missing values,



the two columns would differ more clearly.) A final reminder point here is that this table, too, is worth scrutinising for odd values that might be the result of miscodings or miskeyings. Output: Part Three

Bar Chart of data on New Body Question

Q1NW_BOD

6.005.004.003.002.001.00Missing

Cou

nt

50

40

30

20

10

0

And here, finally, is the sort of bar chart you produced earlier. The only difference is that this bar chart refers to the data on the “new body ….. is necessary” question. Reflecting the figures you have just been looking at, you can see that there are relatively small numbers of people who choose the ranks ‘1’ and ‘2’, slight increases for ‘3’ and ‘4’ and then most people choose the values ‘5’ or ‘6’, which is the most commonly occurring value. The picture tells the same story as the numbers!

Task 2.10 Questions 3 to 8 all probe the functions that a British Institute should have from the perspective of the profession as a whole. Write a half-page report to show the relative importance that is given to these by the questionnaire respondents. There are many formats for a potential report of this sort, so what is shown in the feedback section is not the only way to approach this problem in portraying data. There is also the issue that although you (should) have taken only a few moments to get the information that you needed, there is quite a lot of data, and so you have to be selective: you don’t have space to include everything in your discussion. In that respect, the report in the Feedback section ignores the cumulative frequency plots, and even the bar charts, and only reports the mean scores and standard deviations. As a result, the emphasis is on a very small data table, and then the text which follows.



Feedback

Report: The Functions of British Institute for the Profession Summary of Questionnaire Responses

The results of the questionnaire survey are as follows: For the profession, it should: Mean St.Dev. Q3: Act as the sovereign and constitutive body –

overarching and ‘supra’ 4.40 1.31

Q4: Establish an accepted framework of professional qualifications

5.33 .92

Q5: Establish international equivalences for qualifications 5.33 .90 Q6: Establish a Professional Code of Practice for the

protection of the public 5.18 1.08

Q7: Provide a lobbying and public relations force for British ELT

5.17 1.02

Q8: Form official links (e.g. DfEE, EU, TTA, QCA) 4.97 1.04 It is clear that the most popular functions which respondents see for the projected British Institute is that it should facilitate progress in relation to qualifications, both in terms of a general framework as well as in relation to international links. There is also strong support for the establishment of a Code of Practice to protect the public, as well as a role for the Institute in lobbying and representing the profession. There is strong support, (although at a lower level than in the previous areas) for the Institute to form official links, and, interestingly, there is least support for any such body to have a sovereign, constitutive and presumably regulatory role. It would appear that the Institute is seen more as a service than a regulating body. Finally, it should be noted that there is “normal” variation in the expression of these views. The standard deviations are relatively small (mostly around one-sixth the range), and suggest that the mean scores which are shown can effectively stand for most of the respondents to the questionnaire. The responses to Q3 are the only ones to suggest a greater level of variation. Descriptives The Frequencies procedure assumes you want information on the number of cases for each category on whatever measure you are looking at (e.g. the number of people of who responded to each of the rating points on the “new body” item), and then the procedure allows you to make additional choices, which will generate for you a bar chart and simple descriptive information. If, in contrast, you know that all you want is the basic descriptive information, the Descriptives procedure is a better option (just as if you want a bar chart, the procedure Charts, Bar Chart is a better route). So, perhaps disappointingly there is little new with the Descriptives procedure, just a more direct route to certain statistics.



Task 2.11 To use the procedure, choose: Analyse then Descriptive Statistics, then Descriptives, and from this point you simply make choices on the screen which comes up. Specifically, obtain the mean, standard deviation, minimum and maximum for all the items in the section of the questionnaire relating to what the new body should provide for individuals (i.e. questions 10-16). For the sake of economy, when you are at the Descriptives screen, choosing the variables by moving them from one box to another, make sure you move them as a block, i.e. highlight the first, keep the shift key pressed, and then highlight the last of the group. In this way, you make a quicker selection (and probably, in this case, save about half a nanosecond). In addition, use the Options bex to arrange the output in descending order of means

Feedback The basic information is shown in the table below. Note that the order of the display follows the descending size of the means. This disturbs the original order from the questionnaire, but makes the highest means stand out a little more clearly. Basic Descriptive Information on the New Body for the Individual

Descriptive Statistics

96 2.00 6.00 5.3333 .980595 1.00 6.00 4.8105 1.298996 1.00 6.00 4.7500 1.273293 1.00 6.00 4.7204 1.354396 1.00 6.00 4.4583 1.428495 1.00 6.00 4.3474 1.294695 1.00 6.00 3.8000 1.513192

Q12I_INFQ15I_LIBQ11I_CARQ16I_DISQ13I_JOUQ14I_LOCQ10I_STAValid N (listwise)

N Minimum Maximum MeanStd.

Deviation

Remember that if you need to look up the source questionnaire, it was given as Appendix One to Chapter One. These are interesting results. What is of least importance to the respondents in this sub-sample of the main dataset is the new body will enable people to put letters after their name, e.g. M.BIELT. In contrast, there is clearly a much higher perceived need for the body to act as a source of information, both about the profession in general, (perhaps functioning as a centralised place



for job hunting, or informing people about career opportunities in different parts of the world), and also with resources, in the shape of a library, where material can be easily consulted, one assumes. Then, the next highest rating is received by the item on the role of such a new body in a career structure for the profession. The highest ranking items are, clearly, general in scope, and place the projected organisation in a sort of central, information-source role. The next item, in descending rank order, suggests, somewhat unsurprisingly, that respondents wouldn’t be averse to the organisation negotiating discounts on their behalf. Then, there is something of a drop, and the next two items have noticeably lower means, with the existence of an annual conference and the provision of a journal weighing in with 4.46, and assistance with local teachers’ groups at 4.35. It seems that preference exists, with this group of respondents at least, for an organisation which they can consult when they want to, rather than one which impinges into their lives. Once again, it will be interesting to see if this picture is reflected in the larger dataset you will analyse soon. Crosstabs Suppose you are employed within the ELT profession for an organisation or a school, and draw a salary each month. But maybe you have always hankered after the freedom that working for yourself would give you (not to mention the worry!). Perhaps, though, you have been concerned that life as a freelancer might not be lucrative enough to enable you to live comfortably. The BIELT data allows you to check out this possibility.

Task 2.12 Use the Crosstabs procedure to gather some data on the Freelance vs. Employed salary question. Follow these steps: Choose Analyse then Descriptive Statistics then Crosstabs You are now confronted by a (relatively) complex screen. There are four main boxes you have to consider. On the left hand side there is the box showing all the available variables. On the right hand side there are three smaller boxes, the top one labelled Row(s), the middle one labelled Column(s), and the bottom one, (which you are going to ignore for now), “labelled” Layer 1 of 1. Choose Salary and move it across to the Row(s) box, and then choose Employed vs. Freelance and move it across to the Column(s) box. Ignore all the other things on screen, and click O.K. Look at the output table that you get, and try to decide whether freelancing makes financial sense.




Note, first of all, that SPSS provides you with what it calls a “Case Processing Summary”. This is basic validating output on the data that has been processed. Look at it for any indication that strange things have happened. Otherwise, it can be ignored, as it can be here. You should get the following table:

Salary * Employed vs. Freelance Crosstabulation

Count

16 9 2514 1420 3 2311 1 12

5 54 4 8

70 17 87

£10-15k£16-20k£21-25£26-30£31-35K£36+K

Salary

Total

Employed Freelance

Employed vs.Freelance

Total

Starting at the bottom the table shows that there are 70 employed respondents and 17 freelancers who gave responses (as well, we assume, as several people who did not respond to one or both of the categories represented in the table). But what the table shows most interestingly is the number of each of these groups who report the different salary levels. In other words, there are 16 employed people who earned £10-15K, and 4 freelancers who earned £36K, and so on for the other cells in the table. An examination of the table shows that the distribution/pattern of the Employed group more or less follows the same pattern as the total figures shown in the right-hand column, but the freelancer’s pattern of cases is different. In our small sample there are no freelancers who earn either £16-20k or £31-35k. Instead the most numerous categories are the smallest (£10-15k) and the largest (£36+k). It appears that freelancers are either rich or poor, but not so often in-between!

Reflection We can reflect upon the Crosstabs procedure on the basis of this example of its use. • the procedure always uses a minimum of two variables, which it then relates to

one another • the procedure is based on frequency data, i.e. it tallies the number of cases that

fall into each combination of scores/categories of each of the chosen variables • the procedure is useful in that it displays results in such a way that patterns in the

data can become more evident, as was true in the above example.



Really, then, you should think of the program as providing you with untiring clerical assistance when you need to tabulate two things. You have to think how to exploit what the program can be clerical about, and you have to interpret the data. But the program can display all sorts of cross-tabulations of this sort, and you can then explore what they mean. In Task 2.13, the next task, you are going to answer three questions. You will see how the procedure can be used in a more complex manner. In each case, you have to use Crosstabs to answer a question based on the questionnaire data.

Task 2.13 1. Do people who earn higher salaries tend to consider that the annual membership

fee for individuals should be higher? 2. Do people who earn higher salaries tend to consider that the annual fee for

institutions should be higher? 3. Do older respondents think that the formation of a new body to represent British

ELT is more or less important than younger respondents?

Feedback on Task 2.13 In each case, below, we will look at the Crosstabs output table, and then discuss the results. The feedback in quite long, because all three tables are discussed. 1. Do people who earn higher salaries tend to consider that the annual

membership fee for individuals should be higher?

Annual Individual Membership * Salary Crosstabulation

Count

10 5 8 6 3 324 2 4 2 1 138 5 6 2 4 1 26

1 2 31 1 21 1 1 3

1 124 13 20 11 5 7 80

£30£40£50£60£80£1008.00

AnnualIndividualMembership

Total

£10-15k

£16-20k

£21-25k

£26-30k

£31-35k £36+k

Salary

Total

Judged by this data, I would say the answer to this question is “No”: there is little tendency for those who earn more in a year to chose the higher individual fee category. Possibly there is a slight trend for those in the middle salary range to be more willing for higher fees, but in the main, it seems



straightforward and general, with this sample, that all people would rather pay less than more. 2. Do people who earn higher salaries tend to consider that the annual fee

for institutions should be higher?

Annual Institutional Membership * Salary Crosstabulation

Count

4 6 9 9 2 1 315 1 1 76 5 4 1 2 3 21

2 1 32 23 1 4

15 14 19 10 5 5 68

£300£400£500£600£800£1000

AnnualInstitutionalMembership

Total

£10-15 £16-20 £21-25 £26-30 £31-35 £36+

Salary (Thousands)

Total

Basically, it is difficult to say here. Again, the trend is towards everyone preferring the lower values, even though it would be the institutions themselves which would be paying! It is interesting, though, that the £21-25 category of salary earners do seem more attracted by higher annual fees for institutions. Otherwise, the only individual who deviates from the “less is better” rule is someone who does earn more than £36k, but it is difficult to generalise from just one case. We are left wondering if there is something about being in the middle of this salary range which causes views about institutional fees to differ! 3. Do older respondents think that the formation of a new body to represent

British ELT is more or less important than younger respondents?

Q1New Body * Age Crosstabulation

Count

1 1 22 1 3

1 1 1 1 1 1 61 3 3 4 2 1 14

1 5 4 7 6 8 1 321 4 3 8 13 4 4 37

3 10 14 20 25 15 7 94

1.002.003.004.005.006.00

Q1NW_BOD

Total

26-30 31-35 36-40 41-45 46-50 51-55 56-65Age

Total

There is a slight trend for the younger respondents (26-30, 36-40, and slightly 30-35) to give lower numeric ratings, while older people are more likely to give



ratings of ‘5’ or ‘6’. Perhaps more time in the profession leads to stronger views about the need for such a body. It is important to repeat here that the dataset we are currently working with is a small subset of the entire British Institute questionnaire. As a result, we cannot be sure about the representativeness of the patterns we see: perhaps the distribution of (say) age, or location, or salary will not be the same in the larger dataset. Also important is that the data we have, even though based on 100 cases, are already looking pretty small when we break the data down in any way. The Crosstabs procedure, for example, once it is based upon matrices linking one six-category-variable to another six-category variable, soon looks a bit thin, with some cells contained in this matrix having hardly any entries. Basically, Crosstabs enables you to go into finer and finer detail in examining combinations of variables. That would be inappropriate here because currently we don’t have enough cases to justify this. We will return to this in the next chapter. Means So far, in terms of SPSS menus, we have looked at the Analyse > Descriptive Statistics sequence, and then the statistical procedures which “lie behind” it, i.e. Frequencies, Descriptives, Crosstabs. But there is another descriptive technique which is found if you follow the Analyse > Compare Means > Means sequence (the other options under the Means sub-heading will be treated a little bit later in this course).

Task 2.14 As a start here, follow the sequence Analyse > Compare Means > Means. This will bring up a screen with a (now familiar) three box structure. The one on the left, as usual, is the list of variables to be selected from, while the ones on the right are labelled independent and dependent. For now, think of independent variables as the priming, or organising variables (e.g. things like gender, or employment status), which probably have a small number of categories. Dependent variables are the scores on actual measures (e.g. the various answers to the different questionnaire items) which can be thought of as “organised by” the independent variables. (The use of the Means procedure, and the distinction between independent and dependent variables are revisited in Chapter Four.) Choose the very first variable, q1nw_bod and move it across into the dependent box, and then choose Gender and move it into the independent box. Then click O.K.




If, once again, we ignore the Case Processing summary, the main output is shown in the following table:

Report

Q1NW_BOD

4.7727 44 1.21745.0784 51 1.16354.9368 95 1.1923

GenderMaleFemaleTotal

Mean NStd.

Deviation

This table shows that the average rating for the “a new body to represent …” was 4.94 overall (when rounded), but 4.77 for men and 5.08 for women, i.e. that women give a higher rating to the question than do men. (We will see, in a later chapter, whether this is a statistically significant difference, and what statistical significance means.) In other words, what the procedure Means enables you to do is to identify what one might call an organising variable (which is gender, in this case), and then explore the mean score of some other variable, as arranged in terms of the categories of the organising variable, male and female in this case. This is a very powerful procedure, since it allows you to work through your data very effectively indeed, and explore whether the proposed organising variables do produce patterns in the scores which are found for other variables. Here are some tasks/ questions that you can use the procedure to explore:

Task 2.15 1. Is the average salary higher in the U.K., the Rest of Europe, or the Middle East? 2. Do men or women rate the “act as sovereign and constitutive body” (Q3) item

higher? 3. Does age of respondent influence the mean scores on item 36b, on a diploma level

qualification being the baseline qualification for entry into the profession? Task 2.15 simply consists of questions, which you are expected to use your problem-solving abilities to answer. But the feedback does contain the sequence of actions you should follow. Try, therefore, to do all the tasks yourself. But remember that you have the option of looking at the feedback for help if you have to. If you take this option, though, only do it for one question at a time.




Question One: Is the average salary higher in the U.K., the Rest of Europe, or the Middle East? The procedure to follow here is: Analyse, Compare Means, Means. Then you should choose Salary as the dependent variable, and Country/World Location, as the independent variable, clicking O.K. to generate the actual results. The data in the report table bears upon this question. Remember that the scores are in terms of the coded representations of salary (1=£15-20k, 2=£21-25k etc.)

Report

Salary

2.8831 77 1.54732.4286 7 1.98811.5000 2 .70712.8140 86 1.5756

World locationU.KRest of EuropeMiddle EastTotal

Mean NStd.

Deviation

The most striking thing here, of course, is that the table is dominated by the large number of U.K. respondents, and so we cannot be very sure about the worth of the Rest of Europe (seven cases only) or the Middle East means, (based on only two cases). Still, intriguingly, it is the U.K. based people who get the highest value. This is definitely one to re-do in the large dataset. Question Two: Do men or women rate the “act as sovereign and constitutive body” (Q3) item higher? Same procedure as before, but with Q3 (“act as a sovereign body”) as the dependent variable and Gender as the independent.

Report

Q3P_SOV

4.3000 40 1.34364.4792 48 1.30454.3977 88 1.3178

GenderMaleFemaleTotal

Mean NStd.

Deviation

While there isn’t huge variation between the means, we can say two things here: • we have a reasonable number of cases in each category (40 and 48) • the women do have a higher mean score, at 4.48, compared to the men,

at 4.30. We can wonder therefore, if women are more disposed than men to accept the existence of what would be a regulatory body.

Does age of respondent influence the mean scores on item 36b, on a diploma level qualification being the baseline qualification for entry into the profession?



Once again, the same basic procedure, then age as the independent variable and Q36b as the dependent variable.

Report

Q36B_DIP

5.0000 3 1.73214.3333 9 1.93653.3077 13 1.88793.6667 18 2.05804.1905 21 2.08854.2222 9 1.48145.3333 6 .81654.0633 79 1.9037

Age26-3031-3536-4041-4546-5051-5556-65Total

Mean NStd.

Deviation

The table reveals an interesting pattern: the highest responses, (and the two categories which generate mean scores of 5.00 or more) are from the youngest and the oldest (although there aren’t too many cases for each of these extremes). Then, if one imagines plotting the remaining means on a graph, there is a clear U-shaped curve, in which the values decline, with age, until the minimum of 36-40, and then increase steadily, with the older respondents being more enthusiastic about a Diploma level qualification being the entry to the profession. (Faced with these means, you might now consider generating a bar chart.) The pattern cries out for interpretation. Bear in mind here that the realistic alternative to a Diploma-level entry (implying a course of a minimum of 10 weeks duration, very probable graduate-level entry, and all this after experience has already been gained) is a Certificate level entry (see Question 36a), with a four week full-time course (or its equivalent) and no certainty of previous experience or graduate status). So it seems that those just entering the profession see a Diploma as a reasonable entry point, as do those who are well established. But those in the stages of building a career are less convinced by this. Of course, from these results we only know that these age groups are less enthusiastic about a Diploma-level baseline. We don’t know what they do think would be a desirable alternative. Extending the Means Procedure We can now extend slightly your understanding of the potential uses of Means. The following section may seem close to a task, but it is, in fact, general exposition, with lots of commentary, and synchronised actions on your part. Follow these steps: Choose Analyse then Compare Means then Means then choose Q1 and put it in the Dependent List box then choose Gender and put it in the Independent List box



So far, so familiar. But now, click on the Next button adjacent to the Layer 1 of 1 label. This will give you a new and empty Independent List box, and into that choose and place the Employed vs Freelance variable. While you are doing this, notice the Layer 1 of 1 section has changed, and become Layer 2 of 2. Now click on O.K. and look at the output you get (except, of course, for the Case Processing Summary). The Report section should look like this:

Report

Q1NW_BOD

4.6129 31 1.25645.0000 11 1.09544.7143 42 1.21555.0488 41 1.24405.2500 8 .70715.0816 49 1.16974.8611 72 1.25945.1053 19 .93664.9121 91 1.1986

Employed vs. FreelanceEmployedFreelanceTotalEmployedFreelanceTotalEmployedFreelanceTotal

GenderMale

Female

Total

Mean NStd.

Deviation

Notice that the first row gives you information about the various columns. As before the second, third, and fourth columns tell you about the mean, the number of cases, and the standard deviation. On the left, though, you have an interesting arrangement. In the left-hand column, first you see Gender and then Employed vs. Freelance, i.e. the same order as the Layer 1 to Layer 2 progression that you defined a moment ago. The Gender variable is the more superordinate, and that mean scores for Employed vs. Freelance are given for each of the categories of Gender. In other words, both Gender and Employed vs. Freelance are organising variables, but Gender is even more organising than Employed vs. Freelance! If you had defined the two layers in the reverse order, you would have got the following output:

Report

Q1NW_BOD

4.6129 31 1.25645.0488 41 1.24404.8611 72 1.25945.0000 11 1.09545.2500 8 .70715.1053 19 .93664.7143 42 1.21555.0816 49 1.16974.9121 91 1.1986

GenderMaleFemaleTotalMaleFemaleTotalMaleFemaleTotal

Employed vs. FreelanceEmployed

Freelance

Total

Mean NStd.

Deviation



Now, using your word processor, you could take this output data and set up your own table, as follows, to bring out the 2x2 arrangement of these two variables:

Mean Scores for Gender and Employed vs. Freelance Status Male Female Employed 4.61 5.05 Freelance 5.00 5.25

The table brings out clearly that men seem to give lower ratings than women, and that employed people give lower ratings than freelancers, leading to the highest cell score (Female, freelancers) and the lowest (Male, employed).

Task 2.16 Use the facility to layer with the Means procedure to examine how (a) possession of a teaching qualification and (b) gender, impact on the mean scores for Questions 1 (a new body) and 4 (qualifications).




The data you should have generated is given in the table below. Or rather, the table has been edited (within SPSS) to remove total scores so that the central information stands out more clearly.

4.7931 5.035729 28

1.2065 1.1701

5.2571 5.342935 35

1.0667 .8726

4.5000 5.750012 12

1.3143 .4523

4.6000 5.357115 14

1.2984 .6333

MeanNStd.DeviationMeanNStd.DeviationMeanNStd.DeviationMeanNStd.Deviation

GenderMale

Female

Male

Female

TeacherNoteachingqualification

Teacherqualifiedstatus

Q1NW_BOD Q4P_QUAL

A new body; Qualifications Framework: By Teaching Qualificationand Gender

Regarding the formation of a new body for British English Language Teaching, it is interesting that the group who do not have a teaching qualification, i.e. do not have a Diploma or P.G.C.E., give higher ratings to this question, with the women’s average rating being particularly high. There is little to choose between men and women for the qualified group, with each giving a lower mean score. Slightly surprisingly, (to me), then, the unqualified members of the profession would like the new organisation more. In contrast, we have a reverse situation regarding rating of the new body’s role with a qualifications system. Here the qualified group, especially the men, give the higher ratings, and unqualified group, especially the men, give the lower ratings. The figures, in other words, bring out clearly that the ratings which are given are worth studying in some detail, since such an approach brings out findings which would otherwise not be obvious.



Reflection The layering, or nesting quality of the Means procedure is a very very powerful tool. It allows you to explore your data at a remarkable level of delicacy. You can layer at more than two levels, and so you can imagine, if you have a number of organising variables in your data, that you generate mean scores for quite complex combinations of conditions. The result is that if you routinely use the Means procedure with datasets, you will maximise the chances that if there are patterns in the data, you will uncover them. It’s worth getting into the habit, with datasets, of exploring the data with several uses of the means procedure. The next chapter will be concerned with three things: • the large BIELT dataset, containing almost 1200 cases • pursuit of the answers to a series of questions about this dataset. You will find that

on many occasions the Means procedure is crucial in building answers to these questions, and that the layering capacity that Means has will be very useful indeed.

• A revisiting of the lateralisation questionnaire.

chapter two the subset questionnaire · the entire questionnaire, you are able to address this...

Documents