What is data in educational research?

Paul Standish

Centre for Philosophy of Education UCL Institute of Education

Data - the sine qua non of enquiry into education? On research methods courses great effort goes into becoming clear about : different kinds of data

how they are gathered

their appropriateness to different research purposes.

The gathering of data is regarded as an essential basis for enquiry into educational practice;

and some will claim that without data research is not legitimate.

The desire for foundations

Such views are often based on the apparently common-sense principle that any form of enquiry and any argument must rest on foundations. There must be a starting point, a rock on which to build.

It may even be thought that foundational thinking is necessary for clear thought of any kind, and that this can be seen in developmental terms.

Don’t we learn that 2+2=4 before we learn more complicated arithmetic? Don’t we learn to count even before we learn this? And isn’t it true of all ways of thinking that there are first steps upon which others are built?

But how can we really know? The problem of scepticism

A problem with foundations, however, is where we find them, and this has been a source of perplexity through the ages – for scientists, historians, philosophers, theologians and religious believers. . .

“What is the basis of your claim to know?” – This seems like a very reasonable question. But what if we can’t answer it?

Let’s consider one famous attempt to think this through.

René Descartes and scepticism

From Descartes, Meditation II (1641): “Of the nature of the human mind; and that it is more easily known than the body”:

1. The Meditation of yesterday has filled my mind with so many doubts, that it is no longer in my power to forget them. Nor do I see, meanwhile, any principle on which they can be resolved; and, just as if I had fallen all of a sudden into very deep water, I am so greatly disconcerted as to be unable either to plant my feet firmly on the bottom or sustain myself by swimming on the surface.

I will, nevertheless, make an effort, and try anew the same path on which I had entered yesterday, that is, proceed by casting aside all that admits of the slightest doubt, not less than if I had discovered it to be absolutely false; and I will continue always in this track until I shall find something that is certain, or at least, if I can do nothing more, until I shall know with certainty that there is nothing certain.

2. I suppose, accordingly, that all the things which I see are false (fictitious); I believe that none of those objects which my fallacious memory represents ever existed; I suppose that I possess no senses; I believe that body, figure, extension, motion, and place are merely fictions of my mind. What is there, then, that can be esteemed true? Perhaps this only, that there is absolutely nothing certain.

3. . . . I had the persuasion that there was absolutely nothing in the world, that there was no sky and no earth, neither minds nor bodies; was I not, therefore, at the same time, persuaded that I did not exist?

Far from it; I assuredly existed, since I was persuaded. But there is I know not what being, who is possessed at once of the highest power and the deepest cunning, who is constantly employing all his ingenuity in deceiving me. Doubtless, then, I exist, since I am deceived; and, let him deceive me as he may, he can never bring it about that I am nothing, so long as I shall be conscious that I am something. So that it must, in fine, be maintained, all things being maturely and carefully considered, that this proposition (pronunciatum) I am, I exist, is necessarily true each time it is expressed by me, or conceived in my mind.

Cogito ergo sum. I think, therefore, I am.

Descartes’ legacy

The separation of mind and body The spectatorial stance in relation to the world (greatly accelerated with the rise of science and its technology) Changing social structures in the modern period (1600+), with the individual becoming a free agent You might think of these things as promoting: A different metaphysics: things become objects; the world becomes

neutralised or disenchanted; nature as dynamic turns into nature as resources.

A different ontology: the human being becomes detached from the natural world, acting on that world, manipulating it, controlling it, and out of harmony with it.

Further consequences for facts and values

Charles Taylor speaks of the medieval world as semiologically structured. Value and meaning was inherent in the world.

But with the change to the modern world, this sense of meaning disappears, and the world comes to seem neutral and inert.

With these changes there comes also a separation of fact and value. Values are something conferred on an inert world by human beings. Values are primarily “in the mind”. They contrast with a world of facts, which is “out there”.

And how do facts and values relate to questions of objectivity and subjectivity?

David Hume and the is/ought problem From the Treatise of Human Nature (1739) – in paraphrase:

You cannot get an “ought” from an “is”.

G.E. Moore, Principia Ethica (1903)

Moore: If someone confuses “good” with any natural object whatever, then there is reason for calling that the “naturalistic fallacy”.

The naturalistic fallacy, in philosophy, is the (supposed) fallacy of inferring an “ought” from an “is”.

Suppose someone says:

Torturing cats causes them unnecessary pain.

(A factual or “is” statement.)

Therefore, you shouldn’t torture cats.

(A value or “ought” statement.)

According to the naturalistic fallacy, this is not valid reasoning.

To make it valid you need a further premise (italicized below):

Torturing cats causes them unnecessary pain. (Factual or “is” S.)

You ought not to cause unnecessary pain. (General value or “ought” S.)

Therefore, you shouldn’t torture cats. (Particular value or “ought” S.)

Objectivity and subjectivity

So the naturalistic fallacy is based on the assumption of a fact/value dichotomy:

facts are out there in the world;

values are somehow added to this.

This encourages a certain conception of objectivity (the realm of facts) and subjectivity (the realm of values).

Can the fact/value dichotomy be challenged? Look again at the example of torturing cats and the explanation of the fallacy.

Torturing cats causes them unnecessary pain. Therefore, you shouldn’t torture cats.

Is there anything that can be said against this? The desire for clarity of thought prompts us to seek clear distinctions. Does the distinction between factual and value statements hold up? Or do values go “all the way down”?

But surely there are at least some factual statements that are value free. E.g. Oak trees have green leaves. Iron is a metal. . . These seem purely factual and free from value judgements.

But these facts are related to values to the extent that they would never have been identified if they did not in some way relate to human purposes (which are, of course, matters of value).

In fact the world in its most general and basic sense seems already to be characterised in terms of this human perspective.

Where does this leave the idea that objectivity relates to the realm of facts and subjectivity to the realm of values.

3. Refuting the sceptic

Refuting the sceptic Scepticism involves the tendency to submit things to doubt: How do I know that I am not dreaming? How do I know that I exist? How do I know that you are telling the truth? How do I know that learning is taking place? The later Wittgenstein (especially the Philosophical Investigations, 1953) is normally taken to provide a refutation of scepticism. The sceptic asks “How do I know that this is a table, that I exist, etc.?”, but the very asking of this question cannot occur without a background (of ordinary life, of language, of other things), which the question has suppressed. Acknowledging that background undermines the sceptic’s position and dispenses with the doubt.

Doubts about the refutation (or about its significance in scepticism) Stanley Cavell has questioned this reading of Wittgenstein (e.g., The Claim of Reason, 1979). If Wittgenstein successfully refutes the sceptic, why does he let the doubt come back, again and again? The doubt is not just resolved. It won’t just go away. The itch returns.

What Cavell tries to show is that sceptical doubt in epistemology reflects something deep in the human condition: our compulsion to call what we ordinarily know into question. Hence, there is an existential truth in scepticism.

Cavell shows how this compulsion to doubt permeates the human condition – through our ordinary experience and into the more extreme phases of our moral lives. Take the cases of King Lear and of Othello (in Shakespeare’s plays).

One of the manifestations of this compulsion to doubt is found in what Cavell calls our “dissatisfaction with criteria”. It’s as though we cannot accept the ordinary circumstances of our lives: we cannot acknowledge what we know. Take, as an example of this, the modern orthodoxy that we cannot know if learning is taking place unless outcomes are formally measured.

What is real? What is truth?

Theories of truth have typically divided into two camps:

Correspondence theories are based on the view that truth consists in the correlation of statements with states of affairs in the world. (“The cat is on the mat” + the cat is actually on the mat.)

Coherence theories have taken the view that statements have their sense in virtue of their interconnections with other statements. (Thus, “The cat is on the mat” is held in place and given meaning by its (dis)connections with “The dog is on the mat”, etc.)

Ludwig Wittgenstein is suspicious of questions such as “What is truth?” on the grounds that they are not doing real work. Language has gone on holiday. He wants to return our enquiries to the kinds of circumstances where we would ordinarily speak about things, including the ways we might speak about the truth.

Martin Heidegger draws a distinction between the characteristic preoccupation of philosophy with truth as correctness (adequatio) and an older and in some senses more originary idea of truth as revealing (aletheia).

Thought and language

What is the nature of this relation? Two prominent ideas:

(i) Language is a means of communication, a vehicle for concepts.

(ii) Underlying language there is logic. To think well is to cut through the messiness of ordinary language in order to uncover the basis of our reasoning.

Can these ideas be criticized? If so, what is the consequence, for philosophy, for education, and educational research?

So what is data?

A datum = something that is given.

But “given” in what sense?

Think of the problem in geometry: “Let a be an angle of 80 degrees.”

This seems to be a factual starting point, but it is presented in the subjunctive mood – the mood not of fact but of possibility.

But surely educational research is different. We can compare boys’ and girls’ achievement in maths by gathering factual evidence of the difference.

And yet how many assumptions and hypotheses are embedded here, how much background?

So let’s think more simply of basic research in biology into the genetic structure of different living things.

Surely genes are factual enough.

And yet what biologists mean by “genes” changes with time, as new hypotheses develop. . .

In fact genes are not so much things that are lying about embedded in the world, waiting for us to discover them, as concepts that we bring to the understanding of the world, and these concepts are less stable, more embedded in hypotheses, than we might imagine.

Remember what we said above about oak trees, for example.

And remember also that the very idea of a cause is not something found in the word but rather the means of explanation of what is found.

A return to scepticism?

“So you are saying then that in the end we can never really know? You are saying that the sceptic is in the end always right – that we can never know anything?”

No. Think of the ways in which we actually use the word “know”. Think about what we actually do. The mistake arises at times when we press too hard for foundations: “How do you really know?” What secures our ways of knowing and acting are networks of thought in which no single proposition or fact about the world is our starting point: there is no rock on which we build.

Behaviourism (in some forms) would be an example of distrusting what we ordinarily know and in consequence reducing human life to behaviour.

Data in educational research

Empirical research involves three stages: (i) identification of a topic and formulation of research questions; (ii) the gathering of data; (iii) analysis and interpretation of the data.

Educational research methods courses typically give extensive attention to methods of data collection, and quite often this is the sole way in which questions of ethics (values) are considered.

But consider the following:

this tends to deflect attention from the questions of values and justification that are there at the start of enquiry, in which reason and judgement are paramount

it gives little attention to the nature of the concerns, argument, and speculation that will shape the subsequent analysis and interpretation.

So this is a question of positionality and bias?

No. Certainly it is reasonable to consider the influence of one’s position and the possibility of bias, but this is not the point.

What matters is rather that these aspects of the research centrally involve questions of reflection and justification – the head-on addressing of questions of value.

And here data cannot be our starting point, because what will count as data will depend upon the kinds of values we bring to the fore.

None of this occurs in a vacuum: the background is always there.

But it shows the fallacy of thinking that research begins in data.

It helps to reveal what Wilfred Sellars called “the myth of the given”.

What works?

Sometimes politicians and policy-makers will justify what they do in terms of what works best. There is nothing necessarily wrong with this provided that thought has been given to the intended outcome.

But talk of outcomes can also encourage a separation of means and ends in a kind of instrumental rationality. This derives partly from the metaphysical and ontological Cartesian legacy. It obscures so much about the human condition and the world we live in that is important.

Sometimes speaking about means has an air of robust efficiency and practicality. But again it can block forms of practical reason (Aristotle’s phronesis). And it is easy to fall into its grip (cf. Lyotard and performativity).

The allure of neuroscience

Neuroscience is sometimes seen as the great new breakthrough in research, and certainly the amount of data that is being produced is astonishing. Of course some have seen this as the potential solution to education’s problems (cf neuroscientific research into moral education).

Is the mind the same as the brain? Neuroscience tends to be dominated by a rejection of mind/brain dualism, with the implication that human action is fully analysable at the level of brain processes.

Against this it might be said that this involves a “category mistake” (cf Hume).

An example of a category mistake might be the view that the change from major to minor keys in a piece of music is fully explainable in terms of changes in sound waves (the language of physics).

Looking for evidence in the wrong place losing confidence in judgement The problems we have considered in relation to data are manifested in behaviourism, uncritical faith in evidence, and instrumental reason. The attraction of these ways of thinking is that they remove the need for judgement (and destroy our confidence in our ability to judge).

One way they do this is through their reinforcement of a dichotomisation of objectivity and subjectivity, which borrows from the fact/value divide:

objectivity is out there and to do with facts in the world;

subjectivity is in the mind, and it is purely personal.

But this is to misconceive both and to hide the relation of judgement to reasoning and truth.

The idea of a social science

It also destroys the basis of social science and indeed science itself.

As we saw, science depends upon the construction of hypotheses and on taking some things as given (not just recording evidence).

A further point should also be made, following Peter Winch’s The Idea of a Social Science (1957). While in, say, geology, we have the vocabulary of the geologists, which is addressed to the rocks, in social science we have the social scientist’s vocabulary addressed to the people being studied. People are self-interpreting beings, with their own vocabularies. This is a massive complication, and it means that measurement will never be enough.

So must all research be empirical?

Manifestly, on the present argument, the gathering of data will never be enough. We shall always need the exercise of judgement, involving interpretation and reasoned argument.

And manifestly again, some research is needed that is not empirical at all: research that addresses directy the kinds of values and justifications upon which education (and so much else) relies.