Research into Practice:Qualitative Research in Information

Systems

slides from: avison@essec.fr

David Avison, ESSEC, Fr

Michael Myers, University of Auckland, NZ

SEISN

• Research in Practice

• Qualitative research as well as quantitative -

• But this does not imply poor research

Qualitative Research Methods

• Case study

• Action research

• Grounded theory

• Ethnography

Case Study

A case study is an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident (Yin, p.13)

Lee (1989)

• Suggests a scientific methodology for IS case studies

• Argues that research based on a single case study is able to satisfy positivist criteria for scientific research

• Uses Markus (1983) as exemplar– She discusses how various theories help to explain

power and politics in the implementation of management information systems

Markus (1983) Starting Point

- To avoid resistance, get top management support and obtain user involvement in the design process (Lucas, 1974)

- Technically sound systems are less likely to be resisted than those with frequent downtime and poor response time (Alter, 1975)

- Users resist systems that are not ‘user friendly’ (assertions by IT equipment vendors)

- All other things being equal, people will resist change (received wisdom)

- People will resist an application when the costs outweigh the benefits (received wisdom).

Markus (1983) Conclusion

People-determined

System-determined

Interaction theory

Educate users (training)

Educate designers(better technology)

Fix organizational problem before introducing systems

Coerce users (edicts,policies)

Improve human factors

Restructure incentives for users

Persuade users Modify packages to conform toorganizational procedures

Restructure relationships between users and designers

User participation (toobtain commitment)

User participation (to obtain better design)

User participation is not always appropriate

Markus (1983) In Between

• Interviews with over 30 designers and users of the systems

• Documentary evidence about the systems and the organizations– corporate annual reports – organizational charts, – system training manuals, – design documents, – internal correspondence about the systems.

• Paper includes many quotations, some long• Markus makes a prediction based on her

analysis

Action Research

Action research aims to contribute both to the practical concerns of people in an immediate problematic situation and to the goals of social science by joint collaboration within a mutually acceptable ethical framework (Rapoport, 1970, p. 499).

Baskerville and Wood-Harper (1996)

• Proposes what the authors call ‘a rigorous approach’ to action research

• Suggests certain domains of ideal use (such as systems development methodology

• Discusses various problems, opportunities and strategies for those faced with conducting, reviewing or examining the research method.

Problems and Opportunities

• The lack of impartiality of the researcher has led to rejection of the action research method by a number of researchers

• Some of the action research offered to the scientific community lacks rigour

• Action research is sometimes branded as ‘consulting masquerading as research’

• Action research is context-bound, and not context-free

Grounded Theory

• A research method that seeks to develop theory that is grounded in data systematically gathered and analyzed

• An inductive, theory discovery methodology that allows the researcher to develop a theoretical account of the general features of a topic while simultaneously grounding the account in empirical observations or data (Martin and Turner, 1986)

• Suggests that there should be a continuous interplay between data collection and analysis

• Develops context-based, process-oriented descriptions and explanations of the phenomenon under study.

Orlikowski (1993)

• Presents the findings of an empirical study into two organizations’ experiences with the adoption and use of computer-aided software engineering (CASE) tools over time

• Characterizes the organizations’ experiences in terms of processes of incremental or radical organizational change

• Findings stress the social context of systems development, the intentions and actions of key players, and the implementation process followed by the organization

Type and amount of interviews conducted at one company

PositionApplications Development

CASE Tools Support

Total

Partner (Senior Manager)

5 5 10

Manager (Project Manager)

27 15 42

Senior Consultant (Senior Analyst)

13 18 31

Staff Consultant (Junior Analyst)

11 4 15

Client (User) 11 10 21

TOTAL 67 52 119

Modes of analysis

• Different approaches to gathering, analyzing and interpreting qualitative data– Hermeneutics

– Symbolism

– Grounded theory

– Semiology

• The common thread is that all qualitative modes of analysis are concerned primarily with textual analysis (whether verbal or written).

Hermeneutics

• As a philosophical approach to human understanding, it provides the philosophical grounding for interpretivism

• As a mode of analysis, it suggests a way of understanding textual data

• Hermeneutics is primarily concerned with the meaning of a text or text-analogue (an example of a text-analogue is an organization, which the researcher comes to understand through oral or written text)

• The text might be confused, incomplete, cloudy, and seemingly contradictory: the interpretation aims to reveal an underlying coherence or sense

Boland (1991)

• Makes an interpretive reading of the way four experienced managers analyze and give meaning to the output of a personnel evaluation information system

• Views the output from an information system as a text being read and interpreted by the information system user

• Identifies some unique features of the user as an active reader of the information system text

The Milne Data

• Ratings on 57 required activities

• Activities performed in addition to the 57 required activities

• Expense data

Ratings on 57 required activities Ratings By Year Number of Activities Receiving Each Rating East Division West Division1977Unsatisfactory 0 0Marginal 2 2Satisfactory 45 50Satisfactory Plus 10 51978Unsatisfactory 0 0Marginal 0 0Satisfactory 40 49Satisfactory Plus 17 81979Unsatisfactory 0 0Marginal 0 0Satisfactory 30 48Satisfactory Plus 27 91980Unsatisfactory 0 0Marginal 0 0Satisfactory 27 45Satisfactory Plus 30 12

Activities performed in addition to the 57 required activities 

East Division West DivisionNumber of Total

Number of TotalYear Activities Man-hours Used Activities Man-

hours Used 1977 8 1160 3 3091978 12 1740 5 4201979 17 2280 7 6051980 21 2740 9 790

Expense data 

(Thousands of Dollars)

EAST DIVISION WEST DIVISION 1977Salaries $417 $267Equipment 68 33Other 21 31

Total 506 3311978Salaries $595 $290Equipment 69 39Other 11 31

Total 675 3601979Salaries $564 $329Equipment 31 51Other 31 36

Total 626 4161980Salaries $611 $371Equipment 107 47Other 36 41

Total 754 459 

Subject 6 1: Choosing East

‘My preference was for the East manager because he appeared to be more outgoing. Company wise, he performed a lot more than the 57 required activities. Also, those activities were scored on the average more ‘satisfactory plus’ than the West manager. Although the West manager is doing good, he’s only so-so. It seems he should become more active and try harder. Maybe he had the satisfactory ratings because of the people who work for him. It seems the East manager does a better job.’

Subject 67: Choosing West

‘I picked the West manager because he ran a tighter ship as far as budget was concerned and seemed to be more time organized. If he used the amount of money East did, he would definitely have better results. Both had approximately the same number of employees but salaries were very different. West didn’t spend an inordinate amount of time on activities other than the required since it is wasteful given the fact there is little direction as to what is important.’

Subject 46: Choosing East

‘Although they do not have any obvious differences, I would select East…His Division had a higher number of satisfactory plus ratings on the 57 required activities. Although he did have two marginal ratings in 1977, this problem did not recur again. This indicates a ‘cracking down’ on his staff….East also had more extra activities which, I feel, demonstrates initiative and ambition….Although East’s expenses were higher, I feel this is partly due to the fact that he had no budget. I feel if he were promoted, and given a budget to stick to, that he would do very well.’

Subject 53: Choosing West

‘The manager in the West had consistently higher satisfactory ratings than the East, but the East had higher satisfactory plus ratings….It seems that the East manager tried harder to get the satisfactory plus ratings and achieved it, but in addition they had twice as many additional activities which might imply they tried too hard for the satisfactory plus ratings and had to go back…Financially the West operated at a lower cost and accomplished more satisfactorily without having to go back and do things over. ’

Narrative and Metaphor

• Narrative is a ‘tale, story, recital of facts, especially a story told in the first person’

• Metaphor is the application of a name or descriptive term or phrase to an object or action to which it is not literally applicable

Hirschheim and Newman (1991)

• Challenges the commonly-held assumption that information systems development (ISD) can be conceived of as a normative process reflecting conventional economic rationality

• Suggests that even the basic assumptions about the rationality of the actors and the social processes they engage in need to be critically appraised, and suggest that the role of myth, metaphor, and magic hold more promise as an explanation

• Four main case studies used to demonstrate conclusions – again, lots of quotations

Myths

• User involvement is beneficial and should be encouraged

• Resistance to systems development is dysfunctional and should be eradicated

• Information systems should be integrated wherever possible

• The systems developer is generally the best person for making decisions about the system

• Politics should not be the concern of the system developer

• The key to successful design is the use of a top‑down approach

Metaphors

• Information is structured data • Organization is information • Information is power • Information is intelligence • Information is perfectible• Technology as tool and man as craftsman • Technology as governor and man as

machine • The battle metaphor

Magic

• The systems developer as high priest • Expert systems as the embodiment of

the human expert • Sign‑off as magic • Ownership of data as magic

Conclusions

• Progress the field of information systems• Empirical research appropriate• Greater interest and appropriate use of

qualitative approaches in IS• Not an excuse for poor research• Should always be a response to the question

‘what research approach was used in the IS research?’