A framework for the analysis of interview data from multiple field research sites

Anne M. Lillis

Afzaal Ali

Business School, University of International Business and Economics, Beijng, China.

Anne researches and teaches in Management Accounting. Her research interests focus on field studies of the design and behavioural influence of performance management and control systems. Her teaching interests are in Management Accounting, with a particular focus on capstone and case-based teaching. 

Anne Lillis is a Professor of Management Accounting and Head Department of Accounting and Finance, University of Melbourne, Melbourne, Victoria, Australia.

BCom, University of Melbourne 1980 MCom, University of Melbourne 1990 PhD, University of Melbourne 1998

Anne is an Associate Editor of Journal of Management Accounting & Research, and a member of the editorial boards of Accounting, Organizations and Society, Contemporary Accounting Research, Management Accounting Research, Behavioral Research in Accounting, Accounting and Finance, Abacus and the European Accounting Review.

Summary of the article:Quantitative and qualitative research are commonly considered to differ fundamentally.

In this paper, the author basically tried to focus the issue with qualitative data. In general, papers reporting the results of research studies disclose little detail regarding attributes of study design, analytical processes and methods actually used by researchers.

The major focus of the paper is the application of a systematic analytical protocol designed to encourage completeness and impartiality in the collection and analysis of qualitative data.

Conti…

This paper describes in some depth the method choices and analytical protocol used in a field study project.

To overcome this issue, the author follows a systematic analytical protocol proposed by Miles and Huberman (1994) in their book “Qualitative data analysis: An expanded sourcebook”.

Background of the problem:

Over the past 15 years, in the Accounting (Management Accounting) literature researchers have been urged to shift their study of the role and functioning of accounting to its natural contexts i.e. case/ field study research.

While the call for fieldwork has been persistent, the penetration of this research into top-ranking journals, particularly in the United States, has been very limited.

Shields (1997) attributes the lack of publication of case/ field study research to several factors, with lack of knowledge about how to do good studies' leading the list.

Similarly Foster and Young (1997) argue that few studies meet the criteria of high quality field research applied in other disciplines.

Conti…It may be that some of the problem lies with the way field study papers are written.

Compared with surveys and laboratory studies, qualitative research, at least in accounting, lacks an accepted language in which to communicate study design parameters and the critical links between design and credible research outcomes.

It seems that at least part of the difficulty in publishing field research lies in convincing reviewers that the study is not only relevant and interesting, but also trustworthy.

Section-01: Already discussed

Second-02: The description of the field study protocol commences in this study with a discussion of the general research question and implications for method and design choices.

Section-03: and Section-04: Provide a brief overview of the study design and method and an outline of the population and sample.

Section-05: Outlines the general design characteristics of the interview guide.

Sections of the paper:

Section-06: Details the analytical method applied to the qualitative data.

Section-07: Describes the way qualitative and quantitative data are linked in this study to provide tests of convergent validity at several levels.

Conti:

Objectives of the paper:This paper describes the application of qualitative study design features that can enhance trustworthiness. The paper examines the design of a systematic field study protocol.

Rather than an abstract discussion of method, the protocol is described within the context of a particular study in order to link explicitly the research question, research design and analytical protocol.

Thus this paper focuses primarily on the design of an analytical protocol that can support an audit trail from field data to the testing or development of propositions.

An audit trail is a transparent description of the research steps taken from the start of a research project to the development and

reporting of findings.

This paper describes part of a larger study of strategy and performance measurement in manufacturing.

While the larger study involved the collection and analysis of both quantitative and qualitative data.

This paper reports on the qualitative elements of the study.

The study was motivated from contingency- type studies linking strategy and management control system design and from the more recent management accounting literature on strategic performance measurement.

2. The motivation for field research:

Two issues are particularly relevant to the design of the qualitative elements of the study:

1st one:- The findings of contingency studies linking strategy

with the use and characteristics of management accounting controls have been notoriously conflicting (Langfield-Smith, 1997).

Past literature does not support predictions about the roles of any specific performance measurement techniques in particular strategic contexts.

Conti…

In the 1980s, the manufacturing side witnessed radical change in context of competitiveness.

In this situation, traditional performance measurement systems were proved to be insufficient and to some extent become irrelevant.

As a response, some new solutions emerged in the form of the Balanced Scorecard (Kaplan and Norton, 1992; 1996) and Integrated Performance Measurement (Dixon et al., 1992).

However, the emphasis shifted to promoting the development of non-financial dimensions of performance such as quality and customer responsiveness etc.

2nd one:-

Examples include: Profitability Revenue Return on investment (ROI) Cash flow Standard deviation or VarianceCoefficient of variationRate of return or return on investmentWorking capital

Financial Performance Measures:

Examples include: Customer satisfaction Employee satisfactionQualityMarket shareNumber of new products

Non- Financial Performance Measures:

Both of these issues suggest the need for an exploration of competitive strategyand its implications for accounting performance measurement in practice.

Moreover, there is an apparent need to gain greater insight into the empirical experience of variablessuch as competitive strategy and the role of accounting performance measures (Langfield-Smith, 1997).

3. Overview of study design and method:

Data were collected using a semi-structured interview in combination with a structured questionnaire administered to 50 managers in 36 manufacturing firms in Victoria, Australia.

The use of a semi-structured interview schedule in combination with a structured questionnaire aimed to derive the benefits of quantitative and qualitative methods, and to apply appropriate methods to the questions of interest.

Conventional statistical analyses were applied to data collected using the structured questionnaire.

This paper focuses on the qualitative data collected during the semi-structured interviews.

Conti… The researcher used several approaches to limit

biasness both during the interview and in the analysis of transcripts.

First, researcher designed an interview guide with the aim of ensuring complete and consistent coverage in each interview session.

Secondly, researcher in advanced developed a list of neutral questions and queries in order to minimize interference.

Fourth, a systematic auditable process developed by Miles and Huberman (1994) was used.

Finally, the use of structured questions provided quantitative response from the respondents on established scales.

Time and cost considerations limited the sample of participant firms to the population of manufacturers in the State of Victoria, Australia.

The population was further restricted to relatively large manufacturing firms with more than 200 employees.

Researcher only consider firms that would have formal performance measurement systems in place.

A listing of 340 Victorian manufacturing firms with more than 200 employees was obtained in May 1994, from the Victorian State Department of Business and Employment.

4. The population:

The sample: Several features of the study design determined the

sample size.

The sample was required to be sufficiently large to be able to observe patterns across firms.

To conduct simple inferential statistical evaluation of results.

However, the sample size was to be limited because the study design required semi-structured interviews and a structured questionnaire to be administered by a single researcher.

A sample size of 30-40 manufacturing firms was determined to be appropriate in this context.

The firms were selected across industry categories with the aim of obtaining the participation of firms of comparable size, and with a variety of strategic orientations. .

In total, researcher contacted 55 profit centre managers. .

Out of 55, finally 36 managers were available to give feedback. While 19 managers refused to participate in this study, and final response rate was 65 percent.

Conti..

Follow-up sites visits:

In order to confirm and find possible contradiction in the data collected from profit centre managers, researcher selected 12 manufacturing managers from sample firms.

Researcher formally got permission from all of the profit centre managers to pursue a follow-up interview with a manufacturing manager.

The final sample consisted of 36 profit centre managers and 12 manufacturing managers.

The interviews were conducted between June 1994 and January 1995.

5. Instrument design- the interview guide:

The study was conducted with a semi-structured interview and structured questionnaire.

The interview guide used in this study is divided into four sections. For example, the performance measures used, the constitution of cost benchmarks and competitive priorities represent three separate themes.

The interview protocol was developed from methods documented by McCracken (1988) and Brenner, Brown and Canter (1985) based on their qualitative research experiences.

In addition to the potential interviewer-induced bias in the collection of qualitative data, the analysis of qualitative data is subject potentially to significant bias as it relies on interpretations and classifications imposed by the researcher.

Among the reasons why readers experience `lack of trust' in qualitative studies lies in the absence of established techniques for ensuring that data analysis is both complete and impartial.

On the other hand, the analysis of qualitative data is less structured.

6. The qualitative analysis protocol:

Readers are left questioning the extent to which propositions raised by the researcher are supported by the data; whether potential alternative propositions have been `overlooked‘; whether all cases were systematically evaluated before propositions were raised; whether the researcher saw only what they set out to see in the data; and whether the process is reproducible.

The use of a systematic analytical protocol such as the Miles and Huberman (1994) data display approach used here, enhances trust in the results of qualitative analysis.

Conti…

To analyzed and interpret qualitative data, researcher followed a four steps qualitative analysis protocol.

The variable constitution of performance benchmarks was operationalized in the qualitative data, and this variable was hypothesized to be related to strategy. The structure of the matrix reflects this theoretical proposition.

Step 1: Transcripts were coded using the qualitative analysis package QSR NUDIST. NUDIST was used in this study to code all of the raw, unsummarised interview transcripts, by associating the sentences in the transcript with one or more themes defined in a hierarchical coding structure. Each sentence was given a text-unit number in NUDIST. In this way, each text unit is identified as relating to one or more pre- determined thematic codes.

NUDIST is an acronym for Non-numerical Unstructured Data Indexing, Searching and Theorizing. . It is a qualitative analysis software package distributed by Qualitative solutions and Research Pty Ltd, La Trobe University, Bundoora, Victoria, Australia.

Step 2: A data sheet was prepared manually for each transcript. The text in the transcripts was reconfigured and reported under conceptual headings using NUDIST. The reorganized transcripts were then summarized manually under similar conceptual headings to those used in the coding scheme.

Step 3: The data sheets prepared for each transcript were then combined in a multi-case matrix format.

For this study, these matrices and the analyses they supported were focused on the variables to be intercorrelated.

Step 4: The content of the matrix was then built up case by case. Each case was classified initially as `predominantly cost minimisation', `mixed' or `predominantly differentiation' based on scores on the scaled strategy variable.

7. Linking qualitative and quantitative data:

Thus, the findings were triangulated against the quantitative analysis of participant-rated data on variables similar to those explored in the qualitative data.

For example, scaled responses were collected on strategy using Porter‘s (1980) typology in a form similar to that used in survey-based research (Govindarajan and Fisher, 1990).

The scaled responses reflected an over- whelming importance of differentiation strategies in the sample. This quantitative analysis was compared with the elaborated responses on the firms' competitive edge and strategic priorities.

These quantitative and qualitative analyses `converged' to reflect a similar strategic profile across the sample.Data triangulation validates data and research by cross verifying the same

information

The proposition emerged that firms experience greater difficulty integrating performance criteria relating to customer responsiveness with efficiency criteria than they did integrating quality and efficiency criteria.

Proposition:

Retrospective evaluation of the method adopted: First, the study design achieved its purpose

successfully.

Beyond such re-examination of traditional contingency models, the findings from this study raised new unhypothesised relations that formed the foundation for theory building.

The protocol used here establishes a disciplined approach to data extraction and analysis that at least promotes completeness and impartiality.

A retrospective evaluation may assess the degree to which goals were met.

Conti…

She mentioned some researcher-related problems specifically in qualitative study.

There are practical matters such as scheduling too many interviews too close together, not gathering sufficient archival and other data while on site, getting too tired, occasionally losing enthusiasm and concentration and thus finding loose threads in interviews which should have been followed up.

THANKS