Design Research (DR): A Critical Review

C.VoigtCentre for Social Innovation

- MASI 2013

Three Questions

• Taxonomy: What is DR?

• Practicability: When to use DR?

• Generalisability: What is a design?

C.Voigt - MASI 2013

1st Question: What is DR ?

Motivation:• Research uses different categories for its instruments. The category

tells you what to expect and what not to expect. This is important to know in order to evaluate DR against the appropriate standards.

Question:• What category does DR belong to? Is it a methodology, a method, a

framework, a theory or something else?

Answer:• Formalistic; define and compare

C.Voigt - MASI 2013

Definitions ‘Jog-Through’

• Methodology: specifies a general set of assumptions about what can be observed vs what is inferred, how do we justify our inferences and how do we justify our research (Onto-, Epistemo-, Axio-logy)(Sarantakos 1998, p.33)

• Method: describes the interplay between data collection, data reduction, data display and conclusion drawing/verification (Miles & Huberman 1994, p.10)

• Framework: graphical or textual description of key constructs to be studied and their presumed relationship. They may be commonsensical, theory- or data driven. (Miles & Huberman 1994, p.18)

• Theory (e.g. about design improvement) is a framework plus agreed upon standards to evaluate its content, e.g. comprehensiveness, parsimony and conservatism (Quine 1978, Web of beliefs)

C.Voigt - MASI 2013

Maturity of DRMethodology Method Theory

Ontology Axiology Data Collection

Data Display

Conclusion

Verification

Key constructs &

Relationships

Pos. - ** pragmatic

user centered

*Iterative

Reactive

*algorithms

prototypes

*Artifact

centered

Replication

*Design

Pos. and negative Design

knowledge

Neg. No Definition

what a design is.

Partially technolog.

Deter-ministic

No guidelines about what data to collect? E.g. Use-cases, User-feedback

Etc.

No guidelines about how to visualize design changes over time.

At times:

One-sided

Deterministic

Lacks construct definitions

C.Voigt

2nd Question: When to use DR?

Motivation:• Every research involves a design. Does this mean you do design

research?

Question:• What are DR’s strengths?

Answer:• Good for developmental research, aimed at mainly technical

systems. There might be better alternatives for socio-technical

systems.

C.Voigt - MASI 2013

Focus on emergent, multidisciplinary design issues

What can researchers tell practitioners who want to implement effective online collaboration in a virtual meeting room (Features: VoIP, shared applications, opinion poll etc.) ?

Critical success factors:

a) Functionalities of an application (RE, SE)

b) The interface of an application (HCI)

c) Users knowledge of the interface (HCI)

d) Shared understanding of objectives (Cognitive sciences)

e) Practice of online collaboration (Social sciences)

f) Institutional backup (Organisational sciences)

g) ...

C.Voigt - MASI 2013

Artefact centred: Between Edison and Pasteur

Quest for fundamental understanding?

Considerations of use?

Research is inspired by:

Stokes, D. E. (1997). Pasteur's quadrant: Basic science and technological innovation. Washington Brookings Institution Press (available online).

(Prescriptive)

(Descriptive)

Design needs to develop generalisable knowledge.

C.Voigt - MASI 2013

DR: Interventionist, Iterative and Contextual

• Improvements of technology (any form of practical implementations)

C.Voigt - MASI 2013

Vijay Vaishnavi or Bill Kuechler (2006)

Negative Design Knowledge

Epistemic Cultures: how the sciences make knowledge By K. (Karin) Knorr-Cetina

C.Voigt - MASI 2013

3rd Question: What is design?

Motivation: • Design can be understood as a process or as product.

Question: • Does the definition of design matter?

Answer: • Different definitions suggest different means of design.

C.Voigt - MASI 2013

Design Definitions

• A design is a plan to transform a given situation into a desired one (Simon 1996).

– a procedure

• A design is the fixation of knowledge (Perkins 1986). – a knife (shape, sawing function etc)

C.Voigt - MASI 2013

Design Implications

Situated Design – Context Dependency (Winograd 1987)

Embodied Design – Proactive Users, Multiple Meaning Designs

(Dourish 2004)

Shared Design – The usage of a design is negotiated(Hutchinson 1995)

None of the above design approaches thinks of design as an isolated artifact.

C.Voigt - MASI 2013

References

• Dourish, P. (2004). Where the action is : the foundations of embodied interaction. Cambridge, Mass.: MIT Press.

• Hutchins, E. (1995). Cognition in the wild. Cambridge, Mass.: MIT Press.• Knorr-Cetina, K. (1999). Epistemic cultures : how the sciences make

knowledge. Cambridge, Mass.: Harvard University Press.• Perkins, D. N. (1986). Knowledge as design. Hillsdale, N.J.: L. Erlbaum

Associates.• Sarantakos, S. (1998). Social Research (2nd ed.): Macmillan Education

Australia.• Simon, H. A. (1996). The sciences of the artificial (Third edition ed.).

Cambridge, Mass.: MIT Press.• Stokes, D. E. (1997). Pasteur's quadrant: Basic sience and technological

innovation. Washington: Brookings Institution Press.• Voigt, C. & Swatman, P. M. C. (2006). Learning through Interaction:

improving practice with design-based research. International Journal of Interactive Technology and Smart Education, 3(3), 207-224.

• Winograd, T. & Flores, C. F. (1987). Understanding computers and cognition : a new foundation for design. Reading, Mass. Sydney: Addison-Wesley.

C.Voigt - MASI 2013