1 challenge the future protocol analysis & peer review instructions

1Challenge the future

Protocol analysis & peer reviewInstructions


Background information

• Embodiment design manifests as a non-predetermined sequence of activities, where individual cognition and affection (feelings) interact and feed back into the choices and activities that follow. As such, the embodiment design of embodiment design is self-regulating.

• Normally individuals work tacitly and do not produce any traces of this activity, nor are they able to observe themselves while designing (without irreparably biasing and damaging the pace and quality of their own process). In team settings, natural team communication forces tacit ideas and activities into become explicit, observable and discussible/ debatable.

• The explicit and objective audiovisual records produced by observation contain only a fragment of the actual process. The rest resides in the short-term memory of the team members.


Motivation

1. Achieving excellence in embodiment design requires excellent situational-awareness and self-awareness:a. recognising which activity

is effective at any given

situation in the design

process...

b. also in consideration of the

individual and team

competences;

c. which method is

appropriate for this

activity and whether it is

used competently;

d) how/ when the need for

information emerges and

how effective its

acquisition is;

e) how (effectively) time,

focus, motivation and

other resources are

leveraged

2. Subjective opinions should be informed by objective measurements

3. Subjectivity is evaluated by aggregating and analysing more opinions


What are the learning goals

• Learn who you are and how your mind works (by default) in a team setting to tackle embodiment design. Specifically:• how you sequence activities

• in what sequence you experience different states• Learn when and what patterns characterise team

behaviour:• understand the extent of such correlations

• hypothesise and determine causalities behind the patterns• Learn and quantify how a) advanced methods and b)

accumulating experience with advanced methods change these patterns:• Assess effectiveness and efficiency of the design process;

develop your own meaningful metrics;

• Improve in a knowledge-based way your own activities, patterns

of activities...


...and ultimately:

• Learn to scientifically self-assess and self-reflect using objective observation coupled with advanced protocol analysis methods, thereby• developing the competency to sustainably grow

henceforth by yourself, even when lacking instructors• developing the competency to help your (future) team(s)

to grow likewise• become a highly effective engineer, designer and leader


How to process a design record into a protocol (individual)

• Prepare: Open on the same computer screen in two non-overlapping windows both the pencast in Livescribe Desktop and the protocol template in Excel. Keep the latter as the active window and only leave a few rows visible to avoid visual bias from previous entries.

• Re-live: Start the pencast playback and observe what is being written/ sketched and said. Feel free to also recall spontaneously other memories/ impressions you have from that particular moment being played back.

• Classify-interpret: Fill-in the protocol entries for the particular one-minute-interval based on your most direct recollection/ interpretation of the events played back. Pause if necessary. Continue until the end of the pencast. Re-check your protocol


How to process a design record into a protocol (individual)

enter your observations in-real time during protocol playback; pause if

necessary; do a second play-through of important/ questionable passages

to double check


How to analyse embodiment design (team)

• Compare individual protocols and develop a shared understanding and interpretation of the team process. Tip: don’t be quick to conform: debate! Deviations between your individual subjective assessments are seldom the result of mistakes, they carry a lot of hidden insights- find them.

• Statistically process the protocol entries to detect correlations. Explain them. If you think the analysis data is not showing a correlation that should exist, explain that too.

• Hypothesise and explain causalities behind the correlations. What causes the (patterns of) activities that you observe?


How to analyse embodiment design (team)

• What are the ‘strengths and weaknesses’ of your final design (TDP?). How convincing and accurate (uncertainty!) is your validation report? What would you do better if you could do it again? Observe the effect of the taught methods in your design outcomes and process...

• ...and assess the effectiveness and efficiency of the design process you followed; to this end develop/ evolve your own meaningful metrics

• Suggest in a knowledge-based way improvements to your observed activities and patterns of activities: How can you learn from this performance to become better designers? How much better (quantitatively- see previous metrics) do you think you can get?


Sample protocol layoutreferences any number of non-

contradicting items, internal or external, that collectively and unambiguously

describe the final designNO NEW IDEAS CREATED HERE

references any number of non-contradicting items, internal or

external, that collectively validate the final design

NO NEW IDEAS CREATED HERE

contains all internal items (sketches, calculations, etc), referenced or not,

that emerged during the design process

contains all referenced external items (CAD drawings, FEA/ Excel

calculations, internet search results etc), that emerged during the design

process


Important information!

• When in doubt, ask one of your instructors.

1 challenge the future protocol analysis & peer review instructions

Documents