015-0135
Critical Thinking and its Role in Effective Problem Solving
Dr Pauline Found, Cardiff University, Lean Enterprise Research Centre, Cardiff, CF24 4AY, United Kingdom. Tel: +44(0)29 2064 7022. Email: [email protected]
Lyndon Hughes, ConvaTec Limited, First Avenue, Deeside Industrial Park, Deeside, Flintshire, CH5 2NU, United Kingdom. Tel: +44(0)1244 584017. Email: [email protected]
POMS 21st Annual Conference
Vancouver, Canada
May 7 to May 10, 2010
Abstract
Many organisations striving to become a Lean Enterprise have set their end goal as achieving
perfection. Whilst perceived as unattainable; this fifth and final principle of Lean ensures a
continual drive to improve the way the organisation does business. These organizations
understand that to achieve perfection they need to have a culture of identifying and resolving
problems so they do not re-occur. In the authors’ experience organizations faltering in their
Lean transformation often cite a failure to capitalise on the benefits of problem solving as one
of the primary reasons. It can be argued that there are three key elements to effective problem
solving, one of which is critical thinking and the others motivation and knowledge; this paper
explores the interface between these.
Introduction
In striving to become a Lean Enterprise many organizations pursue the fifth and final
principle of Lean which, according to Womack and Jones (2003) is perfection, thus ensuring
a continual drive to improve the way the organization does business. These organizations
understand that to achieve perfection they need to have a culture of actively seeking and
resolving problems so they do not re-occur, Wilmott (unpub) refers to this as applying the
‘100 year fix’. The Toyota motor company is possibly the most well known for applying this
philosophy, however in their book, The Toyota Way Field Book, Liker and Meier (2006)
explain that “calling the process ‘problem solving’ may be a misnomer, since the process
goes well beyond the basics of solving problems”. They explain how the method
encompasses critical and logical thinking processes.
The ability to solve problems is seen as a key driver in a lean enterprise; in this context
attention is turned to critical thinking within a manufacturing environment. Thinking
critically is not a new phenomenon, 2500 years ago Socrates began to develop the principles
of critical thinking when he “established the importance of seeking evidence, closely
examining reasoning and assumptions, analyzing basic concepts, and tracing out
implications not only of what is said but of what is done as well. His method of questioning
is now known as ‘Socratic Questioning’ and is the best known critical thinking strategy. In
his mode of questioning, Socrates highlighted the need in thinking for clarity and logical
consistency” (Paul, 2009a). Critical thinking helps foster a healthy democracy, it is part of
what it means to be a developing person and without it our work places would remain
organized as they were 20 years ago (Brookfield, 1987). The need to develop critical
thinking skills within education has long been agreed, but it has also long been debated and
consensus has not yet been achieved on how to define or assess a student’s ability, which in
some instances has lead to inaction. This inaction results in the development of students who
are ill equipped for today’s complex, fast paced, information laden environment. Sampson et
al., (2007) discuss the findings of the 1991 Departments of Labour report ‘what work requires
of school’ in which they note that “critical thinking skills were reported to be a fundamental
requirement for competing in the contemporary global environment”. There would be a
resounding ‘yes’ from industry leaders to the question, “do first line supervisors in a
manufacturing environment also need these skills?” The likelihood is however that this
demographic of the organization have not had exposure to the concept of critical thinking nor
the encouragement to develop these skills. This is despite the fact that they have to contend
with an ever increasing pace of modern day manufacturing and an almost exponential
increase in the complexity, availability and quantity of information. As a result, the need for
front line manufacturing supervisors to posses the traits and skills of a critical thinker is
paramount if they are to continuously improve their environments. These individuals must be
capable of interpreting the reams of information before them and by applying purposeful and
reflective judgment determine the meaning and significance of what has been observed,
expressed or inferred. They must determine whether there is adequate justification to accept a
conclusion as true and decide on the appropriate course of action.
Whilst business leaders acknowledge the need for this ability, within a manufacturing
supervisory environment the term critical thinking is practically an unknown phrase. It is the
opinion of this researcher that critical thinking has been replaced with the term problem
solving; prior to this research a ‘straw pole’ within this researcher’s organisation drew many
blank faces when supervisors were asked their understanding of critical thinking. This lack of
recognition of terminology does not restrict itself to the shop floor; literature too fails to
adequately cover the concept within a manufacturing environment and focuses almost
exclusively on pedagogy.
Relationship to Existing Literature
In reviewing the literature on critical thinking one thing is apparent at a very early stage; this
is the lack of a single definitive definition of what critical thinking actually is. Each author,
philosopher, psychologist and professor has a slightly different opinion as to the meaning of
critical thinking, what is included within its scope and even whether it should be given the
title critical thinking. With this lack of agreement on the definition of critical thinking comes
an opportunity for confusion in what is a difficult and in-depth subject. This thought is
captured by Boychuk-Duchscher (1999) who believes that “existing literature is confusing,
in it’s description of the process and is ambiguous in drawing relationships between
critical thinking and the language used to illustrate the process…”
Despite academia failing to agree on a single definition of critical thinking and the skills
required to achieve this higher order thinking, there is sufficient agreement within both
business and educational leaders that there is a need to develop the abilities to apply sound
thinking in everyday lives. The benefits to industry and society as a result of critical thinking
are evident and ‘vital’ if we are to continue to develop both culturally and technologically.
This said, and despite the fact that the skills and attributes required to become a critical
thinking person are well documented, there is a definite gap in the development of these skills
within industry. There is little evidence that a critically thinking disposition is either
appreciated or recognized, and in some organisations is feared (Paul and Elder, 2002). This
lack of acknowledgement and industries desire to apply tools and techniques rather than
philosophies (lean implementation is a classic example), it appears that critical thinking has
been somewhat replaced by problem solving. As a result industry teach ‘toolbox’ skills such
as Ishikawa or Process Mapping to the masses with minimal effort required on coaching and
mentoring rather than nurturing critical thinking. This lack of mentoring has resulted in
organisations applying tools sporadically which often results in problems being solved in an
uncritical manner.
Literature is also split on the generalizability of critical thinking and whether or not subject
specific knowledge is required in order to apply critical thinking. Whilst acknowledging in
some instances critical thinking may occur if there is no former knowledge, within a
manufacturing environment some level of knowledge is an essential part in order to apply
sound critical thinking. For this subject specific knowledge to be of benefit organisations
must foster an environment that supports critical thinking and provide the right motivational
drivers for a person to learn and apply the skills of critical thinking. These skills can be
audited through the application of assessments such as the array of Insight Assessment
critical thinking assessments or Watson Glaser’s critical thinking appraisal test, although the
later should be escorted with a caveat as it does not assess a person disposition to use these
skills. From the literature reviewed there is a definite absence of the terminology of critical
thinking including the title itself and it is unclear if this has been purposefully or naturally
transitioned. It is however clear that within industry critical thinking has been replaced with
problem solving. From the literature reviewed this researcher believes there are three key
elements to application of problem solving these are Critical Thinking, Motivation and
Knowledge and therefore it is the objective of this paper to understand how these three
manifest themselves in first line supervisors within a medical devices manufacturer.
Based on the literature reviewed it is apparent that critical thinking at the coal face of
manufacturing has manifested itself into problem solving and from this it can be argued that
there are three key elements to effective problem solving, one of which is critical thinking the
other two are motivation and knowledge. Therefore it can be hypothesized that the following
model can be used to understand an individual’s ability to effectively problem solve by
empirically testing each of the individual components of the hypothesis within a medical
device manufacturing environment.
Critical Thinking
MotivationKnowledge
ProblemSolving
Without critical thinking skills sound thought cannot be applied
to effectively solve problems.
Without knowledge of the situation or subject it is not possible to effectively solve
problems
Unless a person is motivated by the process of solving problems they will not effectively do so.
Figure 1: Three Key Ingredients to Effective Problem Solving.
Research Methods
The purpose of this study was to empirically test first line supervision within a medical
device manufacturer to challenge the hypothesis that effective problem solving is a
combination of sound critical thinking, knowledge and the correct motivational traits (Fig 1).
This hypothesis was tested by measuring and comparing the abilities of first line
manufacturing supervisors to apply the concept of critical thinking and their dispositions
towards the appropriate business attitudes along with their motivational drivers and their
knowledge, both tacit and explicit.
The need to challenge the multiple facets of the hypothesis requires qualitative and
quantitative data to be collated and analyzed. This requirement therefore calls for a mixed
method choice (Saunders et al., 2007) and as this research is time constrained the time
horizon will be Cross Sectional in that it is a ‘snap shot’ of reality at a particular moment in
time rather than a longitudinal process taken over a longer time frame which is outside the
capability of this research period.
This study represents an embedded case study, focusing on the abilities of manufacturing
based supervisors within the researcher’s organization and therefore the sample selection is
100 percent of the population of supervisors. There are 12 supervisors, eight of which are
responsible for manufacturing activities and four engineering. The supervisors predominantly
work a 37 hour, 5 day working week rotating between shifts starting at 6am or 2pm each
week, there are 2 who are exceptions to this rule and work a permanent 37 hour night shift.
As a result if the identification of the need to conduct a mixed method process for data
collection the following instruments were used.
1. SHL Motivation Questionnaire (MQ.M5)
2. Critical Thinking Assessment
o Business Critical Thinking Skills Test (BCTST)
o Business Attitude Inventory (BAI)
3. Knowledge Assessment
o Tacit Knowledge Inventory for Managers (TKIM)
o Explicit Knowledge Assessment (Personal)
o Explicit Knowledge Assessment (Line Manager)
4. Problem Solving Assessment
Critical Thinking
MotivationKnowledge
ProblemSolving
Business Critical Thinking Skills Test & Business Attitude Inventory Test
TKIM Assessment & Explicit Knowledge Assessments
MQ.M5 AssessmentGridlock and Plates Assessments.
o Gridlock Activity
o Plates Activity
The above instruments will provide the data to challenge the hypothesis as shown in Figure 2..
Figure 1: An Illustration of the Assessment Instruments to Test the Hypothesis
Findings
1. Research Results and Analysis for Motivation Assessment (MQ.M5)To obtain data for this portion of the hypothesis the research participants completed the
motivation questionnaire, MQ.M5 which measures motivation against four groupings, these
are; energy and dynamism, synergy, intrinsic and extrinsic. Table 1 provides summaries of
the sten scores for each of these groups and their respective sub categories by participant.
Table1 – Sten Scores by Participant by Category
To understand the sample groups motivational drivers as a whole the average sten scores
were compared to that of the norm group. These are shown in Table 2
Sub Category S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 Mean SD
E1 Level of Activity 8 6 3 6 5 6 7 10 6 6 10 6 6.6 2.0
E2 Achievement 7 7 5 7 4 4 4 10 6 7 8 7 6.3 1.8
E3 Competition 5 6 5 7 6 8 6 10 6 6 9 7 6.8 1.5
E4 Fear of Failure 7 5 2 4 8 7 7 8 8 5 10 9 6.7 2.3
E5 Power 8 6 7 4 3 5 7 10 6 6 9 6 6.4 2.0
E6 Immersion 4 10 3 5 7 8 9 2 7 10 7 7 6.6 2.6
E7 Commercial Outlook 5 7 6 8 4 4 6 7 7 7 9 6 6.3 1.5
S1 Affiliation 6 7 7 4 5 5 5 9 5 7 6 5 5.9 1.4
S2 Recognition 4 10 10 7 6 5 5 6 3 10 6 4 6.3 2.5
S3 Personal principles 6 4 6 9 4 6 3 6 3 4 4 6 5.1 1.7
S4 Ease and Security 4 4 10 5 3 4 3 8 6 4 6 2 4.9 2.3
S5 Personal Growth 9 7 5 8 4 6 5 9 6 7 9 7 6.8 1.7
I1 Interest 8 4 5 9 4 4 5 6 4 4 6 9 5.7 2.0
I2 Flexibility 4 3 2 4 4 3 6 2 8 3 7 8 4.5 2.2
I3 Autonomy 5 5 10 5 4 1 3 7 7 5 3 8 5.3 2.5
X1 Material Reward 4 4 8 8 3 3 2 8 5 4 6 6 5.1 2.1
X2 Progression 5 3 4 10 3 4 4 10 5 3 10 5 5.5 2.8
X3 Status 4 6 8 7 2 5 3 9 5 6 5 5 5.4 2.0
Intr
insi
cEx
trin
sic
Participant ID
Ener
gy a
nd D
ynam
ism
Syne
rgy
Table 2 – Motivational Sten Scores by Participant
Motivation Optimum for Problem Solving.
To understand how the motivational scores of the sample group impact effective problem
solving, each category has been assessed by the researcher and it has been deemed
appropriate that for each of the sten scores, a score as high as possible would be appropriate
when identifying an effective problem solver. Whilst Baron et al, (2002) highlight that a high
sten score does not mean an individual is more motivated, the objective of this assessment is
to understand at what level the individuals motivational score is in comparison to that of an
Motivation Sub CategoryMean of Sample
Mean of Norm Group Difference SD
E1 Level of Activity 6.6 6.0 0.6 1.98
E2 Achievement 6.3 5.0 1.3 1.83
E3 Competition 6.8 6.0 0.8 1.54
E4 Fear of Failure 6.7 6.0 0.7 2.27
E5 Power 6.4 6.0 0.4 1.98
E6 Immersion 6.6 5.0 1.6 2.61
E7 Commercial Outlook 6.3 6.0 0.3 1.50
S1 Affiliation 5.9 6.0 -0.1 1.38
S2 Recognition 6.3 5.0 1.3 2.46
S3 Personal principles 5.1 6.0 -0.9 1.73
S4 Ease and Securtiy 4.9 5.0 -0.1 2.27
S5 Personal Growth 6.8 6.0 0.8 1.70
I1 Interest 5.7 5.0 0.7 1.97
I2 Flexibility 4.5 6.0 -1.5 2.20
I3 Autonomy 5.3 5.0 0.3 2.45
X1 Material Reward 5.1 5.0 0.1 2.11
X2 Progression 5.5 5.0 0.5 2.81
X3 Status 5.4 6.0 -0.6 1.98
Intr
insi
cEx
trin
sic
Ener
gy &
Dyn
amis
m G
roup
Syne
rgy
effective problem solver. For example, it would be unrealistic to believe that a person who is
demotivated when faced with failure would be an effective problem solver, much learning is
gained from failure and this demotivation will be a barrier to trying alternatives during the
problem solving process. Figure 3 below provides the overall scores of each of the
participants.
Figure 3 – Comparison of Sten Scores and Problem Solving (PS) Optimum
Through comparison of each individual participant’s scores to the maximum available sten
score the individuals with a potential motivational ‘fit’ to problem solving can be determined
this can be shown by reconfiguring the data into Pareto format as shown in Figure 4.
Motivation Category Sten Scores by Participant
0
20
40
60
80
100
120
140
160
Participant
Cum
ulat
ive
Sten
Sco
re
Extrinsic 13 13 20 25 8 12 9 27 15 13 21 16
Intrinsic 17 12 17 18 12 8 14 15 19 12 16 25
Synergy 29 32 38 33 22 26 21 38 23 32 31 24
Energy and Dynamism 44 47 31 41 37 42 46 57 46 47 62 48
S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12
Figure 4 –Motivational Sten Scores Against Maximum by Participant.
It can be seen that participants S8 and S11 have the highest overall sten score and are
therefore more likely to be motivated by the challenges that problem solving presents. This
however does not mean that participant S5 would not be able to solve problems, it implies
that this individual will show a lesser desire to actively seek and solve problems those
problems.
2. Critical Thinking Assessment
2.1 Research Results and Analysis for Business Critical Thinking Skills Test (BCTST)
The BCTST has been devised and provided by Insight Assessment (IA). IA are part of the
California Academic Press group and was borne from the Delphi report - Critical Thinking: A
statement of expert consensus for purposes of educational assessment and instruction
(Facione, 1990). IA’s focus is on the international advancement of critical thinking and their
pioneering work in the measurement of critical thinking skills and dispositions has developed
the BCTST assessment. The BCTST is a case-based reasoning skills assessment tool which is
specifically designed to evaluate the critical thinking skills of working professionals. The
BCTST provides an objective measure of critical thinking skills applied to business and
Motivation Sten Score - Pareto by Participant
0
20
40
60
80
100
120
140
160
180
200
S8 S11 S4 S12 S3 S2 S10 S1 S9 S7 S6 S5
Participant
Ove
rall
Sten
Sco
re
Sten Score
Max Sten Score
workplace, professional and workplace reasoning contexts. It uses mini-cases and vignettes
drawn from common business and workplace contexts. The online timed (50 minutes)
assessment comprises of 35 multiple choice test items which range in difficulty and
complexity. Questions are presented in business contexts with all specialized information
needed to respond correctly provided within the question. With business-relevant topics and,
in some cases, with data presented in images and diagrams, the BCTST provides items that
broadly represent reasoning ability required to be a skilful critical thinker in the business
professions.
BCTST requires the test taker to accurately interpret specific questions or to draw the correct
inferences from a set of assumptions with the more complex questions requiring an iteration
of these two cognitive skills.
The results from the test are presented in two ways, there is an overall critical thinking score,
this first score has been shown to be a predictor of a person’s success within a work
environment (Facione et al., 2008). The second score is individual rankings against the
critical thinking skills of analysis, evaluation, inference and deductive reasoning. All twelve
of the research participants completed this assessment.
The BCTST assessment produces six individual scores that demonstrate a person’s ability in
each of the five critical thinking skills as discussed by Facione (2007) plus a total critical
thinking score. Through comparison of these individual scores against the 2008 scale norms
for the overall sample of test takers using the BCTST (Facione et al., 2008) and the
descriptions of the category as provided by Facione et al, a judgement can be made on the
critical thinking skill levels of each test taker.
Inductive Reasoning Abilities.
The results obtained for the inductive reasoning portion of the assessment have been
summarised in Figure 5. The mean score was 9.75 with minimum and maximum scores of 5
and 13 respectively and a standard deviation of 2.99.
Figure 5 – Graphical Statistical Summary of Results for BCTST Inductive Reasoning Scores
Using the norm group for inductive reasoning and applying the results from the sample the
following percentile curve can be created from this it can be seen that the inductive ability of
the supervisors range from the 1st to the 45th percentile of the norm group. A person who
scores highly on this scale can be said to be skilled in the interpretation and evaluation of
inductive forms of reason.
Deductive Reasoning Abilities.
The deductive reasoning scores summarised in Figure 6 below provide a mean score of 5.42
with minimum and maximum scores of 1 and 13 respectively and a standard deviation of
3.12.
1815129630
Median
Mean
1211109876
1st Quartile 6.2500Median 11.00003rd Quartile 12.0000Maximum 13.0000
7.8511 11.6489
6.2631 12.0000
2.1171 5.0743
A-Squared 0.72P-Value 0.043Mean 9.7500StDev 2.9886Variance 8.9318Skewness -0.51953Kurtosis -1.53079N 12Minimum 5.0000
Anderson-Darling Normality Test
95% Confidence Interval for Mean
95% Confidence I nterval for Median
95% Confidence I nterval for StDev95% Confidence I ntervals
Summary for Induction (out of 20)
Figure 6 – Histogram of Results for BCTST Deductive Reasoning Scores
Using the norm group for deductive reasoning and applying the results from the sample group
from this it can be seen that the ability of the supervisors to apply deductive reasoning to test
the validity of claims and arguments ranges from the 2nd to the 90th percentile of the norm
group. A person who scores high in this norm group can be expected to show confidence in
their ability to logically determine conclusions based upon arguments where the premises are
known to be or must be taken as true.
Analysis Abilities.
The analysis scores summarised in Figure 7 below provide a mean score of 5.83 with
minimum and maximum scores of 3 and 9 respectively and a standard deviation of 2.25.
14121086420
Median
Mean
876543
1st Quartile 3.0000Median 5.00003rd Quartile 7.7500Maximum 11.0000
3.4358 7.3975
3.0000 7.7369
2.2085 5.2934
A-Squared 0.22P-Value 0.798Mean 5.4167StDev 3.1176Variance 9.7197Skewness 0.484158Kurtosis -0.608819N 12Minimum 1.0000
Anderson-Darling Normality Test
95% Confidence I nterval for Mean
95% Confidence Interval for Median
95% Confidence Interval for StDev95% Confidence Intervals
Summary for Deduction (out of 15)
Figure 7 – Histogram of Results for BCTST Deductive Reasoning Scores.
The analysis scale measures the skills of the individual to comprehend and express the
meaning and significance of a wide variety of situations, experiences, data, events,
judgements, beliefs, rules or criteria.
Inference Abilities.
The analysis scores summarised in Figure 8 below provide a mean score of 5.42 with
minimum and maximum scores of 3 and 10 respectively and a standard deviation of 2.27.
9876543
Median
Mean
87654
1st Quartile 4.0000Median 5.00003rd Quartile 8.0000Maximum 9.0000
4.4040 7.2626
4.0000 8.0000
1.5936 3.8195
A-Squared 0.53P-Value 0.138Mean 5.8333StDev 2.2496Variance 5.0606Skewness 0.24808Kurtosis -1.57232N 12Minimum 3.0000
Anderson-Darling Normality Test
95% Confidence Interval for Mean
95% Confidence I nterval for Median
95% Confidence I nterval for StDev95% Confidence I ntervals
Summary for Analysis (Out of 10)
14121086420
Median
Mean
76543
1st Quartile 3.2500Median 5.00003rd Quartile 6.7500Maximum 10.0000
3.9714 6.8619
3.2631 6.7369
1.6114 3.8622
A-Squared 0.58P-Value 0.106Mean 5.4167StDev 2.2747Variance 5.1742Skewness 0.936009Kurtosis 0.198698N 12Minimum 3.0000
Anderson-Darling Normality Test
95% Confidence I nterval for Mean
95% Confidence I nterval for Median
95% Confidence I nterval for StDev95% Confidence I ntervals
Summary for Inference (Out of 15)
Figure 8– Histogram of Results for BCTST Inference Scores
Using the norm group percentiles for inference, which is the ability to identify and consider
relevant information to draw reasonable conclusions and hypotheses the results show that the
supervisory sample group ranged from the 10th to the 87th percentile of the norm group.
Evaluation Abilities.
The evaluation scores summarised in Figure 9 below provide a mean score of 3.92 with
minimum and maximum scores of 1 and 6 respectively and a standard deviation of 1.62.
Figure 9 – Histogram of Results for BCTST Evaluation Scores
Using the norm group for evaluation and applying the results from the sample the following
percentile curve can be created, this indicates that the supervisors ability to evaluate
information to aid their decision making process ranges from the 1st to the 65th percentile of
the norm group.
14121086420
Median
Mean
5.04.54.03.53.02.52.0
1st Quartile 2.2500Median 4.00003rd Quartile 5.0000Maximum 6.0000
2.8865 4.9468
2.2631 5.0000
1.1486 2.7529
A-Squared 0.38P-Value 0.342Mean 3.9167StDev 1.6214Variance 2.6288Skewness -0.454667Kurtosis -0.798299N 12Minimum 1.0000
Anderson-Darling Normality Test
95% Confidence Interval for Mean
95% Confidence I nterval for Median
95% Confidence I nterval for StDev95% Confidence I ntervals
Summary for Evaluation (Out of 15)
BCTST Total Scores.
The total score is the best measure of critical thinking skills and is ideal to compare
individuals and identify those that think at a higher level. The summary in Figure 10 below
shows a mean score of 15.17 with minimum and maximum scores of 8 and 23 respectively
and a standard deviation of 5.31.
Figure 10 – Histogram of Results for BCTST Total Scores
3024181260
Median
Mean
201816141210
1st Quartile 9.250Median 15.5003rd Quartile 20.250Maximum 23.000
11.796 18.538
9.263 20.211
3.759 9.009
A-Squared 0.33P-Value 0.460Mean 15.167StDev 5.306Variance 28.152Skewness 0.03733Kurtosis -1.36415N 12Minimum 8.000
Anderson-Darling Normality Test
95% Confidence I nterval for Mean
95% Confidence Interval for Median
95% Confidence Interval for StDev95% Confidence I ntervals
Summary for Total (out of 35)
Facione et al, provide recommended groupings of score ranges in which they suggest provide
an indication of ‘like minded’ individuals. This is shown in Figure 11.
Figure 11 – Distribution of Total BCTST Results as Recommended by Facione et al.
Using the national scale norm as a comparison, Figure 12 shows the percentile results ranging
from the 1st to the 75th percentile.
% Distribution of Total BCTST Scores
0%
0%
25%
8%
25%
17%
25%
0%
0%
0%
0-2
3-6
7-9
10-13
14-16
17-20
21-23
24-27
28-30
31-34
Scor
e Ra
nge
of P
artic
ipan
t
% of Test Takers 0%0%25%8%25%17%25%0%0%0%
0-23-67-910-1314-1617-2021-2324-2728-3031-34
Figure 12– Percentile Curve for BCTST Total Scores
Table 3 below shows the percentage scores for each of the participants with the overall
percentage based on a potential score of 110.
Table 3 – BCTST Assessment Score by Participants
Percentile Curve for Total Score Using 2008 BCTST National Scale Norm Sample Group
0
10
20
30
40
50
60
70
80
90
100
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34
Total Score
Perc
entil
e
0
1
2
3
4
5
6
7
8
9
10
Num
ber o
f Sam
ples
per
Pe
rcen
tile
Poin
t
Norm Group Percentile
Sample Percentiles
Number of Samples at Percentile Point
Participant ID
Induction (out of 20)
Deduction (out of 15)
Analysis (Out of 10)
Inference (Out of 15)
Evaluation (Out of 15)
Total (out of 35)
S1 60% 27% 50% 33% 40% 46%
S2 30% 20% 40% 20% 13% 26%
S3 55% 47% 70% 40% 33% 51%
S4 60% 67% 80% 60% 33% 63%
S5 65% 13% 50% 27% 40% 43%
S6 25% 33% 40% 20% 20% 29%
S7 30% 20% 30% 33% 7% 26%
S8 45% 33% 50% 33% 27% 40%
S9 35% 7% 30% 20% 13% 23%
S10 65% 53% 90% 47% 33% 60%
S11 60% 73% 90% 67% 27% 66%
S12 55% 40% 80% 33% 27% 49%Average of
group 49% 36% 58% 36% 26% 43%
Considering the statement by Facione et al (2008), that total critical thinking scores can be
predictors of success within the workplace, it can be assumed that in the assessment of
whether the sample group have the ability to effectively problem solve, supervisors S11, S4
and S10 should have a distinct advantage over the other candidates assuming the research
hypothesis is valid. This will be discussed later in this paper.
2.2 Research Results and Analysis for Business Attitude Inventory Assessment (BAI)
The Business Attitude Inventory is part of the suite of tests offered by IA and focuses on the
attitudes and dispositions toward using thinking. This tool focuses on an array of attitudes
and values that influence a person's capacity to learn and to effectively apply critical thinking
skills. Critical thinking disposition and skills go hand in hand: the "'willing and able"
(Facione, 2009) of human reasoning. As with the BCTST, BAI is an online assessment,
however the questions are not timed nor are they multiple-choice. The responses for this
assessment are scored against the desired response for the question posed.
The BAI assessment produces a score for an individual’s critical thinking style plus eight
other attributes required for key employees. These additional eight are dependability,
commitment, honesty, desire to work, willingness to learn, flexibility, sociability and
tolerance. According to Facione et al, (2008) there are three types of trait for each of the BAI
attitudes. Positive traits demonstrate that individuals have a desirable tendency towards the
particular attitude. Negative traits mean hostility is shown towards the attitude and
ambivalence suggests an inconsistency in their expression towards the attitude. Table 4
(below) shows the scores for each participant against the individual attributes and also the
average of the total scores. Scores between 30 and 40 are deemed as positive, between 21 and
29 are ambivalent and between 10 and 20 are hostile towards the attribute.
Table 4: Individual BAI Assessment Results
Of the 12 participants, none exhibited negativity towards any of the business attitudes or the
critical thinking style however ambivalence was the overriding trait with only willingness to
learn demonstrating a strong positive attitude from the group with 92%. Whilst this data is of
concern there is an opportunity that must be capitalised upon as they are all open to learning,
and to quote Thomas R. Dewar, “minds are like parachutes, they only work when they are
open”.
3. Research Results and Analysis for Knowledge Assessment
3.1 Tacit Knowledge Assessment Results.
There are two forms of knowledge that are focussed upon in the field of knowledge
management, tacit and explicit (Sanchez, 2004). Table 5 below demonstrates the differences
between these two forms of knowledge and how they are managed.
S1 30.91 29.09 30.91 23.64 34.00 27.50 34.55 30.00 31.67 30.25
S2 30.00 28.18 27.27 25.45 36.00 25.83 30.00 26.36 30.00 28.79
S3 29.09 30.00 31.82 24.55 32.00 29.17 27.27 30.91 30.83 29.52
S4 30.00 25.45 26.36 31.82 32.00 27.50 30.91 24.55 25.00 28.18
S5 29.09 24.55 27.27 26.36 29.00 24.17 30.00 26.36 26.67 27.05
S6 28.18 32.73 24.55 28.18 33.00 26.67 30.00 28.18 28.33 28.87
S7 28.18 32.73 26.36 29.09 34.00 29.17 31.82 33.64 30.83 30.65
S8 36.36 30.00 30.91 30.00 35.00 31.67 33.64 30.91 28.33 31.87
S9 30.91 28.18 25.45 32.73 32.00 24.17 28.18 23.64 25.00 27.81
S10 29.09 28.18 29.09 26.36 33.00 29.17 30.00 30.00 32.50 29.71
S11 30.91 30.91 30.91 26.36 35.00 30.00 33.64 29.09 32.50 31.04
S12 29.09 26.36 26.36 25.45 34.00 27.50 28.18 31.82 30.00 28.75Average Score 30.15 28.86 28.11 27.50 33.25 27.71 30.68 28.79 29.31
Tacit Knowledge Explicit KnowledgeKnowledge is personal in nature and very difficult to extract from people.Knowledge must be transferred by moving people within or between organisations.Learning time must be encouraged by bringing the right people together under the right circumstances.
Knowledge can be articulated and codified to create explicit knowledge assets.Knowledge can be disseminated (using information technologies) in the form of document drawings, best practices etc.Learning can be designed to remedy knowledge deficiencies through structured, managed, scientific processes.
Table 5.: Source: Adapted from (Sanchez, 2004)
To measure these differing forms of knowledge within the sample group, two knowledge
assessments were administered.
Tacit Knowledge
HEC School of Management, Paris, conducted a tacit management project to benchmark the
management and selling skills characteristics of high mobility managers, engineers, and
salespeople (Segalla et al., 2009), they believe that if managers have a better level of tacit
knowledge about how to get things done then their companies will prosper. An outcome of
this research project is the availability of an on-line tacit knowledge assessment tool to allow
individuals to benchmark themselves against the sample used in the research. The Tacit
Knowledge for Managers’ (TKIM) assessment has been adapted from the original version
created by Sternberg and Wagner at Yale University.
The TKIM Project enables respondents to be compared to an expert panel of over 70
European C-level executives selected by Boyden, a leading executive search consultancy, and
over 2,000 managers and business students from around the world.
All twelve of the research participants completed this assessment.
Table 6 – Summary of TKIM scores
For this particular measure the total TKIM score is the indicator of a persons tacit knowledge,
Figure 13 provides a Pareto of the total TKIM scores by participant, the Pareto is grouped in
ascending order as the lower the total TKIM score the higher the level of tacit knowledge.
Figure 13 – Pareto of TKIM Total Scores for Each Participant
Participant Managing
Yourself ScoreManaging
Others ScoreManaging Tasks Score
Total TKIM Score
S1 27 40 31 98S2 45 52 60 157S3 28 35 26 89S4 33 52 29 114S5 34 31 29 94S6 46 47 48 141S7 55 57 72 184S8 39 66 67 172S9 48 51 40 139S10 39 39 47 125S11 62 68 71 201S12 50 41 38 129
Minimum 27.0 31.0 26.0 89.0Mean 42.2 48.3 46.5 136.9
Median 42.0 49.0 43.5 134.0Maximum 62.0 68.0 72.0 201.0
SD 10.8 11.7 17.2 36.1n 12.0 12.0 12.0 12.0
Total TKIM Pareto
0
50
100
150
200
250
S3 S5 S1 S4 S10 S12 S9 S6 S2 S8 S7 S11
Participant
Tota
l TK
IM S
core
Total TKIM ScoreAverage TKIM Score
The lower the TKIM score the higher the level of tacit knowledge.
The results for the TKIM assessment show a wide ranging level of tacit knowledge; this has
resulted in a standard deviation (SD) for the total score of 36.1. Participants S3, S5 and S1
show the strongest performance whilst candidates S7 and S11 showed very little tacit
knowledge ability.
Explicit Knowledge Assessment Results.
Understanding the required skills and knowledge within an organisation is an essential part of
knowledge management and a key factor to challenging the hypothesis of this research,
observing knowledge however is not possible (Hunt, 2003). Hunt explains the need for a
form of test in order to assess a person’s knowledge, and to this the sample group were asked
to complete a knowledge self assessment questionnaire. The questionnaire was developed
from discussions with the sample groups, their line leaders as well as department heads. Hunt
also explains that measurement is difficult as often people are unaware of what they should
know and therefore cannot state their true knowledge. Likewise they may well have been
misinformed, this results in their belief of what the ‘knowledge’ is being incorrect In an
attempt to overcome this each participant’s line manager were also asked to complete the
questionnaire to enable a comparison between perceptions of knowledge levels.
All twelve of the research participants completed this assessment.
A summary of how the individuals scored their own individual levels of explicit knowledge
and also how their line managers scored them against the same criteria is shown in Figure 14
Table 7 below summarises the overall individual and line manager’s responses for each
category. Presenting this data further into each individual participant and their line manager’s
responses, Figure 41 shows how there is a definite shift in the scoring. In every instance the
individual’s perceived personal knowledge was greater than their line manager’s perception
of that person’s knowledge.
Table 7 – Explicit Knowledge Category Summary
Figure 14– Explicit Knowledge Results – Individual and Line Manager Comparison
Category Scores Min ScoreAverage
ScoreMax
Score Min ScoreAverage
ScoreMax
Score
Role Specific 6.2 8.3 10.0 5.0 6.6 9.0
Site EHS 7.3 8.9 10.0 4.8 7.4 10.0
Product Knowledge 6.0 8.0 9.3 4.3 6.8 9.0
Quality, Regulatory 6.9 8.2 10.0 4.5 6.1 7.1
Accounting 1.0 4.3 9.0 1.0 4.3 7.0
Project management 6.3 7.9 10.0 3.7 6.7 10.0
Use of data 4.3 7.6 10.0 3.3 6.8 10.0
Information technology 4.0 8.1 10.0 3.7 6.6 8.3
Customer Service 6.7 8.3 10.0 5.5 6.4 7.7
Leadership and Management 7.6 8.5 10.0 4.1 5.9 7.0
Overal Score 1.0 7.8 10.0 1.0 6.4 10.0
Individual Scores Line Manager Scores
Explicit Knowledge Score Comparison - Individual and Line Manager Assessment
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12
Participant
Expl
icit
Kno
wle
dge
Scor
e
Individual AssessmentLine Manager Assessment
Despite the difference in scores there is a high level of correlation (0.7) between those of the
individual and those of the line manager. Therefore for the purpose of this research and to
compare explicit knowledge levels of each individual it can be assumed that the mean of the
two scores can be used as the explicit knowledge value. Therefore Table 8 presents these
scores for each participant, it is important to note that participant S3 is relatively new to the
role and therefore their perceived knowledge and the knowledge of their line manager are
lower than maybe expected.
Table 8 – Explicit Knowledge Comparison and Final Adjusted Scores.
A summary of the explicit knowledge levels is shown in Figure 15. The mean explicit
knowledge score is 70.78 and the standard deviation of the sample group is 7.5.
Participant ID Individual ScoreLine Managers
ScoresFinal Explicit
Score
S1 88.2 72.3 80.3
S2 67.6 56.6 62.1
S3 66.3 51.3 58.8
S4 72.9 71.0 72.0
S5 74.3 58.8 66.6
S6 74.5 65.1 69.8
S7 80.9 70.6 75.8
S8 90.9 74.6 82.8
S9 77.9 70.6 74.3
S10 88.8 65.5 77.2
S11 75.9 57.0 66.5
S12 76.9 50.2 63.6
Figure 15 – Summary of Adjusted Explicit Knowledge Scores.
Overall Knowledge Assessment Results
To establish the overall knowledge ranking of each participant the two scores were combined.
Figure 16 – Pareto of Overall Knowledge Scores.
858075706560
Median
Mean
77.575.072.570.067.565.0
1st Quartile 64.275Median 70.8753rd Quartile 76.800Maximum 82.750
66.008 75.551
64.313 76.782
5.320 12.751
A-Squared 0.15P-Value 0.948Mean 70.779StDev 7.510Variance 56.399Skewness 0.02097Kurtosis -1.02851N 12Minimum 58.800
Anderson-Darling Normality Test
95% Confidence Interval for Mean
95% Confidence Interval for Median
95% Confidence Interval for StDev95% Confidence I ntervals
Summary for Final Explicit Score
Overall Knowledge Score by Participant
0.0
50.0
100.0
150.0
200.0
250.0
S1 S5 S3 S4 S10 S9 S12 S6 S8 S2 S7 S11
Paticipant
Know
ledg
e Sc
ore
Lower scores are desirable for this assessment
4. Research Results and Analysis for Problem Solving Assessment
To establish the problem solving ability, measurement was conducted with the application of
two practical activities. The resulting scores provide a ranking in terms of problem solving
ability
The first of the problem solving activities is known as Gridlock, this challenge is usually used
in a team context in which the team is observed in their application of thought, knowledge,
collaboration and leadership whilst they create the image presented on the instruction sheet.
This activity has been adapted for the purpose of this research in that it is an individual
activity and through timing and observation the researcher can assess how well the
individuals apply themselves to the issue.
The second activity, developed as part of the apprentice and technical operator recruitment
process at the case study organization, known as Plates, assesses how individuals interpret
instruction and conduct themselves to complete a task within a given timescale.
4.1 Gridlock Activity Results
All 12 sample members attempted the activity, of which only two completed it within the
time allocation of 40 minutes. Within the group of the ten who failed to complete only one
did not utilise their full time allocation and ‘gave up’ relatively quickly once they had
reached a point of ‘impasse’ despite having 22 minutes remaining.
During the observations, two distinct categories of person became apparent in the ‘did not
finish’ group. There were those who were unable to progress the activity (6 participants) and
were multiple planks from completion and those who managed to assemble the planks (4
participants) with the exception of the final plank which was unable to be fitted, they then
utilised their remaining time to attempt to resolve this error. The error is as a result of the
orientation of plank 13 when initially assembled. Of the two individuals that completed the
activity they too witnessed this final plank issue but managed to find a resolution relatively
quickly.
Of those who completed the activity the average completion time was 19 minutes. And of
those who failed to complete the activity but had reached a point where this was to just the
last plank the average time was 21 minutes.
The performance of each of the individuals is shown in Table 9, a Pareto of the total scores
are shown in Figure 17.
Table 9 – Gridlock Activity Performance Scores and timings by Participant
Participant No.
Time Taken
(Minutes)Completed
Task
Assembled Except One
Piece.Time to Just
1 PieceAssessment
Mark
Bonus for Getting to Final Piece
Penalty for Not using
Time Allowed
Bonus for Completing (1 point per min under
time) ScoreS1 40 N Y 36 25 5 30S2 40 N 9 9S3 40 N 27 5 32S4 16 Y 14 23 5 24 52S5 40 N 14 14S6 40 N 16 16S7 40 N 24 24S8 28 - DNF N Y 14 22 5 -3 24S9 40 N 21 21S10 22 Y 14 26 5 18 49S11 40 N Y 18 23 5 28S12 40 N Y 18 22 5 27
Minimum 16 9 9.0Mean 36.2 21 27.2
Maximum 40 27 52.0
Figure 17– Gridlock Activity Performance Score by Participant (Pareto)
In considering Figure 17, participants S4 and S10 stand out as high performing in this activity
and this is rightly so as they were the only two individuals to complete the activity. However,
if the breakdown of the scores is analysed further, this shows that the successful completion
has a large influence on the scores. Figure 18 shows the overall data but with the added
detail of the score allocations, this Pareto has been sorted on the assessment and bonus for
getting to the final piece. Here it can be seen that participants S4 and S10 are no longer
leading the field and the impact of the ability to resolve the issue of the incorrect final piece
becomes much more apparent. S4 and S10 however both resolved the issue of the final piece
in which the others failed and therefore the overall scores, including the finish time bonus
should remain the measure.
Pareto of Total Score for Gridlock Activity
0
10
20
30
40
50
60
S4 S10 S3 S1 S11 S12 S7 S8 S9 S6 S5 S2
Participant
Poin
ts A
war
ded
Score
Figure 18 – Gridlock Activity Performance Score breakdown.
4.2 Plates Activity Results
Of the 12 participants to attempt this activity three failed to complete it, S1 took the full 20
minute allocation, he had assembled plate 2 incorrectly and could not resolve this.
Participants S6 and S2 stopped their test at 16 and 18 minutes respectively as a result of being
unable to follow the instructions or understand the task. Of those who completed the activity
the average completion time was 14 minutes. The performance of each of the individuals is
shown in Table 10, a Pareto of the total scores are shown in Figure 19.
Breakdown of Points Awarded to Participants - Gridlock Activity
-10
-5
0
5
10
15
20
25
30
35
40
45
50
55
S3 S10 S1 S4 S11 S12 S8 S7 S9 S6 S5 S2
Participant
Num
ber o
f Poi
nts
Bonus for Completing (1 point per min under time)Penalty for Not using Time AllowedBonus for Getting to Final PieceAssessment Mark
Table 10 – Plates Activity Performance Scores and Timings by Participant
Figure 19 – Plates Activity Performance Score by Participant (Pareto)
Participant No.Time Taken (Minutes)
Completed Task
Assessment Mark
Bonus for Completing (1 point per min under
time) ScoreS1 20 -DNF N 24 24S2 18 - DNF N 15 15S3 17 Y 19 3 22S4 10 Y 28 10 38S5 18 Y 21 2 23S6 16 - DNF N 10 10S7 9 Y 35 11 46S8 11 Y 31 9 40S9 15 Y 26 5 31
S10 15 Y 25 5 30S11 18 Y 22 2 24S12 14 Y 20 6 26
Minimum to Complete 9 10 2.0 10.0
Mean to Complete 14.1 23.0 5.9 27.4
Maximum to Complete 18 35 11.0 46.0
Pareto of Total Score for Plates Activity
0
10
20
30
40
50
60
70
S7 S8 S4 S9 S10 S12 S11 S3 S5 S1 S2 S6
Participant
Poin
ts A
war
ded
Score
Gridlock and Plates Activity Combined Scores
To understand the overall ability of the participants to effectively solve problems the two
scores have been combined. Figure 20 below provides a Pareto summary of each participant’s
problem solving ability. From this it can be seen that S4, S10 and S7 are the strongest overall
participants whilst S2 and S6 performed poorly in both activities. The overall correlation of
performance in the two activities is 0.43, this value falls into the region of a medium
correlation and therefore it is not conclusive that ability in the plates activity guarantees
success in the gridlock activity. It does however provide a predictor for those who may be
stronger in testing the hypothesis.
Figure 20 – Combined Problem Solving Score by Participant
Pareto of Combined Scores for Plates and Gridlock Activities by Participant
0
10
20
30
40
50
60
70
80
90
100
S4 S10 S7 S8 S1 S3 S9 S12 S11 S5 S6 S2
Participant
Com
bine
d Pr
oble
m S
olvi
ng S
core
Gridlock ScorePlates Score
Conclusions
The problem solving assessment results can be used along with the motivational assessment,
knowledge assessment and critical thinking assessment to establish a correlation. To enable
the comparison, the participant’s scores for each individual assessment have been ranked in
order of their position compared to the rest of the sample (Table 11). The overall assessment
and problem solving rankings are represented graphically in Figure 21.
Table 11 – Rankings for Each Assessment by Participant
ParticipantMotivation
RankingKnowledge
Ranking
Critcal Thinking Ranking
Overall Assessment
Ranking
Problem Solving Ranking
S1 8 1 5 3 5
S2 6 10 11 9 11
S3 5 3 4 2 5
S4 3 4 3 1 1
S5 12 2 8 8 9
S6 11 8 10 11 10
S7 10 11 9 12 3
S8 1 9 6 6 4
S9 9 6 12 10 7
S10 7 5 2 4 2
S11 2 12 1 5 8
S12 4 7 7 7 7
Figure 21 – Comparison of Participant Assessment Rankings
These rankings provide a platform to establish a correlation between the overall performance
in the assessments and that of the problem solving tasks.
Utilising the Pearson correlation calculation function within Minitab the correlation between
a participants overall ranking in the sample group for their motivation, knowledge and critical
thinking assessments and their overall ranking for the problems solving activities is:
Correlation (r) = 0.52
To understand whether this is significant, reference to the table of critical values for
Pearson’s r is required utilising the following criteria:
Test Type: One Tailed Test
Level of Significance (): 0.05
Comparison of Overall Assessment Rankings and Problem Solving Rankings
0 2 4 6 8 10 12 14
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
S11
S12
Parti
cipa
nt
Ranking
Overall Assessment Ranking Problem Solving Ranking
Degrees of Freedom (df): N-2, (N = 12) = 12-2 = 10
Therefore the critical value = 0.497
With the correlation value of 0.52, which is deemed a medium level of correlation in the
broad environment, and comparison to the critical value of 0.497 it can be seen that the
correlation between the overall ranking for the combined assessments and that of problem
solving is marginally significant.
At this point it is now appropriate to compare the individual assessments and the correlation
to the problems solving results. Table 12 below shows the correlation matrix of the
knowledge, motivation and critical thinking assessments to that of the problem solving
assessments.
Table 12 – Correlation Matrix of Individual Assessments to Problem Solving
Focusing purely on correlation and significance with problem solving and using the same
criteria as above, the correlation between critical thinking and problem solving is the only
combination of individual tasks that returns an r value (0.55) greater than that of the critical
value. However opening the comparisons wider, motivation and critical thinking also yield a
marginally significant correlation of 0.57.
Motivation Ranking
Knowledge Ranking
Critcal Thinking Ranking
Problem Solving
Overall Assessment
Ranking
Motivation Ranking 1.00
Knowledge Ranking -0.28 1.00
Critcal Thinking Ranking 0.57 0.17 1.00
Problem Solving 0.31 0.24 0.55 1.00 0.52
Knowledge20% Weighting
Motivation30% Weighting
Critical Thinking50% Weighting
ProblemSolving
By ranking each of the participants following their performance in the three individual
assessments and also in the problem solving activities, correlations were evaluated to
establish if the hypothesis discussed is valid. Through correlation of the overall assessment
rankings and the problem solving ranking a correlation value of 0.52 can be deemed as
significant when applying a confidence factor of 95%. It can therefore be concluded that the
combination of motivation, knowledge and critical thinking has a direct significance on the
ability of an individual to effectively solve problems.
Further individual correlations demonstrate that the strongest contributor to this correlation is
that of critical thinking, followed by motivation and finally knowledge. This would support
the generalizability of critical thinking and suggests these skills are generalizable. Taking this
into consideration the weighting of the model would shift and may best be represented by a
weighted Venn diagram such as:
Table 22 – The weighted model to Problem Solving
References
BAILIN, S., CASE, R., COOMBS, J. R. & DANIELS, L. B. (1999) Conceptualizing Critical Thinking. Journal of Curriculum Studies, 31,3, 285 - 302.
BARON, H., HENLEY, S., MCGIBBON, A., MCCARTHY, T. & WATT, L. (2002) Motivational Questionnaire. 2nd ed., SHL.
BOYCHUK-DUCHSCHER, J. E. (1999) Catching the wave: understanding the concept of critical thinking. Journal of Advanced Nursing, 29,3.
BROOKFIELD, S. D. (1987) Developing Critical Thinkers, Milton Keynes, Open University Press.
FACIONE, P. (2009) Insight Assessment: Measuring Critical Thinking World Wide http://www.insightassessment.com/home.html
FACIONE, P. A. (1990) Critical Thinking: A statement of expert consensus for purposes of educational assessment and instruction. Fullerton, Santa Clara University.
FACIONE, P. A. (2007) Critical Thinking: What it is and why it counts. Insight Assessment.
FACIONE, P. A., FACIONE, N. C. & GIANCARLO, C. A. F. (1997) The Motivation to Think in Working and Learning http://www.insightassessment.com/9articles2.html#FF
FACIONE, P. A., WINTERHALTER, M. A., FACIONE, N. C. & BLOHM, S. (2008) Business Critical Skills Test and Business Attitude Inventory Test Manual, Millbrae, California Academic Press.
FISHER, A. (2001) Critical Thinking - An Introduction, Cambridge, Cambridge University Press.
FRANCIS, D. (1990) Effective Problem Solving, Guilford, Routledge.INSIGHT ASSESSMENT. (2009) Insight Assessment: Measuring Critical
Thinking World Wide http://www.insightassessment.com/home.htmlLIKER, J. (2004) The Toyota Way, New York, McGraw Hill.
LIKER, J. K. & MEIER, D. (2006) The Toyota Way Field Book, New York, McGraw-Hill.
MCPECK, J. (1981) Critical Thinking and Education, New York, St Martin's Press.
MINTZBERG, H. (1979) The Structuring of Organisations, Englewood Cliffs, Prentice Hall.
MINTZBERG, H. & QUINN, J. B. (1996) The Strategy Process: Concepts, Contexts and Case Studies, New Jersey, Prentice Hall.
MOON, J. (2008) Critical Thinking : An exploration of theory and practice, Abingdon, Routledge.
MULLINS, L. J. (2005) Management and Organisational Behaviour, Pearson Education UK.
NATALE, S. & RICCI, F. (2006) Critical Thinking in Organizations. Team Performance Management, 12,7/8, 272.
PASCARELLA, P. (1997) The Secret of Turning Thinking into Action. Management Review, 86,5, 38.
PAUL, R. W. & ELDER, L. (2002) Critical Thinking: Tools for taking charge of your professional and personal life, New Jersey, Financial Times Prentice Hall.
PHILLEY, J. (2005) Critical Thinking Concepts - Applications to Incident Investigations. Professional Safety, 50,3, 6.
REED, J. H. (1998) Effect of a Model for Critical Thinking on Student Achievement in Primary Source Document Analysis and Interpretation, Argumentative Reasoning, Critical Thinking Dispositions and History Content in a Community College Course. Philosophy. Tampa, University of South Florida.
SAMPSON, D. C., MOORE, R. & JACKSON, M. J. (2007) Critical Thinking: Do They Really Have 'it' if They Don't Know What 'it' is? Allied Academies International Conference. Academy of Educational Leadership. Proceedings, 12,1, 47.
SANCHEZ, R. (2004) Tacit Knowledge versus Explicit Knowledge: Approaches to Knowledge Management Practice. Copenhagen, Copenhagen Business School.
SAUNDERS, M., LEWIS, P. & THORNHILL, A. (2003) Research Methods for Business Students 3rd edition, Harlow, Pearson Education Limited.
SAUNDERS, M., LEWIS, P. & THORNHILL, A. (2007) Research Methods for Business Students 4th Edition, Harlow, Pearson Education Limited.
SEGALLA, M., GLINOW, M. A. V., BARTON A. WEITZ, FISCHER, L., AU, K. Y.-F., PLOSZAJSKI, P., SPARROW, P. & BELL, R. (2009) Tacit Knowledge in Management and Sales http://www.tacit-knowledge.org/
SIEGEL, H. (1991) The Generalizability of Critical Thinking. Educational Philosophy and Theory, 23,1, 18-30.
SMITH, G. F. (2003) Beyond Critical Thinking and Decision Making: Teaching business students how to think. Journal of Management Education, 27,1, 24.
WEST, T. (2009) Motivation and Change. Cardiff University, Lean Operations MSc 2007-2009, Leadership and Change Module. Cardiff.
WILMOTT, P. (2008) Total Productive Maintenance. Cardiff University, Lean Operations MSc 2007-2009, Total Productive Maintenance and Layout. Barnstable.
WITTGENSTEIN, L. (1972) The Blue and Brown Books, Oxford, Basil Blackwell.
WOMACK, J. & JONES, D. (2003) Lean Thinking: Banish Waste and Create Wealth in your Corporation., London, Simon & Schuster UK Ltd.