test & evaluation = the knowledge factory
DESCRIPTION
Test & Evaluation (T&E) is presented in the prespective of knowledge creation, distribution, absorbtion and management. Possible "soft tools" are derived from the insight of the sciences of learning and knowledge management.TRANSCRIPT
1
Test & Evaluationas a
“Knowledge factory”
Outline
2
Knowledge-centered perspective on T&E
Shifting focus to the people
Some observations
Glimpse to theory and its applications
System Debugging Loop
3
Discovery
T&E, Reqs
Systems
Knowledge
ActionSE
T&E ~ Empirical Research
4
Make predictions based on theory/design
Verify/Falsify predictions by experimenting
Search for unpredicted behavior
Reporting findings formally
5
Should test engineers
be trained
as empirical scientists?
Should T&E be structured
as scientific discovery
process?
T&E ≠ Empirical ResearchIt requires:
6
Maximal coverage of phenomena
Scheduled delivery of knowledge
Immediate usefulness of knowledge
7
Test & Evaluation asa personal and group quest for knowledge
8
Project / Organization
Test Team
Test Engineer
9
Some observationson how the Test
Engineers think…
Knowledge iscreated by thinking
10
Using all available sources
Creating a mental model
In order to be able
to “tell a story” about the system
Without documentati
on
11
Need to Learn arises when
The production of required
documentsis impeded
The evidence
contradicts the mental
model
12
Test & Evaluation require effective and efficient
knowledge creation and learning
Test & Evaluation is reallya “Knowledge Factory”
13
How to leverage the science and practice of
working with knowledgeto better perform T&E?
14
What is Knowledge?
Knowledge is experience. Everything else is just information.Albert Einstein
15
Planning T&E
Plans
Designing Tests
Test Procedures
Executing Tests Test Protocols
Evaluating Test Reports
Planner Experience
Designer Experience
Tester Experience
Analyst Experience
UndocumentedTacit
Knowledge
DocumentedKnowledge
16
SECI Model (I. Nonaka)
Socialization in T&E
17
i iDevelopers on-site
Customer on-site
Testing in teams
Learn by shared
experience
Externalization in T&E
18
Talking with developers
Talking with customers/users
Meetings and discussions
Writing reports
i
i
ii
i
i
g
Learn by articulating in language
Combination in T&E
Complex Adaptive Integrator19
g
g
g
g oAnalyzing requirements specs
Analyzing design documents
Designing test plans
Designing test procedures
Learn by analysis and synthesis of information
Internalization in T&E
20
i
go Building “mental models” of the
system
“Playing” with the system simulations
Unstructured and exploratory testing
“Playing around” planned tests
Learn by exercise
and practice
“Soft” T&E Practices
21
Actively question the developers, customers and users to externalize their tacit knowledge
Encourage active participation of developers, customers and fellow testers in running tests
Take meetings and non-formal discussions seriously and use them to learn and share knowledge
22
Learning PreferencesEveryone learns better
when the learning matches
their preferences
Teachers should engage multiple “learning muscles” in their audience
Learners should direct their learning to sources that match their preferences
“Soft” T&E Practices
23
Know your learning preferences and use them while studying the SUTs
Augment reports with multimedia
Organize presentations of the results for relevant audiences
Conclusions
24
Test & Evaluation is a knowledge creating and sharing process
Test Engineers are professional learners
T&E should use the insights and tools of knowledge and learning theory and practice
Questions for further research
25
How the knowledge is created and shared by test engineers and test teams?
How do the knowledge processes integrate with testing activities?
What are the knowledge-related factors that contribute to or impede better testing?
What factors facilitate more effective and efficient learning of Test Engineers?
26
Any answers?