test architects at siemens - hochschule bremerhaventest architect (tea), 2016 test architect...
Post on 30-May-2020
5 Views
Preview:
TRANSCRIPT
Test Architects at SiemensPeter Zimmerer
Siemens Corporate TechnologyUnrestricted © Siemens AG 2019 Bremerhaven, GermanyFebruary 21–22, 2019
https://fg-tav.gi.de/veranstaltung/43-tav/https://informatik.hs-bremerhaven.de/seuh2019/
Unrestricted © Siemens AG 2019February 2019Page 2 Corporate TechnologyPeter Zimmerer
Agenda
Why do we need a Test Architect?
What is a Test Architect?
What are the responsibilities and tasks of a Test Architect?
How can a Test Architect provide value and create impact on the business?Examples on strategies, tactics, and practices
Unrestricted © Siemens AG 2019February 2019Page 3 Corporate TechnologyPeter Zimmerer
Systems we build today at Siemens for our customersVision 2020+New company organization will go live on April 1, 2019
Gas and Power
Smart Infrastructure
Digital Industries
Operating Companies Strategic Companies
Service Companies
Financial Services Global Business Services Real Estate Services
Mobility
Unrestricted © Siemens AG 2019February 2019Page 4 Corporate TechnologyPeter Zimmerer
Siemens Core Learning Programs and beyond
Software Architect (SWA), 2009System Architect (SyA), 2012
R&D Manager, 2012
Senior Software Architect (SSWA), 2007
Softw
are
Arch
itect
Syst
em
Arch
itect
Seni
or
Softw
are
Arch
itect
Functional Trainings
Test Architect (TeA), 2016 Test
A
rchi
tect
Requirements Engineeringfor project / product managers, 2010
Unrestricted © Siemens AG 2019February 2019Page 5 Corporate TechnologyPeter Zimmerer
Key parts of the core learning programs
Fostering best practice sharingand experience exchange
Architects’network
Assuring practical experiencefor critical projects Certification
Setting standards forsoftware and system development
Qualification(courses andon-the-job training) So
ftwar
eA
rchi
tect
Unrestricted © Siemens AG 2019February 2019Page 6 Corporate TechnologyPeter Zimmerer
Test Architect – Some external references (1)
Page, A., Johnston, K. and Rollison, Bj. 2008. How We Test at Microsoft, Chapter 2. 2008
Alan Page (Microsoft), What is a Test Architect?, 2008https://blogs.msdn.microsoft.com/alanpa/2008/05/01/what-is-a-test-architect/
John Morrison (Oracle), Test Architect, Oct 8, 2007https://blogs.oracle.com/johnmorrison/entry/test_architect
James Brewer et.al. (Raytheon), The Value of the Test Architect, March 2012http://www.dtic.mil/ndia/2012TEST/13942_Brewer.pdf
ICST InSTA workshops (International Workshop on Software Test Architecture)http://www.aster.or.jp/workshops/insta2015/
UML Testing Profile (UTP) Version 1.2. 2013. Chapter 8, “Test Architecture”, pp. 19-24http://utp.omg.org/, http://www.omg.org/spec/UTP/
Some of the links are expired!
Unrestricted © Siemens AG 2019February 2019Page 7 Corporate TechnologyPeter Zimmerer
Test Architect – Some external references (2)
ISTQB Glossary (http://www.istqb.org/)
Test architect
(1) A person who provides guidance and strategic direction for a test organization and for its relationship with other disciplines.
(2) A person who defines the way testing is structured for a given system, including topics such as test tools and test data management.
That’s ok … but there is more …
Unrestricted © Siemens AG 2019February 2019Page 8 Corporate TechnologyPeter Zimmerer
Driving triumviratefor testing & quality engineering
Test Manager
Process quality,certification and audits,regulations and norms
across lifecycles and versions
Budget, people,plans, logistics
for the test organization
Quality goals
Productivity
Process
Stra
tegy
Quality Manager
Unrestricted © Siemens AG 2019February 2019Page 9 Corporate TechnologyPeter Zimmerer
Driving triumviratefor testing & quality engineering
Test Manager Test Architect
Process quality,certification and audits,regulations and norms
across lifecycles and versions
Budget, people,plans, logistics
for the test organization
Cost efficient, accelerated, innovative, sustainable
testing solutions
for test methodology and test infrastructure
Quality goals
Productivity
Process
Stra
tegy
Quality Manager
Unrestricted © Siemens AG 2019February 2019Page 10 Corporate TechnologyPeter Zimmerer
As a Test ArchitectYou have one Role – but you are wearing two Hats
Test Expertfor the system under test (SUT)
• Design the test approach• Apply innovative test technologies• Drive the quality of the SUT
Software / System Architectfor the test system
• Design and realize the test architecture• Apply innovative software technologies• Drive the quality of the test system
This is the architect's job!
Unrestricted © Siemens AG 2019February 2019Page 11 Corporate TechnologyPeter Zimmerer
A Test Architect needs a broad set of competencies
tricttteededddddee ©©©©©©©©©©© SSSSSSSiieieieiememememensnsnsns AAAAAGGGGG 20202020201919191919February 20191 CorporaPeter Zimmerer
BusinessUnderstanding
Testingand Quality
RequirementsEngineering
Architecture andDevelopment
Social Skills& Leadership
Business modelunderstanding
Global developmentStandards, Regulations, Legal issues
Quality requirements
Product & Systemrequirements
Arch.& Design
Social skillsProject management
Development processes
TeA Test ArchitectTeM Test Manager
Lifecycle issuesDomain
knowledge
Leadership
Ecosystems,Partnering, Suppliers
Architecture Quality
Test automation
Integration & Deployment
Test & Quality management
L
Advanced
Expert
Basic
Basic knows aboutAdvanced can applyExpert drives, coaches and is able to
improve in own domain & scope
Test strategy
Lean & Agile testing
Testing Governance & Guidance
Test architecture
Test process improvement
Test methods & technologies
Design for Testability
Unrestricted © Siemens AG 2019February 2019Page 12 Corporate TechnologyPeter Zimmerer
Test Architect – Role profileResponsibilities and tasks
UnUUnUUnUUnrerereresssststttttttttttttttttttttttsssssssstssssssssss ririrriririiiririiiiiiiiririiiiirirrrriiirrrriiiirriiiirrrriiiiririrriirictcctctcttttcttctttcccttctctttccttctcctctctctctcccccttccccccccccccccttcct dedededededeeddeddddedddddededdeddddddedddddedededeeddedddddddddeeeedddedddedddedddedededdeeedddeeeedddddeeeeeeeeeddedddedddeded ©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©© SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSieieiieieieieeieiieiieiiieieiiieiieiiieieeeeiieeeeeeiiieeeeeeeeeeeeiememememmemmmemememeememmememmmmemmmmmeeememeemememmmmmemmmeeeemmmmmemmmemmeeeeeeeemmmmmmmmmmmeeemeememmmememennsnsnnnsnsnnsssnssnsssnsnnnnnnnnsnssnnnnnnnssnsnsnnnsnssssssnsnnnsnnnnssssnsnnnnnsssssssnsssssns AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG 0202020020202002020222020202220202000020202222222020200200020220222200002220202022202002020220000202020200000222222000000222200000022200000201111911111111111111111111111111111111111919
Test Architect – Role profileResponsibilities and tasks
Unrestricted © Siemens AG 2019February 2019Page 13 Corporate TechnologyPeter Zimmerer
Test Architect – Role profileResponsibilities and tasks (extract)
• Analyze the current testing process and practices and drive improvements• Evaluate and select technologies (e.g. by prototyping)• Identify and manage technical product risks• Perform make-or-buy decisions
• Ensure testability of requirements engineering artifacts• Define testability requirements• Foster the use of test-driven approaches in requirements engineering
• Manage the internal quality of the whole testing environment• Provide comprehensive documentation of the testing infrastructure's architecture
RequirementsEngineering
Testing & QualityBusinessUnderstanding
RequirementsEngineering
Architectureand Design Maintenance
BusinessUnderstanding
Maintenance
Unrestricted © Siemens AG 2019February 2019Page 14 Corporate TechnologyPeter Zimmerer
Test Architect – Role profileResponsibilities and tasks (extract)
• Ensure that testability requirements are incorporated into product design• Provide testing infrastructure requirements• Identify and manage technical risks for the testing environment• Drive integration of testing infrastructure with the SUT• Guide/Coach system/software architects in testing related topics• Review architecture artifacts• Define and implement a justified, effective test strategy for the project, aligned with the business and quality
goals• Measure, control and continuously optimize coverage and degree of test automation• Define and maintain an effective and efficient test architecture in close cooperation
with system integration and test managers• Govern the design and improvement of test automation framework and test suites• Ensure quality of the test code
Architectureand Design
Testing & QualityBusinessUnderstanding
RequirementsEngineering
Architectureand Design Maintenance
Testing & Quality
Unrestricted © Siemens AG 2019February 2019Page 15 Corporate TechnologyPeter Zimmerer
Test Architect Expert Training (TeA) Topics
Business Understanding (~15%) Testing & Quality (~40%)
Requirements Engineering (~10%) Social Skills (~20%)
• Product Lifecycle Models (PLM)• Lifecycle models and processes• Portfolio management• Risk & Uncertainty• Estimation and prioritization
• Communication with stakeholders about testing and quality
• Situational leadership model• Leadership and motivation• Conflict management
• Good requirements• Reviewing requirements• Configuration and change request
management
Architecture (~15%)• Systematic architecting• Design strategies & tactics• Design patterns• Architectural views and documentation• Architecture quality• Architecture reviews• Refactoring
• Value of testing• Test strategies, risk-based testing• Design for testability• Test management• Agile testing• Test design techniques• Test automation• Test environment, test architecture• Test documentation• Internal quality• Test Architect revisited
Cross-Cutting• Role of the Test Architect• Dealing with quality attributes
(non-functional requirements)• Agile / Lean development
Unrestricted © Siemens AG 2019February 2019Page 16 Corporate TechnologyPeter Zimmerer
Test Architect Expert Training (TeA) Schedule
Preparationphase
~2 months
CertificationWorkshop 1Live as a TeA
4.5 days
Workshop 2Work as a TeA
3.5 days
Workshop 3Survive as a TeA
4 days
Projectphase 1
~2 months
Projectphase 2
~2 months
Selection of candidates(self perception)
Experiencephase
>6 weeks
In parallel setup and maintenance of networking bycentrally organized events and exchange platforms
Unrestricted © Siemens AG 2019February 2019Page 17 Corporate TechnologyPeter Zimmerer
Example: Domain Model for TestingMetamodel of the UML Testing Profile (UTP)
http://www.omg.org/spec/UTP/http://utp.omg.org/http://utp.zen-tools.com/
In software engineering, a domain model is a conceptual model of the domainthat incorporates both behavior and data.
Unrestricted © Siemens AG 2019February 2019Page 18 Corporate TechnologyPeter Zimmerer
Test architecture
Test architecture refers to the high level structures of a test system that is used/required to test a system, the discipline of creating such structures, and the documentation of these structures. These structures are needed to reason about the test system. Each structure comprises test elements, relations among them, and properties of both elements and relations … (adapted from Wikipedia)Test architecture is the architecture of the test system
Two kinds of scope: test system architecture and test suite architecture• Test system architecture is for test system, system/software to be tested (SUT), platform where SUT is
executed, generator of test cases etc. (see UTP)• Test suite architecture is for test suite inside, which mainly consists of test levels, test types, test
conditions, test cycles, groups of test cases, etc.
A good test architecture and design for testability is a precondition/enabler/driver for effective and cost-efficient test automation that is easy to maintain and evolveA test architecture is needed to design and manage the quality attributes (NFRs) of a test system
http://www.aster.or.jp/workshops/insta2015//i t 2015/
Unrestricted © Siemens AG 2019February 2019Page 19 Corporate TechnologyPeter Zimmerer
Test Architect & “Architecture”
Test Architect
Product / Systemarchitecture Test architecture
TestDefineDesign
Test infrastructurearchitecture
contains
ReviewCoach
Unrestricted © Siemens AG 2019February 2019Page 20 Corporate TechnologyPeter Zimmerer
Systematic architecting
Performuser requirements
elicitation
Createdomain model
Model dynamicsof scenarios
Determine scope& boundaries
Create first conceptual draft
Structure thebaseline
architecture
Introducedeployment
views
Defineprinciples &guidelines
Plan and realizeincrements
Product / System / Service
SWA / SyA
Unrestricted © Siemens AG 2019February 2019Page 21 Corporate TechnologyPeter Zimmerer
Systematic architectingLinks between Test Architect (TeA) and Software / System Architect (SWA / SyA)
Performuser requirements
elicitation
Createdomain model
Model dynamicsof scenarios
Determine scope& boundaries
Create first conceptual draft
Structure thebaseline
architecture
Introducedeployment
views
Defineprinciples &guidelines
Plan and realizeincrements
Domain model Product / System / Service
SWA / SyA
IdentifyASR
“Companion” domain model
Conceptual draft of test system
Model dynamics of test system
Scope & boundaries of test system
Refine architecture of
test system
Define test deployment architecture
Testing principles
governance
Incremental evolution
Define qualities
Dynamics
Boundaries
Increments,
Integration TestsReview both
architectures
SynchronizeTest
Sys
tem
–Te
A
Test strategy
Unrestricted © Siemens AG 2019February 2019Page 22 Corporate TechnologyPeter Zimmerer
Parallel Architecture for Component Testing (PACT)
ApplicabilityThis approach is useful at class and cluster scope for unit and integration testing of an application class hierarchy
IntentImplement a driver as a class hierarchy that is symmetric to the hierarchy of the class under test. Each driver class sends a test message to an object of the class under test
John D. McGregorClemson University, South Carolina
Unrestricted © Siemens AG 2019February 2019Page 23 Corporate TechnologyPeter Zimmerer
ISTQB Generic test automation architecture (gTAA)IIIISSSSSSSSSSSSSSSSSSSSSSSSSSSTTTTTQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQBBB
http://www.istqb.org/
Unrestricted © Siemens AG 2019February 2019Page 24 Corporate TechnologyPeter Zimmerer
TTCN-3 test architecture
System Under Test (SUT)
System Adapter Platform Adapter
TTCN-3 Executable
Test Management
External Codecs Test Logging
Test System User
TTCN-3 ControlInterface (TCI)
TTCN-3 RuntimeInterface (TRI)
Provided by the test environment
Compiled TTCN-3module
System-dependent
Standardized interface
Standardized interface
Test Control Component Handler
TTCN-3 Test System
http://www.ttcn-3.org/
Unrestricted © Siemens AG 2019February 2019Page 25 Corporate TechnologyPeter Zimmerer
Testing strategies in a microservice architecture
Toby Clemson, ThoughtWorks: Testing Strategies in a Microservice Architecturehttps://martinfowler.com/articles/microservice-testing/
Unrestricted © Siemens AG 2019February 2019Page 26 Corporate TechnologyPeter Zimmerer
Utility tree and scenario descriptions
A Utility Tree• describes the most important quality attributes• details each quality attribute in relevant facets• relates scenarios to quality attributes or their facets and thus allows to prioritize them
Scenarios are used to• represent stakeholders’ interests• understand and clarify quality attribute requirementsUtility
Reliability
...
Scalability
PerformanceThroughput
Signal delay
Target: The track allows a frequency of at least 12 trainsper hour in normal operation
(H,M)
(L,H) Outstanding: The track allows a frequency of at least 15 trains per
hour in normal operation
Train length
Numberof seats
Trains can be configured with 2, 4 or 6 wagons.
(H,L)
(L,L) 30% of the seats shall be removable in the service station.
Safety
Source of stimulus
StimulusArtifact
Environment
Response
Responsemeasure
Source of stimulus
StimulusArtifact
Environment
Response
Responsemeasure
Unrestricted © Siemens AG 2019February 2019Page 27 Corporate TechnologyPeter Zimmerer
Design alternatives for a quality attributeDesign strategies and tactics
Performance of IT Systems
• Increase computation efficiency
• Reduce computational overhead
• Manage event rate• Control frequency of
events / raw data
Resourcedemand
• Introduce concurrency
• Maintain multiple copies of data/services
• Adapt HW resources(processor, memory, network)
Resourcemanagement
• Scheduling policy• Distribution• Localization
Resourcearbitration
The Quality in focus
What could be done
How could it be done
Design patterns for implementing the tactics• Caching• Lazy Evaluation• Coordinator• Half Sync / Half Async• Command Processor
Unrestricted © Siemens AG 2019February 2019Page 28 Corporate TechnologyPeter Zimmerer
As a Test ArchitectYou have one Role – but you are wearing two Hats
Test Expertfor the system under test (SUT)
• Design the test approach• Apply innovative test technologies• Drive the quality of the SUT
Software / System Architectfor the test system
• Design and realize the test architecture• Apply innovative software technologies• Drive the quality of the test system
This is the architect's job!
Quality Attributesof the test system (test code)
Scope of Dealing with Quality Attributes
Quality Attributesof the SUT (production code)
Unrestricted © Siemens AG 2019February 2019Page 29 Corporate TechnologyPeter Zimmerer
As a Test ArchitectYou have one Role – but you are wearing two Hats
Test Expertfor the system under test (SUT)
• Design the test approach• Apply innovative test technologies• Drive the quality of the SUT
Software / System Architectfor the test system
• Design and realize the test architecture• Apply innovative software technologies• Drive the quality of the test system
This is the architect's job!
Internal Qualityof the test system (test code)
Scope of Internal Quality
Internal Qualityof the SUT (production code)
Unrestricted © Siemens AG 2019February 2019Page 30 Corporate TechnologyPeter Zimmerer
Some numbers
• Started in 2016
• Currently in RUN 6 (typically 2 RUNs per year)
• Overall ca. 90 participants in the trainings
• >30 Certified Siemens Test Architects (TeA)
Invented – Defined – Designed – Implemented – Piloted – Established – Grow
Unrestricted © Siemens AG 2019February 2019Page 31 Corporate TechnologyPeter Zimmerer
Summary Test Architect
A new key role Test Architect on eye level with software / system architects
Diverse responsibilities and tasks that require proper skills, competencies, capabilities
A new career path for testers
A good Test Architect provides value and creates impact on the business
Test Architect
This is the architect's job!
Unrestricted © Siemens AG 2019February 2019Page 32 Corporate TechnologyPeter Zimmerer
Peter ZimmererPrincipal Key Expert EngineerSiemens AGCorporate Technology Research and Development for Digitalization and AutomationSoftware & Systems InnovationsCT RDA SSI
Otto-Hahn-Ring 681739 Muenchen
Phone: +49 (89) 636 633509Fax: +49 (89) 636 45450Mobile: +49 (172) 4328121
E-mail: peter.zimmerer@siemens.com
Internetsiemens.com/innovation siemens.com/corporate-technology
top related