addq testautomatiseringserfarenheter
TRANSCRIPT
![Page 1: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/1.jpg)
Testautomatiseringserfarenheter
![Page 2: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/2.jpg)
![Page 3: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/3.jpg)
Presentation
Michael Albrecht Lars Wahlberg (Aguil)
![Page 4: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/4.jpg)
Vad innehåller boken?
![Page 5: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/5.jpg)
• Michael Albrecht – Anecdote 29.12 Cooperation Can Overcome
Resource Limitations • Lars Wahlberg
– Chapter 18 Automated Tests for Marketplace Systems: Ten years and Three Frameworks
Nu blir det vissa slides på Engelska !!!
Våra bidrag
![Page 6: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/6.jpg)
• Hur gick det till – Hur vi blev kontaktade
• Arbetet med att ta fram bok – Massor med reviewer & uppdateringar – Tekniska saker som att pillra med bilder
• Legal process – Skriva på kontrakt osv
• Känslan att få boken (tillslut)
Framtagning av en bok
![Page 7: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/7.jpg)
Första stegen
![Page 8: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/8.jpg)
Anecdote 29.12 Cooperation Can Overcome Resource Limitations
![Page 9: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/9.jpg)
• Sit together • Daily status meetings • Learn some basic programming (same
language as the developers) • Select tools already used by the developers
for unit testing if possible • Create semi-automatic and automatic tests • Learn from the past • Get going!
Before the agile hype
![Page 10: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/10.jpg)
• 55% a good tester • 35% development skills • 10% business or domain knowledge
Technical automater
![Page 11: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/11.jpg)
• http://www.addq.se/nyheter/back-to-the-future-an-agile-story-from-the-past/
• http://www.automatedtestinginstitute.com
.and the rest of the story?
![Page 12: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/12.jpg)
Chapter 18 Automated Tests for Marketplace Systems: Ten years and Three Frameworks
• Market Place Systems • Automated Tests • Three frameworks • ROI
![Page 13: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/13.jpg)
Market Place Systems (MPS)
Gwy A
Gwy B
Gwy C
Gwy D
Matching Engine
Central Data
Query / History Server(s)
Trading Clients Protocol X
Trading Clients Protocol Y
Market Data
Admin Client
Primary Standby
Typical instruments: Equities, Commodities, Bonds, Derivatives …
![Page 14: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/14.jpg)
Drivers for large nr of tests … • Complex functionality • Interference of functionality • Critical functionality
– Wrong auto match L – Wrong transparency L – Possible to “game” the system L
• High uptime requirements • Many customers (adaptations of a product)
![Page 15: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/15.jpg)
Why automate?
• Large nr of tests • Development methods – many releases!
– Incremental (10 years ago) – Agile (now days)
• Time to market important (rapid deliveries) – Enhancements (new business opportunity) – Patch (maintenance)
Solution => automate tests ...
![Page 16: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/16.jpg)
Development of Test Framework
Abstraction Layer
Framework for automated tests
System
Test Scripts
Test Engine
Configuration
(ex JUnit)
![Page 17: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/17.jpg)
Test Script
Enter Buy Order (25 SEK)
Enter Sell Order (24 SEK)
Verify Trade (25 SEK)
Enter Buy Order(price)
Enter Order (price, True, ...)
Enter Order (price, isBuy, …)
• Create Enter Order Message
• Fill in Price X
• Fill in Buy or Sell
• Fill in …
• Send Transaction
Changes in the protocol can be handled in the abstraction layer, instead of rewriting ~100 of tests scripts …
Abstraction Layer – Example
![Page 18: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/18.jpg)
Abstraction of Configuration
• Predicates can be used Test Script
myList.add(new PredicateInstrument(“Equity”) )
myList.add(new PredicateCurrency(“USD”) )
myList.add(new PredicateTrader(myTrader1) )
myList.add(new PredicateTrader(myTrader2) )
OrderBook myOb = TestConfig.getAnyOb(myList)
Hint: Use TDD to design the Abstraction Layer !
If not found =>
Test Skipped !
Not same as Failed !
![Page 19: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/19.jpg)
Development of Test Tool
• Abstraction Layer is important – Thin / Thick? Requirements on people skills! – Who can modify? Who will maintain? – When tests increased problems will be revealed!
• Abstraction against test data – Hard coded in a file, version controlled with tests – Predicates may be used to increase robustness
Tip: Very good reference go to => www.stickyminds.com Search for “mexus” (author Lars-Ivar Sellberg)
![Page 20: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/20.jpg)
Development of Tests Automatic Tests
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Week
Test
s
ActualPlanned
![Page 21: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/21.jpg)
Test Execution (then)
![Page 22: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/22.jpg)
Test Execution (then)
TestCase written before implementation
Now it works ! :)
Oops, it fails! :( Regression tests are good !
Oops, vacation!
Now it works again! :)
![Page 23: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/23.jpg)
Framework - 1 • Developed under much time pressure 1998 • Tests written in Java
– New language, attracted developers to test – Thin Abstraction Layer
• Own Test Engine (JUnit not available) • Predicated used • Simulations used (later more on that) • After ~1 year: 2000 tests, ~3 year: 15 000 tests • A real success !!!
![Page 24: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/24.jpg)
Framework - 2 • Developed 1999 based on success of -1 • Development took some time …too much
– Another system => new abstraction layer, still thin – Predicates not used – Switched to JUnit – Focus on Tool instead of tests … few tests L
• Heavy investment during 2002 – Partly due to one senior Manager J – Larger team => 50 000 tests after 3 years!
![Page 25: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/25.jpg)
Framework - 3
• Developed around 2003 • Based on ideas from mostly from 1
– But should be better! Func & Non Func tests, Preds – Thick Layer – Tests in XML action language
• Test Tool Architects became bottlenecks – Difficult to do modifications of abstraction layer L
• Re-done, Thin layer & No Predicates • Now ~ 20 000 Tests
![Page 26: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/26.jpg)
ROI - Savings Saving of Automization
-350
-300
-250
-200
-150
-100
-50
0
50
100
150
500
700
900
1100
1300
1500
1700
1900
nr of tests in batch
savi
ngs
in % Yearly
MonthlyWeeklyDaily
Effort dev manual test 2 h/testEffort dev automatic test 2 h/testTest Tool Maintanence 3000 h/yearEffort Execute manual test 0,1 h / testEffort Execute automatic test 8 h/batch
![Page 27: AddQ Testautomatiseringserfarenheter](https://reader034.vdocuments.us/reader034/viewer/2022052602/559b334f1a28ab34638b460c/html5/thumbnails/27.jpg)
Utlottning av böcker !