testing - mdse research group€¦ · o all other systems thoroughly tested component-by-component....
TRANSCRIPT
![Page 1: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/1.jpg)
TestingAn Introduction to Software Testing
Gholamhossein Tavasoli @ ZNU
![Page 2: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/2.jpg)
Who Am I?
o Gholamhossein Tavasoli
o PhD Student in Artificial Intelligence
o System Developer for 8 yearso High Performance IDS / IPSo Industrial IDSo
o Security (Semi) Expert, Member of ZNU Cert for 3 years
![Page 3: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/3.jpg)
Type of Bugso Security
o Regression
o Business Logic
o Accessibility
o Performance
o User Interface
o Integration
o Internationalization
o Scaling
![Page 4: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/4.jpg)
Some Famous Software Failures
o Heartbleedo Heartbeat Extension of TLS is used to keep Datagram TLS sessions open.o Simple request and response scheme:o “send me back the following (padded) string which is n bytes long.”
o An attacker just had to request a long string, while telling it is short.o Other party responded with short string, then leaked potentially confidential data.
o Video (https://youtu.be/WgrBrPW_Zn4)
![Page 5: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/5.jpg)
Some Famous Software Failures
o Ariane-5o Inertial navigation software taken from Ariane-4. Untested.o All other systems thoroughly tested component-by-component.o Ariane-5 had a different trajectory than Ariane 4.o Converting 64-bit floating-point data into 16-bit unsigned integer values. → Arithmetic overflow.o There was an exception handler for that. It had been disabled.o Not even 40 seconds after launch, Ariane-5 literally self-destructed. Successfully.
o Video (https://youtu.be/WgrBrPW_Zn4)
![Page 6: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/6.jpg)
Why Testing?
o Software contains defects. The hide.
o Defects can cause software failures.
o Failures can cost money, and even be mortal!
o Testing assures the quality of the software.
o Testing accelerates software development.
o And much, much more.
![Page 7: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/7.jpg)
Why Testing?
o Software contains defects. The hide.
o Defects can cause software failures.
o Failures can cost money, and even be mortal!
o Testing assures the quality of the software.
o Testing accelerates software development.
o And much, much more.
![Page 8: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/8.jpg)
What is Testing?
The process consisting of all life cycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software products and related work products to determine that they
satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects.
![Page 9: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/9.jpg)
Do Software Testing. Right.
o Testing has to be planned.
o Testing costs easily up to 40% of a project's budget.
o Testing has to be performed in a reasonable way.
o Testing shall be independent and objective.
o Testing has to be managed.
Software testing is a fundamental part of professional software development!
![Page 10: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/10.jpg)
Type of Tests
o Unit Testing
o Integration Testing
o E2E Testing
o Penetration Testing
o Performance Testing
o Regression Testing
o a11y Testing
o Stress Testing
o Fuzz Testing
o A/B Testing
o User Acceptance Testing
o Usability Testing
o i18n Testing
o Smoke Testing
![Page 11: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/11.jpg)
Test Levels
o Unit Testingo Test individual units in isolation.o Find defects in software components (e.g., functions, classes).o Done by developers.o In general, white-box tests.
![Page 12: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/12.jpg)
What is a Unit Test?
![Page 13: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/13.jpg)
Test Levels
o Unit Testing
o Integration Testingo Test communication/interaction of units (i.e., their interfaces). o Maybe separate unit integration and system integration tests. o Done by developers. o Primarily white-box tests, but also black-box tests.
![Page 14: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/14.jpg)
What is a Integration Test?
![Page 15: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/15.jpg)
Test Levels
o Unit Testing
o Integration Testing
o System Testingo Test complete, integrated system. o Evaluate system compliance with specified requirements. o Stress, performance, usability etc. testing. o Done by (external) testers. o In general, black-box tests. Additional white-box tests possible.
![Page 16: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/16.jpg)
Test Levels
o Unit Testing
o Integration Testing
o System Testing
o Acceptance Testing o Test complete, integrated system. o Evaluate system compliance with specified acceptance criteria. o May be performed at various times during development. o Done by customers/users. o Only black-box tests
![Page 17: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/17.jpg)
App
o Technologieso Python
![Page 18: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/18.jpg)
Python in Business
![Page 19: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/19.jpg)
App
o Technologieso Pythono Flasko Flaskro NGINXo uWSGI
o Environmento Ubuntu Server (VM)o PyCharm
![Page 20: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/20.jpg)
Setup
git clone https://github.com/gtavasoli/testing-workshop.git
cd testing-workshop
![Page 21: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/21.jpg)
App Structure
o Unit Tests
o Simple Flask Apps
o Flaskr
o JMeter Profiles
![Page 22: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/22.jpg)
Unit TestUnit test in python
![Page 23: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/23.jpg)
Unit Testing
o Sample 1 – Calculatoro Test operatorso Floating point divisiono Division by zero
o Sample 2 – Employeeo Setup & Teardowno Setup & Teardown Class
![Page 24: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/24.jpg)
Test Double
![Page 25: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/25.jpg)
Test Double
o Dummy → I do nothing at all but to fill parameter list
o Stub → canned Answers
o Spy → stubs + interaction recording (for late interaction expectations verification)
o Mock → stubs + expectations on interaction
o Fake → I seem real but no
![Page 26: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/26.jpg)
What Are Mocks?
o The term 'Mock Objects' has become a popular one to describe special case objects that mimic real objects for testing.
o Mocking makes unit testing easier!
o Verifies behavior (calls) to a method.
o A mock object simulates the behavior of real objects in controlled ways.
![Page 27: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/27.jpg)
What Are Mocks?
![Page 28: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/28.jpg)
Problems Mocks Solve
o Eliminates dependencies in the CUT (class under test)o Isolate Unite Tests
foo(x) bar(x)
Copyright: Based on “Mocking Strategies” – slides from 26 to end of mocking sample
![Page 29: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/29.jpg)
Problems Mocks Solve
o Tests methods that have no return value
How do we know that bar(x) has been called?
![Page 30: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/30.jpg)
Problems Mocks Solve
o Tests error handling
How do we generate this exception???
![Page 31: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/31.jpg)
Problems Mocks Solve
o Eliminate dependency on database callso Speed up testing!
o Reduce test complexityo Don’t have to write complex logic to handle behavior of methods not under test
o Don’t have to wait to implement other methods
![Page 32: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/32.jpg)
Problem: Friend Finder App
o Create a method to return a new, random friendship candidateo Must not show the same persono Must not show someone the user has already visit it.
![Page 33: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/33.jpg)
Easy Method
get_next_person() get_random_person()
“Surely no one could
have seen EVERYONE
in the database!!!”
![Page 34: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/34.jpg)
Write a Unit Test...
![Page 35: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/35.jpg)
May be it Works!!!
![Page 36: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/36.jpg)
May be don’t!
![Page 37: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/37.jpg)
![Page 38: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/38.jpg)
Easy Method
What if knew the result of get_random_person()???
![Page 39: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/39.jpg)
![Page 40: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/40.jpg)
Patching
![Page 41: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/41.jpg)
But what if we call it multiple times???
![Page 42: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/42.jpg)
Problem: Matching in Friend Finder App
o When a user send friend request…
o If the other user “likes” it:o Send them both a message with contact info
o If the other user “dislikes” it:o Let the user down gently…
o If the other user hasn’t evaluated yet:o Display the “give it time” message
![Page 43: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/43.jpg)
Implementation
How do we test this??? No return values!!!
![Page 44: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/44.jpg)
Behavior Verification
![Page 45: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/45.jpg)
Mocking Exceptions
![Page 46: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/46.jpg)
Simple JSON reader
How do we test something like this???
![Page 47: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/47.jpg)
Testing JSON reader
o Test parsing a valid file
o Test an IOError (i.e. file missing)
o Test a ValueError (i.e invalid JSON)
![Page 48: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/48.jpg)
How do I test open()
o Let’s just create a sample file!
![Page 49: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/49.jpg)
Can you even mock a builtin?
![Page 50: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/50.jpg)
What? Why Not???
o open() returns a File object
o open(filename).read()
o So we really need to mock File.read()o But it’s an instance!!! Oh no!
o Have you tried solving it with Mocks???
![Page 51: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/51.jpg)
Mocks returning Mocks?
![Page 52: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/52.jpg)
What about IOError?
![Page 53: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/53.jpg)
What about ValueError?
![Page 54: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/54.jpg)
Warning
o A warning about over-isolatingo Many people fall for relentless isolationo Magic tool for helping in isolating parts
o Mocking is about object communication and interface discoveryo Using it for isolation is a misuseo Specially from 3rd party code
o General rule of thumb: "do not mock what you don’t own"
![Page 55: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/55.jpg)
Warning
o Avoid over-mockingo Duplicating implementation code in the testso Keep production and test code in sync
o We should refrain from this and try as much as possible to use real collaborators when possible.
![Page 56: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/56.jpg)
Testing/Coding is so hard!!
![Page 57: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/57.jpg)
![Page 58: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/58.jpg)
Type of Software Testing
![Page 59: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/59.jpg)
Non-functional Testing
o Performance Testingo To identify any bottlenecks or performance issues rather than finding bugs in a softwareo Speedo Capacityo Stabilityo Scalability
![Page 60: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/60.jpg)
Non-functional Testing
o Performance Testing
o Load Testingo The process of testing the behavior of a software by applying maximum load in terms of software accessing and manipulating large input datao Toolso Load Runner, AppLoader, IBM Rational Performance Tester, Apache
JMeter, Silk Performer, Visual Studio Load Test, etc.
![Page 61: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/61.jpg)
Non-functional Testing
o Performance Testing
o Load Testing
o Stress Testingo Includes testing the behavior of a software under abnormal conditionso Applying the load to the system and taking over the resources used by the software to identify the breaking point
![Page 62: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/62.jpg)
Non-functional Testing
o Performance Testing
o Load Testing
o Stress Testingo Shutdown or restart of network ports randomlyo Turning the database on or offo Running different processes that consume resources such as CPU, memory, server, etc.
![Page 63: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/63.jpg)
Non-functional Testing
o Performance Testing
o Security Testingo Confidentialityo Integrityo Authenticationo Availabilityo Authorizationo …
![Page 64: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/64.jpg)
Performance Testingo Methodologieso Real User Monitoring - RUMo Synthetic Testing
o The Recipeo Step 1 – Collect Datao Step 2 – Analyzeo Step 3 – Experimento Step 4 – GoTo 1
![Page 65: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/65.jpg)
Targets in Performance Testing
o Things we measureo Availability or uptimeo Concurrencyo Throughputo Response timeo Network utilizationo Server utilization
![Page 66: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/66.jpg)
Tips
o Beware of failures
o Be aware of caching
o Automate bookkeeping
![Page 67: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/67.jpg)
Tutorial 1 – Simple Test
o Step 1 – Start JMeter
o Step 2 – Create a TestPlan
o Step 3 – Create a Thread Group (Users)
o Step 4 – Add Sampler (HTTP)
o Step 5 – Add Listeners
o Step 6 – To Run the Test
![Page 68: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/68.jpg)
Tutorial 2 – Using Assertions
o Response Assertion
o Duration Assertion
o Size Assertion
o HTML Assertion
![Page 69: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/69.jpg)
Tutorial 3 – Flaskr
o Simple Microblog With Flask
o Parametric Request
![Page 71: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/71.jpg)
Tutorial 4 – JMeter Plugins
o JMeter Plugin: Response Times Percentiles / Distribution
oTail Latency
The Tail at Scale – Dean and Barroso 2013
![Page 72: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/72.jpg)
Tutorial 4 – JMeter Plugins
o What happens when there is not just a single server involved in servicing a request, but 100! (fan-out query, contribution of a large number of microservices)
![Page 73: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/73.jpg)
Tutorial 4 – JMeter Plugins
![Page 74: Testing - MDSE Research Group€¦ · o All other systems thoroughly tested component-by-component. o Ariane-5 had a different trajectory than Ariane 4. o Converting 64-bit floating-point](https://reader034.vdocuments.us/reader034/viewer/2022051814/603b089213dd0c070765d5bb/html5/thumbnails/74.jpg)
Tutorial 4 – JMeter Plugins
o Tail Latency
o Root Causeso Shared resources (memory, CPU cores, CPU caches, shared file system)o Background tasks running on the servero Maintenance activities (GC, Log rotation)o Queues in general
https://blog.acolyer.org/2015/01/15/the-tail-at-scale/