Download - Ipc2008 Slide Qa In Depth Best Practises
Quality Assurance in PHP Projects
In-Depth Best-Practises
Max Horváth & Wojciech Duda, studiVZ Ltd. 2
Who are we?
• Max Horváth
• Actively involved in web development for the past 10 years.
• Quality Manager at studiVZ Ltd.
Max Horváth & Wojciech Duda, studiVZ Ltd. 3
Who are we?
• Wojciech Duda
• Involved in software testing for the past 6 years.
• QA Engineer at studiVZ Ltd.
Max Horváth & Wojciech Duda, studiVZ Ltd. 4
Who are YOU?
• What is your experience with: • QA in PHP projects?
• Metrics of software?
• Testing of software?
Max Horváth & Wojciech Duda, studiVZ Ltd. 5
Schedule
Max Horváth & Wojciech Duda, studiVZ Ltd. 6
Schedule / Lots of coffee
• 10:45am – 11:15am Coffee Break
• 12:15am – 01:30pm Lunch
• 03:15pm – 03:45pm Coffee Break
Max Horváth & Wojciech Duda, studiVZ Ltd. 7
Schedule (Part I)
• 09:30am – 10:00am Intro / Tutorial
• 10:45am – 11:15am Coffee Break
• 11:15am – 12:15am Tutorial
• 12:15am – 01:30pm Lunch
Max Horváth & Wojciech Duda, studiVZ Ltd. 8
Schedule (Part II)
• 01:30pm – 02:30pm Tutorial
• 03:15pm – 03:45pm Coffee Break
• 03:45pm – 05:00pm Q&A
Max Horváth & Wojciech Duda, studiVZ Ltd. 9
Objectives of this workshop
• To introduce you to the quality assurance process and key quality assurance activities
Max Horváth & Wojciech Duda, studiVZ Ltd. 10
Objectives of this workshop
• To introduce you to the quality assurance process and key quality assurance activities
• To explain the role of standards in quality assurance
Max Horváth & Wojciech Duda, studiVZ Ltd. 11
Objectives of this workshop
• To explain the concept of a software metric, predictor metrics and control metrics
Max Horváth & Wojciech Duda, studiVZ Ltd. 12
Objectives of this workshop
• To explain the concept of a software metric, predictor metrics and control metrics
• To explain how measurement may be used in assessing software quality
Max Horváth & Wojciech Duda, studiVZ Ltd. 13
Topics covered
• Quality assurance and standards
Max Horváth & Wojciech Duda, studiVZ Ltd. 14
Topics covered
• Quality assurance and standards
• Quality planning
Max Horváth & Wojciech Duda, studiVZ Ltd. 15
Topics covered
• Quality assurance and standards
• Quality planning
• Quality control
Max Horváth & Wojciech Duda, studiVZ Ltd. 16
Topics covered
• Quality assurance and standards
• Quality planning
• Quality control
• Software measurement and metrics
Max Horváth & Wojciech Duda, studiVZ Ltd. 17
What is Quality Assurance?
• QA is part of all good production processes.
Max Horváth & Wojciech Duda, studiVZ Ltd. 18
What is Quality Assurance?
• QA is part of all good production processes.
• Quality Assurance (QA) is part of the process that ensures:
Max Horváth & Wojciech Duda, studiVZ Ltd. 19
What is Quality Assurance?
• QA is part of all good production processes.
• Quality Assurance (QA) is part of the process that ensures: • quality in work
Max Horváth & Wojciech Duda, studiVZ Ltd. 20
What is Quality Assurance?
• QA is part of all good production processes.
• Quality Assurance (QA) is part of the process that ensures: • quality in work
• activities are being performed effectively
Max Horváth & Wojciech Duda, studiVZ Ltd. 21
What is Quality Assurance?
• QA is part of all good production processes.
• Quality Assurance (QA) is part of the process that ensures: • quality in work
• activities are being performed effectively
• the product meets requirements
Max Horváth & Wojciech Duda, studiVZ Ltd. 22
What is Quality Assurance?
• Involves defining appropriate quality standards and procedures and ensuring that these are followed
Max Horváth & Wojciech Duda, studiVZ Ltd. 23
What is Quality Assurance?
• Involves defining appropriate quality standards and procedures and ensuring that these are followed
• Should aim to develop a ‘quality culture’ where quality is seen as everyone’s responsibility
Max Horváth & Wojciech Duda, studiVZ Ltd. 24
What is quality?
• Quality, simplistically, means that a product should meet its specification
Max Horváth & Wojciech Duda, studiVZ Ltd. 25
What is unique about Software QA?
Max Horváth & Wojciech Duda, studiVZ Ltd. 26
What is unique about Software QA?
Max Horváth & Wojciech Duda, studiVZ Ltd. 27
What is unique about Software QA?
!=
Max Horváth & Wojciech Duda, studiVZ Ltd. 28
It is not equal
• Software QA has some differences:
Max Horváth & Wojciech Duda, studiVZ Ltd. 29
It is not equal
• Software QA has some differences: • Functions, benefits and production costs are not as
easily measured as a physical product.
Max Horváth & Wojciech Duda, studiVZ Ltd. 30
It is not equal
• Software QA has some differences: • Functions, benefits and production costs are not as
easily measured as a physical product.
• When a software product is delivered, it is not necessarily complete ... it lives, it evolves, it gets updated.
Max Horváth & Wojciech Duda, studiVZ Ltd. 31
It is not equal
• Software QA has some differences: • Functions, benefits and production costs are not as
easily measured as a physical product.
• When a software product is delivered, it is not necessarily complete ... it lives, it evolves, it gets updated.
• Finished is a term applied for the purpose of releasing it for use.
Max Horváth & Wojciech Duda, studiVZ Ltd. 32
It is not equal
• Software QA has some differences: • Functions, benefits and production costs are not as
easily measured as a physical product.
• When a software product is delivered, it is not necessarily complete ... it lives, it evolves, it gets updated.
• Finished is a term applied for the purpose of releasing it for use.
• Software specifications are usually incomplete and often inconsistent
Max Horváth & Wojciech Duda, studiVZ Ltd. 33
The challenge
• Processes and methods to manage, monitor, and measure quality in web applications can be as fluid and elusive as the defects they are meant to keep in check.
Max Horváth & Wojciech Duda, studiVZ Ltd. 34
The challenge
• Processes and methods to manage, monitor, and measure quality in web applications can be as fluid and elusive as the defects they are meant to keep in check.
• For processes and methods to be most effective, you need to implement both standards and company guidelines.
Max Horváth & Wojciech Duda, studiVZ Ltd. 35
Quality Assurance is not limited to Quality Control
• Testing is an essential part of QA
Max Horváth & Wojciech Duda, studiVZ Ltd. 36
Quality Assurance is not limited to Quality Control
• Testing is an essential part of QA
• But QA tests the entire process:
Max Horváth & Wojciech Duda, studiVZ Ltd. 37
Quality Assurance is not limited to Quality Control
• Testing is an essential part of QA
• But QA tests the entire process: • Your development process / efficiency
Max Horváth & Wojciech Duda, studiVZ Ltd. 38
Quality Assurance is not limited to Quality Control
• Testing is an essential part of QA
• But QA tests the entire process: • Your development process / efficiency
• Functional accuracy of your software releases
Max Horváth & Wojciech Duda, studiVZ Ltd. 39
Quality Assurance is not limited to Quality Control
• Testing is an essential part of QA
• But QA tests the entire process: • Your development process / efficiency
• Functional accuracy of your software releases
• Ideally preventing future software defects
Max Horváth & Wojciech Duda, studiVZ Ltd. 40
The benefits of QA
• Improved client satisfaction:
Max Horváth & Wojciech Duda, studiVZ Ltd. 41
The benefits of QA
• Improved client satisfaction: • profitable relationships
Max Horváth & Wojciech Duda, studiVZ Ltd. 42
The benefits of QA
• Improved client satisfaction: • profitable relationships
• good testimonials
Max Horváth & Wojciech Duda, studiVZ Ltd. 43
The benefits of QA
• Improved client satisfaction: • profitable relationships
• good testimonials
• waves of referrals
Max Horváth & Wojciech Duda, studiVZ Ltd. 44
The benefits of QA
• Reduced cost of development:
Max Horváth & Wojciech Duda, studiVZ Ltd. 45
The benefits of QA
• Reduced cost of development: • fewer defects
Max Horváth & Wojciech Duda, studiVZ Ltd. 46
The benefits of QA
• Reduced cost of development: • fewer defects
• streamlined and simplified stages of development
Max Horváth & Wojciech Duda, studiVZ Ltd. 47
The benefits of QA
• Reduced cost of development: • fewer defects
• streamlined and simplified stages of development
• retesting runs smoothly
Max Horváth & Wojciech Duda, studiVZ Ltd. 48
The benefits of QA
• Reduced cost of development: • fewer defects
• streamlined and simplified stages of development
• retesting runs smoothly
• deployment on time
Max Horváth & Wojciech Duda, studiVZ Ltd. 49
The benefits of QA
• Reduced cost of development: • fewer defects
• streamlined and simplified stages of development
• retesting runs smoothly
• deployment on time
• on budget
Max Horváth & Wojciech Duda, studiVZ Ltd. 50
The benefits of QA
• Reduced cost of maintenance:
Max Horváth & Wojciech Duda, studiVZ Ltd. 51
The benefits of QA
• Reduced cost of maintenance: • good development is less troublesome to support
Max Horváth & Wojciech Duda, studiVZ Ltd. 52
The benefits of QA
• Reduced cost of maintenance: • good development is less troublesome to support
• support is costly
Max Horváth & Wojciech Duda, studiVZ Ltd. 53
Quality Assurance activities
• Quality management
Max Horváth & Wojciech Duda, studiVZ Ltd. 54
Quality Assurance activities
• Quality management • Establish organizational procedures and standards for
quality
Max Horváth & Wojciech Duda, studiVZ Ltd. 55
Quality Assurance activities
• Quality management • Establish organizational procedures and standards for
quality
• Quality planning
Max Horváth & Wojciech Duda, studiVZ Ltd. 56
Quality Assurance activities
• Quality management • Establish organizational procedures and standards for
quality
• Quality planning • Select applicable procedures and standards for a
particular project and modify these as required
Max Horváth & Wojciech Duda, studiVZ Ltd. 57
Quality Assurance activities
• Quality management • Establish organizational procedures and standards for
quality
• Quality planning • Select applicable procedures and standards for a
particular project and modify these as required
• Quality control
Max Horváth & Wojciech Duda, studiVZ Ltd. 58
Quality Assurance activities
• Quality management • Establish organizational procedures and standards for
quality
• Quality planning • Select applicable procedures and standards for a
particular project and modify these as required
• Quality control • Ensure that procedures and standards are followed by
the software development team
Max Horváth & Wojciech Duda, studiVZ Ltd. 59
Quality Assurance activities
• Important:
Max Horváth & Wojciech Duda, studiVZ Ltd. 60
Quality Assurance activities
• Important: • Quality management should be separate from project
management to ensure independence
Max Horváth & Wojciech Duda, studiVZ Ltd. 61
Quality assurance and standards
• Standards are the key to effective quality management
Max Horváth & Wojciech Duda, studiVZ Ltd. 62
Quality assurance and standards
• Standards are the key to effective quality management
• They may be international, national, organizational or project standards
Max Horváth & Wojciech Duda, studiVZ Ltd. 63
Quality assurance and standards
• Product standards define characteristics that all components should exhibit e.g. a common programming style
Max Horváth & Wojciech Duda, studiVZ Ltd. 64
Quality assurance and standards
• Product standards define characteristics that all components should exhibit e.g. a common programming style
• Process standards define how the software process should be enacted
Max Horváth & Wojciech Duda, studiVZ Ltd. 65
Importance of standards
• Encapsulation of best practice - avoids repetition of past mistakes
Max Horváth & Wojciech Duda, studiVZ Ltd. 66
Importance of standards
• Encapsulation of best practice - avoids repetition of past mistakes
• Framework for quality assurance process - it involves checking standard compliance
Max Horváth & Wojciech Duda, studiVZ Ltd. 67
Importance of standards
• Encapsulation of best practice - avoids repetition of past mistakes
• Framework for quality assurance process - it involves checking standard compliance
• Provide continuity - new staff can understand the organization by understand the standards applied
Max Horváth & Wojciech Duda, studiVZ Ltd. 68
Problems with standards
• Not seen as relevant and up-to-date by software engineers
Max Horváth & Wojciech Duda, studiVZ Ltd. 69
Problems with standards
• Not seen as relevant and up-to-date by software engineers
• Involve too much bureaucratic form filling
Max Horváth & Wojciech Duda, studiVZ Ltd. 70
Problems with standards
• Not seen as relevant and up-to-date by software engineers
• Involve too much bureaucratic form filling
• Unsupported by software tools so tedious manual work is involved to maintain standards
Max Horváth & Wojciech Duda, studiVZ Ltd. 71
Testing in PHP Projects
• Testing is one of the most important tasks of QA.
Max Horváth & Wojciech Duda, studiVZ Ltd. 72
Testing in PHP Projects
• Testing is one of the most important tasks of QA.
• We are going to talk about these testing techniques and tools:
Max Horváth & Wojciech Duda, studiVZ Ltd. 73
Testing in PHP Projects
• Testing is one of the most important tasks of QA.
• We are going to talk about these testing techniques and tools: • Unit Tests (PHPUnit)
Max Horváth & Wojciech Duda, studiVZ Ltd. 74
Testing in PHP Projects
• Testing is one of the most important tasks of QA.
• We are going to talk about these testing techniques and tools: • Unit Tests (PHPUnit)
• Continuous Integration (phpUnderControl)
Max Horváth & Wojciech Duda, studiVZ Ltd. 75
Testing in PHP Projects
• Testing is one of the most important tasks of QA.
• We are going to talk about these testing techniques and tools: • Unit Tests (PHPUnit)
• Continuous Integration (phpUnderControl)
• Software Metrics (PHPUnit / PHP_CodeSniffer)
Max Horváth & Wojciech Duda, studiVZ Ltd. 76
Testing in PHP Projects
• Testing is one of the most important tasks of QA.
• We are going to talk about these testing techniques and tools: • Unit Tests (PHPUnit)
• Continuous Integration (phpUnderControl)
• Software Metrics (PHPUnit / PHP_CodeSniffer)
• Acceptance Tests (PHPUnit / Selenium)
Max Horváth & Wojciech Duda, studiVZ Ltd. 77
Testing Types
• Unit Tests
Max Horváth & Wojciech Duda, studiVZ Ltd. 78
Testing Types
• Unit Tests • Executable code fragments test the correctness of
parts – units – of the software (system under test).
Max Horváth & Wojciech Duda, studiVZ Ltd. 79
Testing Types
• Unit Tests • Executable code fragments test the correctness of
parts – units – of the software (system under test).
• System / Integration Tests
Max Horváth & Wojciech Duda, studiVZ Ltd. 80
Testing Types
• Unit Tests • Executable code fragments test the correctness of
parts – units – of the software (system under test).
• System / Integration Tests • Conducted on a complete, integrated system to
evaluate the system's compliance with ist specified requirements.
Max Horváth & Wojciech Duda, studiVZ Ltd. 81
Testing Types
• Non-Functional Tests
Max Horváth & Wojciech Duda, studiVZ Ltd. 82
Testing Types
• Non-Functional Tests • Test of performance, stability or security of
application.
Max Horváth & Wojciech Duda, studiVZ Ltd. 83
Testing Types
• Non-Functional Tests • Test of performance, stability or security of
application.
• Acceptance Tests
Max Horváth & Wojciech Duda, studiVZ Ltd. 84
Testing Types
• Non-Functional Tests • Test of performance, stability or security of
application.
• Acceptance Tests • Ensure that the code does what the customer wants.
Max Horváth & Wojciech Duda, studiVZ Ltd. 85
Testing Tools
• To make testing viable, good tool support is needed.
Max Horváth & Wojciech Duda, studiVZ Ltd. 86
Testing Tools
• To make testing viable, good tool support is needed.
• This is where a testing framework such as PHPUnit comes into play.
Max Horváth & Wojciech Duda, studiVZ Ltd. 87
Testing Tools
• Unit Tests • PHPUnit
• PHPT
• SimpleTest
Max Horváth & Wojciech Duda, studiVZ Ltd. 88
Testing Tools
• Acceptance Tests • Selenium Suite
• Selenium Remote Control • PHPUnit to talk Selenese to Selenium RC
Max Horváth & Wojciech Duda, studiVZ Ltd. 89
Testing Tools
• Non-Functional Tests • Performance Tests
• ab • Apache JMeter
Max Horváth & Wojciech Duda, studiVZ Ltd. 90
Testing Tools
• Non-Functional Tests • Performance Tests
• ab • Apache Jmeter
• Security Tests • Chorizo • Selenium and Chorizo
Max Horváth & Wojciech Duda, studiVZ Ltd. 91
PHPUnit
• PHPUnit is a member of the xUnit family of testing frameworks (SUnit, jUnit, pyUnit...).
Max Horváth & Wojciech Duda, studiVZ Ltd. 92
PHPUnit
• PHPUnit is a member of the xUnit family of testing frameworks (SUnit, jUnit, pyUnit...).
• Written for PHP 5.
Max Horváth & Wojciech Duda, studiVZ Ltd. 93
PHPUnit
• The xUnit frameworks provide a way to automate unit tests.
Max Horváth & Wojciech Duda, studiVZ Ltd. 94
PHPUnit
• Make it easy to test, and you'll do it more often.
Max Horváth & Wojciech Duda, studiVZ Ltd. 95
PHPUnit
• Make it easy to test, and you'll do it more often.
• If you use version control, test before every check-in.
Max Horváth & Wojciech Duda, studiVZ Ltd. 96
PHPUnit
• The PHPUnit Pocket Guide
Max Horváth & Wojciech Duda, studiVZ Ltd. 97
PHPUnit
• The PHPUnit Pocket Guide
• Tells you everything you need to know when writing PHPUnit test cases.
Max Horváth & Wojciech Duda, studiVZ Ltd. 98
PHPUnit
• The PHPUnit Pocket Guide
• Tells you everything you need to know when writing PHPUnit test cases.
• Buy the book, or read the HTML version at: • http://www.phpunit.de/pocket_guide/index.en.php
Max Horváth & Wojciech Duda, studiVZ Ltd. 99
PHPUnit
Let's look at a test case!
Max Horváth & Wojciech Duda, studiVZ Ltd. 100
PHPUnit <?php require_once 'PHPUnit/Framework.php';
class ArrayTest extends PHPUnit_Framework_TestCase { protected $fixture;
protected function setUp() { // Create the Array fixture. $this->fixture = array(); }
public function testNewArrayIsEmpty() { // Assert that the size of the Array fixture is 0. $this->assertEquals(0, sizeof($this->fixture)); }
public function testArrayContainsAnElement() { // Add an element to the Array fixture. $this->fixture[] = 'Element';
// Assert that the size of the Array fixture is 1. $this->assertEquals(1, sizeof($this->fixture)); } } ?>
Max Horváth & Wojciech Duda, studiVZ Ltd. 101
PHPUnit
• Run a PHPUnit test case with the phpunit command.
Max Horváth & Wojciech Duda, studiVZ Ltd. 102
PHPUnit
• Run a PHPUnit test case with the phpunit command.
phpunit UnitTest UnitTest.php
Max Horváth & Wojciech Duda, studiVZ Ltd. 103
PHPUnit
• Run a PHPUnit test case with the phpunit command.
phpunit UnitTest UnitTest.php
• Tells PHPUnit to look for a class UnitTest in the UnitTest.php file (filename is optional).
Max Horváth & Wojciech Duda, studiVZ Ltd. 104
PHPUnit
• PHPUnit can output test logs to a variety of formats.
Max Horváth & Wojciech Duda, studiVZ Ltd. 105
PHPUnit
• PHPUnit can output test logs to a variety of formats.
• This allows results to be used for Continuous Integration.
Max Horváth & Wojciech Duda, studiVZ Ltd. 106
Continuous Integration
• Continuous integration is a way to automatically run your test suite regularly.
Max Horváth & Wojciech Duda, studiVZ Ltd. 107
Continuous Integration
• Continuous integration is a way to automatically run your test suite regularly.
• Once a day …
Max Horváth & Wojciech Duda, studiVZ Ltd. 108
Continuous Integration
• Continuous integration is a way to automatically run your test suite regularly.
• Once a day …
• Once an hour …
Max Horváth & Wojciech Duda, studiVZ Ltd. 109
Continuous Integration
• Continuous integration is a way to automatically run your test suite regularly.
• Once a day …
• Once an hour …
• Every time someone checks in code …
Max Horváth & Wojciech Duda, studiVZ Ltd. 110
What is integration?
• Integrating a change (code, database structure …) into the source code repository without breaking the project.
Max Horváth & Wojciech Duda, studiVZ Ltd. 111
When is integration becoming important?
• For any change that might break functionality.
Max Horváth & Wojciech Duda, studiVZ Ltd. 112
When is integration becoming important?
• For any change that might break functionality.
• More than one developer and/or multiple components.
Max Horváth & Wojciech Duda, studiVZ Ltd. 113
When is integration becoming important?
• For any change that might break functionality.
• More than one developer and/or multiple components.
• More developers, more components = more complexity.
Max Horváth & Wojciech Duda, studiVZ Ltd. 114
Continuous Integration
• It is a software engineering practice.
Max Horváth & Wojciech Duda, studiVZ Ltd. 115
Continuous Integration
• It is a software engineering practice.
• It describes how to integrate changes frequently.
Max Horváth & Wojciech Duda, studiVZ Ltd. 116
Continuous Integration
• It is a software engineering practice.
• It describes how to integrate changes frequently.
• It stabilizes the product through self-testing code.
Max Horváth & Wojciech Duda, studiVZ Ltd. 117
Continuous Integration
• It uses rapid feedback to solve integration issues fast and putting the pain upfront the development cycle. (Remember: software defects are more expensive the later they are fixed.)
Max Horváth & Wojciech Duda, studiVZ Ltd. 118
Continuous Integration
• It uses rapid feedback to solve integration issues fast and putting the pain upfront the development cycle. (Remember: software defects are more expensive the later they are fixed.)
• Goal: produce as many working builds as possible.
Max Horváth & Wojciech Duda, studiVZ Ltd. 119
Continuous Integration
• Without unit tests Continuous Integration does not help a lot.
Max Horváth & Wojciech Duda, studiVZ Ltd. 120
Continuous Integration
• Without unit tests Continuous Integration does not help a lot.
• Start using PHPUnit or extend your test suite:
Max Horváth & Wojciech Duda, studiVZ Ltd. 121
Continuous Integration
• Without unit tests Continuous Integration does not help a lot.
• Start using PHPUnit or extend your test suite: • Each identified bug deserves a unit test to prevent
regression.
Max Horváth & Wojciech Duda, studiVZ Ltd. 122
Continuous Integration
• Without unit tests Continuous Integration does not help a lot.
• Start using PHPUnit or extend your test suite: • Each identified bug deserves a unit test to prevent
regression.
• Develop unit tests for each new feature.
Max Horváth & Wojciech Duda, studiVZ Ltd. 123
Continuous Integration
• Without unit tests Continuous Integration does not help a lot.
• Start using PHPUnit or extend your test suite: • Each identified bug deserves a unit test to prevent
regression.
• Develop unit tests for each new feature.
• You don’t need full code coverage. Find the right level!
Max Horváth & Wojciech Duda, studiVZ Ltd. 124
How to get started with Continuous Integration
• Keep everything under version control.
Max Horváth & Wojciech Duda, studiVZ Ltd. 125
How to get started with Continuous Integration
• Keep everything under version control. • Automation does not work if pieces are missing.
Max Horváth & Wojciech Duda, studiVZ Ltd. 126
How to get started with Continuous Integration
• Keep everything under version control. • Automation does not work if pieces are missing.
• Clone your production environment:
Max Horváth & Wojciech Duda, studiVZ Ltd. 127
How to get started with Continuous Integration
• Keep everything under version control. • Automation does not work if pieces are missing.
• Clone your production environment: • At least versions of main components should match.
Max Horváth & Wojciech Duda, studiVZ Ltd. 128
How to get started with Continuous Integration
• Keep everything under version control. • Automation does not work if pieces are missing.
• Clone your production environment: • At least versions of main components should match.
• Automate the deployment of that environment.
Max Horváth & Wojciech Duda, studiVZ Ltd. 129
How to get started with Continuous Integration
• Continuous Integration is more than setting up a CI server.
Max Horváth & Wojciech Duda, studiVZ Ltd. 130
How to get started with Continuous Integration
• Continuous Integration is more than setting up a CI server.
• You have to live Continuous Integration.
Max Horváth & Wojciech Duda, studiVZ Ltd. 131
How to get started with Continuous Integration
• Continuous Integration is more than setting up a CI server.
• You have to live Continuous Integration.
• Keep in mind: CI is a customized , tailored process:
Max Horváth & Wojciech Duda, studiVZ Ltd. 132
How to get started with Continuous Integration
• Continuous Integration is more than setting up a CI server.
• You have to live Continuous Integration.
• Keep in mind: CI is a customized , tailored process: • CI in company A is not necessarily the same as in
company B.
Max Horváth & Wojciech Duda, studiVZ Ltd. 133
Benefits of Continuous Integration
• Enforce your build policies.
Max Horváth & Wojciech Duda, studiVZ Ltd. 134
Benefits of Continuous Integration
• Enforce your build policies.
• Inform developers of build statuses.
Max Horváth & Wojciech Duda, studiVZ Ltd. 135
Benefits of Continuous Integration
• Enforce your build policies.
• Inform developers of build statuses.
• Make builds and reports available to everyone.
Max Horváth & Wojciech Duda, studiVZ Ltd. 136
Benefits of Continuous Integration
• Enforce your build policies.
• Inform developers of build statuses.
• Make builds and reports available to everyone.
• Automate deployments to various testing environments.
Max Horváth & Wojciech Duda, studiVZ Ltd. 137
Your team and Continuous Integration
• The team should always see the monitor.
Max Horváth & Wojciech Duda, studiVZ Ltd. 138
Your team and Continuous Integration
• The team should always see the monitor.
• Transparency is important – Developers need to see status and bugs.
Max Horváth & Wojciech Duda, studiVZ Ltd. 139
Your team and Continuous Integration
• The team should always see the monitor.
• Transparency is important – Developers need to see status and bugs.
• Define some punishment for breaking the build.
Max Horváth & Wojciech Duda, studiVZ Ltd. 140
CruiseControl / phpUnderControl
• The mother of all CI servers.
Max Horváth & Wojciech Duda, studiVZ Ltd. 141
CruiseControl / phpUnderControl
• The mother of all CI servers.
• phpUnderControl features:
Max Horváth & Wojciech Duda, studiVZ Ltd. 142
CruiseControl / phpUnderControl
• The mother of all CI servers.
• phpUnderControl features: • Support for different code metric views.
Max Horváth & Wojciech Duda, studiVZ Ltd. 143
CruiseControl / phpUnderControl
• The mother of all CI servers.
• phpUnderControl features: • Support for different code metric views.
• It includes, but is not limited to, plugins for email notification, Ant, Phing, and various source control tools.
Max Horváth & Wojciech Duda, studiVZ Ltd. 144
CruiseControl / phpUnderControl
• The mother of all CI servers.
• phpUnderControl features: • Support for different code metric views.
• It includes, but is not limited to, plugins for email notification, Ant, Phing, and various source control tools.
• A web interface is provided to view the details of the current and previous builds.
Max Horváth & Wojciech Duda, studiVZ Ltd. 145
CruiseControl / phpUnderControl
• phpUnderControl is customization of CruiseControl that caters to the needs of PHP projects:
Max Horváth & Wojciech Duda, studiVZ Ltd. 146
CruiseControl / phpUnderControl
• phpUnderControl is customization of CruiseControl that caters to the needs of PHP projects: • PHPUnit
Max Horváth & Wojciech Duda, studiVZ Ltd. 147
CruiseControl / phpUnderControl
• phpUnderControl is customization of CruiseControl that caters to the needs of PHP projects: • PHPUnit
• PHPDocumentor
Max Horváth & Wojciech Duda, studiVZ Ltd. 148
CruiseControl / phpUnderControl
• phpUnderControl is customization of CruiseControl that caters to the needs of PHP projects: • PHPUnit
• PHPDocumentor
• PHP_CodeSniffer
Max Horváth & Wojciech Duda, studiVZ Ltd. 149
CruiseControl / phpUnderControl
• phpUnderControl is customization of CruiseControl that caters to the needs of PHP projects: • PHPUnit
• PHPDocumentor
• PHP_CodeSniffer
• PHP_Depend
Max Horváth & Wojciech Duda, studiVZ Ltd. 150
CruiseControl / phpUnderControl
• phpUnderControl is customization of CruiseControl that caters to the needs of PHP projects: • PHPUnit
• PHPDocumentor
• PHP_CodeSniffer
• PHP_Depend • PHP_CompatInfo
Max Horváth & Wojciech Duda, studiVZ Ltd. 151
CruiseControl / phpUnderControl
• phpUnderControl is customization of CruiseControl that caters to the needs of PHP projects: • PHPUnit
• PHPDocumentor
• PHP_CodeSniffer
• PHP_Depend • PHP_CompatInfo
• … your own tools
Max Horváth & Wojciech Duda, studiVZ Ltd. 152
Code metrics
• A software metric is a measured value for a quality of software.
Max Horváth & Wojciech Duda, studiVZ Ltd. 153
Code metrics
• A software metric is a measure of some property of a piece of software or is specifications.
• You cannot control what you cannot measure.
Max Horváth & Wojciech Duda, studiVZ Ltd. 154
Measuring code quality
• Lines of code.
Max Horváth & Wojciech Duda, studiVZ Ltd. 155
Measuring code quality
• Lines of code.
• Text-based metric for code size
Max Horváth & Wojciech Duda, studiVZ Ltd. 156
Measuring code quality
• Lines of code.
• Text-based metric for code size
• Several definitions:
Max Horváth & Wojciech Duda, studiVZ Ltd. 157
Measuring code quality
• Lines of code.
• Text-based metric for code size
• Several definitions: • Lines of Code (LOC)
Max Horváth & Wojciech Duda, studiVZ Ltd. 158
Measuring code quality
• Lines of code.
• Text-based metric for code size
• Several definitions: • Lines of Code (LOC)
• Comment Lines of Code (CLOC)
Max Horváth & Wojciech Duda, studiVZ Ltd. 159
Measuring code quality
• Lines of code.
• Text-based metric for code size
• Several definitions: • Lines of Code (LOC)
• Comment Lines of Code (CLOC)
• Non-Comment Lines of Code (NCLOC)
Max Horváth & Wojciech Duda, studiVZ Ltd. 160
Measuring code quality
• Lines of code.
• Text-based metric for code size
• Several definitions: • Lines of Code (LOC)
• Comment Lines of Code (CLOC)
• Non-Comment Lines of Code (NCLOC)
• Executable Lines of Code (ELOC)
Max Horváth & Wojciech Duda, studiVZ Ltd. 161
Measuring code quality
• Lines of code.
• Ratios are of interest:
Max Horváth & Wojciech Duda, studiVZ Ltd. 162
Measuring code quality
• Lines of code.
• Ratios are of interest: • I.e. ratio of CLOC / (E)LOC
Max Horváth & Wojciech Duda, studiVZ Ltd. 163
Measuring code quality
• Code coverage
Max Horváth & Wojciech Duda, studiVZ Ltd. 164
Measuring code quality
• Code coverage
• Tells how many lines of code are covered by your unit tests.
Max Horváth & Wojciech Duda, studiVZ Ltd. 165
Measuring code quality
• Code coverage
• Tells how many lines of code are covered by your unit tests.
• 100% Code coverage is a required criteria, but it doesn’t tell if your tests do test your code sufficient enough.
Max Horváth & Wojciech Duda, studiVZ Ltd. 166
Measuring code quality
• Code coverage
• Which statements, branches, and paths are executed when the tests run?
Max Horváth & Wojciech Duda, studiVZ Ltd. 167
Measuring code quality
• Code coverage
• Which statements, branches, and paths are executed when the tests run? • C0-Coverage: Statement Coverage
Max Horváth & Wojciech Duda, studiVZ Ltd. 168
Measuring code quality
• Code coverage
• Which statements, branches, and paths are executed when the tests run? • C0-Coverage: Statement Coverage
• C1-Coverage: Branch Coverage
Max Horváth & Wojciech Duda, studiVZ Ltd. 169
Measuring code quality
• Code coverage
• Which statements, branches, and paths are executed when the tests run? • C0-Coverage: Statement Coverage
• C1-Coverage: Branch Coverage
• C2-Coverage: Path Coverage
Max Horváth & Wojciech Duda, studiVZ Ltd. 170
Measuring code quality
• Cyclomatic Complexity
Max Horváth & Wojciech Duda, studiVZ Ltd. 171
Measuring code quality
• Cyclomatic Complexity
• Metric for the complexity of a code unit
Max Horváth & Wojciech Duda, studiVZ Ltd. 172
Measuring code quality
• Cyclomatic Complexity
• Metric for the complexity of a code unit
• Counting the branching points:
Max Horváth & Wojciech Duda, studiVZ Ltd. 173
Measuring code quality
• Cyclomatic Complexity
• Metric for the complexity of a code unit
• Counting the branching points: • if, for, foreach, while, case, catch, &&, ||, ternary
operator (?:)
Max Horváth & Wojciech Duda, studiVZ Ltd. 174
Measuring code quality
• Cyclomatic Complexity
• Interpretation:
Max Horváth & Wojciech Duda, studiVZ Ltd. 175
Measuring code quality
• Cyclomatic Complexity
• Interpretation: • Higher complexity leads to more errors
Max Horváth & Wojciech Duda, studiVZ Ltd. 176
Measuring code quality
• Cyclomatic Complexity
• Interpretation: • Higher complexity leads to more errors
• Higher complexity makes testing harder
Max Horváth & Wojciech Duda, studiVZ Ltd. 177
Analyze your code quality
• Use the following tools:
Max Horváth & Wojciech Duda, studiVZ Ltd. 178
Analyze your code quality
• Use the following tools: • PHPUnit
Max Horváth & Wojciech Duda, studiVZ Ltd. 179
Analyze your code quality
• Use the following tools: • PHPUnit
• PhpDocumentor • Generate API documentation
Max Horváth & Wojciech Duda, studiVZ Ltd. 180
Analyze your code quality
• Use the following tools: • PHPUnit
• PhpDocumentor • Generate API documentation
• PHP_CodeSniffer • Coding Style • Correctness of API Documentation
Max Horváth & Wojciech Duda, studiVZ Ltd. 181
Analyze your code quality
• Use the following tools: • PHPUnit
• PhpDocumentor • Generate API documentation
• PHP_CodeSniffer • Coding Style • Correctness of API Documentation
• Php_Depend • Analyze code dependencies
Max Horváth & Wojciech Duda, studiVZ Ltd. 182
Analyze your code quality
• Use the following tools: • PHPUnit
• PhpDocumentor • Generate API documentation
• PHP_CodeSniffer • Coding Style • Correctness of API Documentation
• Php_Depend • Analyze code dependencies
• Zend Code Analyzer • Unreachable Code • Unused parameters and variables
Max Horváth & Wojciech Duda, studiVZ Ltd. 183
Collect code analysis
• Either collect all analysis results in your CI server or build a tool to let your developers see a wrap-up of all analyzer results.
Max Horváth & Wojciech Duda, studiVZ Ltd. 184
Collect code analysis
• Either collect all analysis results in your CI server or build a tool to let your developers see a wrap-up of all analyzer results.
• Categorize code analysis results in different levels.
Max Horváth & Wojciech Duda, studiVZ Ltd. 185
Collect code analysis
• Either collect all analysis results in your CI server or build a tool to let your developers see a wrap-up of all analyzer results.
• Categorize code analysis results in different levels.
• Higher rated issues need to be fixed first.
Max Horváth & Wojciech Duda, studiVZ Ltd. 186
Continual metrics
• Metric values are not definite values.
Max Horváth & Wojciech Duda, studiVZ Ltd. 187
Continual metrics
• Metric values are not definite values.
• Wire metrics and analysis into continuous integration.
Max Horváth & Wojciech Duda, studiVZ Ltd. 188
Continual metrics
• Metric values are not definite values.
• Wire metrics and analysis into continuous integration.
• Discuss on iteration boundaries.
Max Horváth & Wojciech Duda, studiVZ Ltd. 189
Continual metrics
• Metric values are not definite values.
• Wire metrics and analysis into continuous integration.
• Discuss on iteration boundaries.
• It is important to note that variations of metrics need more attention than its values.
Max Horváth & Wojciech Duda, studiVZ Ltd. 190
Issue tracking
• Issues and tasks need to be:
Max Horváth & Wojciech Duda, studiVZ Ltd. 191
Issue tracking
• Issues and tasks need to be: • organized
Max Horváth & Wojciech Duda, studiVZ Ltd. 192
Issue tracking
• Issues and tasks need to be: • organized
• allocated
Max Horváth & Wojciech Duda, studiVZ Ltd. 193
Issue tracking
• Issues and tasks need to be: • organized
• allocated
• scheduled
Max Horváth & Wojciech Duda, studiVZ Ltd. 194
Issue tracking
• Issues and tasks need to be: • organized
• allocated
• scheduled
• tracked
Max Horváth & Wojciech Duda, studiVZ Ltd. 195
Issue tracking
• Issues and tasks need to be: • organized
• allocated
• scheduled
• tracked
• That’s where an issue tracker comes into play.
Max Horváth & Wojciech Duda, studiVZ Ltd. 196
Issue tracking
• An Issue tracker is a software application that is designed to help quality assurance and programmers keep track of reported software bugs in their work.
Max Horváth & Wojciech Duda, studiVZ Ltd. 197
Issue tracking
• An Issue tracker is a software application that is designed to help quality assurance and programmers keep track of reported software bugs in their work.
• Issues may include the time a bug was reported, its severity, the erroneous program behavior, and details on how to reproduce the bug; as well as any programmers who may be working on fixing it.
Max Horváth & Wojciech Duda, studiVZ Ltd. 198
Issue tracking
• Typical issue tracking systems support the concept of the life cycle for a bug/feature which is tracked through the status assigned to the issue.
Max Horváth & Wojciech Duda, studiVZ Ltd. 199
Issue tracking
• Typical issue tracking systems support the concept of the life cycle for a bug/feature which is tracked through the status assigned to the issue.
• You can also assign the expected time needed to fix a bug so you can plan how many issues may be solved in one iteration.
Max Horváth & Wojciech Duda, studiVZ Ltd. 200
Issue tracking
• If supported by your issue tracker and version control system use a hook script to let developers mark commits to an active issue.
Max Horváth & Wojciech Duda, studiVZ Ltd. 201
Issue tracking
• If supported by your issue tracker and version control system use a hook script to let developers mark commits to an active issue.
• If you require an issue ID in the commit message developers are forced to use the issue tracking software.
Max Horváth & Wojciech Duda, studiVZ Ltd. 202
Pair Programming
• Helps write better code by dividing responsibility between two people.
Max Horváth & Wojciech Duda, studiVZ Ltd. 203
Pair Programming
• Helps write better code by dividing responsibility between two people.
• Observer – strategy
Max Horváth & Wojciech Duda, studiVZ Ltd. 204
Pair Programming
• Helps write better code by dividing responsibility between two people.
• Observer – strategy • Improvements
Max Horváth & Wojciech Duda, studiVZ Ltd. 205
Pair Programming
• Helps write better code by dividing responsibility between two people.
• Observer – strategy • Improvements
• Future problems
Max Horváth & Wojciech Duda, studiVZ Ltd. 206
Pair Programming
• Helps write better code by dividing responsibility between two people.
• Observer – strategy • Improvements
• Future problems
• Driver – tactics
Max Horváth & Wojciech Duda, studiVZ Ltd. 207
Pair Programming
• Helps write better code by dividing responsibility between two people.
• Observer – strategy • Improvements
• Future problems
• Driver – tactics • Completing the task
Max Horváth & Wojciech Duda, studiVZ Ltd. 208
Pair Programming
• Observer functions as safety net and guide.
Max Horváth & Wojciech Duda, studiVZ Ltd. 209
Pair Programming
• Observer functions as safety net and guide.
• Observer and Driver can exchange roles.
Max Horváth & Wojciech Duda, studiVZ Ltd. 210
Pair Programming
• Benefits
Max Horváth & Wojciech Duda, studiVZ Ltd. 211
Pair Programming
• Benefits • Design quality
Max Horváth & Wojciech Duda, studiVZ Ltd. 212
Pair Programming
• Benefits • Design quality
• Reduced cost due to fewer defects
Max Horváth & Wojciech Duda, studiVZ Ltd. 213
Pair Programming
• Benefits • Design quality
• Reduced cost due to fewer defects
• Learning & Training
Max Horváth & Wojciech Duda, studiVZ Ltd. 214
Pair Programming
• Benefits • Design quality
• Reduced cost due to fewer defects
• Learning & Training
• Shared knowledge of product
Max Horváth & Wojciech Duda, studiVZ Ltd. 215
Pair Programming
• Benefits • Design quality
• Reduced cost due to fewer defects
• Learning & Training
• Shared knowledge of product
• Improved discipline and time management
Max Horváth & Wojciech Duda, studiVZ Ltd. 216
Pair Review/Buddy System
• Review your code/tests regularly with different partners in short meetings.
Max Horváth & Wojciech Duda, studiVZ Ltd. 217
Pair Review/Buddy System
• Review your code/tests regularly with different partners in short meetings.
• Solve difficult problems.
Max Horváth & Wojciech Duda, studiVZ Ltd. 218
Pair Review/Buddy System
• Review your code/tests regularly with different partners in short meetings.
• Solve difficult problems.
• Exchange best practices, ideas, elegant solutions.
Max Horváth & Wojciech Duda, studiVZ Ltd. 219
Pair Review/Buddy System
• Review your code/tests regularly with different partners in short meetings.
• Solve difficult problems.
• Exchange best practices, ideas, elegant solutions.
• Just spread knowledge.
Max Horváth & Wojciech Duda, studiVZ Ltd. 220
Code Reviews: Too time consuming?
• Code Reviews may seem to be to be too time consuming when time pressure increases.
Max Horváth & Wojciech Duda, studiVZ Ltd. 221
Code Reviews: Too time consuming?
• Code Reviews may seem to be to be too time consuming when time pressure increases.
• Without specific approaches to manage the reviews they might be in fact too time consuming.
Max Horváth & Wojciech Duda, studiVZ Ltd. 222
Code Reviews: Too time consuming?
• Code Reviews may seem to be to be too time consuming when time pressure increases.
• Without specific approaches to manage the reviews they might be in fact too time consuming.
• These issues are simply problems with some review processes.
Max Horváth & Wojciech Duda, studiVZ Ltd. 223
Code Reviews: Your return on investment
• Regular peer review is a proven process with demonstrable ROI.
Max Horváth & Wojciech Duda, studiVZ Ltd. 224
Code Reviews: Your return on investment
• Regular peer review is a proven process with demonstrable ROI.
• The benefits vary from team to team but commonly include:
Max Horváth & Wojciech Duda, studiVZ Ltd. 225
Code Reviews: Your return on investment
• Regular peer review is a proven process with demonstrable ROI.
• The benefits vary from team to team but commonly include: • Identifying bugs and defects early.
Max Horváth & Wojciech Duda, studiVZ Ltd. 226
Code Reviews: Your return on investment
• Regular peer review is a proven process with demonstrable ROI.
• The benefits vary from team to team but commonly include: • Identifying bugs and defects early.
• Sharing expertise and encouraging knowledge transfer.
Max Horváth & Wojciech Duda, studiVZ Ltd. 227
Code Reviews: Your return on investment
• Regular peer review is a proven process with demonstrable ROI.
• The benefits vary from team to team but commonly include: • Identifying bugs and defects early.
• Sharing expertise and encouraging knowledge transfer.
• Encouraging adherence to internal standards and style conventions.
Max Horváth & Wojciech Duda, studiVZ Ltd. 228
Code Reviews: Your return on investment
• Regular peer review is a proven process with demonstrable ROI.
• The benefits vary from team to team but commonly include: • Identifying bugs and defects early.
• Sharing expertise and encouraging knowledge transfer.
• Encouraging adherence to internal standards and style conventions.
• Identifying individual strengths and weaknesses.
Max Horváth & Wojciech Duda, studiVZ Ltd. 229
Code Reviews: Processes are your friend!
• Defining Code Review Iterations is as important as defining how to do them!
Max Horváth & Wojciech Duda, studiVZ Ltd. 230
Code Reviews: Processes are your friend!
• Defining Code Review Iterations is as important as defining how to do them!
• Don‘t skip any Code Reviews as changes to your source code are more expensive the later you do them!
Max Horváth & Wojciech Duda, studiVZ Ltd. 231
Code Reviews: What are the best tools?
• A human being!
Max Horváth & Wojciech Duda, studiVZ Ltd. 232
Code Reviews: What are the best tools?
• A human being!
• There is pair programming – why not make it pair reviewing?
Max Horváth & Wojciech Duda, studiVZ Ltd. 233
Code Reviews: What are the best tools?
• A human being!
• There is pair programming – why not make it pair reviewing?
• An initial Code Review is faster and more reliable to be done, if both Reviewer and Developer can communicate directly with each other.
Max Horváth & Wojciech Duda, studiVZ Ltd. 234
Code Reviews: Web-based tool: Crucible
• Crucible is a web based Code Review tool by Atlassian Software Systems Pty Ltd.
Max Horváth & Wojciech Duda, studiVZ Ltd. 235
Code Reviews: Web-based tool: Crucible
• Crucible is a web based Code Review tool by Atlassian Software Systems Pty Ltd.
• Commercial tool – but available free of charge for Open Source projects.
Max Horváth & Wojciech Duda, studiVZ Ltd. 236
Code Reviews: Web-based tool: Crucible
• Crucible is a web based Code Review tool by Atlassian Software Systems Pty Ltd.
• Commercial tool – but available free of charge for Open Source projects.
• Crucible's mission is to streamline the process aspects so development teams can access the benefits.
Max Horváth & Wojciech Duda, studiVZ Ltd. 237
Code Reviews: Web-based tool: Crucible
• This is how it looks like:
Max Horváth & Wojciech Duda, studiVZ Ltd. 238
Code Reviews: Web-based tool: Crucible
• You create a new review from your version control system:
Max Horváth & Wojciech Duda, studiVZ Ltd. 239
Code Reviews: Web-based tool: Crucible
• Your review is open:
Max Horváth & Wojciech Duda, studiVZ Ltd. 240
Code Reviews: Web-based tool: Crucible
• Both reviewer and developer can communicate about the code:
Max Horváth & Wojciech Duda, studiVZ Ltd. 241
Code Reviews: Web-based tool: Crucible
• Both reviewer and developer can communicate about the code:
Max Horváth & Wojciech Duda, studiVZ Ltd. 242
Code Reviews: Web-based tool: Crucible
• Commits can either be done after committing to a version control system or using pre-commit diff files.
Max Horváth & Wojciech Duda, studiVZ Ltd. 243
Testing your web application
• Now that we covered the code analysis what about the product you are creating?
Max Horváth & Wojciech Duda, studiVZ Ltd. 244
Testing your web application
• Now that we covered the code analysis what about the product you are creating?
• We want to have automated tests for our web application that do regression testing, compare results and check whether our application runs flawlessly in all of our supported browsers.
Max Horváth & Wojciech Duda, studiVZ Ltd. 245
Testing your web application
• Challenges:
Max Horváth & Wojciech Duda, studiVZ Ltd. 246
Testing your web application
• Challenges: • Modern GUI app testing (AJAX & Co.)
Max Horváth & Wojciech Duda, studiVZ Ltd. 247
Testing your web application
• Challenges: • Modern GUI app testing (AJAX & Co.)
• OS / browser / platform
Max Horváth & Wojciech Duda, studiVZ Ltd. 248
Testing your web application
• Challenges: • Modern GUI app testing (AJAX & Co.)
• OS / browser / platform
• JavaScript engines / versions / levels
Max Horváth & Wojciech Duda, studiVZ Ltd. 249
Testing your web application
• There is the Selenium product family:
Max Horváth & Wojciech Duda, studiVZ Ltd. 250
Testing your web application
• There is the Selenium product family:
• Selenium Core
Max Horváth & Wojciech Duda, studiVZ Ltd. 251
Testing your web application
• There is the Selenium product family:
• Selenium Core
• Selenium IDE
Max Horváth & Wojciech Duda, studiVZ Ltd. 252
Testing your web application
• There is the Selenium product family:
• Selenium Core
• Selenium IDE
• Selenium RC
Max Horváth & Wojciech Duda, studiVZ Ltd. 253
Testing your web application
• Selenium Core:
Max Horváth & Wojciech Duda, studiVZ Ltd. 254
Testing your web application
• Selenium Core: • JavaScript engine
Max Horváth & Wojciech Duda, studiVZ Ltd. 255
Testing your web application
• Selenium Core: • JavaScript engine
• runs in browser
Max Horváth & Wojciech Duda, studiVZ Ltd. 256
Testing your web application
• Selenium Core: • JavaScript engine
• runs in browser
• controls web application in other frame
Max Horváth & Wojciech Duda, studiVZ Ltd. 257
Testing your web application
• Selenium Core: • JavaScript engine
• runs in browser
• controls web application in other frame
• exposes its API
Max Horváth & Wojciech Duda, studiVZ Ltd. 258
Testing your web application
• Selenium IDE:
Max Horváth & Wojciech Duda, studiVZ Ltd. 259
Testing your web application
• Selenium IDE: • Firefox plugin
Max Horváth & Wojciech Duda, studiVZ Ltd. 260
Testing your web application
• Selenium IDE: • Firefox plugin
• directly communicates with Selenium Core in the browser
Max Horváth & Wojciech Duda, studiVZ Ltd. 261
Testing your web application
• Selenium IDE: • Firefox plugin
• directly communicates with Selenium Core in the browser
• Records / replays scenarios
Max Horváth & Wojciech Duda, studiVZ Ltd. 262
Testing your web application
• Selenium IDE: • Firefox plugin
• directly communicates with Selenium Core in the browser
• Records / replays scenarios
• great to get started (interactively)
Max Horváth & Wojciech Duda, studiVZ Ltd. 263
Testing your web application
• Selenium RC:
Max Horváth & Wojciech Duda, studiVZ Ltd. 264
Testing your web application
• Selenium RC: • remote control
Max Horváth & Wojciech Duda, studiVZ Ltd. 265
Testing your web application
• Selenium RC: • remote control
• server component and proxy
Max Horváth & Wojciech Duda, studiVZ Ltd. 266
Testing your web application
• Selenium RC: • remote control
• server component and proxy
• allows test automation in various ways
Max Horváth & Wojciech Duda, studiVZ Ltd. 267
Testing your web application
• Selenium RC: • remote control
• server component and proxy
• allows test automation in various ways
• has a capability called Selenium Grid, to drive multiple instances of the Selenium RC server
Max Horváth & Wojciech Duda, studiVZ Ltd. 268
Advantages of Selenium
• Tests run directly in browser
Max Horváth & Wojciech Duda, studiVZ Ltd. 269
Advantages of Selenium
• Tests run directly in browser
• Implemented entirely using browser technologies:
Max Horváth & Wojciech Duda, studiVZ Ltd. 270
Advantages of Selenium
• Tests run directly in browser
• Implemented entirely using browser technologies: • JavaScript
Max Horváth & Wojciech Duda, studiVZ Ltd. 271
Advantages of Selenium
• Tests run directly in browser
• Implemented entirely using browser technologies: • JavaScript
• DHTML
Max Horváth & Wojciech Duda, studiVZ Ltd. 272
Advantages of Selenium
• Tests run directly in browser
• Implemented entirely using browser technologies: • JavaScript
• DHTML
• Frames
Max Horváth & Wojciech Duda, studiVZ Ltd. 273
Advantages of Selenium
• Selenium interacts with the DOM served to the browser
Max Horváth & Wojciech Duda, studiVZ Ltd. 274
Advantages of Selenium
• Selenium interacts with the DOM served to the browser
• Selenium is agnostic of server side technology
Max Horváth & Wojciech Duda, studiVZ Ltd. 275
Concepts of Selenium
• Element Locators:
Max Horváth & Wojciech Duda, studiVZ Ltd. 276
Concepts of Selenium
• Element Locators: • specify HTML elements
Max Horváth & Wojciech Duda, studiVZ Ltd. 277
Concepts of Selenium
• Element Locators: • specify HTML elements
• Patterns:
Max Horváth & Wojciech Duda, studiVZ Ltd. 278
Concepts of Selenium
• Element Locators: • specify HTML elements
• Patterns: • for pattern matching values
Max Horváth & Wojciech Duda, studiVZ Ltd. 279
Concepts of Selenium
• Element Locators: • specify HTML elements
• Patterns: • for pattern matching values
• Actions:
Max Horváth & Wojciech Duda, studiVZ Ltd. 280
Concepts of Selenium
• Element Locators: • specify HTML elements
• Patterns: • for pattern matching values
• Actions: • manipulate application state
Max Horváth & Wojciech Duda, studiVZ Ltd. 281
Concepts of Selenium
• Accessors:
Max Horváth & Wojciech Duda, studiVZ Ltd. 282
Concepts of Selenium
• Accessors: • store results in variables
Max Horváth & Wojciech Duda, studiVZ Ltd. 283
Concepts of Selenium
• Accessors: • store results in variables
• Assertions:
Max Horváth & Wojciech Duda, studiVZ Ltd. 284
Concepts of Selenium
• Accessors: • store results in variables
• Assertions: • verify that the application is in a certain state
Max Horváth & Wojciech Duda, studiVZ Ltd. 285
Useful Selenium Tools
• For Firefox!
Max Horváth & Wojciech Duda, studiVZ Ltd. 286
Useful Selenium Tools
• For Firefox! • Web Developer Toolbar
Max Horváth & Wojciech Duda, studiVZ Ltd. 287
Useful Selenium Tools
• For Firefox! • Web Developer Toolbar
• Firebug
Max Horváth & Wojciech Duda, studiVZ Ltd. 288
Useful Selenium Tools
• For Firefox! • Web Developer Toolbar
• Firebug
• XPather
Max Horváth & Wojciech Duda, studiVZ Ltd. 289
Useful Selenium Tools
• For Firefox! • Web Developer Toolbar
• Firebug
• XPather
• Xpath Checker
Max Horváth & Wojciech Duda, studiVZ Ltd. 290
Maintaining Selenium Tests
• Test code is just like production code.
Max Horváth & Wojciech Duda, studiVZ Ltd. 291
Maintaining Selenium Tests
• Test code is just like production code.
• Needs refactoring, abstraction.
Max Horváth & Wojciech Duda, studiVZ Ltd. 292
Maintaining Selenium Tests
• Test code is just like production code.
• Needs refactoring, abstraction.
• Try to maintain meaning – in test names, variable names, comments, and test suite organization.
Max Horváth & Wojciech Duda, studiVZ Ltd. 293
Test Fragility
• Tests are fragile.
Max Horváth & Wojciech Duda, studiVZ Ltd. 294
Test Fragility
• Tests are fragile.
• They break when your code changes.
Max Horváth & Wojciech Duda, studiVZ Ltd. 295
Test Fragility
• Tests are fragile.
• They break when your code changes.
• Stabilize them!
Max Horváth & Wojciech Duda, studiVZ Ltd. 296
Test Fragility
• Minimize influences:
Max Horváth & Wojciech Duda, studiVZ Ltd. 297
Test Fragility
• Minimize influences: • Use Mock Objects
Max Horváth & Wojciech Duda, studiVZ Ltd. 298
Test Fragility
• Minimize influences: • Use Mock Objects
• Recreate dependencies – SetUp
Max Horváth & Wojciech Duda, studiVZ Ltd. 299
Test Fragility
• Minimize influences: • Use Mock Objects
• Recreate dependencies – SetUp
• Clean up your test environment – TearDown
Max Horváth & Wojciech Duda, studiVZ Ltd. 300
Test Fragility
• Minimize influences: • Use Mock Objects
• Recreate dependencies – SetUp
• Clean up your test environment – TearDown
• Use non-fragile locators
Max Horváth & Wojciech Duda, studiVZ Ltd. 301
Mock Objects
• Let Unit Tests test the code not the dependencies:
Max Horváth & Wojciech Duda, studiVZ Ltd. 302
Mock Objects
• Let Unit Tests test the code not the dependencies: • Simulate complex objects
Max Horváth & Wojciech Duda, studiVZ Ltd. 303
Mock Objects
• Let Unit Tests test the code not the dependencies: • Simulate complex objects
• Simulate database connections
Max Horváth & Wojciech Duda, studiVZ Ltd. 304
Mock Objects
• Let Unit Tests test the code not the dependencies: • Simulate complex objects
• Simulate database connections
• Simulate uncontrollable variables – time, temperature etc.
Max Horváth & Wojciech Duda, studiVZ Ltd. 305
Mock Objects
• Let Unit Tests test the code not the dependencies: • Simulate complex objects
• Simulate database connections
• Simulate uncontrollable variables – time, temperature etc.
• Simulate objects that do not exist yet
Max Horváth & Wojciech Duda, studiVZ Ltd. 306
Test SetUp
• Use the SetUp method to recreate testable states for functional tests:
Max Horváth & Wojciech Duda, studiVZ Ltd. 307
Test SetUp
• Use the SetUp method to recreate testable states for functional tests: • Use methods that are unit-tested
Max Horváth & Wojciech Duda, studiVZ Ltd. 308
Test SetUp
• Use the SetUp method to recreate testable states for functional tests: • Use methods that are unit-tested
• Fill database with necessary data
Max Horváth & Wojciech Duda, studiVZ Ltd. 309
Test SetUp
• Use the SetUp method to recreate testable states for functional tests: • Use methods that are unit-tested
• Fill database with necessary data
• Tests can be run on additional fresh test environments.
Max Horváth & Wojciech Duda, studiVZ Ltd. 310
Test TearDown
• Use the TearDown method to clean up your test environment:
Max Horváth & Wojciech Duda, studiVZ Ltd. 311
Test TearDown
• Use the TearDown method to clean up your test environment: • Remove no longer necessary database entries
Max Horváth & Wojciech Duda, studiVZ Ltd. 312
Test TearDown
• Use the TearDown method to clean up your test environment: • Remove no longer necessary database entries
• Clean caches, no longer needed log files
Max Horváth & Wojciech Duda, studiVZ Ltd. 313
Test TearDown
• Use the TearDown method to clean up your test environment: • Remove no longer necessary database entries
• Clean caches, no longer needed log files
• Reduces uncontrollable influences on tests.
Max Horváth & Wojciech Duda, studiVZ Ltd. 314
Test TearDown
• Use the TearDown method to clean up your test environment: • Remove no longer necessary database entries
• Clean caches, no longer needed log files
• Reduces uncontrollable influences on tests.
• Keeps test execution quick.
Max Horváth & Wojciech Duda, studiVZ Ltd. 315
Non-Fragile Locators
• When using Selenium, use non-fragile locators:
Max Horváth & Wojciech Duda, studiVZ Ltd. 316
Non-Fragile Locators
• When using Selenium, use non-fragile locators: • Keep locators independent from DOM
Max Horváth & Wojciech Duda, studiVZ Ltd. 317
Non-Fragile Locators
• When using Selenium, use non-fragile locators: • Keep locators independent from DOM
• Use ids or names
Max Horváth & Wojciech Duda, studiVZ Ltd. 318
Non-Fragile Locators
• When using Selenium, use non-fragile locators: • Keep locators independent from DOM
• Use ids or names
• Use wildcards where applicable
Max Horváth & Wojciech Duda, studiVZ Ltd. 319
Non-Fragile Locators
• When using Selenium, use non-fragile locators: • Keep locators independent from DOM
• Use ids or names
• Use wildcards where applicable
• Dom-Locators (JavaScript expressions)
Max Horváth & Wojciech Duda, studiVZ Ltd. 320
Non-Fragile Locators
• When using Selenium, use non-fragile locators: • Keep locators independent from DOM
• Use ids or names
• Use wildcards where applicable
• Dom-Locators (JavaScript expressions)
• Page elements can be moved around, but your tests will still work.
Max Horváth & Wojciech Duda, studiVZ Ltd. 321
Release process
• Make sure you have a standard procedure for releasing / launching new versions of your product.
Max Horváth & Wojciech Duda, studiVZ Ltd. 322
Release process
• Make sure you have a standard procedure for releasing / launching new versions of your product.
• Assign responsibility or have a dedicated release management.
Max Horváth & Wojciech Duda, studiVZ Ltd. 323
Release process
• Make sure you have a standard procedure for releasing / launching new versions of your product.
• Assign responsibility or have a dedicated release management.
• Use the tools to the fullest.
Max Horváth & Wojciech Duda, studiVZ Ltd. 324
Release process: Versioning
• Use the versioning system – choose a stable revision and branch.
Max Horváth & Wojciech Duda, studiVZ Ltd. 325
Release process: Versioning
• Use the versioning system – choose a stable revision and branch.
• Development can continue without breaking the stable branch.
Max Horváth & Wojciech Duda, studiVZ Ltd. 326
Release process: Versioning
• If you find defects before next major version you can release stable versions by fixing them in the stable branch.
Max Horváth & Wojciech Duda, studiVZ Ltd. 327
Release process: Versioning
• If you find defects before next major version you can release stable versions by fixing them in the stable branch.
• Reintegrate fixes for bugs into the trunk so they are do not resurface.
Max Horváth & Wojciech Duda, studiVZ Ltd. 328
Release process: Delta Builder
• Tool to collect data about stable branch.
Max Horváth & Wojciech Duda, studiVZ Ltd. 329
Release process: Delta Builder
• Tool to collect data about stable branch.
• List changes to the codebase since last release.
Max Horváth & Wojciech Duda, studiVZ Ltd. 330
Release process: Delta Builder
• Tool to collect data about stable branch.
• List changes to the codebase since last release.
• Have developers review their changes.
Max Horváth & Wojciech Duda, studiVZ Ltd. 331
Release process: Delta Builder
• Highlight important pieces of your product – configuration files, most used libraries etc.
Max Horváth & Wojciech Duda, studiVZ Ltd. 332
Release process: Delta Builder
• Highlight important pieces of your product – configuration files, most used libraries etc.
• Match versioning data with issue tracker – have all completed tasks been really checked into the versioning system?
Max Horváth & Wojciech Duda, studiVZ Ltd. 333
Release process: Tests
• Dedicate time to test the stable branch before release.
Max Horváth & Wojciech Duda, studiVZ Ltd. 334
Release process: Tests
• Dedicate time to test the stable branch before release.
• Run all your tests as first step – last remaining defects found can be fixed before release.
Max Horváth & Wojciech Duda, studiVZ Ltd. 335
Release process: Tests
• Make sure all your tests pass.
Max Horváth & Wojciech Duda, studiVZ Ltd. 336
Release process: Tests
• Make sure all your tests pass.
• A manual test can help you assure stability – find defects not identifiable by tests.
Max Horváth & Wojciech Duda, studiVZ Ltd. 337
Release process: Checklists
• Minimize human error
Max Horváth & Wojciech Duda, studiVZ Ltd. 338
Release process: Checklists
• Minimize human error
• Collect all tasks for the release
Max Horváth & Wojciech Duda, studiVZ Ltd. 339
Release process: Checklists
• Minimize human error
• Collect all tasks for the release • Running tests
Max Horváth & Wojciech Duda, studiVZ Ltd. 340
Release process: Checklists
• Minimize human error
• Collect all tasks for the release • Running tests
• Building dependencies
Max Horváth & Wojciech Duda, studiVZ Ltd. 341
Release process: Checklists
• Minimize human error
• Collect all tasks for the release • Running tests
• Building dependencies
• Deploying all components of your product
Max Horváth & Wojciech Duda, studiVZ Ltd. 342
Release process: Checklists
• Minimize human error
• Collect all tasks for the release • Running tests
• Building dependencies
• Deploying all components of your product
• Communicating – notify all those that need to know!
Max Horváth & Wojciech Duda, studiVZ Ltd. 343
Checklists in general
• Once again - Minimize human error!
Max Horváth & Wojciech Duda, studiVZ Ltd. 344
Checklists in general
• Once again - Minimize human error!
• All repetitive but complicated tasks:
Max Horváth & Wojciech Duda, studiVZ Ltd. 345
Checklists in general
• Once again - Minimize human error!
• All repetitive but complicated tasks: • Creating the skeleton of a new module
Max Horváth & Wojciech Duda, studiVZ Ltd. 346
Checklists in general
• Once again - Minimize human error!
• All repetitive but complicated tasks: • Creating the skeleton of a new module
• Manually check the functionality of a module
Max Horváth & Wojciech Duda, studiVZ Ltd. 347
Checklists in general
• Once again - Minimize human error!
• All repetitive but complicated tasks: • Creating the skeleton of a new module
• Manually check the functionality of a module
• Set up a new system ready to be shipped
Max Horváth & Wojciech Duda, studiVZ Ltd. 348
Checklists in general
• Once again - Minimize human error!
• All repetitive but complicated tasks: • Creating the skeleton of a new module
• Manually check the functionality of a module
• Set up a new system ready to be shipped
• …
Max Horváth & Wojciech Duda, studiVZ Ltd. 349
Questions?
Max Horváth & Wojciech Duda, studiVZ Ltd. 350
Thank You!