AUTOMATED TESTING IN ANGULARJSWITH JIM LYNCHAUGUST 2016

OVERVIEW

▸ Why write tests?

▸ UI / E2e Testing

▸ Unit Testing & TDD

▸ Acceptance Testing

WHY USE AUTOMATED TESTING AT ALL?

IF YOU DON’T HAVE AUTOMATED TESTS…

▸ Refactoring becomes dangerous since you can break something and not be alerted of it.

▸ The team must rely on heavily on manual testing to ensure quality which is expensive and lengthens each iteration.

▸ Anyone reading your code has nothing to help him or her understand the code besides that source code itself.

▸ Every push to production leaves you with fear and worry in the back of your mind that something might go wrong.

“OUR HIGHEST PRIORITY IS TO SATISFY THE CUSTOMER WITH EARLY AND CONTINUOUS DELIVERY OF VALUABLE SOFTWARE”

#1 Principle of Agile Manifesto 12 Principles

IT ALL COMES BACK TO THIS

MANUAL TESTING

▸ Can you think of a manual test that can’t be automated?

▸ Manual Testing is the Devil!

▸ Quickly becomes expensive, error prone, time consuming…

▸ Don’t rely only on “____ serve” for development because then you need to test manually!

▸ Used only for exploratory / usability testing and should NOT be a step in the deployment process.

AVOID IT!

PROTRACTOR

SETTING UP PROTRACTOR

‣ Create a config file:

‣ You might also need to update webdriver-manager:

‣ Install the library:

‣ And then run it:

AN EXAMPLE OF A PROTRACTOR TEST

PAGE OBJECTS

▸ These are the mysterious .po.js files.

▸ They allow you to separate the DOM selection code from the core test logic.

▸ Existed way before Protractor was even a thing. Tons of info has been written on the web about page objects.

▸ Example:

KEEP YOUR UI TESTS MAINTAINABLE!

PROTRACTOR IS AWESOME!

▸ Test on virtually any browser via Sauce Labs!

▸ Allows you to control Selenium WebDriver with friendly, JavaScript API.

▸ Tip: Start with functional testing of user interactions and flow through the app.

▸ Keep it real! Instead of mocking backend services, just run your tests on the real app and call it e2e testing too!

WE ARE SO LUCKY.

TEST ON ALL THE BROWSERS WITH SAUCE LABS!

ALL THE THINGS!

PROTRACTOR REPORTSInstall the package:

And then add this to your protractor config file exports.config object:

Then on every run you’ll get a generated report with screenshots!

FOR WHEN YOU CARE WHAT ACTUALLY HAPPENED

KARMA

SETTING UP KARMAInstall Karma:

Install any plugins you’ll need:

Create a config file:

And run it like this:

AN EXAMPLE OF A UNIT TEST

THE CODE COVERAGE REPORT

TEST DRIVEN DEVELOPMENT

IF YOU WANT TO REFACTOR, THE ESSENTIAL PRECONDITION IS HAVING SOLID TESTS.

Martin Fowlerauthor of Refactoring: Improving the Design of Existing Code

TDD: WHAT IS IT?

▸ TDD is all about writing unit tests for your source code.

▸ When you write the failing test first and then make it pass with source code you can be confident that the test is really covering the piece of functionality that it should.

▸ Ideally you should avoid writing testless lines of code.

▸ Write the tests first, inventing the names of controllers, services, and methods in your tests. Then generate them (or create them by hand), and implement them in a way that makes the tests pass.

LET THE TESTS DRIVE THE DEVELOPMENT (DUH)

WHY IS TDD SO AWESOME?

▸ Unit tests can expose runtime errors immediately after every change of the code which helps to keep quality high and prevent regression.

▸ Seeing a unit test fail, making a change in the source, and seeing it pass helps show that the test is indeed checking what it should.

▸ You don’t have to “go in after you’re done” and add in unit tests.

▸ It encourages prolific refactoring with confidence.

TDD: YOU SHOULD LOVE IT

▸ You appreciate the importance of DELETING code.

TIPS FOR TDD & UNIT TESTING

▸ TDD is hard. Getting into the right mindset comes with practice.

▸ Try “Reverse TDD” (green-red-green-refactor) or “Transplant TDD”.

▸ Use mocks and spies. Check only one thing per test.

▸ Imagine testing each of your Angular objects in a black hole in outer space separately from the rest of the application.

▸ Don’t worry about the UI details. Focus on testing the functions, logic, and calculations of your source code.

▸ Focus on explicitly calling your “public methods” which in turn will use the private methods to do it’s work.

THE HOLE IN PURE TDD

▸ You can sometimes find that it’s unclear “what test to write next”.

▸ TDD Tests only that that software is free of errors but doesn’t necessarily test that it provides real business value.

▸ It’s unclear how or when ui testing tools like Protractor fit into the TDD development cycle.

▸ It says nothing about how to actually turn the requirements into unit tests.

WHAT’S MISSING WITH TDD?

WOULDN’T IT BE NICE IF:

▸ we could save our requirements in source control also have them accessible, readable, and easily understandable for all stakeholders?

▸ the requirements were able to be run against our codebase quickly and easily, recording which ones passed and which ones failed?

▸ recordings could be visualized in a report similar to a code coverage report but for show if our requirements tests are passing?

IF ONLY WE HAD TDD FOR OUR REQUIREMENTS

YES!

IT IS A THING!

TERMS THAT ALL REALLY REFER TO THIS SAME IDEA:

▸ Specification by Example

▸ Behavior Driven Development

▸ Acceptance-Test Driven Development

▸ Agile Acceptance Testing

WHAT SHOULD WE CALL IT?

“SPECIFICATION BY EXAMPLE IS TDD APPLIED TO BUSINESS PROBLEMS”

Gojko Adzicfrom Specification by Example

OK, SHOW ME AN

EXAMPLE.

A SNIPPET OF GHERKIN CODE

GHERKIN

▸ Meant to be easily readable by non-programmers.

▸ Aids in building a common understanding of the requirements between all parties and is accessible to non-programmers.

▸ Becomes incredibly nice as documentation and progress reporting once the project begins.

▸ Meant to be written during a “Three Amigos” meeting.

Scenario: eat 5 out of 12

Given there are 12 cucumbers

When I eat 5 cucumbers

Then I should have 7 cucumbers

Scenario: Adding items to basket

Given I have no items in my shopping cart.

When I add 2 pumpkins

And remove two pumpkins

Then I should no items in my shopping cart.

FOR TEAMS PRACTICING BDD, THE REQUIREMENTS AND THE EXECUTABLE SPECIFICATIONS ARE THE SAME THING.

John Ferguson Smartfrom BDD In Action

OK... BUT HOW IS AN ENGLISH SENTENCE

“EXECUTED”? A. STEP DEFINITIONS

STEP DEFINITIONS

▸ Uses a Given-When-Then syntax which separate the test logic into setup code, action code, and assertion code, respectively..

▸ The Gherkin is mapped to a step definition function that uses a regex to match the Gherkin text.

▸ These functions then implement what the Gherkin says should happen and checks these business requirements against the source code.

MAPPING GHERKIN STEPS TO JAVASCRIPT FUNCTIONS

BUT HOW DOES ONE SIMPLY

“IMPLEMENT A STEP DEFINITION” IN ANGULARJS?

UNDERSTANDING CUCUMBERJS

▸ Instead of running the acceptance tests through cucumber itself, we can “cucumberize” or existing test runners.

▸ The most common JavaScript acceptance test running framework is called, “Cucumber-JS”.

▸ Running CucumberJS itself might be fine for a NodeJS project, but AngularJS testing requires special setup that CucumberJS does not know how to do.

CUCUMBERIZING YOUR TEST RUNNERS

▸ For Karma I recommend, “karma-cucumber-js”: https://github.com/eugene-sea/karma-cucumber-js

▸ Karma and Protractor both know how to execute gherkin.

▸ Both have a “frameworks” option that gets changed

▸ For Protractor, check out “protractor-cucumber-framework”:https://github.com/mattfritz/protractor-cucumber-framework

CUCUMBERIZING PROTRACTOR

Install the library via npm:

And then edit the framework parameter in your config block:

RUNNING CUCUMBERIZED PROTRACTOR TESTS

▸ You can also set up gulp / grunt tasks for acceptance tests.

▸ Tell Protractor to only look for the files with BDD suffixes like .feature or .step.js

▸ Run the Protractor executable and pass in your config file:

CUCUMBERIZING KARMA

‣ Notice that this project uses a slightly different syntax from Protractor step definitions:

▸ I recommend “karma-cucumber-js” (by eugene-sea) and not “karma-cucumberjs” (works with newer versions on cucumberjs).

WRITING KARMA STEP DEFINITIONS

▸ The Then is where you do the assertion that what you expected to happen did happen.

▸ The Given is similar to your “beforeEach” regular unit tests. It is where you can inject Angular objects and create mocks.

▸ The When is where you do some action, in this case normally calling a public method of some controller, service, etc.

THE CUCUMBER “WORLD”

▸ Cucumber has the concept of a “World object” that is somewhat of a global object for your acceptance tests.

▸ You can set frameworks like chai to the World object, allowing you to use chai API in your step definitions.

▸ Depending on your cucumber framework it’s either an object or a function that you can implement.

▸ It’s purpose is to allow you to define “utility properties” that can be used in step definitions.

WHY YOU SHOULD ALSO STILL USE YOUR NON-CUCUMBERIZED TEST RUNNERS

▸ If you’re worried about having two separate code coverage reports, you can combine them with “Instanbul-combine”.

▸ Use cucumberized karma when you don’t need the UI to test the BUSINESS CASE of the scenario.

▸ It’s important to be able to increase your code coverage without needing to write “boring scenarios”.

TEXT

A CUCUMBER REPORT EXAMPLE

GOOD REPORTS ARE KEY TO SUCCESS

▸ Cucumber reports allow you to see the progress of the application at any point in time.

▸ The reports should be seen as incredibly useful.

▸ Reports are all just-in-time generated (ideally from a build server) giving everyone up-to-date reports available at any time without the devs ever needing to “write documentation”.

▸ Code coverage report lets you know what has’t been covered by unit tests.

YOUR LIVING DOCUMENTATION!

▸ Great for the BAU team trying to understand your code.

THE MOST SUCCESSFUL TEAMS DISCOVERED THE LONG-TERM BENEFIT ON IMPLEMENTING SPECIFICATION BY EXAMPLE COMES FROM LIVING DOCUMENTATION.

Gojko Adzicfrom “Specification by Example”

A LOT OF PREPARATION GOES A LONG WAY

▸ But it’s worth it!

▸ Jenkins pipeline to run all of these tests, hosts the reports, and deploy the final build.

▸ Creating a cucumberized Protractor config file.

▸ Creating a cucumberized Karma config file.

▸ Learning and understanding gherkin / step definitions.

CAN WE START YET?

▸ More complicated file globbing patterns.

TEXT

EXAMPLE OF A TRAVIS.YML FILE

BDD WITHOUT CUCUMBER

▸ Some downsides to not cucumberizing your runners include:

▸ You can still practice some form of Behavior Driven Development even without cucumberized test runners.

▸ You can still have “Three Amigos” conversations about the requirements and even create Gherkin feature files without executing them.

KEEP AN ACCEPTANCE TESTING MINDSET

1) You don’t get reports (ie living documentation).2) You don’t know if your code passes or fails the scenarios.3) You don’t have explicit step definition functions.

ADDITIONAL BENEFITS OF ACCEPTANCE TESTS

▸ Misunderstandings get thrashed out early.

▸ Better team communication and ubiquitous language.

▸ Requirements and code are always in sync

▸ Removes the manual testing bottleneck.

▸ JIT documentation for development and beyond.

IF YOU’RE STILL NOT CONVINCED

▸ It helps developers think more in the business domain than just falling back to computer science patterns.

BUT WE DON’T HAVE TIME FOR TESTING

▸ Remind him that with the all of the rework, regression, and manual testing it could easily take longer to build the project without tests.

▸ Remind the boss that this is a long-term project, and the reports increase in value as the project ages.

▸ The 99% perfect rule: Nobody wants 99% perfect software. If it’s not bug-free then it’s not really finished.

▸ Is it that your team doesn’t have time, or is it that they just don’t know how and no one wants to pair program on it?

BUILD IT QUICKLY VS BUILD IT FOR LONGEVITY

HIGH QUALITY -> HIGH PERFORMANCE

PERFORMANCE TESTING

▸ Manual performance testing available through Chrome dev tools, WebPageTest, Google Pagespeed…

▸ Command line utility “Benchpress” that can be used for getting stats on and benchmarking any web application.

▸ In general, favor automated performance tests for the same reasons we automate our other tests.

▸ You could even host your benchpress report(s) on every build as part of your living documentation!

WHERE DO I PUT ALL OF THESE FILES?

▸ Avoid parallel directory structures or you might find yourself with “scrolling like mad syndrome”.

▸ It usually makes sense to work on one component at a time, and it’s nice to have all the things relating to that component in one place.

DIRECTORY DIVING

IT’S “LIVING” DOCUMENTATION, SET IT LIVE!

THE IMPORTANCE OF HOSTING YOUR REPORTS

▸ Without hosting your reports at a url for everyone to see, no one on your team can see the reports!

▸ Reports should NOT be generated and hosted manually. Generating the reports (by running the tests) and hosting them should be steps in your CI pipeline.

▸ As a CEO or business leader, if you can create a culture that highly values and utilizes these three types of reports, everything else will* fall into place.

▸ Access to up-to-date reports for non-programmers is CRITICAL for them to do their job properly.

GREAT BOOKS ON BDD / AUTOMATED ACCEPTANCE TESTING

UGAT

github.com/JimTheMan/AngularJS-UGAT

AN EXAMPLE PROJECT

OFFICIAL UGAT PROJECT: NG-NJ

THANKS!!Send any questions to twitter.com/webWhizJim

(or just ask them right now!)