strategies for end-to-end web apps testing

Strategies for End-to-EndWeb Apps Testing Ariya Hidayat

Amazon KindleCloud Reader

Financial Times

Edit

Develop

QA

Code Editing

Avoid silly mistakes

Write readable code

Do not provoke ambiguities

Improve future maintenance

Learn better code pattern

JavaScript Engine Building Blocks

Virtual Machine/

InterpreterParser

Runtime

Source

Syntax Tree

Built-in objects,host objects, ...

Fast and conservative

Parsing Stage

keyword equal sign semicolon

identifier number

var answer = 42;

Variable Declaration

Identifier Literal Constant

JavaScript Parser (Written in JavaScript)

UglifyJS

ZeParser

Esprima

TraceurNarcissus

Es-Lab

Syntax Tree Visualization

answer = 42

Style Formatter https://github.com/fawek/codepainter

CodePainterSource

Sample code

Formatted source

Infer coding styles

IndentationQuote for string literalWhitespace

Code Outline Eclipse

FunctionsVariables

Application Structure

Ext.define('My.sample.Person', { name: 'Mr. Unknown',  age: 42,

constructor: function(name) {},  walk: function(steps) {} run: function(steps) {}

});

Class manifest

{ className: 'My.sample.Person', functions: ['walk', 'run'], properties: ['name', 'age']}

Content Assist (aka Intellisense aka Autocomplete)

Eclipse

http://contraptionsforprogramming.blogspot.com/2012/02/better-javascript-content-assist-in.html

Error Tolerant

var msg = "Hello’;

person..age = 18;

 if (person.

 'use strict';with (person) {}

Mismatched quote

Too many dots

Incomplete, still typing

Strict mode violation

Code Linting

var fs = require('fs'), esprima = require('./esprima'), files = process.argv.splice(2); files.forEach(function (filename) { var content = fs.readFileSync(filename, 'utf-8'), syntax = esprima.parse(content, { loc: true });  JSON.stringify(syntax, function (key, value) { if (key === 'test' && value.operator === '==') console.log('Line', value.loc.start.line); return value; });});

if (x == 9) { // do Something}

Not a strict equal

Copy Paste Mistake

function inside(point, rect) { return (point.x >= rect.x1) && (point.y >= rect.y1) && (point.x <= rect.x2) && (point.y <= rect.y1);}

Wrong check

“Boolean Trap” Finder

Can you make up your mind? treeItem.setState(true, false);

event.initKeyEvent("keypress", true, true, null, null, false, false, false, false, 9, 0);

The more the merrier

Obfuscated choice var volumeSlider = new Slider(false);

Double-negative component.setHidden(false);filter.setCaseInsensitive(false);

http://ariya.ofilabs.com/2011/08/hall-of-api-shame-boolean-trap.html

Refactoring Helper

// Add two numbersfunction add(firt, two) { return firt + two;}

// Add two numbersfunction add(first, two) { return first + two;}

Syntax = Message

var Foo = (function () { return { bar: function (x, y) { // do Something } };};

module Foo { export function bar (x, y) { // do something }}

ES 5.1 HarmonyToday ES 6/7, Future

Transpilation

// Object literal property shorthand.function Point(x, y) { return { x: x, y: y };}

// Object literal property shorthand.function Point(x, y) { return { x, y };}

Harmony

ES 5.1

Development

Unit Test Framework

http://www.trollquotes.org/image/5-gandalf-troll-quote.jpg

Automatic Reload

Modified source Reload content

Remote Interactionhttp://www.sencha.com/blog/remote-javascript-debugging-on-android/

Weinre, Adobe Shadow, Remote Web Inspector,...

Fast Smoke Test

Feedback Cycle

Precommit Check

forget to run

the tests

Typical Scenario

This is awesome!

Git: precommit hookhttp://ariya.ofilabs.com/2012/03/git-pre-commit-hook-and-smoke-testing.html

Zombie.js

var Browser = require("zombie"); browser = new Browser();browser.visit("http://mobileconference.nl", function () { console.log(browser.text("title"));});

PhantomJS (Headless WebKit)

Display

Paint

Layout

Normal Browser

Display

Paint

Layout

XHeadless Operation

Screen Capture = Pixel-Perfect Comparison

phantomjs rasterize.js URL output

CasperJS

var casper = require('casper').create();

casper.start('http://www.mobileconference.nl/', function() { this.echo('Page title is: ' + this.evaluate(function() { return document.title; }), 'INFO');});

casper.run();

Postcommit Quality Metrics

Quality Factor 1

Quality Factor 2

Check-in

Identifier Length Distribution

http://ariya.ofilabs.com/2012/05/javascript-identifier-length-distribution.html

0

250

500

750

0 5 10 15 20 25 30 35 40 45

mean of the identifier length is 8.27 characters

prototype-1.7.0.0.js SCRIPT_ELEMENT_REJECTS_TEXTNODE_APPENDINGprototype-1.7.0.0.js MOUSEENTER_MOUSELEAVE_EVENTS_SUPPORTED jquery-1.7.1.js subtractsBorderForOverflowNotVisiblejquery.mobile-1.0.js getClosestElementWithVirtualBindingprototype-1.7.0.0.js HAS_EXTENDED_CREATE_ELEMENT_SYNTAX

Statement Distribution

http://ariya.ofilabs.com/2012/04/most-popular-javascript-statements.html

ExpressionStatementBlockStatement

IfStatementReturnStatement

VariableDeclarationFunctionDeclaration

ForStatementForInStatementWhileStatementBreakStatement

TryStatementEmptyStatementThrowStatementSwitchStatement

ContinueStatementDoWhileStatementLabeledStatement 6

12

25

35

38

66

84

115

131

143

293

371

2116

2878

3063

6353

6728

var fs = require('fs'), esprima = require('esprima'), files = process.argv.splice(2); files.forEach(function (filename) { var content = fs.readFileSync(filename, 'utf-8'), syntax = esprima.parse(content);  JSON.stringify(syntax, function (key, value) { if (key === 'type') { if (value.match(/Declaration$/) || value.match(/Statement$/)) { console.log(value); } } return value; });});

QA

Platform Proliferation

BusterJS

Server Interface

Test Suite

Test Server

Selenium

WebDriver

WebDriver Client

Opera WebDriver

Firefox WebDriver Chrome

WebDriver

Code Coverage

http://ariya.ofilabs.com/2012/03/javascript-code-coverage-and-esprima.html

x = 42;if (false) x = -1;

https://github.com/itay/node-coverhttps://github.com/coveraje/coverajehttps://github.com/pmlopes/coberturajs

Instrumentation for Code Coverage

http://itay.github.com/snug_codecoverage_slides/

var a = 5;{ __statement_ZyyqFc(1); var a = 5;}

foo();{ __statement_ZyyqFc(2); __expression_kC$jur(3), foo();}

function foo() { ...};

function foo() { __block_n53cJc(1); ...}

Statement

Expression

Block

Performance via Benchmarks.js

var suite = new Benchmark.Suite;

suite.add('String#indexOf', function() { 'Hello World!'.indexOf('o') > -1;}).on('complete', function() { console.log('Fastest is ' + this.filter('fastest').pluck('name'));}).run();

JSPerf.com

Performance Baseline

Application revision

Exec

utio

n tim

e

Baseline

Scalability via Run-time Complexity Analysis

http://ariya.ofilabs.com/2012/01/scalable-web-apps-the-complexity-issue.html

Array.prototype.swap = function (i, j) { var k = this[i]; this[i] = this[j]; this[j] = k;}

Array.prototype.swap = function (i, j) {Log({ name: 'Array.prototype.swap', lineNumber: 1, range: [23, 94] }); var k = this[i]; this[i] = this[j]; this[j] = k;}

Execution Tracing

http://ariya.ofilabs.com/2012/02/tracking-javascript-execution-during-startup.html

https://gist.github.com/1823129jQuery Mobile startup log

4640 function calls

jquery.js 26 jQuery jquery.js 103 init undefined, undefined, [object Object] jquery.js 274 each (Function) jquery.js 631 each [object Object], (Function), undefined jquery.js 495 isFunction [object Object] jquery.js 512 type [object Object]jquery.mobile.js 1857 [Anonymous]jquery.mobile.js 642 [Anonymous]jquery.mobile.js 624 enableMouseBindingsjquery.mobile.js 620 disableTouchBindings

Wrap-up

Edit

Develop

QA

Coding style/lint, semantic outline, autocomplete, API usage, refactoring, better syntax

Test framework, remote control, precommit check, automatic reload, fast smoke testing, preflight metrics

Comprehensive browser driver, code coverage, performance baseline, execution tracking

To boldly analyze what no man has analyzed before...

Thank You

ariya.hidayat@gmail.com

@AriyaHidayat

ariya.ofilabs.com