Creating a Smooth Development Workflow for High-Quality Modular Open-Source PHP Libraries

by Greg Anderson @greg_1_anderson

Pantheon.io

Goal: Code re-use

Drupal Site

Drupal Module

Re-usable PHP Library

2

Pantheon.io

Optional: Command-line tool

http://robo.li/framework/

3

Pantheon.io

Use a PHP library from a module

{

"name": "drupal/lcache",

"description": "LCache module.",

"type": "drupal-module",

"license": "GPLv2",

"require": {

"lcache/lcache": "0.3.*"

}

}

● Add a minimal composer.json to module

○ n.b. Drupal 8 has an implicit

autoloader entry

Drupal/modulename

● Use Composer to manage Drupal site

○ drupal-composer/drupal-project

4

Pantheon.io

What is our Workflow?

REPEAT

5

Pantheon.io

Focus of this Session

HOWEVER, we also need:

● Collaboration● Reproducibility● Analysis of

○ Test coverage○ Code Quality○ Open Source License Compliance

● Documentation

6

Pantheon.io

What is a smooth workflow?

Just push the button, and the right thing happens.

7

Pantheon.io

All things take effort

Spend time up-front to make things easier later: WIN

8

Pantheon.io

Is it worth the effort?

Also remember:

You will be able to respond to high-priority requests faster.

Onboarding new team members will be easier.

Consistent processes will lead to consistent results and fewer defects.

https://xkcd.com/1205/

9

Pantheon.io

Services to the rescue!

Collaborate with other engineers across multiple branches.Keep a record of all work done. Integrate With All Of The Things.

Run tests andother tasks.

Calculate test coverage.

Analyze code complexity.

Track dependency versions and licenses.

Package manager for composer.

Publish documentation site from markdown.

10

Pantheon.io

Follow the example of existing projects

Some projects we will examine:

● lcache/lcache

● consolidation/*

● westkingdom/website

http://dilbert.com/strip/1996-01-31

11

Pantheon.io

Github

● What can you do with it?○ Collaborate through browser without git○ Set up lots of integrations

● How do you use it effectively?○ Add README and COLLABORATING○ Make issue and PR templates○ Advertise integrations with badges

● How do you fix mistakes?

https://github.com

12

Pantheon.io

Collaborate through browser interface

In-browser collaboration is even more powerful with automated builds.

13

Pantheon.io

Make your own copy for local work

14

Pantheon.io

Basics of using git locally

Make a local copy$ git clone … working-copy

$ cd working-copy

Create a branch$ git checkout -b mywork

Commit changes$ git add -A .

$ git commit -m "Awesomesauce."

Push branch$ git push origin mywork

15

Pantheon.io

Recovering from Git-tastrophes

Oh , git!http://ohshitgit.com/

16

Pantheon.io

Add a README

https://github.com/matiassingers/awesome-readme

● Short project description.

● Badges with links to services.

● Reasons for using project.

● Build and test instructions.

● Installation instructions.

● Usage documentation.

● How to contribute.

The README may either contain this information directly, or contain links to other documents where this information can be found.

17

Pantheon.io

Add a CONTRIBUTING document

18

Pantheon.io

Add issue and PR templates

### Steps to reproduce

What did you do?

### Expected behavior

Tell us what should happen

### Actual behavior

Tell us what happens instead

### Overview

This pull request:

- [ ] Fixes a bug

- [ ] Adds a feature

- [ ] Breaks backwards compat

- [ ] Needs tests

### Description

Any additional information.

.github/pull_request_template.md.github/issue_template.md

19

Pantheon.io

Packagist

● What can you do with it?○ Register projects so they may be easily

found

● How do you use it effectively?○ Define a branch alias○ Use search URLs to find projects of a

certain type (e.g. plugins)

https://packagist.org

20

Pantheon.io

Set up composer.json

{ "name": "consolidation/bootstrap", "description": "Locate and bootstrap components ...", "autoload":{ "psr-4":{ "Consolidation\\Bootstrap\\": "src" } }, "require": { "php": ">=5.5.0", "psr/log": "~1.0", "symfony/console": "~2.5|~3.0" }, "extra": { "branch-alias": { "dev-master": "1.x-dev" } }}

21

Pantheon.io

Register project with packagist

22

Pantheon.io

Confirm submission

23

Pantheon.io

Packagist project without auto-update

24

Pantheon.io

Get Packagist API token

25

Pantheon.io

Add Packagist service in GitHub

26

Pantheon.io

Enter Packagist service information

27

Pantheon.io

Test Packagist service

28

Pantheon.io

Confirm auto-update warning is gone

29

Pantheon.io

Searching for project types

https://packagist.org/search/?type=drupal-drush

30

Pantheon.io

Badges for Packagist

https://poser.pugx.org

31

Pantheon.io

Travis

● What can you do with it?○ Run tests on multiple versions of PHP○ Generate artifacts when tests pass

● How do you use it effectively?○ Provide a functional phpunit.xml.dist○ Test code style for PSR-2 conformance○ Use composer install --prefer dist○ Avoid testing PRs twice○ Cache dependencies for faster builds○ Commit composer.lock○ Provide scripts to run tests locally

https://travis-ci.org

32

Pantheon.io

Set up phpunit.xml.dist

<phpunit bootstrap="vendor/autoload.php" colors="true"> <testsuites> <testsuite name="annotation-command"> <directory prefix="test" suffix=".php">tests</directory> </testsuite> </testsuites></phpunit>

33

Pantheon.io

Install phpcs

$ composer require squizlabs/php_codesniffer

34

Pantheon.io

Set up .travis.yml

language: php

php: - 7.0 - 5.6 - 5.5 - 5.4

before_script: - composer install --prefer-dist

script: - vendor/bin/phpunit - vendor/bin/phpcs --standard=PSR2 -n src

35

Pantheon.io

Test PRs only one time

branches: # Only test the master branch and SemVer tags. only: - master - /^[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+.*$/

36

Pantheon.io

Cache dependencies

sudo: false

cache: directories: - $HOME/.composer/cache

37

Pantheon.io

Commit composer.lock

matrix: include: - php: 7.0 env: deps=highest - php: 5.6 - php: 5.5 - php: 5.4 env: deps=lowest

before_script: - if [ -z "$deps" ]; then composer install --prefer-dist; fi; - if [ "$deps" = "lowest" ]; then composer update --prefer-dist --prefer-lowest -n; fi; - if [ "$deps" = "highest" ]; then composer update --prefer-dist -n; fi;

https://blog.wyrihaximus.net/2015/06/test-lowest-current-and-highest-possible-on-travis/

38

Pantheon.io

Highest / lowest dependency test results

39

Pantheon.io

Create a phar using box2

Define a box.json file to define phar contents, then run:

$ composer require kherge/box$ vendor/bin/box build .

40

Pantheon.io

Contents of box.json for box2

{ "alias": "robo.phar", "chmod": "0755", "compactors": ["Herrera\\Box\\Compactor\\Php"], "directories": ["src"], "files": ["RoboFile.php"], "finder": [ { "name": [ "*.php", "*.exe", "GeneratedWrapper.tmpl" ], "exclude": [ "test", "tests", "Test", "Tests", "Tester" ], "in": "vendor" } ], "git-commit": "git-commit", "git-version": "git-version", "output": "robo.phar", "main": "robo", "stub": true}

https://github.com/consolidation/Robo/pull/411

41

Pantheon.io

Deploy phar on every releases

# Prior to a deploy, build a fresh robo.phar

before_deploy:

- vendor/bin/box build .

deploy:

provider: releases

api_key:

secure: REPLACE_WITH_OAUTH_TOKEN

file: robo.phar

skip_cleanup: true

on:

tags: true

https://help.github.com/articles/creating-an-access-token-for-command-line-use/42

Pantheon.io

Provide scripts to run tests locally

{ "name": "consolidation/annotated-command", "scripts": { "phar": "vendor/bin/box build .", "cs": "phpcs --standard=PSR2 -n src", "cbf": "phpcbf --standard=PSR2 -n src", "unit": "SHELL_INTERACTIVE=true phpunit --colors=always", "test": [ "@unit", "@cs" ] }}

43

Pantheon.io

Build status badge

44

Pantheon.io

Build status summary pages

Paste in badge image URLs any place HTML can be rendered (e.g. wiki pages) to create summary pages.

45

Pantheon.io

Coveralls

● What can you do with it?○ Keep a log of test coverage over time○ See line-by-line what parts of the code are

tested, and what parts are not

● How do you use it effectively?○ Periodically review untested functions, and

prioritize time to write new tests based on importance

https://coveralls.io

46

Pantheon.io

Add coverage to phpunit.xml.dist

<phpunit bootstrap="vendor/autoload.php" colors="true"> … <logging> <!-- <log type="coverage-html" target="build/logs/coverage" lowUpperBound="35" highLowerBound="70"/> --> <log type="coverage-clover" target="build/logs/clover.xml"/> </logging> <filter> <whitelist processUncoveredFilesFromWhitelist="true"> <directory suffix=".php">src</directory> </whitelist> </filter></phpunit>

47

Pantheon.io

Install xdebug and coveralls PHP runner

$ brew install php70-xdebug$ composer require satooshi/php-coveralls

48

Pantheon.io

Add Coveralls to .travis.yml

after_success: - travis_retry php vendor/bin/coveralls -v

49

Pantheon.io

Turn on Github integration

50

Pantheon.io

View change in coverage over time

51

Pantheon.io

View change in coverage for each test

52

Pantheon.io

View tested and untested lines in source

53

Pantheon.io

Code coverage badge

54

Pantheon.io

Scrutinizer

● What can you do with it?○ Analyze code for complexity and

duplication○ Run tests and calculate coverage

(Travis and Coveralls better, though)

● How do you use it effectively?○ Set up GitHub integration○ Always fix reported “bugs”○ Review “hot spots” and refactor○ Learn from provided advice○ Ignore advice you think is wrong

https://coveralls.io

55

Pantheon.io

Set up Scrutinizer GitHub Integration

Type the name of the organization and project to inspect; Scrutinizer will set up the GitHub integration for you.

56

Pantheon.io

Inline Comments with Chrome Plugin

https://scrutinizer-ci.com/docs/integration/chrome-extension

57

Pantheon.io

Inspection Report

58

Pantheon.io

Hot Spots in Code

59

Pantheon.io

Class Analysis

60

Pantheon.io

Scrutinizer code analysis badge

61

Pantheon.io

Version Eye

● What can you do with it?○ Track dependencies that have new versions○ Confirm OSS license compliance

● How do you use it effectively?○ Set up GitHub integration○ Point your project’s license badge at the

VersionEye license overview page

https://www.versioneye.com

62

Pantheon.io

Set up license info in composer.json

{ "name": "consolidation/annotated-command", "description": "Initialize Symfony Console commands …", "license": "MIT", "authors": [ { "name": "Greg Anderson", "email": "greg.1.anderson@greenknowe.org" } ],

… }

63

Pantheon.io

Set up VersionEye GitHub Integration

VersionEye will set up the GitHub integrations for your project automatically; just select your project from a list.

64

Pantheon.io

Dependencies tab

Highlights dependencies that have newer versions available.

65

Pantheon.io

Licenses tab

Spot licenses in the list that are not like the others.

n.b. Clicking on patchwork/jsqueeze reveals that it is dual-licensed Apache-2.0 / GPL-2

66

Pantheon.io

Dependencies badge

67

Pantheon.io

License badge

https://poser.pugx.orghttps://…#tab-licenses

[](https://www.versioneye.com/user/projects/ID#tab-licenses)

68

Pantheon.io

ReadTheDocs

● What can you do with it?○ Publish documentation from

markdown files in source repository

● How do you use it effectively?○ Set up GitHub integration○ Link to your project documentation

from your project README page

https://readthedocs.org

69

Pantheon.io

Create a mkdocs.yml file

70

Pantheon.io

Add documentation markdown files

71

Pantheon.io

Set up Github integration

72

Pantheon.io

Define a custom domain

73

Pantheon.io

Read your documentation

74

Pantheon.io

Add Markdown API on ReadTheDocs

http://outputformatters.readthedocs.io/en/latest

75

Pantheon.io

Create simple API markdown docs

$ composer require victorjonsson/markdowndocs$ vendor/bin/phpdoc-md phpdoc-md generate src > docs/api.md$ git add docs/api.md$ git commit -m "Add API documentation."

OH NO! It’s not automated!

● ReadTheDocs is a python service; it can’t run php.

● Can’t easily build from Travis and commit back to the repository, as that would create a separate commit (not part of the release, might cause another test run, etc.)

76

Pantheon.io

GitHub pages

● What can you do with it?○ Serve static html directly from GitHub○ Automatically update generated

documentation from Travis

● How do you use it effectively?

https://pages.github.com

All sorts of advanced techniques possible!

77

Pantheon.io

GitHub pages automatic generator

78

Pantheon.io

Choose one of the provided themes

79

Pantheon.io

Resulting page using selected GitHub theme

80

Pantheon.io

After manual css fiddling (and cursing)

Alternatives to hand-editing HTML:

● http://www.mkdocs.org○ mkdocs gh-deploy

● https://jekyllrb.com○ jekyll build

● https://sculpin.io○ sculpin generate

Many others!

81

Pantheon.io

API docs with Sami and GitHub pages

Install Sami$ curl --output $HOME/bin/sami.phar http://get.sensiolabs.org/sami.phar

In .travis.ymlafter_success: # Publish updated API documentation on every push to the master branch - git config --global user.email $GITHUB_USER_EMAIL - git config --global user.name "Travis LCache Documentation Bot" - sami.phar --ansi update sami-config.php - git clone --branch=gh-pages https://${TOKEN}@github.com/org/proj work - rm -rf work/api - cp -R docs/api work/api - cd work - git add -A api - git commit -m "API docs from $TRAVIS_BUILD_NUMBER/$TRAVIS_COMMIT" - git push

82

Pantheon.io

Travis environment variables for GitHub pages

83

Pantheon.io

Sami API documentation on GitHub pages

https://lcache.github.io/lcache/api/master

84

Pantheon.io

Q & A / Please take diversity survey

85