Technical Culture

Who am I?Aleksey Yashchenko

Software Engineer &Engineering Manager at Grammarly

@tuxslayer

The CompanyUkrainian bootstrapped startup

60 engineers

Offices in Kyiv and San-Francisco

The ProductGrammar

Spell

Style

Plagiarism

… checker

5M weekly active users on Chrome and growing

10M+ WebSocket connections/day

Services in Java, Lisp, Scala and Python

6 UI applications in JS, TypeScript and F#

The Product

Our Apps: UI3 Browser Plugins: Chrome, Safari, FF

Web App

M$ Office Add-in

Desktop Editor

Funnels at www.grammarly.com

Our Growthrapid user base growth: 5M in 1.5 years

new products

60 engineers

We can only allow ourselves intensive growth

Any tech culture won’t do the thingTech Culture

Tech ExcellenceJust excellence ‘all the time’ is hard

Technical excellence culture is needed to grow intensely

Technical Excellence Culture

HOW?

Our Approach

Orgstructure

TechnicalmeansT.E.C.

The Foundation

Orgstructure

Technicalmeans

The Problem: Time-Eater[s]Low development pace

Buggy releases =>

regular outages =>

high downtime

Our SolutionEngineers at

Cross-functional Teams

Fully own services and releases, while

Tools solve all the hassle

Cross-func Teams

QA => SE in Test => Feature Teams

Story #1

XP

Continuos Delivery

Monitoring

ORG: crossfit TECH: CI

Solution #1: org

Feature Cross-functional Teams

Solution #1: org

Catch bugs

Build for quality

Write code

… well… catch bugs

no-QA: from QA to SET

Solution #1: techXP

pair programming

defensive programming

code review

tests for new code / bugs found

… and more

Solution #1: techCI/CD: Release often

Feature branches on git

CI Server (TeamCity)

Unit / Integration / Load / … tests

Continuous Deployment

Solution #1: techCI/CD: Release often

Feature branches on git

CI Server (TeamCity)

Unit / Integration / Load / … tests

Continuous Deployment

Doesn’t work

Monitor All the Things!

Make it an engineering practice

Measure everything, choose what to monitor then

Validate monitoring data!

Monitor All the Things!

Logs: back-end and front-end

Time-series metrics: counters, timers

More: www.slideshare.net/tuxSlayer/frontend-monitoring-grammarly

Monitor All the Things!

Results #1Zero ping-pong time between teams

Developers can release “at will”

QA-people are involved from day 1

Everybody writes tests

Everybody* can fix a bug

* almost

Results #1Time to release from 1-2h to 1-10 min

Rollback instead of fixing => Short outages

Whole team does not get distracted by outages

Release Calendar

Platform Team

Embedded Ops in Teams

Story #2

docker

rocker

Terraform

ORG: no-Ops TECH: containers

Solution #2: Platform TeamNot a “service” team

PaaS

Strategic rather than tactical

Software Engineers + DevOps

Embedded Ops for Feature Teams

Solution #2: Containers

Container Platform

Static content

Nginx configs

Nginx

Responsible Dev Team Happy Ops Kitchen

ship it! Container

Container

Container

…

Results #2Wait no more for DevOps tasks

Reproducible environments => Safe deployments

Configuration as a code (versioning, automation, templating etc)

Multi-service deployments (e.g. DB+Redis+App+Nginx)

… soEngineers at

Cross-functional Teams

Fully own services and releases, while

Tools solve all the hassle

Our Approach

Orgstructure

TechnicalmeansT.E.C.

T.E.C.: the FoundationOptimize every single phase of dev cycle

Create an env for dev practices

Apply best practices to

Stop fire-fighting and free more time and mindshare for improvements

T.E.C.: let’s do it!

Build ExpertiseHire the best

Take coursesmake internal talks afterwards

Mentorship including external mentors

Free Books

Fun Days for learning

Internal

Hackathons

Build ExcellenceDefinition of Done

Tech Excellence Check Lists (forever!)

SLOs / SLAs

Check Lists: Practices

Check List: Quality Aspects (50+)

Build CultureTechnical Leadership Guild

2 tech-leads + 2 managers

Set expectations

Praise: pride & recognition

Create more space for communication

T.E.C.…

…

Profit: time and space for intensive growth

TakeawaysIt’s hard

It’s expensive

It’s slow

You don’t always need it

But when you have it – it’s a pure joy!

Thank you!Aleksey Yashchenko

Software Engineer &Engineering Manager at Grammarly

@tuxslayer