Postgresis the new defaultHow we transitioned our platform and why you should too

I work with Data!

Agenda

•Where we were

•Our relationship with Postgres

•Challenges

•Our future together

We live in The Cloud!

Engine Yard

• Languages

•Amazing Support

• Team DBAs & me

• 2 Products *

> 1 Petabyte

No multi-tenancy.

...A year (or so) ago

Engine Yard Cloud•Only MySQL 5.0

• 8.3 in perpetual Alpha

•No Slaves, No Docs

• Increasing interest

• Stack Expansion - DIRTT

Our Relationship

with Postgres

Infatuation

Stage 1

Why love Postgres?

• Flexible Replication: streaming & hot standby

• Features & Extensions

• Versatile Data Types

• FDWs

Why love Postgres?

•All of the Indexes!

• B-tree, GiST, GIN, SP-Gist*

• Procedural Languages (PL)

• Full-Text Search

Why love Postgres?

• Feature parity with Managed

• Outstanding community & commercial support

• Let’s make it our default!

Understanding

Stage 2

Re-Tool

•MySQL Shop

•DBA Training: DBAs & Engineers

•OSS to help us gain velocity

Customer Education

•Customer Facing Docs focused on MySQL

• Technical Writers FTW!

Disturbances

Stage 3

Roadmap issues

• Shiny new process for DB releases

•Alpha, Beta, GA

• Surprise we got it WRONG!

Multiple Architectures

• EBS Volumes & 32-bit instance sizes

•Dump & Restore only way to vertically scale

Bad Assumptions

• Product codebase tied to old assumptions

•MySQL everywhere

• Refactor!

Moulding

Stage 5

Company-Wide Effort

Installation

• Engine Yard Cloud

•Chef > EBS

• Engine Yard Managed

• Puppet > Storage

Configuration

• $PGDATA on EBS

•We don’t RAID

• Snapshots to S3

• pg_xlog same volume

Replication

•Archiving on db_master

• hot_standby slaves

•All slaves <> AZs

• Smaller slaves

Monitoring

• Bucardo’s check_postgres

•Collectd & Nagios

• Process health, replication, checkpoints - MORE

Backups

• Internal tool wraps around pg_dump

• Snapshots & dumps to S3

•Only backup from slave (if present)

Allow Extensions

•Dedicated instances

•Customizations OK!

• Ship recipes for ~30 extensions

•Curated & tested

Architectures

•New instance types: 64-bit small, mediums

• Standardized instances architecture 64-bit

Happiness & Trust

Stage 9

Future Directions

• 9.2 Support

•AWS Optimizations

• Provisioned IOPS

• EBS optimized instances

• SSD-backed instances

Future Directions

• Painless upgrades

• Better extensions

•Ugh to cookbooks!

•HA/DR

Future Directions•Customer education

still a problem

• Low adoption of slaves

•~146 year ago

•~165 now

Give us a try!

Questions? @RandomMood