building infrastructure with terraform (google)

Building infrastructure with Terraform

Radek Simko

$ whoami

radeksimko}twitter.com/

google.com/+

linkedin.com/in/

github.com/

Provisioning of the past

● manual● Shell, Perl● extra knowledge required

○ bottleneck for team growth● pure-ops task

○ devs & ops talking over a wall

Provisioning today

● Chef, Puppet, Salt, Ansible● knowledge codified● de facto serves as documentation● faster● less error prone● brings devs and ops closer

Infrastructure today

Hashicorp

Vagrant

● Virtualbox● VMWare● Docker● AWS● Google Cloud● ...

Terraform

● AWS● Azure● Digital Ocean● Google Cloud● Docker● OpenStack● ...

Other solutions

● AWS CloudFormation● Google Deployment Manager● Heat (OpenStack)● Puppet Cloud Provisioner● Ansible● SaltStack● ...

XML

JSON

YAML- states: - QC: Quebec - ON: Ontario - BC: British Columbia - YT: Yukon Territory

[ { "states": [ { "QC": "Quebec" }, { "true": "Ontario" }, { "BC": "British Columbia" }, { "YT": "Yukon Territory" } ] }]

YAML#cloud-config

_discovery_url: &ETCD_DISCOVERY_URL

url: "https://discovery.etcd.io/5416cf2db"

coreos:

fleet:

<<: *FLEET_METADATA

public-ip: $private_ipv4

etcd:

<<: *ETCD_DISCOVERY_URL

addr: $private_ipv4:4001

peer-addr: $private_ipv4:7001

jsonnet

● 20% project of Dave Cunningham● Turing-complete language● allows building more abstraction layers● compatible w/ Terraform

DSL

● referencing● reusability (DRY)● human-readability

Demo 1Basic two-tier app (POC)

https://github.com/radeksimko/terraform-examples/tree/master/google-two-tier-simple

DSL● provider● resource (ID + reference name)● count attribute● variable● output● provisioner● Reference: ${TYPE.NAME.ATTRIBUTE}● Expansion: ${TYPE.NAME.*.ATTRIBUTE}

Provisioners

● local-exec● remote-exec● chef● …

Demo 2Two-tier app (Scalable)

https://github.com/radeksimko/terraform-examples/tree/master/google-two-tier-scalable

DSL - built-in functions● file(path)● format("web-%03d", count.index+1)● formatlist("https://%s:%s/", aws_instance.

foo.*.public_dns, var.port)● lookup(map, key)● ...

How does it work?

Why Terraform? (overview)

● Provider-agnostic● DSL (yet JSON-compatible)

Why Terraform?● open to community

○ missing feature or bug != support ticket

Why Terraform?● doesn’t “own” your whole account or resource “type”

Why Terraform?

● core features○ aware of dependency graph between resources○ transparent state (pros/cons)

■ resources import (in the future)■ detailed plan■ maintenance & atomicity & sharing in the team

State..."google_compute_firewall.consul_admin_node": { "type": "google_compute_firewall", "primary": { "id": "consul-admin-firewall", "attributes": { "allow.#": "1", "allow.803338340.ports.#": "1", "allow.803338340.ports.1685985038": "22", "allow.803338340.protocol": "tcp", "id": "consul-admin-firewall", "name": "consul-admin-firewall", "network": "default", "self_link": "https://www.googleapis.com/compute/v1/projects/skilled-bee-777/global/firewalls/consul-admin-firewall", "source_tags.#": "1", "source_tags.3928264908": "consul-web-ui", "target_tags.#": "1", "target_tags.1747926572": "consul-node" } }},...

State

● terraform.tfstate (default)● Atlas● S3● Consul● HTTP● OpenStack’ Swift

State - setupterraform remote config \

-backend=consul \

-backend-config="address=demo.consul.io:80" \

-backend-config="path=tf"

State - referencingresource "terraform_remote_state" "network" {

backend = "atlas"

config {

name = "timeinc/network-prod"

}

}

resource "google_compute_firewall" "default" {

network = "${terraform_remote_state.network.network-name}"

...

Modulesmodule "consul" {

source = "github.com/hashicorp/consul/terraform/aws"

servers = 3

}

variable "servers" {

default = 2

}

$ terraform get

Modules./module/outputs.tf

output "ip" {

value = "${google_compute_instance.default.public_ip}"

}

./main.tf

module "consul" {

…

${module.consul.ip}

Demo 3Consul cluster

https://github.com/radeksimko/terraform-examples/tree/master/google-consul-module

New provider?~/.terraformrc:

providers {

privatecloud = "/path/to/privatecloud"

}

Custom provider?package main

import (

"github.com/hashicorp/terraform/plugin"

)

func main() {

plugin.Serve(new(MyPlugin))

}

Under the hood - provider

● helper/schema

What’s next?

● Kubernetes provider● Google backend service (HTTP LB)● …

↓ SLIDES ↓

$ whoami

radeksimko}twitter.com/

google.com/+

linkedin.com/in/

github.com/

slideshare.net/