cloud foundry api for fun and ops

EXPLORING THE CLOUD FOUNDRY API FOR FUN AND OPS

Josh Kruckprofessional doer of things

github.com/krujos@krujos

http://github.com/krujos

WHO AM I?• Today

• pivotal customer success

• cf cli plugin author

• service broker author

• lots of little utilities for getting shit done

• Past

• a lot of storage and application management

• Why

• use software to make work more fun and be happier

maybe you have heard?

https://lynxbat.files.wordpress.com/2012/06/api.jpg

https://lynxbat.files.wordpress.com/2012/06/api.jpg

apidocs.cloudfoundry.org

http://apidocs.cloudfoundry.org

done. good game. thanks everyone!

http://www.wired.com/2013/11/pivotal-one/

http://www.wired.com/2013/11/pivotal-one/

now what?

SOME IDEAS

• apps (dashboard, provisioner, automation, crypt keeper)

• cli plugins

• scripts (why aren’t you writing a cli plugin?)

• A one time thing? (jq will change your life)

SCRIPTS ARE AN ANTI PATTERNCLI plugins FTW!

What are the things that got API on them?

Service Brokers(more of a spec really)

Lots of other people talk about service brokers and how to author them, i won’t be covering them.

UAA(User Account and Authentication Server)

This is where we go to login. If you’re writing CLI plugins and scripts, it’s a non issue, but for any kind of app we’ll have to deal with it.

Firehose(the tattletale)

the emitter of most of the things

Collector(older tattletale)

emits a lot of the readable things

Cloud Controller(the brains)

The CC is where we got to listen and act.

but wait!Router

(eventually)

A feature that’s coming down the pipe is “router services”. While I don’t have anything to say on this today it is something to be aware of. It will allow us to do some inline things at the router that we previously had to insert between our LB’s and routers.

THE DETAILS

UAA(User Account and Authentication Server)

This is where we go to login. If you’re writing CLI plugins and scripts, it’s a non issue, but for any kind of app we’ll have to deal with it.

WHEN/WHY DO WE INTERACT WITH UAA?

• When we’re writing an application and DON’T want to inherit the credentials of the user

• Dashboards, system automation and on boarding

System automation examples: Clean up orphaned services Check for rouge users Reports (new URL’s for nessus, apps pushed in the last day, crash summaries… whatevs).

WHEN/WHY DO WE INTERACT WITH UAA?

• When we’re writing an (web) application and we DO want to inherit the credentials of the user

• cli plugins and scripts can usually get this for free

System automation examples: Onboarding, cleanup, CI stuff (slackbots)

UN HUMANS

• For apps that do not act on behalf of a human we use client_credentials

• See the OAuth 2.0 spec for more information

• https://tools.ietf.org/html/rfc6749#section-1.3.4

• http://tools.ietf.org/html/rfc6749#section-4.4

If you google, you will find people who have translated this into somewhat more complicated english than the spec provides.

https://tools.ietf.org/html/rfc6749#section-1.3.4

http://tools.ietf.org/html/rfc6749#section-4.4

GOuri, _ := url.Parse(uaaURI.String() + "/oauth/token?grant_type=client_credentials") creds := &UaaClientCredentials{ uaaURI: uri, clientID: clientID, clientSecret: clientSecret, } client := http.Client{tls.Config{}} req, _ := http.NewRequest("POST", creds.uaaURI.String(), nil) req.SetBasicAuth(creds.clientID, creds.clientSecret) resp, _ := client.Do(req)

auth = requests.auth.HTTPBasicAuth(client_id, client_secret) r = requests.post( url=uaa_uri, headers={'accept': 'application/json'}, params={'grant_type': ‘client_credentials'}, auth=client_auth)

PYTHON

both of those examples assume certificate validation, see the docs for skipping it.

➜ curl “https://user:[email protected]/oauth/token" -d grant_type=client_credentials -X POST | jq “.” { "access_token": “eyJhbG…C6Fvww”, "token_type": "bearer", "expires_in": 43199, "scope": "doppler.firehose", "jti": "2d56398d-ba65-4dbd-9f43-4d1a302709a1" }

STOP WITH YOUR TRICKERY AND GIVE ME A SHELL MAN

THE IMPORTANT THINGS

1. UAA exposes an API protected by HTTP Basic Auth that gives us a token http(s)://uaa.example.com/oauth/token

2. We use a client_id & client_secret for auth

3. We pass it the grant type as a parameter grant_type=client_credentials

4. It gives us back a token and time to live

note that you can indeed send your credentials in clear text. that’s a bad idea.you can, and should prevent http requests from getting to uaa as your traffic can be sniffed.

http://uaa.example.com/

UAA IS FOR PEOPLE TOO?• CLI Plugins you’ve already assumed the identity of the user

• Scripts should use & assume the identity of the logged in user via cf curl

• password_grant otherwise (but srsly, why do it the hard way?)

• Web apps use the regular OAuth flow

For applications that act on behalf of humans we have a few options.

For scripts cf curl is a good option as it means you can avoid (hopefully) managing usernames and passwords in the scripts.

require 'bundler' require 'sinatra' require 'omniauth' require 'omniauth-uaa-oauth2'

use Rack::Session::Cookie, :secret => ‘its a secret' use OmniAuth::Builder do provider :cloudfoundry, 'app', 'appclientsecret', {:auth_server_url => “https://login.run.cf.io", :token_server_url => “https://uaa.run.cf.io"} end

class App < Sinatra::Base get '/' do <ul><li><a href='/auth/cloudfoundry'>Sign in with Cloud Foundry</a></li></ul> end

get '/auth/cloudfoundry/callback' do content_type 'application/json' request.env['omniauth.auth'].to_hash.to_json rescue "No Data" end get '/auth/failure' do content_type 'text/plain' request.env['omniauth.auth'].to_hash.inspect rescue "No Data" end end

run App.new

This one doesn’t do a redirect like I normally do, use has to click the auth link.

Firehose(the tattletale)

WHAT IS THE FIREHOSE?

• The firehose is where we go to listen

• Combined stream of logs from all apps, plus metrics data from CF components

• Exposed as a web socket endpoint

• Accessed via doppler.firehose scope

The scope means it’s unlikely your user accounts have access to it. Login w/ adminOr create user in uaac.

WHATS IN THE FIREHOSE• A lot, but not everything

• https://github.com/cloudfoundry/loggregator/wiki/Firehose-Metric-Catalog

• https://docs.google.com/spreadsheets/d/176yIaJChXEmvm-CjybmwopdRGQfDGrSzo3J_Mx8mMnk/edit#gid=472741134

• heartbeats are being removed (rejoice!)

• https://lists.cloudfoundry.org/archives/list/[email protected]/message/EPTBPAJLBV4LAG72GOSK3KYO7MKZPL7B/

I find the most interesting things in the firehose to be http related. I can tell the status of every request in and out of the system, as well as it’s response time. This helps me paint a picture of “normal”. It also helps me figure out “lights are on but no one’s home” situations. CF health check just make a connection on a socket, they don’t check http response codes. So the firehose http metrics give me a good view of what my users are seeing.

https://docs.google.com/spreadsheets/d/176yIaJChXEmvm-CjybmwopdRGQfDGrSzo3J_Mx8mMnk/edit#gid=472741134

https://lists.cloudfoundry.org/archives/list/[email protected]/message/EPTBPAJLBV4LAG72GOSK3KYO7MKZPL7B/

HOW TO DISCOVER STUFF?

• noaa firehose sample app and grep

• ➜ export DOPPLER_ADDR=wss://doppler.10.244.0.34.xip.io:443

• ➜ CF_ACCESS_TOKEN=`cf oauth-token | grep bearer` ./firehose_sample

• https://github.com/cloudfoundry/noaa/blob/master/firehose_sample/main.go

*note the app ignores cert validation errors.

wss://doppler.10.244.0.34.xip.io:443

https://github.com/cloudfoundry/noaa/blob/master/firehose_sample/main.go

WHAT’S INTERESTING• POST’s to the Cloud Controller

origin:"router__0" eventType:HttpStartStop timestamp:1439755833436264212 deployment:"cf-warden" job:"router_z1" index:"0" ip:"10.244.0.22" httpStartStop:<startTimestamp:1439755833419943436 stopTimestamp:1439755833436264212 requestId:<low:8452822747789629966 high:15632988695117975649 > peerType:Client method:POST uri:"api.10.244.0.34.xip.io/v2/organizations" remoteAddress:"10.0.2.15:39470" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 contentLength:1298 parentRequestId:<low:5137755037457773227 high:6003671325381183557 > instanceId:"10.244.0.134:9022" >

WHAT’S INTERESTING• POST’s to the Cloud Controller

origin:"router__0" eventType:HttpStartStop timestamp:1439755833436264212 … peerType:Client method:POST uri:"api.10.244.0.34.xip.io/v2/organizations" remoteAddress:"10.0.2.15:39470" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 … >

Someone just created an org successfully.

WHATS INTERESTING➜ egrep "method:PUT|method:POST" /tmp/firehose.out | egrep -v "uaa|login" origin:"router__0" eventType:HttpStartStop timestamp:1439756656563113538 deployment:"cf-warden" job:"router_z1" index:"0" ip:"10.244.0.22" httpStartStop:<startTimestamp:1439756656523306795 stopTimestamp:1439756656563113538 requestId:<low:5639804705379164088 high:13655764414813434448 > peerType:Client method:POST uri:"api.10.244.0.34.xip.io/v2/apps" remoteAddress:"10.0.2.15:43117" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 contentLength:1485 parentRequestId:<low:9677111003368022686 high:16762836456174757743 > instanceId:"10.244.0.134:9022" > origin:"router__0" eventType:HttpStartStop timestamp:1439756656563296157 deployment:"cf-warden" job:"router_z1" index:"0" ip:"10.244.0.22" httpStartStop:<startTimestamp:1439756656523150050 stopTimestamp:1439756656563296157 requestId:<low:9677111003368022686 high:16762836456174757743 > peerType:Server method:POST uri:"api.10.244.0.34.xip.io/v2/apps" remoteAddress:"10.0.2.15:43117" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 contentLength:1485 > origin:"router__0" eventType:HttpStartStop timestamp:1439756656712256742 deployment:"cf-warden" job:"router_z1" index:"0" ip:"10.244.0.22" httpStartStop:<startTimestamp:1439756656666492905 stopTimestamp:1439756656712256742 requestId:<low:8304746205643063272 high:4491838666105706059 > peerType:Client method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79/routes/15879527-b813-4165-9b04-8dc3f5af2a31" remoteAddress:"10.0.2.15:43129" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 contentLength:1503 parentRequestId:<low:16449840403178608098 high:13013670532211215483 > instanceId:"10.244.0.134:9022" > origin:"legacy" eventType:LogMessage deployment:"cf-warden" job:"api_z1" index:"0" ip:"10.244.0.134" logMessage:<message:"Updated app with guid d2cd4770-64e0-44fe-a997-f2e7e6dc5c79 ({\"route\"=>\"15879527-b813-4165-9b04-8dc3f5af2a31\"})" message_type:OUT timestamp:143975\ 6656694498836 app_id:"d2cd4770-64e0-44fe-a997-f2e7e6dc5c79" source_type:"API" source_instance:"0" > origin:"router__0" eventType:HttpStartStop timestamp:1439756656712519692 deployment:"cf-warden" job:"router_z1" index:"0" ip:"10.244.0.22" httpStartStop:<startTimestamp:1439756656666355668 stopTimestamp:1439756656712519692 requestId:<low:16449840403178608098 high:13013670532211215483 > peerType:Server method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79/routes/15879527-b813-4165-9b04-8dc3f5af2a31" remoteAddress:"10.0.2.15:43129" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 contentLength:1503 > origin:"router__0" eventType:HttpStartStop timestamp:1439756656786930549 deployment:"cf-warden" job:"router_z1" index:"0" ip:"10.244.0.22" httpStartStop:<startTimestamp:1439756656765375916 stopTimestamp:1439756656786930549 requestId:<low:15224145774697211575 high:9071476560656655465 > peerType:Client method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79/bits" remoteAddress:"10.0.2.15:43133" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 contentLength:270 parentRequestId:<low:13565402856673872720 high:11582024932942486143 > instanceId:"10.244.0.134:9022" > origin:"router__0" eventType:HttpStartStop timestamp:1439756656787167491 deployment:"cf-warden" job:"router_z1" index:"0" ip:"10.244.0.22" httpStartStop:<startTimestamp:1439756656765234958 stopTimestamp:1439756656787167491 requestId:<low:13565402856673872720 high:11582024932942486143 > peerType:Server method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79/bits" remoteAddress:"10.0.2.15:43133" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 contentLength:270 > origin:"router__0" eventType:HttpStartStop timestamp:1439756663113749547 deployment:"cf-warden" job:"router_z1" index:"0" ip:"10.244.0.22" httpStartStop:<startTimestamp:1439756661872770559 stopTimestamp:1439756663113749547 requestId:<low:10468161744519186019 high:3081438611027230039 > peerType:Client method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79" remoteAddress:"10.0.2.15:43167" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 contentLength:4073 parentRequestId:<low:16375959389728685162 high:5351609031812398929 > instanceId:"10.244.0.134:9022" > origin:"router__0" eventType:HttpStartStop timestamp:1439756663114085243 deployment:"cf-warden" job:"router_z1" index:"0" ip:"10.244.0.22" httpStartStop:<startTimestamp:1439756661872568200 stopTimestamp:1439756663114085243 requestId:<low:16375959389728685162 high:5351609031812398929 > peerType:Server method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79" remoteAddress:"10.0.2.15:43167" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 contentLength:4073 >

lets distill this and just look at app interactions

WHATS INTERESTINGtimestamp:1439756656563113538 httpStartStop:<startTimestamp:1439756656523306795 stopTimestamp:1439756656563113538 peerType:Client method:POST uri:"api.10.244.0.34.xip.io/v2/apps" remoteAddress:"10.0.2.15:43117" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 instanceId:"10.244.0.134:9022" > timestamp:1439756656563296157 httpStartStop:<startTimestamp:1439756656523150050 stopTimestamp:1439756656563296157 peerType:Server method:POST uri:"api.10.244.0.34.xip.io/v2/apps" remoteAddress:"10.0.2.15:43117" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 > timestamp:1439756656712256742 httpStartStop:<startTimestamp:1439756656666492905 stopTimestamp:1439756656712256742 peerType:Client method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79/routes/15879527-b813-4165-9b04-8dc3f5af2a31" remoteAddress:"10.0.2.15:43129" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 instanceId:"10.244.0.134:9022" > timestamp:1439756656712519692 httpStartStop:<startTimestamp:1439756656666355668 stopTimestamp:1439756656712519692 peerType:Server method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79/routes/15879527-b813-4165-9b04-8dc3f5af2a31" remoteAddress:"10.0.2.15:43129" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 > timestamp:1439756656786930549 httpStartStop:<startTimestamp:1439756656765375916 stopTimestamp:1439756656786930549 peerType:Client method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79/bits" remoteAddress:"10.0.2.15:43133" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 instanceId:"10.244.0.134:9022" > timestamp:1439756656787167491 httpStartStop:<startTimestamp:1439756656765234958 stopTimestamp:1439756656787167491 peerType:Server method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79/bits" remoteAddress:"10.0.2.15:43133" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 > timestamp:1439756663113749547 httpStartStop:<startTimestamp:1439756661872770559 stopTimestamp:1439756663113749547 peerType:Client method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79" remoteAddress:"10.0.2.15:43167" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 instanceId:"10.244.0.134:9022" > timestamp:1439756663114085243 httpStartStop:<startTimestamp:1439756661872568200 stopTimestamp:1439756663114085243 peerType:Server method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79" remoteAddress:"10.0.2.15:43167" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 >

lets distill further and just grab some of the fields we’re interested in. We can see some requests being passed through the system by looking at the http traffic. Kind of a poor mans after the fact zipkin.

WHATS INTERESTINGtimestamp:1439756656563113538 httpStartStop:<startTimestamp:1439756656523306795 stopTimestamp:1439756656563113538 peerType:Client method:POST uri:"api.10.244.0.34.xip.io/v2/apps" remoteAddress:"10.0.2.15:43117" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 instanceId:"10.244.0.134:9022" > timestamp:1439756656563296157 httpStartStop:<startTimestamp:1439756656523150050 stopTimestamp:1439756656563296157 peerType:Server method:POST uri:"api.10.244.0.34.xip.io/v2/apps" remoteAddress:"10.0.2.15:43117" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 > timestamp:1439756656712256742 httpStartStop:<startTimestamp:1439756656666492905 stopTimestamp:1439756656712256742 peerType:Client method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79/routes/15879527-b813-4165-9b04-8dc3f5af2a31" remoteAddress:"10.0.2.15:43129" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 instanceId:"10.244.0.134:9022" > timestamp:1439756656712519692 httpStartStop:<startTimestamp:1439756656666355668 stopTimestamp:1439756656712519692 peerType:Server method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79/routes/15879527-b813-4165-9b04-8dc3f5af2a31" remoteAddress:"10.0.2.15:43129" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 > timestamp:1439756656786930549 httpStartStop:<startTimestamp:1439756656765375916 stopTimestamp:1439756656786930549 peerType:Client method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79/bits" remoteAddress:"10.0.2.15:43133" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 instanceId:"10.244.0.134:9022" > timestamp:1439756656787167491 httpStartStop:<startTimestamp:1439756656765234958 stopTimestamp:1439756656787167491 peerType:Server method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79/bits" remoteAddress:"10.0.2.15:43133" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 > timestamp:1439756663113749547 httpStartStop:<startTimestamp:1439756661872770559 stopTimestamp:1439756663113749547 peerType:Client method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79" remoteAddress:"10.0.2.15:43167" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 instanceId:"10.244.0.134:9022" > timestamp:1439756663114085243 httpStartStop:<startTimestamp:1439756661872568200 stopTimestamp:1439756663114085243 peerType:Server method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79" remoteAddress:"10.0.2.15:43167" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 >

So now we can see that we have a client talking to a server and a server talking to a server

WHATS INTERESTINGtimestamp:1439756656563113538 httpStartStop:<startTimestamp:1439756656523306795 stopTimestamp:1439756656563113538 peerType:Client method:POST uri:"api.10.244.0.34.xip.io/v2/apps" remoteAddress:"10.0.2.15:43117" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 instanceId:"10.244.0.134:9022" > timestamp:1439756656712256742 httpStartStop:<startTimestamp:1439756656666492905 stopTimestamp:1439756656712256742 peerType:Client method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79/routes/15879527-b813-4165-9b04-8dc3f5af2a31" remoteAddress:"10.0.2.15:43129" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 instanceId:"10.244.0.134:9022" > timestamp:1439756656786930549 httpStartStop:<startTimestamp:1439756656765375916 stopTimestamp:1439756656786930549 peerType:Client method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79/bits" remoteAddress:"10.0.2.15:43133" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 instanceId:"10.244.0.134:9022" > timestamp:1439756663113749547 httpStartStop:<startTimestamp:1439756661872770559 stopTimestamp:1439756663113749547 peerType:Client method:PUT uri:"api.10.244.0.34.xip.io/v2/apps/d2cd4770-64e0-44fe-a997-f2e7e6dc5c79" remoteAddress:"10.0.2.15:43167" userAgent:"go-cli 6.11.2-2a26d55 / darwin" statusCode:201 instanceId:"10.244.0.134:9022" >

lets distill further, to just client operations.


Lets make sure we’re tracing one user’s experience through the system.

note the port changed? Why? I suspect the CLI had to open a new connection..


lets distill further, to just client operations, we can trace one user’s experience through the system.

How long did their operation take? (6.5 seconds), firehose timestamps are nano seconds.


What did they do what was the output?

First one crates the app. second one uploads the bitsthird one does a state change to start apps.

WHY IS THAT INTERESTING?

• If I can learn what normal interaction patterns are, I can identify abnormal ones and investigate

• Are apps failing to start after a push?

• For all teams?

to me anyway.

this lets us identify and view actions across the entirety of the user base.

Could you do this with platform logs? Probably. The beauty of the firehose is it gives you all of it, I omitted the logs from my example for readability and dramatic effect.

Collector(older tattletale)

WHAT IS THE COLLECTOR• The collector is the “legacy” aggregator of CF metrics

• No log messages

• Grabbed all the /varz and /healthz data

• The collector still has a lot of relevant runtime information in it

• available_stagers, healthy, system.cpu.wait

Most of the collector usage is for dashboards.

WHAT IS THE COLLECTOR FOR• It’s also how we export metrics in open source via historian plugins

• https://github.com/cloudfoundry/collector/tree/master/lib/collector/historian

• Vendors (well Pivotal at least) have their own spin

• Pivotal Ops Metrics (JMX)

The historian plugins change the way data is presented. They are usually monitoring solution specific.

https://github.com/cloudfoundry/collector/tree/master/lib/collector/historian

COLLECTOR DOWNSIDE

• Vendors don’t install all the plugins in their distribution

• Consumption can be irritating, lots of guesswork

• You may end up pretending to be a historian plugin, or consuming ones output.

Cloud Controller(the brains)

The CC is where we got to listen and act.

CLOUD CONTROLLER

• The CC API is what we talk to the most

• api.run.pivotal.io

• The CC API is where we go to act and report

THE CC API IS UNSURPRISING• Which is to say, the api has all of the things you would expect

• /v2/apps

• /v3/apps (experimental)

• /v2/organizations

• /v2/events

• /v2/spaces

or kinda boring

API DETAILS

• the v3 api is totes HATEOAS compliant!!!

• the v2 api has some links conventions

THE API TEAM COMMITTED TO MAINTAINING BACKWARDS COMPATIBILITY

• v3 api’s will have v2 compatibility layers

• I was there, they said it at cf summit, it’s like they wrote it in their own blood

API DETAILS

• The v2 api is very, very, very verbose

• The v3 api is more targeted

API DETAILS

• v3 api is fun to talk about, but as of today much of it doesn’t exist outside of documentation.

• we will see some examples later.

Pitfalls and gotchas

http://imgur.com/gallery/LaJ9Kmo

http://imgur.com/gallery/LaJ9Kmo

VERSIONSdoes thing you read about exist in your

version

EXPERIMENTAL REALLY MEANS EXPERIMENTAL.

Theres a good chance it won’t be there (or be the same) in your vendors next release

(I’m looking at you async service broker api)

SOMETIMES IT’S NOT PRETTY

➜ cf curl /v2/apps/<a guid> | grep buildpack "buildpack": null, "detected_buildpack": "PHP",

TOKENS MUST BE ACQUIREDAND THEY EXPIRE

https://github.com/cloudfoundry/cf-uaa-lib (Ruby)https://github.com/cloudfoundry/omniauth-uaa-oauth2 (more ruby)

https://github.com/krujos/uaaclientcredentials (Go)

https://github.com/cloudfoundry/cf-uaa-lib

https://github.com/cloudfoundry/omniauth-uaa-oauth2

https://github.com/krujos/uaaclientcredentials

CLIENTS VS USERS

https://docs.google.com/document/d/1YU01vrGmIhSHCKOFIRY7EhU8qCbUHJsw3ZXgdnUIfeI/editTODO: move that somewhere permanent, and more public.

What’s a client?What’s a user?

https://docs.google.com/document/d/1YU01vrGmIhSHCKOFIRY7EhU8qCbUHJsw3ZXgdnUIfeI/edit

UAA

http? https?be safe!

SCOPES, AUTHORITY AND THE GRANTING OF POWER

SO HOW DO I BUILD A THING?

• it starts with cf curl and jq

• or the firehose and grep

• lets see what the firehose has to say about started apps?

export DOPPLER_ADDR=wss://doppler.10.244.0.34.xip.io:443CF_ACCESS_TOKEN=`cf oauth-token | grep bearer` ./firehose_sample | grep -i started

cf stop hello-nodecf start hello-node

• how about orphaned user provided services?

cf cups my-svc -p '{"foo":"bar"}'cf curl /v2/user_provided_service_instancescf curl `cf curl /v2/user_provided_service_instances | jq -r '.resources[0].entity.service_bindings_url'` | jq ‘.total_results'

Think about what we would have had to do to report on that with the CLI?

• lets look at something more complex.

• how many ai’s are you running?

• How does the usage-report plugin work?

• https://github.com/krujos/usagereport-plugin

demo the plugin

sometimes its about mashing things up

Why is how many ai’s your running a hard question to answer?Why is how many ai’s your running an important question to answer? As someone who’s paying for IT charge back, am I paying too much?How much ram am I consuming, are my quotas right?make note of the lack of auth, plugin assumes current user, works for everyone based on their cf access.magic starts here: https://github.com/krujos/usagereport-plugin/blob/master/usagereport.go#L57https://github.com/krujos/usagereport-plugin/blob/master/apihelper/apihelper.go

https://github.com/krujos/usagereport-plugin

• What’s fun? what should we try to figure out?

cloud foundry api for fun and ops

Software