Performance Monitoring

Yan Cui

pseudo-

RT

withAOP & AWS

by@theburningmonk

Server-side Developer @

iwi by numbers• 400k+ DAU

• ~100m requests/day

• 25k+ concurrent users

• 1500+ requests/s

• 7000+ IO opts/s

• 100+ commodity servers

• 75ms average latency

What is my code doing at runtime?

Runtime Monitoring != Profiling

Monitoring

• Difficult in a distributed environment

• Lots of effort to set up

Monitoring

• Achievable in 3 simple steps

1. Gather runtime data

2. Publish

3. Visualize

Gathering Data

• Method execution time

• Method execution count

• Error count

• Requires (lots) boilerplate to set up

AOP to the rescue!

AOP

• Aspect Oriented Programming

• Cross-Cutting Concerns

• PostSharp

Image by Mike Rohde

PostSharp

Code demo goes here

Publish

• Where to?...

• Amazon CloudWatch

Visualize

Alarms

Notifications

• Notify via email/SMS

Image by Mike Rohde

CloudWatch

Visualize

• Amazon Management Console

–Min, Max, Average, Count

–Up to 14 days of data

–Down to 1 minute intervals

Amazon CloudWatch

• Cheap

• Supports alarms and notifications

• Visualization tool

AOP + Amazon CloudWatch =

What to Monitor?

• IO operations

• CPU intensive operations

• Service entry-points

Caution

• Overhead

• Only monitor what you need

• Make it configurable/optional?

Thank You!

@theburningmonk