Varnish the Drupal Way

About me

• Boyan Borisov• Team Leader @

Propeople• boyanb@propeople.dk• @boyan_borisov• Skype: boian.borisov• http://linkedin.com/in/

boyanborisov

What is Varnish?

• Reverse proxy cache server • ... and more (load balancer, filter, director, ...)• Open Source (BSD), commercially supported• Typically in front of a LAMP stack

Why do we need Varnish?

• Believe or not Drupal is not very fast by default ;)

• “Large Scale Drupal” is not only a sales phrase but the present and the future of Drupal

• May be this is the best and fastest way to achieve great performance results

Why do we need Varnish?

What does Varnish do?

What can be cached?

• Drupal pages for anonymous users • Static content: images, js, css • Other types of static or generated content• Varnish stores a cache entry for each Host +

URL

What should NOT be cached in Varnish?

• Responses to POST requests • Responses with a Set-Cookie : header • User-specific content (logged-in page

requests) • Very large media files (> 100MB) • Has a Cache-Control: no-cache header

When the cache will expire?

• Response header: Cache-Control: max-age= 300

• If no Cache-Control specified then Varnish uses the Expires date, if specified

• If no Cache-Control or Expires header then uses Varnish's default TTL which is 2 minutes but could be changed

But wait… what about Drupal?

• Install varnish• Install and configure the Drupal varnish module -

https://drupal.org/project/varnish• Update the default Varnish VCL configuration with a Drupal

related one– https://github.com/boyanborisov/varnish-for-drupal– https

://fourkitchens.atlassian.net/wiki/display/TECH/Configure+Varnish+3+for+Drupal+7

– https://drupal.org/node/1427486• And don’t forget to enable the caching for the anonymous

users

Varnish state machine

VCL – Varnish Configuration Language

• Extend and modify Varnish cache policies • Add behaviors during each phase of delivery• Each Request is processed separately and

independently• States are isolated but are related• Return statements exit one state and start

another• VCL defaults are ALWAYS appended below your

own VCL

Other VCL features

• Add/remove/change HTTP headers • Redirects based on request parameters • Rewrite requests • Retry failed requests • Saint and Grace: Deliver cached content when backend

is unresponsive • ACLs: Network-address filtering • Request filtering ESI: Separate cache entries and

separate cache policies for each part of a document• Inline C code

VCL – Subroutines – breaking it down

• vcl_init – VCL is loaded, no request yet; VMOD initialization• vcl_recv – Beginning of request, req is in scope• vcl_pipe – Client & backend data passed unaltered• vcl_pass – Request goes to backend and not cached• vcl_hash – call hash_data to add to the hash• vcl_hit – called on request found in the cache• vcl_miss – called on request not found in the cache• vcl_fetch – called on document retrieved from backend• vcl_deliver – called prior to delivery of cached object• vcl_error – called on errors

Directors

• Directors allow you to talk to the backend servers• Directors are a glorified reverse proxy– Allows for certain types of load balancing – Allows for talking to a cluster

“A director is a logical group of backend servers clustered together for redundancy. The basic role of the director is to let Varnish choose a backend server amongst several so if one is down another can be used.”

Varnish realtime tools

• varnishstat – realtime status and performance metrics

• varnishhist – live frequency plot of response times

• varnishtop – live list of hottest files • varnishlog – analyze incoming HTTP headers • Varnish CLI – live admin capabilities

Drupal specifics

• Drupal emits Vary – Cookie,Accept-Encoding• Cookie – Avoids cache collisions between

users Accept-Encoding : maintain separate cache entries for compressed content (gzip, deflate) vs. uncompressed content

ESI – Edge Side Includes

• ESI is a small markup language much like SSI (server side includes) to include fragments (or dynamic content for that matter)

• Think of it as replacing regions inside of a page as if you were using XHR (AJAX) but single threaded.

• Three Statements can be utilized– esi:include – Include a page– esi:remove – Remove content– <!-- esi --> - ESI disabled, execute normally

How to cache authenticated users?• Authcache - https://drupal.org/project/authcache

“Authcache saves the final rendered HTML of a page to serve visitors. A separate cache is created for each user role as defined by the administrator, so some roles can be excluded if necessary.

Authcache places priority on serving pages to the visitor as fast as possible. After a page is loaded by the browser, a second HTTP request may be performed via Ajax. This initiates a lightweight DRUPAL_BOOTSTRAP_SESSION that allows SQL queries to be executed , and returns any user-customized data to the page (such as form tokens or default values on a contact form).”

How to cache authenticated users?

How to cache authenticated users?

How to cache authenticated users?

“Production graphs from intranet solution developed by Propeople ( around 50 000 users ). Web CPU usage below 1% and varnish hit ratio > 80%. Varnish & Propeople rocks ”

Rumen YordanovOpen Source Department manager @Propeople

Cache invalidation

• Drupal invalidate the whole Varnish cache on node/comment/etc save by default

• Expire module is a solution - https://drupal.org/project/expire

Purging and banning

• Purging – a purge is what happens when you pick out an object from the cache and discard it along with its variants. Usually a purge is invoked through HTTP with the method PURGE.

• Banning – You can think of bans as a sort of a filter on objects already in the cache. You ban certain content from being served from your cache. You can ban content based on any metadata we have. A ban will only work on objects already in the cache, it does not prevent new content from entering the cache or being served.

Demo

Questions?