varnish presentation for the symfony zaragoza user group

r.v. var·nished, var·nish·ing, var·nish·es

1. To cover with varnish.2. To give a smooth and glossy finish to.3. To give a deceptively attractive appearance to; gloss over.

by jnerin 1

Server (cubietruck)

●AllWinnerTech SOC A20○ ARM® Cortex™-A7

Dual-Core●2GB ram●SAMSUNG SpinPoint

F2 EG 1.5TB sata hard disk

●1Gb Ethernet

2

Server (cubietruck)● Apache 2.2.27-r4

○ KeepAlive On○ MaxKeepAliveRequests 100○ KeepAliveTimeout 15○ mpm worker:

StartServers 2MinSpareThreads 25MaxSpareThreads 75ThreadsPerChild 25MaxClients 150MaxRequestsPerChild 10000

● PHP 5.5.16 (opcache enabled by default 64M)

● mariadb 10.0.14 (have_query_cache=YES but query_cache_size=0)

3

Test.sh#!/bin/bashURL="$1"LOG="$URL.$(date +%Y%m%d-%H%M%S).log"HOST="test"IP="192.168.2.3:8080"REQUESTS=10000CONCURRENCY=20exec 6<&1 # Save STDOUT FDexec >$LOGecho "- Simple test, no concurrency"/usr/sbin/ab2 -n $REQUESTS -H "Host: $HOST" http://$IP/$URLecho "- Simple test, concurrency"/usr/sbin/ab2 -n $REQUESTS -c $CONCURRENCY -H "Host: $HOST" http://$IP/$URLecho "- Concurrency & KeepAlive"/usr/sbin/ab2 -k -n $REQUESTS -c $CONCURRENCY -H "Host: $HOST" http://$IP/$URLecho "- Simple test, no concurrency (Accept-Encoding: gzip)"/usr/sbin/ab2 -n $REQUESTS -H "Accept-encoding: gzip" -H "Host: $HOST" http://$IP/$URLecho "- Simple test, concurrency (Accept-Encoding: gzip)"/usr/sbin/ab2 -n $REQUESTS -c $CONCURRENCY -H "Accept-encoding: gzip" -H "Host: $HOST" http://$IP/$URLecho "- Concurrency & KeepAlive (Accept-Encoding: gzip)"/usr/sbin/ab2 -k -n $REQUESTS -c $CONCURRENCY -H "Accept-encoding: gzip" -H "Host: $HOST" http://$IP/$URLexec 1<&6 6<&- # Restore STDOUT FDegrep "^(-|Requests per second)" $LOG

4

Test1.html

<html><head><title>Test 1</title></head><body>Test 1</body></html>

Simple test, no concurrency222.56 req/sSimple test, concurrency197.06 req/sConcurrency & KeepAlive679.81 req/sSimple test, no concurrency (Accept-Encoding: gzip)232.75 req/sSimple test, concurrency (Accept-Encoding: gzip)415.48 req/sConcurrency & KeepAlive (Accept-Encoding: gzip)825.26 req/s

5

Test2.php<html><head><title>Test 2</title></head><body><?php

require_once('defaults.php');$link = mysqli_connect($mysql_server,

$mysql_user, $mysql_password,$mysql_database) or die("Error " . mysqli_error($link));

if ($result = mysqli_query($link, "SELECT temperature FROM oregon_data ORDER BY receiver_timestamp DESC LIMIT 1")) {

echo mysqli_fetch_row($result)[0]; mysqli_free_result($result);

}mysqli_close($link);

?></body></html>


6

mysql -e 'show global variables like "%query_cache%";show status like "Qcache%"'+------------------------------+----------+| Variable_name | Value |+------------------------------+----------+| have_query_cache | YES || query_cache_limit | 1048576 || query_cache_min_res_unit | 4096 || query_cache_size | 33554432 || query_cache_strip_comments | OFF || query_cache_type | ON || query_cache_wlock_invalidate | OFF |+------------------------------+----------++-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 33544000 || Qcache_hits | 60030 || Qcache_inserts | 10 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 97 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+

MySQL Query Cache (Qcache)

7

Test2.php + QCache<html><head><title>Test 2</title></head><body><?php

require_once('defaults.php');$link = mysqli_connect($mysql_server,







8

Test3.php (QCache + pconnect)<html><head><title>Test 3</title></head><body><?php

require_once('defaults.php');$link = mysqli_connect("p:$mysql_server",







9

Let's try varnishvarnishd -a :8080 \-s malloc,200M \-f /etc/varnish.vcl \-T 127.0.0.1:8081 \-u apache -g apache \-t 120

10

Let's try varnishvarnishd -a :8080 \-s malloc,200M \-f /etc/varnish.vcl \-T 127.0.0.1:8081 \-u apache -g apache \-t 120


11

Obligatory Graph"Lies, damned lies, and statistics" (Mark Twain)

12

Varnish Control Language (VCL)https://www.varnish-software.com/static/book/VCL_Basics.html

13

https://www.varnish-software.com/static/book/VCL_Basics.html


1 vcl 4.0; 2 # Based on: https://github.com/mattiasgeniar/varnish-4.0-configuration-templates/blob/master/default.vcl 3 # Corrected & improved for 4.0.2 by [email protected] 4 import std; 5 import directors; 6 backend server1 { # Define one backend 7 .host = "127.0.0.1"; # IP or Hostname of backend 8 .port = "80"; # Port Apache or whatever is listening 9 .max_connections = 300; # That's it 10 .probe = { 11 #.url = "/"; # short easy way (GET /) 12 # We prefer to only do a HEAD / 13 .request = 14 "HEAD / HTTP/1.1" 15 "Host: localhost" 16 "Connection: close"; 17 .interval = 5s; # check the health of each backend every 5 seconds 18 .timeout = 1s; # timing out after 1 second. 19 # If 3 out of the last 5 polls succeeded the backend is considered healthy, otherwise it will be marked as sick 20 .window = 5; 21 .threshold = 3; 22 } 23 .first_byte_timeout = 300s; # How long to wait before we receive a first byte from our backend? 24 .connect_timeout = 5s; # How long to wait for a backend connection? 25 .between_bytes_timeout = 2s; # How long to wait between bytes received from our backend? 26 } 27 acl purge { 28 # ACL we'll use later to allow purges 29 "localhost"; 30 "127.0.0.1"; 31 "::1"; 32 }

16

https://github.com/mattiasgeniar/varnish-4.0-configuration-templates/blob/master/default.vcl




34 /* 35 acl editors { 36 # ACL to honor the "Cache-Control: no-cache" header to force a refresh but only from selected IPs 37 "localhost"; 38 "127.0.0.1"; 39 "::1"; 40 } 41 */ 42 43 sub vcl_init { 44 # Called when VCL is loaded, before any requests pass through it. Typically used to initialize VMODs. 45 46 new vdir = directors.round_robin(); 47 vdir.add_backend(server1); 48 # vdir.add_backend(server...); 49 # vdir.add_backend(servern); 50 } 51 52 sub vcl_recv { 53 # Called at the beginning of a request, after the complete request has been received and parsed. Its purpose is to decide whether or not to serve the request, how to do it, and, if applicable, which backend to use. 54 # also used to modify the request 55 56 set req.backend_hint = vdir.backend(); # send all traffic to the vdir director 57 58 if (req.restarts == 0) { 59 if (req.http.X-Forwarded-For) { # set or append the client.ip to X-Forwarded-For header 60 set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip; 61 } else { 62 set req.http.X-Forwarded-For = client.ip; 63 } 64 } 17

66 # Normalize the header, remove the port (in case you're testing this on various TCP ports) 67 set req.http.Host = regsub(req.http.Host, ":[0-9]+", ""); 68 69 # Normalize the query arguments 70 set req.url = std.querysort(req.url); 71 72 # Allow purging 73 if (req.method == "PURGE") { 74 if (!client.ip ~ purge) { # purge is the ACL defined at the begining 75 # Not from an allowed IP? Then die with an error. 76 return (synth(405, "This IP is not allowed to send PURGE requests.")); 77 } 78 # If you got this stage (and didn't error out above), purge the cached result 79 return (purge); 80 } 81 82 # Only deal with "normal" types 83 if (req.method != "GET" && 84 req.method != "HEAD" && 85 req.method != "PUT" && 86 req.method != "POST" && 87 req.method != "TRACE" && 88 req.method != "OPTIONS" && 89 req.method != "PATCH" && 90 req.method != "DELETE") { 91 /* Non-RFC2616 or CONNECT which is weird. */ 92 return (pipe); 93 } 94 95 # Implementing websocket support (https://www.varnish-cache.org/docs/4.0/users-guide/vcl-example-websockets.html) 96 if (req.http.Upgrade ~ "(?i)websocket") { 97 return (pipe); 98 }

18

https://www.varnish-cache.org/docs/4.0/users-guide/vcl-example-websockets.html)



$ GET -de "http://test:8080/varnish.vcl.html?q1=1&q2=2&q3=3" |egrep "(Cache|Age|ETag|Expires|Date)"Cache-Control: max-age=14400Date: Wed, 08 Oct 2014 14:25:28 GMTAge: 984ETag: W/"3d838b6-b4f1-504e9efa40148"Expires: Wed, 08 Oct 2014 18:25:28 GMTClient-Date: Wed, 08 Oct 2014 14:41:53 GMTX-Cache: HITX-Cache-Hits: 5

$ GET -de "http://test:8080/varnish.vcl.html?q2=2&q1=1&q3=3" |egrep "(Cache|Age|ETag|Expires|Date)"Cache-Control: max-age=14400Date: Wed, 08 Oct 2014 14:25:28 GMTAge: 994ETag: W/"3d838b6-b4f1-504e9efa40148"Expires: Wed, 08 Oct 2014 18:25:28 GMTClient-Date: Wed, 08 Oct 2014 14:42:02 GMTX-Cache: HITX-Cache-Hits: 6

Effect of set req.url = std.querysort(req.url);

19

100 # Only cache GET or HEAD requests. This makes sure the POST requests are always passed.101 if (req.method != "GET" && req.method != "HEAD") {102 return (pass);103 }104 105 # Some generic URL manipulation, useful for all templates that follow106 # First remove the Google Analytics added parameters, useless for our backend107 if (req.url ~ "(\?|&)(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=") {108 set req.url = regsuball(req.url, "&(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "");109 set req.url = regsuball(req.url, "\?(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "?");110 set req.url = regsub(req.url, "\?&", "?");111 set req.url = regsub(req.url, "\?$", "");112 }113 114 # Strip hash, server doesn't need it.115 if (req.url ~ "\#") {116 set req.url = regsub(req.url, "\#.*$", "");117 }118 119 # Strip a trailing ? if it exists120 if (req.url ~ "\?$") {121 set req.url = regsub(req.url, "\?$", "");122 }

20

124 # Some generic cookie manipulation, useful for all templates that follow125 # Remove the "has_js" cookie126 set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");127 128 # Remove any Google Analytics based cookies129 set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");130 set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");131 set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");132 set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");133 set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");134 135 # Remove the Quant Capital cookies (added by some plugin, all __qca)136 set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");137 138 # Remove the AddThis cookies139 set req.http.Cookie = regsuball(req.http.Cookie, "__atuvc=[^;]+(; )?", "");140 141 # Remove a ";" prefix in the cookie if present142 set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");143 144 # Are there cookies left with only spaces or that are empty?145 if (req.http.cookie ~ "^\s*$") {146 unset req.http.cookie;147 }

21

149 # Normalize Accept-Encoding header150 # straight from the manual: https://www.varnish-cache.org/docs/3.0/tutorial/vary.html151 # TODO: Test if it's still needed, Varnish 4 now does this by itself if http_gzip_support = on152 # https://www.varnish-cache.org/docs/trunk/users-guide/compression.html153 # https://www.varnish-cache.org/docs/trunk/phk/gzip.html154 if (req.http.Accept-Encoding) {155 if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {156 # No point in compressing these157 unset req.http.Accept-Encoding;158 } elsif (req.http.Accept-Encoding ~ "gzip") {159 set req.http.Accept-Encoding = "gzip";160 } elsif (req.http.Accept-Encoding ~ "deflate") {161 set req.http.Accept-Encoding = "deflate";162 } else {163 # unkown algorithm164 unset req.http.Accept-Encoding;165 }166 }167 168 if (req.http.Cache-Control ~ "(?i)no-cache") {169 #if (req.http.Cache-Control ~ "(?i)no-cache" && client.ip ~ editors) { # create the acl editors if you want to restrict the Ctrl-F5170 # http://varnish.projects.linpro.no/wiki/VCLExampleEnableForceRefresh171 # Ignore requests via proxy caches and badly behaved crawlers172 # like msnbot that send no-cache with every request.173 if (! (req.http.Via || req.http.User-Agent ~ "(?i)bot" || req.http.X-Purge)) {174 #set req.hash_always_miss = true; # Doesn't seems to refresh the object in the cache175 return(purge); # Couple this with restart in vcl_purge and X-Purge header to avoid loops176 }177 }

22

https://www.varnish-cache.org/docs/3.0/tutorial/vary.html

https://www.varnish-cache.org/docs/3.0/tutorial/vary.html

https://www.varnish-cache.org/docs/trunk/users-guide/compression.html

https://www.varnish-cache.org/docs/trunk/users-guide/compression.html

https://www.varnish-cache.org/docs/trunk/phk/gzip.html

https://www.varnish-cache.org/docs/trunk/phk/gzip.html

http://varnish.projects.linpro.no/wiki/VCLExampleEnableForceRefresh

http://varnish.projects.linpro.no/wiki/VCLExampleEnableForceRefresh

179 # Large static files are delivered directly to the end-user without180 # waiting for Varnish to fully read the file first.181 # Varnish 4 fully supports Streaming, so set do_stream in vcl_backend_response()182 if (req.url ~ "^[^?]*\.(mp[34]|rar|tar|tgz|gz|wav|zip|bz2|xz|7z|avi|mov|ogm|mpe?g|mk[av])(\?.*)?$") {183 unset req.http.Cookie;184 return (hash);185 }186 187 # Remove all cookies for static files188 # A valid discussion could be held on this line: do you really need to cache static files that don't cause load? Only if you have memory left.189 # Sure, there's disk I/O, but chances are your OS will already have these files in their buffers (thus memory).190 # Before you blindly enable this, have a read here: http://mattiasgeniar.be/2012/11/28/stop-caching-static-files/191 if (req.url ~ "^[^?]*\.(bmp|bz2|css|doc|eot|flv|gif|gz|ico|jpeg|jpg|js|less|pdf|png|rtf|swf|txt|woff|xml)(\?.*)?$") {192 unset req.http.Cookie;193 return (hash);194 }195 196 # Send Surrogate-Capability headers to announce ESI support to backend197 set req.http.Surrogate-Capability = "key=ESI/1.0";198 199 if (req.http.Authorization) {200 # Not cacheable by default201 return (pass);202 }203 204 return (hash);205 }

23

http://mattiasgeniar.be/2012/11/28/stop-caching-static-files/




207 sub vcl_pipe {208 # Called upon entering pipe mode. In this mode, the request is passed on to the backend, and any further data from both the client and backend is passed on unaltered until either end closes the connection. Basically, Varnish will degrade into a simple TCP proxy, shuffling bytes back and forth. For a connection in pipe mode, no other VCL subroutine will ever get called after vcl_pipe.209 210 # Note that only the first request to the backend will have211 # X-Forwarded-For set. If you use X-Forwarded-For and want to212 # have it set for all requests, make sure to have:213 # set bereq.http.connection = "close";214 # here. It is not set by default as it might break some broken web215 # applications, like IIS with NTLM authentication.216 217 #set bereq.http.Connection = "Close";218 219 # Implementing websocket support (https://www.varnish-cache.org/docs/4.0/users-guide/vcl-example-websockets.html)220 if (req.http.upgrade) {221 set bereq.http.upgrade = req.http.upgrade;222 }223 224 return (pipe);225 }226 227 sub vcl_pass {228 # Called upon entering pass mode. In this mode, the request is passed on to the backend, and the backend's response is passed on to the client, but is not entered into the cache. Subsequent requests submitted over the same client connection are handled normally.229 230 # return (pass);231 }

24




233 # The data on which the hashing will take place234 sub vcl_hash {235 # Called after vcl_recv to create a hash value for the request. This is used as a key to look up the object in Varnish.236 237 hash_data(req.url);238 239 if (req.http.host) {240 hash_data(req.http.host);241 } else {242 hash_data(server.ip);243 }244 245 # hash cookies for requests that have them246 if (req.http.Cookie) {247 hash_data(req.http.Cookie);248 }249 }250 251 sub vcl_hit {252 # Called when a cache lookup is successful.253 254 if (obj.ttl >= 0s) {255 # A pure unadultered hit, deliver it256 return (deliver);257 }

25

259 # https://www.varnish-cache.org/docs/trunk/users-guide/vcl-grace.html260 # When several clients are requesting the same page Varnish will send one request to the backend and place the others on hold while fetching one copy from the backend. In some products this is called request coalescing and Varnish does this automatically.261 # If you are serving thousands of hits per second the queue of waiting requests can get huge. There are two potential problems - one is a thundering herd problem - suddenly releasing a thousand threads to serve content might send the load sky high. Secondly - nobody likes to wait. To deal with this we can instruct Varnish to keep the objects in cache beyond their TTL and to serve the waiting requests somewhat stale content.262 269 # We have no fresh fish. Lets look at the stale ones.270 if (std.healthy(req.backend_hint)) {271 # Backend is healthy. Limit age to 10s.272 if (obj.ttl + 10s > 0s) {273 #set req.http.grace = "normal(limited)";274 return (deliver);275 } else {276 # No candidate for grace. Fetch a fresh object.277 return(fetch);278 }279 } else {280 # backend is sick - use full grace281 if (obj.ttl + obj.grace > 0s) {282 #set req.http.grace = "full";283 return (deliver);284 } else {285 # no graced object.286 return (fetch);287 }288 }289 290 291 # fetch & deliver once we get the result292 return (fetch); # Dead code, keep as a safeguard293 } 26

https://www.varnish-cache.org/docs/trunk/users-guide/vcl-grace.html

https://www.varnish-cache.org/docs/trunk/users-guide/vcl-grace.html

295 sub vcl_miss {296 # Called after a cache lookup if the requested document was not found in the cache. Its purpose is to decide whether or not to attempt to retrieve the document from the backend, and which backend to use.297 298 return (fetch);299 }300 301 # Handle the HTTP request coming from our backend302 sub vcl_backend_response {303 # Called after the response headers has been successfully retrieved from the backend.304 305 # Pause ESI request and remove Surrogate-Control header306 if (beresp.http.Surrogate-Control ~ "ESI/1.0") {307 unset beresp.http.Surrogate-Control;308 set beresp.do_esi = true;309 }310 311 # Enable cache for all static files312 # The same argument as the static caches from above: monitor your cache size, if you get data nuked out of it, consider giving up the static file cache.313 # Before you blindly enable this, have a read here: http://mattiasgeniar.be/2012/11/28/stop-caching-static-files/314 if (bereq.url ~ "^[^?]*\.(bmp|bz2|css|doc|eot|flv|gif|gz|ico|jpeg|jpg|js|less|mp[34]|pdf|png|rar|rtf|swf|tar|tgz|txt|wav|woff|xml|zip)(\?.*)?$") {315 unset beresp.http.set-cookie;316 }

27





319 # Large static files are delivered directly to the end-user without320 # waiting for Varnish to fully read the file first.321 # Varnish 4 fully supports Streaming, so use streaming here to avoid locking.322 if (bereq.url ~ "^[^?]*\.(mp[34]|rar|tar|tgz|gz|wav|zip|bz2|xz|7z|avi|mov|ogm|mpe?g|mk[av])(\?.*)?$") {323 unset beresp.http.set-cookie;324 set beresp.do_stream = true; # Check memory usage it'll grow in fetch_chunksize blocks (128k by default) if 325 # the backend doesn't send a Content-Length header, so only enable it for big objects326 set beresp.do_gzip = false; # Don't try to compress it for storage327 }329 # Sometimes, a 301 or 302 redirect formed via Apache's mod_rewrite can mess with the HTTP port that is being passed along.330 # This often happens with simple rewrite rules in a scenario where Varnish runs on :80 and Apache on :8080 on the same box.331 # A redirect can then often redirect the end-user to a URL on :8080, where it should be :80.332 # This may need finetuning on your setup.333 #334 # To prevent accidental replace, we only filter the 301/302 redirects for now.335 if (beresp.status == 301 || beresp.status == 302) {336 set beresp.http.Location = regsub(beresp.http.Location, ":[0-9]+", "");337 }339 # Disable queue coalescence for uncacheable objects340 if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") {341 set beresp.uncacheable = true; # hit_for_pass342 set beresp.ttl = 120s; # Remeber this343 return (deliver);344 }346 # Allow stale content, in case the backend goes down.347 # make Varnish keep all objects for 6 hours beyond their TTL348 set beresp.grace = 6h;350 return (deliver);351 } 28

353 # The routine when we deliver the HTTP request to the user354 # Last chance to modify headers that are sent to the client355 sub vcl_deliver {356 # Called before a cached object is delivered to the client.357 358 if (obj.hits > 0) { # Add debug header to see if it's a HIT/MISS and the number of hits, disable when not needed359 set resp.http.X-Cache = "HIT";360 } else {361 set resp.http.X-Cache = "MISS";362 }363 # Please note that obj.hits behaviour changed in 4.0, now it counts per objecthead, not per object364 # and obj.hits may not be reset in some cases where bans are in use. See bug 1492 for details.365 # So take hits with a grain of salt366 set resp.http.X-Cache-Hits = obj.hits;368 # Remove some headers: PHP version369 unset resp.http.X-Powered-By;371 # Remove some headers: Apache version & OS372 unset resp.http.Server;373 unset resp.http.X-Drupal-Cache;374 unset resp.http.X-Varnish;375 unset resp.http.Via;376 unset resp.http.Link;378 return (deliver);379 }380 381 sub vcl_purge {382 # restart request383 set req.http.X-Purge = "Yes";384 return(restart);385 }

29

387 sub vcl_synth {388 if (resp.status == 720) {389 # We use this special error status 720 to force redirects with 301 (permanent) redirects390 # To use this, call the following from anywhere in vcl_recv: error 720 "http://host/new.html"391 set resp.http.Location = resp.reason;392 set resp.status = 301;393 return (deliver);394 } elseif (resp.status == 721) {395 # And we use error status 721 to force redirects with a 302 (temporary) redirect396 # To use this, call the following from anywhere in vcl_recv: error 720 "http://host/new.html"397 set resp.http.Location = resp.reason;398 set resp.status = 302;399 return (deliver);400 }401 402 return (deliver);403 }404 405 406 sub vcl_fini {407 # Called when VCL is discarded only after all requests have exited the VCL. Typically used to clean up VMODs.408 409 return (ok);410 }

30

http://host/new.html






sub vcl_recv {

if (req.url ~ "^/varnish/") {# Lame example to generate content, look down in vcl_synth for status 820return (synth(820, "Test"));

}}

sub vcl_synth {if (resp.status == 820) {

# Lame example showing how to generate synthetic content.# Usually to do this you would use a vmod from https://www.varnish-cache.org/vmods# to do things like memcache/redis/database/... lookups and return meaningful things# expected url /varnish/parameter1/min/max/ -> /varnish/real_random_api/8/32/set resp.http.Content-Type = "text/plain; charset=utf-8";set req.http.X-parameter1 = regsub(req.url, "^/varnish/(.*?)/.*", "\1");set req.http.X-min = regsub(req.url, "^/varnish/.*?/(.*?)/.*", "\1");set req.http.X-max = regsub(req.url, "^/varnish/.*?/.*?/(.*?)/.*", "\1");synthetic("You are " + client.ip + {"

url: "} + req.url + {"reason: "} + resp.reason + {"1st parameter: "} + req.http.X-parameter1 + {"Have your random number between "} + req.http.X-min + " and " + req.http.X-max + " : " + std.random(std.real(req.http.X-min, 0), std.real(req.http.X-max, 100)) + {"End"});

set resp.status = 200; # Be careful to set it after using resp.reason or it'll overwrite itreturn (deliver);

}

return (deliver);}

31

https://www.varnish-cache.org/vmods


$ GET -E http://localhost:8080/varnish/reason/-8/42/GET http://localhost:8080/varnish/reason/-8/42/User-Agent: lwp-request/6.03 libwww-perl/6.05

200 OKConnection: closeDate: Mon, 24 Nov 2014 15:26:12 GMTAccept-Ranges: bytesServer: VarnishContent-Length: 133Content-Type: text/plain; charset=utf-8Client-Date: Mon, 24 Nov 2014 15:26:12 GMTClient-Peer: 127.0.0.1:8080Client-Response-Num: 1X-Varnish: 42

You are 127.0.0.1url: /varnish/reason/-8/42/reason: Test1st parameter: reasonHave a random number between -8 and 42 : -2.204End

Playing with vcl_synth

32

But that's not enough:$ GET -e http://test/test1.html200 OKConnection: closeDate: Fri, 03 Oct 2014 16:37:59 GMTAccept-Ranges: bytesETag: "3d838cc-51-504816b16fe6a"Server: ApacheVary: Accept-EncodingContent-Length: 81Content-Type: text/htmlLast-Modified: Fri, 03 Oct 2014 09:32:57 GMT

33

But that's not enough:# http://httpd.apache.org/docs/2.0/mod/mod_expires.html

ExpiresActive On ExpiresByType image/gif "access plus 4 days"

#<FilesMatch "\.(gif|jpe?g|png)$"># Header merge Cache-Control "public" # Default anyway# ExpiresDefault "access plus 4 days"#</FilesMatch>

ExpiresByType image/jpg "access plus 4 days" ExpiresByType image/jpeg "access plus 4 days" ExpiresByType image/png "access plus 4 days" ExpiresByType image/x-icon "access plus 4 days" ExpiresByType text/css "access plus 3 days" ExpiresByType application/x-javascript "access plus 3 days" ExpiresByType text/html "access plus 4 hours" ExpiresByType application/xml "access plus 1 hours"

34

But that's not enough:$ GET -e http://test/test1.html200 OKCache-Control: max-age=14400Connection: closeDate: Fri, 03 Oct 2014 17:37:42 GMTAccept-Ranges: bytesETag: "3d838cc-51-504816b16fe6a"Server: ApacheVary: Accept-EncodingContent-Length: 81Content-Type: text/htmlExpires: Fri, 03 Oct 2014 21:37:42 GMTLast-Modified: Fri, 03 Oct 2014 09:32:57 GMT

35

Numbers on faster hardware(AMD FX(tm)-8120 Eight-Core Processor 32GB RAM)

/usr/sbin/ab2 -n 500000 -c 200 -k -H "Accept-encoding: gzip" "http://localhost/prueba.html"

Requests per second: 24318.71 [#/sec] (mean)Time per request: 8.224 [ms] (mean)Time per request: 0.041 [ms] (mean, across all concurrent requests)Transfer rate: 8161.06 [Kbytes/sec] received

/usr/sbin/ab2 -n 500000 -c 200 -k -H "Accept-encoding: gzip" "http://localhost:8080/prueba.html"

Requests per second: 25780.13 [#/sec] (mean)Time per request: 7.758 [ms] (mean)Time per request: 0.039 [ms] (mean, across all concurrent requests)Transfer rate: 8611.04 [Kbytes/sec] received 36

Resources● https://www.varnish-cache.org/docs

○ https://www.varnish-software.com/static/book/VCL_Basics.html○ https://www.varnish-cache.org/trac/wiki

● High Performance Browser Networking by Ilya Grigorik (http://chimera.labs.oreilly.com/books/1230000000545/index.html)

● Caching in HTTP (http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html)

● The only moment you should ever cache static files is if you have memory to spare (http://mattiasgeniar.be/2012/11/28/stop-caching-static-files/)

● How to Use Varnish to Speed up my Website (http://symfony.com/doc/current/cookbook/cache/varnish.html)

● https://www.varnish-cache.org/vmods (redis, memcache, MySQL, PostgreSQL, ratelimit, device detection, json, auth, ...)

37

https://www.varnish-cache.org/docs

https://www.varnish-cache.org/docs



https://www.varnish-cache.org/trac/wiki

https://www.varnish-cache.org/trac/wiki

http://chimera.labs.oreilly.com/books/1230000000545/index.html



http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html






http://symfony.com/doc/current/cookbook/cache/varnish.html




HTTP/1.1 200 OK38

varnish presentation for the symfony zaragoza user group

Software