apache-trafﬁcserver-server documentation

Apache-Trafficserver-ServerDocumentation

latest

7 21, 2017

Contents

1 11.1 Apache Traffic Server? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Typographic Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Other Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Getting Started 32.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.3 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.4 Logging and Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.5 Further Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3 Administrator’s Guide 113.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.2 Traffic Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.4 Interacting with Traffic Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543.5 Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543.6 Cache Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583.7 Plugins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643.8 Event and Error Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1793.9 Configuring Traffic Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2133.10 Performance Tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2143.11 Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2183.12 Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

4 Developer’s Guide 2594.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2594.2 Release Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2604.3 Contributing to Traffic Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2624.4 Using Vagrant to Test Traffic Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2624.5 Debugging and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2664.6 Cache Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2684.7 Plugin Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2914.8 Configuration Variable Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3764.9 API Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3794.10 Continuous Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475

i

4.11 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4754.12 Host Resolution Proposal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 488

5 Appendices 4915.1 Command Line Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4915.2 Frequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5055.3 Troubleshooting Tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5115.4 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5145.5 HTTP Status Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516

6 Indices and tables 517

ii

CHAPTER 1

Apache Traffic Server?

Apache Traffic Server™ is a high-performance web proxy cache that improves network efficiency and performanceby caching frequently-accessed information at the edge of the network. This brings content physically closer to endusers, while enabling faster delivery and reduced bandwidth use. Traffic Server is designed to improve content deliveryfor enterprises, Internet service providers (ISPs), backbone providers, and large intranets by maximizing existing andavailable bandwidth.

This manual will explore every aspect of installing, managing, extending, and troubleshooting Traffic Server.

Typographic Conventions

This documentation uses the following typographic conventions:

Italic Used to introduce new terms on their initial appearance.

Example: The Apache Traffic Server™ object storage is based on a cyclone buffer architecture. Cyclone buffersare a form of storage addressing in which a single writer continually reclaims the oldest allocations for useby new updates.

Monospace Represents C/C++ language statements, commands, file paths, file content, and computer output.

Example: The default installation prefix for Traffic Server is /usr/local/ts.

Bracketed Monospace Represents variables for which you should substitute a value in file content or commands.

Example: Running the command traffic_line -r <name> will display the current value of a perfor-mance statistic, where <name> is the statistic whose value you wish to view.

Ellipsis Indicates the omission of irrelevant or unimportant information.

1

Apache-Trafficserver-Server Documentation, latest

Other Resources

Websites

Official Website https://trafficserver.apache.org/

The official Apache Traffic Server™ project website is hosted by the Apache Software Foundation. Documen-tation, software downloads, community resource links, security announcements, and more are located, or linkedto, at the site.

Online Documentation https://docs.trafficserver.apache.org/

The most up to date version of the documentation is hosted at Read the Docs, with built-in search functionality.Documentation for past releases is also available.

Jira Bug Tracker https://issues.apache.org/jira/browse/TS

If you wish to report bugs, or look for open issues on which you may help contribute to the Traffic Server project,please visit the public Jira bug tracker site.

Mailing Lists

User List The user’s mailing list offers support and discussions oriented to users and administrators of the TrafficServer software.

Send an email to [email protected] to join the list.

Developer List If you have questions about, or wish to discuss, the development of Traffic Server, plugins for theserver, or other developer-oriented matters, the developer list offers an active list of both core project membersand external contributors.

Send an email to [email protected] to join the list.

Internet Relay Chat (IRC)

The #traffic-server channel on irc.freenode.net is the official IRC resource for the Traffic Serverproject, and boasts active discussions.

Community Forums

2 Chapter 1.

https://trafficserver.apache.org/

https://docs.trafficserver.apache.org/

https://issues.apache.org/jira/browse/TS

CHAPTER 2

Getting Started

Introduction

Apache Traffic Server™ provides a high-performance and scalable software solution for both forward and reverseproxying of HTTP/HTTPS traffic, and may be configured to run in either or both modes simultaneously. This GettingStarted guide explains the basic steps an administrator new to Traffic Server will need to perform to get the softwareup and running in a minimal configuration as quickly as possible.

Example Scenario

In this guide, we will use the fictional company Acme Widgets as a basis for the configuration examples. AcmeWidgets has a product brochure website (assumed to reside at the domain www.acme.com) that performs verypoorly. The content management software they chose takes an unbearable amount of time to generate pages on everyrequest and their engineering team has chosen Traffic Server as a caching proxy layer to improve site performance.

Separately, Acme Widgets has decided to use Traffic Server to help improve the performance of their office’s Internetaccess, which is hobbled by their reliance on an aging leased line and certain employees’ predilection for extracurric-ular web browsing.

Terminology

This guide uses some terms which may be unfamiliar to administrators new to proxy servers.

Origin Server The server which generates the content you wish to proxy (and optionally cache) with Traffic Server.In a forward proxy configuration, the origin server may be any remote server to which a proxied client attemptsto connect. In a reverse proxy configuration, the origin servers are typically a known set of servers for whichyou are using Traffic Server as a performance-enhancing caching layer.

Reverse Proxy A reverse proxy appears to outside users as if it were the origin server, though it does not generate thecontent itself. Instead, it intercepts the requests and, based on the configured rules and contents of its cache, willeither serve a cached copy of the requested content itself, or forward the request to the origin server, possiblycaching the content returned for use with future requests.

3


Forward Proxy A forward proxy brokers access to external resources, intercepting all matching outbound trafficfrom a network. Forward proxies may be used to speed up external access for locations with slow connections(by caching the external resources and using those cached copies to service requests directly in the future), ormay be used to restrict or monitor external access.

Transparent Proxy A transparent proxy may be either a reverse or forward proxy (though nearly all reverse proxiesare deployed transparently), the defining feature being the use of network routing to send requests through theproxy without any need for clients to configure themselves to do so, and often without the ability for thoseclients to bypass the proxy.

For a more complete list of definitions, please see the Glossary.

Installation

As with many other software packages, you may have the option of installing Apache Traffic Server™ from youroperating system distribution’s packages, or compiling and installing from the source code yourself. Distributionpackages may lag behind the current stable release of Traffic Server, sometimes by a significant amount. While wewill cover briefly the packages required for common operating system distributions, the remainder of this guide willbe assuming the latest stable release of Traffic Server when discussing configuration parameters and available features.

Installing From Distribution Packages

It is recommended that you install from source code, as described in the section below, rather than rely on yourdistribution’s packages if you wish to have access to the latest features and fixes in Traffic Server.

Ubuntu

The Canonical repositories up through, and including, Utopic Unicorn only provide Traffic Server v3.2.x.

sudo apt-get install trafficserver

RHEL / CentOS

Traffic Server is available through the EPEL repositories. If you do not have those configured on your machine yet,you must install them first with the following:

wget https://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpmsudo rpm -Uvh epel-release-7*.rpm

Ensuring that you replace the release number with a value that is appropriate for your system. Once you have EPELinstalled, you may install Traffic Server itself.

sudo yum install trafficserver

OmniOS (illumos)

OmniOS (an illumos-based distribution) provides the Traffic Server package in its public OmniTI-MS publisher.

sudo pkg set-publisher -g http://pkg.omniti.com/omniti-ms/ ms.omniti.comsudo pkg install omniti/server/trafficserver

4 Chapter 2. Getting Started


The latest release published, at the time of this writing, is Traffic Server v4.2.1.

Installing From Source Code

To install from source code, you will need to have the following tools and libraries on the machine used to build TrafficServer:

• pkgconfig

• libtool

• gcc (>= 4.3 or clang > 3.0)

• GNU make

• openssl

• tcl

• expat

• pcre

• libcap

• flex (for TPROXY)

• hwloc

• lua

• curses (for traffic_top)

• curl (for traffic_top)

To build Traffic Server from a Git clone (the method we will be using in this guide), you will also need the following:

• git

• autoconf

• automake

In this guide, Traffic Server will be built to use the default nobody user and group and will be installed to /opt/ts.It is assumed that all of the dependencies are located in standard paths. If this is not the case, you may need to use theappropriate --with-<package> options as detailed in the output of ./configure --help.

1. Clone the repository (you may skip this if you have downloaded an archive of the source code to build a specificofficial release of Traffic Server instead of the HEAD from source control):

git clone https://git-wip-us.apache.org/repos/asf/trafficserver.git

2. Change to the cloned (or unarchived) directory:

cd trafficserver/

3. If you have cloned the repository from Git, you will need to generate the configure script before proceeding:

autoreconf -if

4. Configure the source tree:

./configure --prefix=/opt/ts

2.2. Installation 5


5. Build Traffic Server with the generated Makefiles, and test the results:

makemake check

6. Install Traffic Server to the configured location:

sudo make install

7. Optionally, you may find it prudent to run the regression tests on your newly installed Traffic Server:

cd /opt/tssudo bin/traffic_server -R 1

Configuration

We will be tackling two separate configuration scenarios in the following sections. The first is the most common ap-plication of a performance-enhancing caching proxy for externally-facing websites, a transparent and caching reverseproxy which forwards all requests presented to it to a single origin address and caches the responses based on theircache control headers (as well as some simple heuristics for specific content types when cache control headers are notpresent).

The second configuration we will review is a very basic transparent forward proxy, typically used in situations whereyou either need to improve the performance of a local network’s use of external resources, or you wish to have thecapability of monitoring or filtering the traffic.

Configuring A Reverse Proxy

A minimal reverse proxy configuration requires changes to only a few configuration files, which will all be locatedin the /opt/ts/etc/trafficserver directory if you have configured your installation per the instructions inInstalling From Source Code above.

For these examples, we will be assuming that Traffic Server is running on the same host machine as the origin website.This is not a requirement, and you may choose to run Traffic Server on an entirely different host, even connected toentirely different networks, as long as Traffic Server is able to reach the origin host.

Enable Reverse Proxying

Within the records.config configuration file, ensure that the following settings have been configured as shownbelow:

CONFIG proxy.config.http.cache.http INT 1CONFIG proxy.config.reverse_proxy.enabled INT 1CONFIG proxy.config.url_remap.remap_required INT 1CONFIG proxy.config.url_remap.pristine_host_hdr INT 1CONFIG proxy.config.http.server_ports STRING 8080

:ts:cv:‘proxy.config.http.cache.http‘ Enables caching of proxied HTTP requests.

:ts:cv:‘proxy.config.reverse_proxy.enabled‘ Enables reverse proxying support.

:ts:cv:‘proxy.config.url_remap.remap_required‘ This setting requires that a remap rule exist before Traffic Serverwill proxy the request and ensures that your proxy cannot be used to access the content of arbitrary websites(allowing someone of malicious intent to potentially mask their identity to an unknowning third party).



:ts:cv:‘proxy.config.url_remap.pristine_host_hdr‘ This setting causes Traffic Server to keep the Host: client re-quest header intact which is necessary in cases where your origin servers may be performing domain-basedvirtual hosting, or taking other actions dependent upon the contents of that header.

:ts:cv:‘proxy.config.http.server_ports‘ This configures Traffic Server to bind itself to the port 8080 for HTTP traf-fic.

Configure Origin Location

The previous settings enable reverse proxying (and prevent flagrant abuse of it), but now Traffic Server needs to knowwhat to proxy. This is achieved by writing remap rules, which make use of the core Configuration Remap Plugin.For our Getting Started guide’s Acme Widgets example scenario, we have very simple needs and want little morethan to proxy all requests to our single origin server. This is accomplished with the following rule added to theremap.config configuration:

regex_map http://(.*)/ http://localhost:80/

It is worth pausing at this point to note that in a reverse proxying scenario, it is Traffic Server itself which should beresponding to HTTP requests made to your public domain. While you first configure and evaluate whether TrafficServer will meet your needs, your origin server will continue to reside at the public-facing domain name on the defaultports, should you move your Traffic Server configuration into production your DNS records for the domain(s) youwish to proxy and cache should resolve to the host(s) running Traffic Server (in the event that you run it on a separatehost from your origin). Your origin should be accessible at a different address (or bind to different ports if you arerunning both your origin service and Traffic Server on the same host) and should no longer receive requests sent to theprimary domain on its default ports.

In our Acme Widgets scenario, they ultimately decide to deploy Traffic Server at which point, since they are runningboth Traffic Server and their origin web server on the same host, they reconfigure their origin service to listen on port8080 instead of the default, and change Traffic Server to bind to 80 itself. Updating the remap is thus required, andit should now be:


Now all requests made to www.acme.com are received by Traffic Server which knows to proxy those requests tolocalhost:8080 if it cannot already serve them from its cache. Because we enabled pristine host headers earlier,the origin service will continue to receive Host: www.acme.com in the HTTP request.

Adjust Cache Parameters

The default Traffic Server configuration will provide a 256 MB disk cache, located in var/trafficserver/underneath your install prefix. You may wish to adjust either or both of the size and location of this cache. This isdone with the storage.config configuration file. In our example, Acme Widgets has dedicated a large storagepool on their cache server which is mounted at /cache. To use this, and to disable the default cache storage setting,the following will be the sole entry in storage.config:

/cache/trafficserver 500G

: Changes to the cache configuration require a restart of Traffic Server.

You may also wish to use raw devices, or partition your cache storage. Details on these features may be found in thesection of the Administrator’s Guide.

2.3. Configuration 7


Final Configurations

Once completed, the following configuration files for Acme Widgets contain the following entries:

records.config:

CONFIG proxy.config.http.cache.http INT 1CONFIG proxy.config.reverse_proxy.enabled INT 1CONFIG proxy.config.url_remap.remap_required INT 1CONFIG proxy.config.url_remap.pristine_host_hdr INT 1CONFIG proxy.config.http.server_ports STRING 80

remap.config:


storage.config:

/cache/trafficserver 500G

Configuring A Forward Proxy

Configuring a forward proxy with Traffic Server is somewhat more straightforward, though there are two main ap-proaches for how to direct client traffic through the proxy: explicit or transparent.

More detail on the process is also available in the Administrator’s Guide in the Forward Proxy section. This guide willcover only a minimal configuration.

Enable Forward Proxying

Contrary to a reverse proxy, where you have a defined list of origin servers for which you wish to proxy (and optionallycache), a forward proxy is used to proxy (and optionally cache) for arbitrary remote hosts. As such, the followingsettings in records.config are the base configuration for a minimal forward proxy:

CONFIG proxy.config.url_remap.remap_required INT 0CONFIG proxy.config.http.cache.http INT 1

:ts:cv:‘proxy.config.url_remap.remap_required‘ Disables the requirement for a remap rule to exist and match theincoming request before Traffic Server will proxy the request to the remote host.

:ts:cv:‘proxy.config.http.cache.http‘ Enables caching of proxied HTTP requests.

If your installation will be strictly a forwarding proxy, then reverse proxying should be explicitly disabled:

CONFIG proxy.config.reverse_proxy.enabled INT 0

Explicit Versus Transparent

With forward proxying enabled, the next step is deciding how clients will connect through the proxy server. Explicitforward proxying requires that every client application be configured (manually or through whatever configurationmanagement system you may employ) to use the proxy. The presence of the proxy will be known to clients, and anyclients not configured to use the proxy will bypass it entirely unless you otherwise block network access for unproxiedtraffic.



With transparent proxying, clients require no configuration and may be given no option to bypass the proxy. Thisconfiguration requires your network to route all requests automatically to the proxy. The details of how to accomplishthis routing are entirely dependent upon the layout of your network and the routing devices you use.

For a more detailed discussion of the options, and starting points for configuring each in your network environment,please refer to the Forward Proxy section of the Administrator’s Guide.

Logging and Monitoring

Configuring Log Output

The log formats used by Traffic Server are highly configurable. While this guide will not go into full detail of thisversatility, it is useful to consider what style of logging you would like to perform. If your organization already makesuse of log monitoring or analysis tools that understand, for example, Netscape Extended-2 format you may wish toenable that logging format in addition to, or instead of, the default Traffic Server logs.

The Administrator’s Guide discusses logging options in great detail in Logging.

Using Traffic Top

Using Stats Over HTTP

Using Cache Inspector

Further Steps

By this point, you should have a fully functioning caching proxy, whether to help improve the customer facing per-formance of your website, or to assist in speeding up Internet access for your office while allowing for the possibilityof access control, content filtering, and/or usage monitoring. However, it is quite likely that your installation is notyet tuned properly and may not be providing the best experience Apache Traffic Server™ has to offer. It is stronglyrecommended that you consult the Performance Tuning guide.

You may also want to learn more about Event and Error Monitoring, or ensuring that your installation is properlysecured by reading the Security section. Properly sizing your cache, both the on-disk cache and the companionmemory cache, are important topics covered in .

2.4. Logging and Monitoring 9

CHAPTER 3

Administrator’s Guide

:

Global data networking has become part of everyday life: Internet users request billions of documents and petabytes ofdata, on a daily basis, to and from all parts of the world. Information is free, abundant, and accessible. Unfortunately,global data networking can also be a nightmare for IT professionals as they struggle with overloaded servers andcongested networks. It can be challenging to consistently and reliably accommodate society’s growing data demands.

Traffic Server is a high-performance web proxy cache that improves network efficiency and performance by cachingfrequently-accessed information at the edge of the network. This brings content physically closer to end users, whileenabling faster delivery and reduced bandwidth use. Traffic Server is designed to improve content delivery for enter-prises, Internet service providers (ISPs), backbone providers, and large intranets by maximizing existing and availablebandwidth.

Traffic Server Deployment Options

To best suit your needs, Traffic Server can be deployed in several ways:

• WEB

•

•

The following sections provide a summary of these Traffic Server deployment options. Please keep in mind that withevery one of these options Traffic Server can be run as a single instance, or as a multi-node cluster.

Traffic Server WEB

As a web proxy cache, Traffic Server receives user requests for web content as those requests travel to the destinedweb server (origin server). If Traffic Server contains the requested content, then it serves the content directly. If the

11


requested content is not available from cache, then Traffic Server acts as a proxy: it obtains the content from the originserver on the user’s behalf and also keeps a copy to satisfy future requests.

Traffic Server provides explicit proxy caching, in which the user’s client software must be configured to send requestsdirectly to Traffic Server. Explicit proxy caching is described in the Explicit Proxy Caching chapter.

Traffic Server can also be employed as a transparent caching proxy server, in which the client software needs no specialconfiguration or even knowledge of the proxy’s existence. This setup is described in the Transparent Proxying section.

Traffic Server

As a reverse proxy, Traffic Server is configured to be the origin server to which the user is trying to connect (typically,the origin server’s advertised hostname resolves to Traffic Server, which acts as the real origin server). The reverseproxy feature is also called server acceleration. Reverse proxy is described in more detail in Reverse Proxy and HTTPRedirects.

Traffic Server

Traffic Server can participate in flexible cache hierarchies, in which Internet requests not fulfilled from one cache arerouted to other regional caches, thereby leveraging the contents and proximity of nearby caches. In a hierarchy ofproxy servers, Traffic Server can act either as a parent or a child cache to other Traffic Server systems or to similarcaching products.

Traffic Server supports ICP (Internet Cache Protocol) peering. Hierarchical caching is described in more detail inHierarchical Caching.

There are a number of deployment options that Traffic Server does not support right out of the box. Such functionalitymay be implemented in a plugin, but in some cases Traffic Server’s internal APIs or architectural restrictions won’tmake it easy:

• Load Balancing - note that there is an experimental plugin for this, Balancer Plugin.

Traffic Server Components

Traffic Server consists of several components that work together to form a web proxy cache you can easily monitorand configure.

The Traffic Server Cache

The Traffic Server cache consists of a high-speed object database called the object store. The object store indexesobjects according to URLs and associated headers. Using sophisticated object management, the object store can cachealternate versions of the same object (perhaps in a different language or encoding type). It can also efficiently storevery small and very large objects, thereby minimizing wasted space. When the cache is full, Traffic Server removesstale data to ensure that the most requested objects are readily available and fresh.

Traffic Server is designed to tolerate total disk failures on any of the cache disks. If the disk fails completely, thenTraffic Server marks the entire disk as corrupt and continues to use remaining disks. If all of the cache disks fail, thenTraffic Server switches to proxy-only mode. You can partition the cache to reserve a certain amount of disk space forstoring data for specific protocols and origin servers. For more information about the cache, see HTTP Proxy Caching.

12 Chapter 3. Administrator’s Guide


RAM

Traffic Server maintains a small RAM cache that contains extremely popular objects. This RAM cache serves themost popular objects as fast as possible and reduces load on disks, especially during temporary traffic peaks. You canconfigure the RAM cache size to suit your needs. For detailed information, refer to Changing the Size of the RAMCache.

Host

The Traffic Server host database stores the domain name server (DNS) entries of origin servers to which TrafficServer connects to fulfill user requests. This information is used to adapt future protocol interactions and optimizeperformance. Along with other information, the host database tracks:

• DNS information (for fast conversion of hostnames to IP addresses).

• The HTTP version of each host (so advanced protocol features can be used with hosts running modern servers).

• Host reliability and availability information (so users will not wait for servers that are not running).

DNS

Traffic Server includes a fast, asynchronous DNS resolver to streamline conversion of hostnames to IP addresses.Traffic Server implements the DNS resolver natively by directly issuing DNS command packets rather than relyingon slower, conventional resolver libraries. Since many DNS queries can be issued in parallel and a fast DNS cachemaintains popular bindings in memory, DNS traffic is reduced.

Traffic Server Processes

Traffic Server contains three processes that work together to serve requests and manage, control, and monitor thehealth of the system.

• The traffic_server process is the transaction processing engine of Traffic Server. It is responsible foraccepting connections, processing protocol requests, and serving documents from the cache or origin server.

• The traffic_manager process is the command and control facility of the Traffic Server, responsible forlaunching, monitoring, and reconfiguring the traffic_server process. The traffic_manager processis also responsible for the proxy autoconfiguration port, the statistics interface, cluster administration, and virtualIP failover.

If the traffic_manager process detects a traffic_server process failure, it instantly restarts the pro-cess but also maintains a connection queue of all incoming requests. All incoming connections that arrive inthe several seconds before full server restart are saved in the connection queue and processed in first-come,first-served order. This connection queueing shields users from any server restart downtime.

• The traffic_cop process monitors the health of both the traffic_server and traffic_managerprocesses. The traffic_cop process periodically (several times each minute) queries thetraffic_server and traffic_manager processes by issuing heartbeat requests to fetch synthetic webpages. In the event of failure (if no response is received within a timeout interval or if an incorrect response isreceived), traffic_cop restarts the traffic_manager and traffic_server processes.

The figure below illustrates the three Traffic Server processes.

3.1. 13


3.1: Illustration of the three Traffic Server Processes



Traffic Server offers the following administration options:

• The traffic_ctl command-line interface is a text-based interface from which you can monitor Traffic Serverperformance and network traffic, as well as configure the Traffic Server system. From Traffic Line, you canexecute individual commands or script a series of commands in a shell.

• Various configuration files enable you to configure Traffic Server through a simple file-editing and signal-handling interface. Any changes you make through Traffic Line is automatically made to the configurationfiles as well.

• Finally, there is a clean C API which can be put to good use from a multitude of languages. The Traffic ServerAdmin Client demonstrates this for Perl.

Traffic Analysis Options

Traffic Server provides several options for network traffic analysis and monitoring:

• traffic_ctl enables you to collect and process statistics obtained from network traffic information.

• Transaction logging enables you to record information (in a log file) about every request Traffic Server receivesand every error it detects. By analyzing the log files, you can determine how many clients used the Traffic Servercache, how much information each of them requested, and what pages were most popular. You can also see whya particular transaction was in error and what state the Traffic Server was in at a particular time. For example,you can see that Traffic Server was restarted or that cluster communication timed out.

Traffic Server supports several standard log file formats, such as Squid and Netscape, and its own custom format.You can analyze the standard format log files with off-the-shelf analysis packages. To help with log file analysis,you can separate log files so that they contain information specific to protocol or hosts.

Traffic Server event and error logging, monitoring, and analysis is covered in greater detail in Event and Error Moni-toring.

Traffic Server

Traffic Server provides numerous options that enable you to establish secure communication between the Traffic Serversystem and other computers on the network. Using the security options, you can do the following:

• Control client access to the Traffic Server proxy cache.

• Configure Traffic Server to use multiple DNS servers to match your site’s security configuration. For example,Traffic Server can use different DNS servers, depending on whether it needs to resolve hostnames located insideor outside a firewall. This enables you to keep your internal network configuration secure while continuing toprovide transparent access to external sites on the Internet.

• Configure Traffic Server to verify that clients are authenticated before they can access content from the TrafficServer cache.

• Secure connections in reverse proxy mode between a client and Traffic Server, and Traffic Server and the originserver, using the SSL termination option.

• Control access via SSL (Secure Sockets Layer).

Traffic Server security options are described in more detail in Security.

3.1. 15


Tuning Traffic Server

Finally, this last chapter on Performance Tuning discusses the vast number of options that allow administrators tooptimally tune Apache Traffic Server for maximum performance.

Traffic Server

Supported Platforms

Traffic Server Versioning

Before you get started with Traffic Server you may have to decide which version you want to use. Traffic Serverfollows the Semantic Versioning guidelines.

A complete version number is made of a version-triplet: MAJOR.MINOR.PATCH.

As of v4.0.0, there are no longer any development (or unstable) releases. All releases are considered stable and readyfor production use. Releases within a major version are always upgrade compatible. More details are available on theNew Release Processes wiki page.

Sometimes we speak of trunk, master or HEAD, all of which are used interchangeably. Trunk and master, or some-times TIP or HEAD, refer to the latest code in a Git version control system (also referred to as a repository or Gitrepo). Master is always kept releasable, and compatible with the current major release version. Incompatible changesare sometimes committed on a next-major release branch; for example, we have the 5.0.x branch where changesincompatible with 4.x are managed.

Binary Distributions

Building From Source

Retrieving Tarballs

Compressed archives of the source code for Traffic Server are available on the official website’s Downloads page.From there you may select the version most appropriate for your needs. The Traffic Server project does not providebinary downloads.

Cloning from Version Control

Traffic Server uses a public Git repository for version control. This repository will also provide the most cutting edgesource code if you wish to test out the latest features and bug fixes.

: We do also have a GitHub Mirror that you may use to submit pull requests. However, it may not be entirelyup-to-date, and you should always refer to our official project Git repository for the very latest state of the source code.

Build Dependencies

In order to build Traffic Server from source you will need the following development tools and libraries installed:

• pkgconfig


http://semver.org

https://cwiki.apache.org/confluence/display/TS/Release+Process

https://trafficserver.apache.org/downloads

https://git-wip-us.apache.org/repos/asf/trafficserver.git

https://github.com/apache/trafficserver


• libtool

• gcc (>= 4.3 or clang > 3.0)

• GNU make

• openssl

• tcl

• expat

• pcre

• libcap

• flex (for TPROXY)

• hwloc

• lua

• curses (for traffic_top)

• curl (for traffic_top)

If you’re building from a git clone, you’ll also need:

• git

• autoconf

• automake

Layouts

Preparing the Source Tree

If you are buiding from a checkout of the Git repository, you will need to prepare the source tree by regenerating theconfiguration scripts. This is performed by running:

autoreconf -if

At the base directory of your local clone.

Configuration Options

Traffic Server uses the standard configure script method of configuring the source tree for building. A full list ofavailable options may always be obtained by running the following in the base directory of your unpackaged archiveor Git working copy:

./configure --help

1. A configure script will be generated from configure.ac which may now be used to configure the sourcetree for your build.

./configure --prefix=/opt/ats

By default, Traffic Server will be built to use the nobody user and group. You may change this with the--with-user argument to configure:

3.2. Traffic Server 17


./configure --prefix=/opt/ats --with-user=tserver

If dependencies are not in standard paths (/usr/local or /usr), you may need to pass options toconfigure to account for that:

./configure --prefix=/opt/ats --with-lua=/opt/csw

Most configure path-options accept a format of “INCLUDE_PATH:LIBRARY_PATH”:

./configure --prefix=/opt/ats --with-pcre=/opt/csw/include:/opt/csw/lib/amd64

2. Once the source tree has been configured, you may proceed on to building with the generated Makefiles. Themake check command may be used to perform sanity checks on the resulting build, prior to installation, andit is recommended that you use this.

makemake check

3. With the source built and checked, you may now install all of the binaries, header files, documentation, andother artifacts to their final locations on your system.

sudo make install

4. Finally, it is recommended that you run the regression test suite. Please note that the regression tests will onlybe successful with the default layout.

cd /opt/atssudo bin/traffic_server -R 1

Adding SPDY Support

Traffic Server v5.0.x and above are capable of supporting SPDY, but it is optional and must be explicitly configuredduring the build process. The support of SPDY is deprecated as of ATS v6.2.0, and will be removed in v7.0.0.

1. Clone the spdylay Git repository from tatsuhiro.

git clone https://github.com/tatsuhiro-t/spdylay

2. Generate fresh configure scripts.

autoreconf -if

3. Run the generated configuration script, choosing an installation location for the overlay.

./configure --prefix=/opt/spdylay

4. Build and install the overlay.

make install

5. Set the PKG_CONFIG_PATH enviroment variable to the appropriate path in your newly-installed overlay direc-tory.

export PKG_CONFIG_PATH=/opt/spdylay/lib/pkgconfig/



6. Finally, you may proceed with the regular building of Traffic Server from source, with the addition of a singlenew configuration option to enable SPDY support in the produced binaries.

./configure --enable-spdy

Start Traffic Server

To start Traffic Server manually, issue the trafficserver command, passing in the subcommand start. Thiscommand starts all the processes that work together to process Traffic Server requests as well as manage, control, andmonitor the health of the Traffic Server system.

bin/trafficserver start

The traffic_ctl provides a quick way of viewing Traffic Server statistics and configuring the Traffic Serversystem via a command-line interface.

traffic_ctl commands take the following form:

bin/traffic_ctl COMMAND COMMAND ...

For a list of traffic_ctl commands, enter:

bin/traffic_ctl

Please note that traffic_ctl, while a fine tool for administrators, is a poor choice for automation, especially thatof monitoring. See our chapter on monitoring for how to do that more efficiently and effectively.

Stop Traffic Server

To stop Traffic Server, always use the trafficserver command, passing in the attribute stop. This commandstops all the Traffic Server processes (traffic_manager, traffic_server, and traffic_cop). Do notmanually stop processes, as this can lead to unpredictable results.

bin/trafficserver stop

Session Protocol

Traffic Server supports some session level protocols in place of or on top of HTTP. These can be provided by a plugin(see New Protocol Plugins) or be one that is supported directly by Traffic Server. Note that the SPDY protocol isdeprecated as of v6.2.0, and will be removed in v7.0.0.

Session protocols are specified by explicit names:

• http/0.9

• http/1.0

• http/1.1

• http/2

• spdy/1

3.3. 19


• spdy/2

• spdy/3

• spdy/3.1

The session protocols supported on a proxy port are a subset of these values. For convenience some pseudo-values aredefined in terms of these fundamental protocols:

• http means http/0.9, http/1.0, and http/1.1

• spdy means spdy/3 and spdy/3.1.

• http2 means http/2

Each proxy port can be configured in :ts:cv:‘proxy.config.http.server_ports‘ to support a subset of these sessionprotocols. For TLS enabled connections this configuration controls which protocols are offered by NPN. Protocolsniffing is use for non-TLS proxy ports to determine which protocol is being used by the client. If the detectedprotocol is not supported for that proxy port the connection is dropped.

HTTP Proxy Caching

HTTP proxy caching enables you to store copies of frequently-accessed web objects (such as documents, images, andarticles) and then serve this information to users on demand. It improves performance and frees up Internet bandwidthfor other tasks.

Understanding HTTP Web Proxy Caching

Internet users direct their requests to web servers all over the Internet. A caching server must act as a web proxy serverso it can serve those requests. After a web proxy server receives requests for web objects, it either serves the requestsor forwards them to the origin server (the web server that contains the original copy of the requested information). TheTraffic Server proxy supports explicit proxy caching, in which the user’s client software must be configured to sendrequests directly to the Traffic Server proxy. The following overview illustrates how Traffic Server serves a request.

1. Traffic Server receives a client request for a web object.

2. Using the object address, Traffic Server tries to locate the requested object in its object database (cache).

3. If the object is in the cache, then Traffic Server checks to see if the object is fresh enough to serve. If it is fresh,then Traffic Server serves it to the client as a cache hit (see the figure below).

4. If the data in the cache is stale, then Traffic Server connects to the origin server and checks if the object is stillfresh (a revalidation). If it is, then Traffic Server immediately sends the cached copy to the client.

5. If the object is not in the cache (a cache miss) or if the server indicates the cached copy is no longer valid, thenTraffic Server obtains the object from the origin server. The object is then simultaneously streamed to the clientand the Traffic Server local cache (see the figure below). Subsequent requests for the object can be served fasterbecause the object is retrieved directly from cache.

Caching is typically more complex than the preceding overview suggests. In particular, the overview does not discusshow Traffic Server ensures freshness, serves correct HTTP alternates, and treats requests for objects that cannot orshould not be cached. The following sections discuss these issues in greater detail.

Ensuring Cached Object Freshness

When Traffic Server receives a request for a web object, it first tries to locate the requested object in its cache. If theobject is in cache, then Traffic Server checks to see if the object is fresh enough to serve. For HTTP objects, TrafficServer supports optional author-specified expiration dates. Traffic Server adheres to these expiration dates; otherwise,



3.2: A cache hit

3.3: A cache miss

3.3. 21


it picks an expiration date based on how frequently the object is changing and on administrator-chosen freshnessguidelines. Objects can also be revalidated by checking with the origin server to see if an object is still fresh.

HTTP Object Freshness

Traffic Server determines whether an HTTP object in the cache is fresh by checking the following conditions in order:

• Checking the Expires or max-age header

Some HTTP objects contain Expires headers or max-age headers that explicitly define how long the objectcan be cached. Traffic Server compares the current time with the expiration time to determine if the object isstill fresh.

• Checking the Last-Modified / Date header

If an HTTP object has no Expires header or max-age header, then Traffic Server can calculate a freshnesslimit using the following formula:

freshness_limit = ( date - last_modified ) * 0.10

where date is the date in the object’s server response header and last_modified is the date in theLast-Modified header. If there is no Last-Modified header, then Traffic Server uses the date the objectwas written to cache. The value 0.10 (10 percent) can be increased or reduced to better suit your needs. Referto Modifying Aging Factor for Freshness Computations.

The computed freshness limit is bound by a minimum and maximum value. Refer to Setting Absolute FreshnessLimits for more information.

• Checking the absolute freshness limit

For HTTP objects that do not have Expires headers or do not have both Last-Modified and Date head-ers, Traffic Server uses a maximum and minimum freshness limit. Refer to Setting Absolute Freshness Limits.

• Checking revalidate rules in cache.config

Revalidate rules apply freshness limits to specific HTTP objects. You can set freshness limits for objects orig-inating from particular domains or IP addresses, objects with URLs that contain specified regular expressions,objects requested by particular clients, and so on. Refer to cache.config.

Modifying Aging Factor for Freshness Computations

If an object does not contain any expiration information, then Traffic Server can estimate its freshness from theLast-Modified and Date headers. By default, Traffic Server stores an object for 10% of the time that elapsedsince it last changed. You can increase or reduce the percentage according to your needs.

To modify the aging factor for freshness computations:

1. Change the value for :ts:cv:‘proxy.config.http.cache.heuristic_lm_factor‘.

2. Run the traffic_ctl config reload command to apply the configuration changes.

Setting Absolute Freshness Limits

Some objects do not have Expires headers or do not have both Last-Modified and Date headers. To controlhow long these objects are considered fresh in the cache, specify an absolute freshness limit.

To specify an absolute freshness limit:



1. Edit the variables :ts:cv:‘proxy.config.http.cache.heuristic_min_lifetime‘ and:ts:cv:‘proxy.config.http.cache.heuristic_max_lifetime‘ in records.config.


Specifying Header Requirements

To further ensure freshness of the objects in the cache, configure Traffic Server to cache only objects with specificheaders. By default, Traffic Server caches all objects (including objects with no headers); you should change thedefault setting only for specialized proxy situations. If you configure Traffic Server to cache only HTTP objects withExpires or max-age headers, then the cache hit rate will be noticeably reduced (since very few objects will haveexplicit expiration information).

To configure Traffic Server to cache objects with specific headers:

1. Change the value for :ts:cv:‘proxy.config.http.cache.required_headers‘ in records.config.


Cache-Control Headers

Even though an object might be fresh in the cache, clients or servers often impose their own constraints that precluderetrieval of the object from the cache. For example, a client might request that a object not be retrieved from a cache,or if it does allow cache retrieval, then it cannot have been cached for more than 10 minutes.

Traffic Server bases the servability of a cached object on Cache-Control headers that appear in both client requestsand server responses. The following Cache-Control headers affect whether objects are served from cache:

• The no-cache header, sent by clients, tells Traffic Server that it should not serve any objects directly from thecache. When present in a client request, Traffic Server will always obtain the object from the origin server. Youcan configure Traffic Server to ignore client no-cache headers. Refer to Configuring Traffic Server to IgnoreClient no-cache Headers for more information.

• The max-age header, sent by servers, is compared to the object age. If the age is less than max-age, then theobject is fresh and can be served from the Traffic Server cache.

• The min-fresh header, sent by clients, is an acceptable freshness tolerance. This means that the client wantsthe object to be at least this fresh. Unless a cached object remains fresh at least this long in the future, it isrevalidated.

• The max-stale header, sent by clients, permits Traffic Server to serve stale objects provided they are nottoo old. Some browsers might be willing to take slightly stale objects in exchange for improved performance,especially during periods of poor Internet availability.

Traffic Server applies Cache-Control servability criteria after HTTP freshness criteria. For example, an objectmight be considered fresh but will not be served if its age is greater than its max-age.

Revalidating HTTP Objects

When a client requests an HTTP object that is stale in the cache, Traffic Server revalidates the object. A revalidationis a query to the origin server to check if the object is unchanged. The result of a revalidation is one of the following:

• If the object is still fresh, then Traffic Server resets its freshness limit and serves the object.

• If a new copy of the object is available, then Traffic Server caches the new object (thereby replacing the stalecopy) and simultaneously serves the object to the client.

3.3. 23


• If the object no longer exists on the origin server, then Traffic Server does not serve the cached copy.

• If the origin server does not respond to the revalidation query, then Traffic Server serves the stale object alongwith a 111 Revalidation Failed warning.

By default, Traffic Server revalidates a requested HTTP object in the cache if it considers the object to be stale. TrafficServer evaluates object freshness as described in HTTP Object Freshness. You can reconfigure how Traffic Serverevaluates freshness by selecting one of the following options:

Traffic Server considers all HTTP objects in the cache to be stale: Always revalidate HTTP objects in the cachewith the origin server.

Traffic Server considers all HTTP objects in the cache to be fresh: Never revalidate HTTP objects in the cache withthe origin server.

Traffic Server considers all HTTP objects without Expires or Cache-control headers to be stale: Revalidateall HTTP objects without Expires or Cache-Control headers.

To configure how Traffic Server revalidates objects in the cache, you can set specific revalidation rules in cache.config.

To configure revalidation options

1. Edit the variable :ts:cv:‘proxy.config.http.cache.when_to_revalidate‘ in records.config.


Pushing Content into the Cache

Traffic Server supports the HTTP PUSH method of content delivery. Using HTTP PUSH, you can deliver contentdirectly into the cache without client requests.

Configuring Traffic Server for PUSH Requests

Before you can deliver content into your cache using HTTP PUSH, you must configure Traffic Server to accept PUSHrequests.

1. Edit ip_allow.config to allow PUSH from the appropriate addresses.

2. Update :ts:cv:‘proxy.config.http.push_method_enabled‘ in records.config:

CONFIG proxy.config.http.push_method_enabled INT 1

3. Run the command traffic_ctl config reload to apply the configuration changes.

Understanding HTTP PUSH

PUSH uses the HTTP 1.1 message format. The body of a PUSH request contains the response header and responsebody that you want to place in the cache. The following is an example of a PUSH request:

PUSH http://www.company.com HTTP/1.0Content-length: 84

HTTP/1.0 200 OKContent-type: text/htmlContent-length: 17

<HTML>



a</HTML>

: Your PUSH headers must include Content-length, the value for which must include both headers and body bytecounts. The value is not optional, and an improper (too large or too small) value will result in undesirable behavior.

Tools that will help manage pushing

Traffic Server comes with a Perl script for pushing, tspush, which can assist with understanding how to write scriptsfor pushing content yourself.

Pinning Content in the Cache

The Cache Pinning Option configures Traffic Server to keep certain HTTP objects in the cache for a specified time.You can use this option to ensure that the most popular objects are in cache when needed and to prevent Traffic Serverfrom deleting important objects. Traffic Server observes Cache-Control headers and pins an object in the cacheonly if it is indeed cacheable.

To set cache pinning rules:

1. Enable :ts:cv:‘proxy.config.cache.permit.pinning‘ in records.config:

CONFIG proxy.config.cache.permit.pinning INT 1

2. Add a rule in cache.config for each URL you want Traffic Server to pin in the cache. For example:

url_regex=^https?://(www.)?apache.org/dev/ pin-in-cache=12h


Caching HTTP Objects

When Traffic Server receives a request for a web object that is not in the cache, it retrieves the object from the originserver and serves it to the client. At the same time, Traffic Server checks if the object is cacheable before storing it inits cache to serve future requests.

Traffic Server responds to caching directives from clients and origin servers, as well as directives you specify throughconfiguration options and files.

Client Directives

By default, Traffic Server does not cache objects with the following request headers:

• Authorization

• Cache-Control: no-store

• Cache-Control: no-cache

To configure Traffic Server to ignore this request header, refer to Configuring Traffic Server to Ignore Clientno-cache Headers.

3.3. 25


• Cookie (for text objects)

By default, Traffic Server caches objects served in response to requests that contain cookies (even if the objectis text). You can configure Traffic Server to not cache cookied content of any type, cache all cookied content, orcache cookied content that is of image type only. For more information, refer to Caching Cookied Objects.

Configuring Traffic Server to Ignore Client no-cache Headers

By default, Traffic Server strictly observes client Cache-Control: no-cache directives. If a requested objectcontains a no-cache header, then Traffic Server forwards the request to the origin server even if it has a fresh copyin cache. You can configure Traffic Server to ignore client no-cache directives such that it ignores no-cacheheaders from client requests and serves the object from its cache.

1. Edit :ts:cv:‘proxy.config.http.cache.ignore_client_no_cache‘ in records.config.

CONFIG proxy.config.http.cache.ignore_client_no_cache INT 1


Origin Server Directives

By default, Traffic Server does not cache objects with the following response headers:

• Cache-Control: no-store

• Cache-Control: private

• WWW-Authenticate

To configure Traffic Server to ignore WWW-Authenticate headers, refer to Configuring Traffic Server toIgnore WWW-Authenticate Headers.

• Set-Cookie

• Cache-Control: no-cache

To configure Traffic Server to ignore no-cache headers, refer to Configuring Traffic Server to Ignore Serverno-cache Headers.

• Expires header with a value of 0 (zero) or a past date.

Configuring Traffic Server to Ignore Server no-cache Headers

By default, Traffic Server strictly observes Cache-Control: no-cache directives. A response from an originserver with a no-cache header is not stored in the cache and any previous copy of the object in the cache is removed.If you configure Traffic Server to ignore no-cache headers, then Traffic Server also ignores no-store headers.The default behavior of observing no-cache directives is appropriate in most cases.

To configure Traffic Server to ignore server no-cache headers:

1. Edit :ts:cv:‘proxy.config.http.cache.ignore_server_no_cache‘ in records.config.

CONFIG proxy.config.http.cache.ignore_server_no_cache INT 1




Configuring Traffic Server to Ignore WWW-Authenticate Headers

By default, Traffic Server does not cache objects that contain WWW-Authenticate response headers. TheWWW-Authenticate header contains authentication parameters the client uses when preparing the authenticationchallenge response to an origin server.

When you configure Traffic Server to ignore origin server WWW-Authenticate headers, all objects withWWW-Authenticate headers are stored in the cache for future requests. However, the default behavior of notcaching objects with WWW-Authenticate headers is appropriate in most cases. Only configure Traffic Server toignore server WWW-Authenticate headers if you are knowledgeable about HTTP 1.1.

To configure Traffic Server to ignore server WWW-Authenticate headers:

1. Edit :ts:cv:‘proxy.config.http.cache.ignore_authentication‘ in records.config.

CONFIG proxy.config.http.cache.ignore_authentication INT 1


Configuration Directives

In addition to client and origin server directives, Traffic Server responds to directives you specify through configurationoptions and files.

You can configure Traffic Server to do the following:

• Not cache any HTTP objects. Refer to Disabling HTTP Object Caching.

• Cache dynamic content. That is, objects with URLs that end in .asp or contain a question mark (?), semicolon(;), or cgi. For more information, refer to Caching Dynamic Content.

• Cache objects served in response to the Cookie: header. Refer to Caching Cookied Objects.

• Observe never-cache rules in cache.config.

Disabling HTTP Object Caching

By default, Traffic Server caches all HTTP objects except those for which you have set never-cache as actionrules in cache.config. You can disable HTTP object caching so that all HTTP objects are served directly fromthe origin server and never cached, as detailed below.

To disable HTTP object caching manually:

1. Set :ts:cv:‘proxy.config.http.cache.http‘ to 0 in records.config.

CONFIG proxy.config.http.cache.http INT 0


Caching Dynamic Content

A URL is considered dynamic if it ends in .asp or contains a question mark (?), a semicolon (;), or cgi. By default,Traffic Server caches dynamic content. You can configure the system to ignore dynamic looking content, although thisis recommended only if the content is truly dynamic, but fails to advertise so with appropriate Cache-Controlheaders.

To configure Traffic Server’s cache behaviour in regard to dynamic content:

3.3. 27


1. Edit :ts:cv:‘proxy.config.http.cache.cache_urls_that_look_dynamic‘ in records.config. To disablecaching, set the variable to 0, and to explicitly permit caching use 1.

CONFIG proxy.config.http.cache.cache_urls_that_look_dynamic INT 0


Caching Cookied Objects

By default, Traffic Server caches objects served in response to requests that contain cookies. This is true for all typesof objects including text. Traffic Server does not cache cookied text content because object headers are stored alongwith the object, and personalized cookie header values could be saved with the object. With non-text objects, it isunlikely that personalized headers are delivered or used.

You can reconfigure Traffic Server to:

• Not cache cookied content of any type.

• Cache cookied content that is of image type only.

• Cache all cookied content regardless of type.

To configure how Traffic Server caches cookied content:

1. Edit :ts:cv:‘proxy.config.http.cache.cache_responses_to_cookies‘ in records.config.


Forcing Object Caching

You can force Traffic Server to cache specific URLs (including dynamic URLs) for a specified duration, regardless ofCache-Control response headers.

To force document caching:

1. Add a rule for each URL you want Traffic Server to pin to the cache cache.config:

url_regex=^https?://(www.)?apache.org/dev/ ttl-in-cache=6h


Caching HTTP Alternates

Some origin servers answer requests to the same URL with a variety of objects. The content of these objects can varywidely, according to whether a server delivers content for different languages, targets different browsers with differentpresentation styles, or provides different document formats (HTML, XML). Different versions of the same object aretermed alternates and are cached by Traffic Server based on Vary response headers. You can specify additionalrequest and response headers for specific Content-Type values that Traffic Server will identify as alternates forcaching. You can also limit the number of alternate versions of an object allowed in the cache.

Configuring How Traffic Server Caches Alternates

To configure how Traffic Server caches alternates:

1. Edit the following variables in records.config:

• :ts:cv:‘proxy.config.http.cache.enable_default_vary_headers‘



• :ts:cv:‘proxy.config.http.cache.vary_default_text‘

• :ts:cv:‘proxy.config.http.cache.vary_default_images‘

• :ts:cv:‘proxy.config.http.cache.vary_default_other‘


: If you specify Cookie as the header field on which to vary in the above variables, make sure that the variable:ts:cv:‘proxy.config.http.cache.cache_responses_to_cookies‘ is set appropriately.

Limiting the Number of Alternates for an Object

You can limit the number of alternates Traffic Server can cache per object (the default is 3).

: Large numbers of alternates can affect Traffic Server cache performance because all alternates have the same URL.Although Traffic Server can look up the URL in the index very quickly, it must scan sequentially through availablealternates in the object store.

To alter the limit on the number of alternates:

1. Edit :ts:cv:‘proxy.config.cache.limits.http.max_alts‘ in records.config.

CONFIG proxy.config.cache.limits.http.max_alts INT 5


Using Congestion Control

The Congestion Control option enables you to configure Traffic Server to stop forwarding HTTP requests to originservers when they become congested. Traffic Server then sends the client a message to retry the congested originserver later.

To enable this option:

1. Set :ts:cv:‘proxy.config.http.congestion_control.enabled‘ to 1 in records.config.

CONFIG proxy.config.http.congestion_control.enabled INT 1

2. Create rules in congestion.config to specify:

• Which origin servers Traffic Server tracks for congestion.

• The timeouts Traffic Server uses, depending on whether a server is congested.

• The page Traffic Server sends to the client when a server becomes congested.

• Whether Traffic Server tracks the origin servers by IP address or by hostname.


3.3. 29


Using Transaction Buffering Control

By default, I/O operations are run at full speed, as fast as either Traffic Server, the network, or the cache can support.This can be problematic for large objects if the client side connection is significantly slower. In such cases the contentwill be buffered in ram while waiting to be sent to the client. This could potentially also happen for POST requests ifthe client connection is fast and the origin server connection slow. If very large objects are being used this can causethe memory usage of Traffic Server to become very large.

This problem can be ameliorated by controlling the amount of buffer space used by a transaction. A high water andlow water mark are set in terms of bytes used by the transaction. If the buffer space in use exceeds the high water mark,the connection is throttled to prevent additional external data from arriving. Internal operations continue to proceed atfull speed until the buffer space in use drops below the low water mark and external data I/O is re-enabled.

Although this is intended primarily to limit the memory usage of Traffic Server it can also serve as a crude rate limiterby setting a buffer limit and then throttling the client side connection either externally or via a transform. This willcause the connection to the origin server to be limited to roughly the client side connection speed.

Traffic Server does network I/O in large chunks (32K or so) and therefore the granularity of transaction bufferingcontrol is limited to a similar precision.

The buffer size calculations include all elements in the transaction, including any buffers associated with transformplugins.

Transaction buffering control can be enabled globally by using configuration variables or byTSHttpTxnConfigIntSet() in a plugin.

Value Variable TSHttpTxnConfigIntSet() keyEnablebuffering :ts:cv:‘proxy.config.http.flow_control.enabled‘

TS_CONFIG_HTTP_FLOW_CONTROL_ENABLED

Set high water:ts:cv:‘proxy.config.http.flow_control.high_water‘

TS_CONFIG_HTTP_FLOW_CONTROL_HIGH_WATER

Set low water:ts:cv:‘proxy.config.http.flow_control.low_water‘

TS_CONFIG_HTTP_FLOW_CONTROL_LOW_WATER

Be careful to always have the low water mark equal or less than the high water mark. If you set only one, the otherwill be set to the same value.

If using TSHttpTxnConfigIntSet(), it must be called no later thanTS_HTTP_READ_RESPONSE_HDR_HOOK.

Reducing Origin Server Requests (Avoiding the Thundering Herd)

When an object can not be served from cache, the request will be proxied to the origin server. For a popular object, thiscan result in many near simultaneous requests to the origin server, potentially overwhelming it or associated resources.There are several features in Traffic Server that can be used to avoid this scenario.

Read While Writer

When Traffic Server goes to fetch something from origin, and upon receiving the response, any number of clients canbe allowed to start serving the partially filled cache object once background_fill_completed_threshold % of the objecthas been received.

While some other HTTP proxies permit clients to begin reading the response immediately upon the proxy receivingdata from the origin server, ATS does not begin allowing clients to read until after the complete HTTP response headershave been read and processed. This is a side-effect of ATS making no distinction between a cache refresh and a coldcache, which prevents knowing whether a response is going to be cacheable.


https://issues.apache.org/jira/browse/TS-1496


As non-cacheable responses from an origin server are generally due to that content being unique to different clientrequests, ATS will not enable read-while-writer functionality until it has determined that it will be able to cache theobject.

The following settings must be made in records.config to enable read-while-writer functionality in ATS:

CONFIG proxy.config.cache.enable_read_while_writer INT 1CONFIG proxy.config.http.background_fill_active_timeout INT 0CONFIG proxy.config.http.background_fill_completed_threshold FLOAT 0.000000CONFIG proxy.config.cache.max_doc_size INT 0

All four configurations are required, for the following reasons:

• :ts:cv:‘proxy.config.cache.enable_read_while_writer‘ being set to 1 turns the feature on, as it is off (0) bydefault.

• The background fill feature (both :ts:cv:‘proxy.config.http.background_fill_active_timeout‘ and:ts:cv:‘proxy.config.http.background_fill_completed_threshold‘) should be allowed to kick in for ev-ery possible request. This is necessary in the event the writer (the first client session to request the object, whichtriggered ATS to contact the origin server) goes away. Another client session needs to take over the writer.

As such, you should set the background fill timeouts and threshold to zero; this assures they never time out andare always allowed to kick in.

• The :ts:cv:‘proxy.config.cache.max_doc_size‘ should be unlimited (set to 0), since the object size may beunknown, and going over this limit would cause a disconnect on the objects being served.

Once these are enabled, you have something that is very close, but not quite the same, to Squid’s Collapsed Forwarding.

In addition to the above settings, the settings :ts:cv:‘proxy.config.cache.read_while_writer.max_retries‘ and:ts:cv:‘proxy.config.cache.read_while_writer_retry.delay‘ allow to control the number of retries TS attempts totrigger read-while-writer until the download of first fragment of the object is completed:

CONFIG proxy.config.cache.read_while_writer.max_retries INT 10

CONFIG proxy.config.cache.read_while_writer_retry.delay INT 50

Fuzzy Revalidation

: These options are deprecated as of v6.2.0.

Traffic Server can be set to attempt to revalidate an object before it becomes stale in cache. records.configcontains the settings:

CONFIG proxy.config.http.cache.fuzz.time INT 240CONFIG proxy.config.http.cache.fuzz.min_time INT 0CONFIG proxy.config.http.cache.fuzz.probability FLOAT 0.005

For every request for an object that occurs :ts:cv:‘proxy.config.http.cache.fuzz.time‘ before (inthe example above, 240 seconds) the object is set to become stale, there is a small chance(:ts:cv:‘proxy.config.http.cache.fuzz.probability‘ == 0.5%) that the request will trigger a revalidation requestto the origin.

: When revalidation occurs, the requested object is no longer available to be served from cache. Subsequent requestsfor that object will be proxied to the origin.

3.3. 31


For objects getting a few requests per second, these settings would offer a fairly low probability of revalidating thecached object before it becomes stale. This feature is not typically necessary at those rates, though, since odds areonly one or a small number of connections would hit origin upon the objects going stale.

Once request raise rise, the same fuzz.probability leads to a greater chance the object may be revalidated beforebecoming stale. This can prevent multiple clients simultaneously triggering contact with the origin server under higherloads, as they would do if no fuzziness was employed for revalidations.

These settings are also overridable by remap rules and via plugins, so can be adjusted per request if necessary.

Finally, :ts:cv:‘proxy.config.http.cache.fuzz.min_time‘ allows for different time periods to evaluate the probabilityof revalidation for small TTLs and large TTLs. Objects with small TTLs will start “rolling the revalidation dice” nearthe fuzz.min_time, while objects with large TTLs would start at fuzz.time.

A logarithmic like function between determines the revalidation evaluation start time (which will be between fuzz.min_time and fuzz.time). As the object gets closer to expiring, the window start becomes more likely. Bydefault this setting is not enabled, but should be enabled anytime you have objects with small TTLs. Note that thisoption predates overridable configurations, so you can achieve something similar with a plugin or remap.configsettings.

These configuration options are similar to Squid’s refresh_stale_hit configuration option.

Open Read Retry Timeout

The open read retry configurations attempt to reduce the number of concurrent requests to the ori-gin for a given object. While an object is being fetched from the origin server, subsequent requestswould wait :ts:cv:‘proxy.config.http.cache.open_read_retry_time‘ milliseconds before checking if the ob-ject can be served from cache. If the object is still being fetched, the subsequent requests will retry:ts:cv:‘proxy.config.http.cache.max_open_read_retries‘ times. Thus, subsequent requests may wait a total of(max_open_read_retries x open_read_retry_time) milliseconds before establishing an origin connec-tion of its own. For instance, if they are set to 5 and 10 respectively, connections will wait up to 50ms for a responseto come back from origin from a previous request, until this request is allowed through.

: These settings are inappropriate when objects are uncacheable. In those cases, requests for an object effectivelybecome serialized. The subsequent requests would await at least open_read_retry_time milliseconds beforebeing proxied to the origin.

It is advisable that this setting be used in conjunction with Read While Writer for big (those that take longer than(max_open_read_retries x open_read_retry_time) milliseconds to transfer) cacheable objects. With-out the read-while-writer settings enabled, while the initial fetch is ongoing, not only would subsequent requests bedelayed by the maximum time, but also, those requests would result in unnecessary requests to the origin server.

Since ATS now supports setting these settings per-request or remap rule, you can configure this to be suitable for yoursetup much more easily.

The configurations are (with defaults):

CONFIG proxy.config.http.cache.max_open_read_retries INT -1CONFIG proxy.config.http.cache.open_read_retry_time INT 10

The defaults are such that the feature is disabled and every connection is allowed to go to origin without artificialdelay. When enabled, you will try max_open_read_retries times, each with an open_read_retry_timetimeout.



Open Write Fail Action

In addition to the open read retry settings TS supports a new setting:ts:cv:‘proxy.config.http.cache.open_write_fail_action‘ that allows to further reduce multiple concurrent re-quests hitting the origin for the same object by either returning a stale copy, in case of hit-stale or an error in case ofcache miss for all but one of the requests.

Reverse Proxy and HTTP Redirects

Reverse Proxy and HTTP Redirects

As a reverse proxy cache, Traffic Server serves requests on behalf of origin servers. Traffic Server is configured insuch a way that it appears to clients like a normal origin server.

Understanding Reverse Proxy Caching

With forward proxy caching, Traffic Server handles web requests to origin servers on behalf of the clients requestingthe content. Reverse proxy caching (also known as server acceleration) is different because Traffic Server acts as aproxy cache on behalf of the origin servers that store the content. Traffic Server is configured to behave outwardlyas origin server which the client is trying to connect to. In a typical scenario the advertised hostname of the originserver resolves to Traffic Server, which serves client requests directly, fetching content from the true origin serverwhen necessary.

Reverse Proxy Solutions

There are many ways to use Traffic Server as a reverse proxy. Below are a few example scenarios.

• Offload heavily-used origin servers.

• Deliver content efficiently in geographically distant areas.

• Provide security for origin servers that contain sensitive information.

Offloading Heavily-Used Origin Servers

Traffic Server can accept requests on behalf of the origin server and improve the speed and quality of web serving byreducing load and hot spots on backup origin servers. For example, a web hoster can maintain a scalable Traffic Serversystem with a set of low-cost, low-performance, less-reliable PC origin servers as backup servers. In fact, a singleTraffic Server can act as the virtual origin server for multiple backup origin servers, as shown in the figure below.

Delivering Content in Geographically-Dispersed Areas

Traffic Server can be used in reverse proxy mode to accelerate origin servers that provide content to areas not locatedwithin close geographical proximity. Caches are typically easier to manage and are more cost-effective than replicatingdata. For example, Traffic Server can be used as a mirror site on the far side of a trans-Atlantic link to serve userswithout having to fetch the request and content across expensive, or higher latency, international connections. Unlikereplication, for which hardware must be configured to replicate all data and to handle peak capacity, Traffic Serverdynamically adjusts to optimally use the serving and storing capacity of the hardware. Traffic Server is also designedto keep content fresh automatically, thereby eliminating the complexity of updating remote origin servers.

3.3. 33


3.4: Traffic Server as reverse proxy for a pair of origin servers

Providing Security for an Origin Server

Traffic Server can be used in reverse proxy mode to provide security for an origin server. If an origin server containssensitive information that you want to keep secure inside your firewall, then you can use a Traffic Server outside thefirewall as a reverse proxy for that origin server. When outside clients try to access the origin server, the requestsinstead go to Traffic Server. If the desired content is not sensitive, then it can be served from the cache. If the contentis sensitive and not cacheable, then Traffic Server obtains the content from the origin server (the firewall allows onlyTraffic Server access to the origin server). The sensitive content resides on the origin server, safely inside the firewall.

How Does Reverse Proxy Work?

When a browser makes a request, it normally sends that request directly to the origin server. When Traffic Server isin reverse proxy mode, it intercepts the request before it reaches the origin server. Typically, this is done by setting upthe DNS entry for the origin server (i.e., the origin server’s advertised hostname) so it resolves to the Traffic Server IPaddress. When Traffic Server is configured as the origin server, the browser connects to Traffic Server rather than theorigin server. For additional information, see HTTP Reverse Proxy.

: To avoid a DNS conflict, the origin server’s hostname and its advertised hostname must not be the same.

HTTP Reverse Proxy

In reverse proxy mode, Traffic Server serves HTTP requests on behalf of a web server. The figure below illustrateshow Traffic Server in reverse proxy mode serves an HTTP request from a client browser.



3.5: HTTP reverse proxy

The figure above demonstrates the following steps:

1. A client browser sends an HTTP request addressed to a host called www.host.com on port 80. Traffic Serverreceives the request because it is acting as the origin server (the origin server’s advertised hostname resolves toTraffic Server).

2. Traffic Server locates a map rule in the remap.config file and remaps the request to the specified originserver (realhost.com).

3. If the request cannot be served from cache, Traffic Server opens a connection to the origin server (or more likely,uses an existing connection it has pre-established), retrieves the content, and optionally caches it for future use.

4. If the request was a cache hit and the content is still fresh in the cache, or the content is now available throughTraffic Server because of step 3, Traffic Server sends the requested object to the client from the cache directly.

: Traffic Server, when updating its own cache from the origin server, will simultaneously deliver that content to theclient while updating its cache database. The response to the client containing the requested object will begin as soonas Traffic Server has received and processed the full response headers from the origin server.

To configure HTTP reverse proxy, you must perform the following tasks:

• Create mapping rules in the remap.config file (refer to Creating Mapping Rules for HTTP Requests).

map http://www.host.com http://realhost.com

• Enable the reverse proxy option (refer to Enabling HTTP Reverse Proxy).

In addition to the tasks above, you can also Setting Optional HTTP Reverse Proxy Options.

Handling Origin Server Redirect Responses

Origin servers often send redirect responses back to browsers redirecting them to different pages. For example, if anorigin server is overloaded, then it might redirect browsers to a less loaded server. Origin servers also redirect whenweb pages have moved to different locations. When Traffic Server is configured as a reverse proxy, it must readdressredirects from origin servers so that browsers are redirected to Traffic Server and not to another origin server.

3.3. 35


To readdress redirects, Traffic Server uses reverse-map rules. Unless you have:ts:cv:‘proxy.config.url_remap.pristine_host_hdr‘ enabled (the default) you should generally set up a reverse-maprule for each map rule. To create reverse-map rules, refer to Using Mapping Rules for HTTP Requests.

Using Mapping Rules for HTTP Requests

Traffic Server uses two types of mapping rules for HTTP reverse proxy.

map rule

A map rule translates the URL in client requests into the URL where the content is located. When Traffic Server is inreverse proxy mode and receives an HTTP client request, it first constructs a complete request URL from the relativeURL and its headers. Traffic Server then looks for a match by comparing the complete request URL with its list oftarget URLs in remap.config. For the request URL to match a target URL, the following conditions must be true:

• The scheme of both URLs must be the same.

• The host in both URLs must be the same. If the request URL contains an unqualified hostname, then it willnever match a target URL with a fully-qualified hostname.

• The ports in both URLs must be the same. If no port is specified in a URL, then the default port for the schemeof the URL is used.

• The path portion of the target URL must match a prefix of the request URL path.

If Traffic Server finds a match, then it translates the request URL into the replacement URL listed in the map rule:it sets the host and path of the request URL to match the replacement URL. If the URL contains path prefixes, thenTraffic Server removes the prefix of the path that matches the target URL path and substitutes it with the path fromthe replacement URL. If two mappings match a request URL, then Traffic Server applies the first mapping listed inremap.config.

reverse-map rule

A reverse-map rule translates the URL in origin server redirect responses to point to Traffic Server so that clients areredirected to Traffic Server instead of accessing an origin server directly. For example, if there is a directory /pub onan origin server at www.molasses.com and a client sends a request to that origin server for /pub, then the originserver might reply with a redirect by sending the Header Location: http://realhost.com/pub/ to letthe client know that it was a directory it had requested, not a document (a common use of redirects is to normalizeURLs so that clients can bookmark documents properly).

Traffic Server uses reverse_map rules to prevent clients (that receive redirects from origin servers) from bypassingTraffic Server and directly accessing the origin servers. In many cases the client would be hitting a wall becauserealhost.com actually does not resolve for the client. (E.g.: Because it’s running on a port shielded by a firewall,or because it’s running on a non-routable LAN IP)

Both map and reverse-map rules consist of a target (origin) URL and a replacement (destination) URL. In a map rule,the target URL points to Traffic Server and the replacement URL specifies where the original content is located. Ina reverse-map rule, the target URL specifies where the original content is located and the replacement URL pointsto Traffic Server. Traffic Server stores mapping rules in remap.config located in the Traffic Server configdirectory.

Creating Mapping Rules for HTTP Requests

To create mapping rules:



1. Enter the map and reverse-map rules into remap.config.


Enabling HTTP Reverse Proxy

To enable HTTP reverse proxy:

1. Edit :ts:cv:‘proxy.config.reverse_proxy.enabled‘ in records.config.



Setting Optional HTTP Reverse Proxy Options

Traffic Server provides several reverse proxy configuration options in records.config that enable you to:

• Configure Traffic Server to retain the client host header information in a request during translation. See:ts:cv:‘proxy.config.url_remap.pristine_host_hdr‘.

• Configure Traffic Server to serve requests only to the origin servers listed in the mapping rules.As a result, requests to origin servers not listed in the mapping rules are not served. See:ts:cv:‘proxy.config.url_remap.remap_required‘.

• Specify an alternate URL to which incoming requests from older clients ,such as ones that do not provide Hostheaders, are directed. See :ts:cv:‘proxy.config.header.parse.no_host_url_redirect‘.

Run the command traffic_ctl config reload to apply any of these configuration changes.

Redirecting HTTP Requests

You can configure Traffic Server to redirect HTTP requests without having to contact any origin servers. For example,if you redirect all requests for http://www.ultraseek.com to http://www.server1.com/products/portal/search/, then all HTTP requests for www.ultraseek.com go directly to www.server1.com/products/portal/search.

You can configure Traffic Server to perform permanent or temporary redirects. Permanent redirects notify the browserof the URL change (by returning the HTTP status code 301) so that the browser can update bookmarks. Temporaryredirects notify the browser of the URL change for the current request only (by returning the HTTP status code 307 ).

To set redirect rules:

1. For each redirect you want to set enter a mapping rule in remap.config.


Example

The following permanently redirects all HTTP requests for www.server1.com to www.server2.com:

redirect http://www.server1.com http://www.server2.com

3.3. 37


Explicit Proxy Caching

If you want to use Traffic Server as an explicit proxy cache, you must configure client software (i.e., browsers) to sendrequests directly to Traffic Server.

If you do not configure Traffic Server to use the transparency option (with which client requests are intercepted enroute to origin servers by a switch/router and rerouted to the Traffic Server machine), then clients must configure theirweb browsers to send HTTP requests to the Traffic Server proxy cache by configuring their browsers to downloadproxy configuration instructions from a PAC file (Proxy Auto-Configuration file).

Configuring Browsers Manually

To manually configure a browser to send HTTP requests to Traffic Server, clients must provide the following informa-tion:

• The fully-qualified hostname or IP address of the Traffic Server node

• The Traffic Server proxy server port (by default, 8080)

In addition, clients can specify not to use Traffic Server for certain sites - in such cases, requests to the listed sites godirectly to the origin server. The procedures for manual configuration vary among browser versions; refer to specificbrowser documentation for complete proxy configuration instructions. You do not need to set any special configurationoptions on Traffic Server if you want to accept requests from manually-configured browsers.

Using a PAC File

A PAC file is a specialized JavaScript function definition that a browser calls to determine how requests are handled.Clients must specify (in their browser settings) the URL from which the PAC file is loaded. You can store a PAC fileon Traffic Server (or on any server in your network) and then provide the URL for this file to your clients.

Sample PAC File

The following sample PAC file instructs browsers to connect directly to all hosts without a fully-qualified domain nameand to all hosts in the local domain. All other requests go to the Traffic Server named myproxy.company.com.:

function FindProxyForURL(url, host){

if (isPlainHostName(host)) || (localHostOrDomainIs(host, ".company.com")) {return "DIRECT";

}elsereturn "PROXY myproxy.company.com:8080; DIRECT";

}

Transparent Proxying

Building ATS for transparency

In most cases, if your environment supports transparency then configure will automatically enable it. For otherenvironments you may need to twiddle the configure options.



--enable-posix-cap This enables POSIX capabilities, which are required for transparency. These are compiledin by default. To check your system, look for the header file sys/capability.h and the system librarylibcap. These are in the packages libcap and libcap-devel or libcap-dev (depending on theDistribution) contra-respectively.

--enable-tproxy[=value] Enable TPROXY support, which is the Linux kernel feature used for trans-parency. This should be present in the base installation, there is no package associated with it. * autoDo automatic checks for the the TPROXY header file (linux/in.h) and enable TPROXY support if theIP_TRANSPARENT definition is present. This is the default if this option is not specified or value is omitted.* no Do not check for TPROXY support, disable support for it. * force Do not check for TPROXY support,enable it using the $ats@ built in value for IP_TRANSPARENT. This is useful for systems that have it in thekernel for but some reason do not have the appropriate system header file. * number Do not check for TPROXYsupport, use number as the IP_TRANSPARENT value. There are, at present, no known standard distributionsof Linux that support TPROXY but use a value different from the built in ATS default. However, a custom builtkernel may do so and in that case the specific value can be specified.

In the default case, ATS configuration will automatically check for TPROXY support via the presence of the linux/in.h header file and compile in TPROXY support if it is available. If that fails, you may be able to recover by usingone of the options above. Note that transparency may be built in by default but it is not active unless explicitly enabledin the ATS configuration files.

Inline on a Linux Bridge

A Linux can be configured to operate in bridge mode. Two or more physical interfaces are assigned to the bridge. Asingle IP address is shared across the interfaces. By default any packet that arrives on one interface is immediatelyrouted out another bridge interface.

Linux packages required:

• bridge-utils

• ebtables

In our example of setting up bridge mode we will use a local address of 192.168.1.11/24 and interfaces eth0 andeth1 as the bridge interfaces (more detailed documentation is available here). You may omit the ‘#’ character andeverything after it.

brctl addbr br0 # create bridge devicebrctl stp br0 off # Disable spanning tree protocolbrctl addif br0 eth0 # Add eth0 to bridgebrctl addif br0 eth1 # Add eth1 to bridge

ifconfig eth0 0 0.0.0.0 # Get rid of interface IP addressesifconfig eth1 0 0.0.0.0 # ditto # Set the bridge IP address and enable itifconfig br0 192.168.1.11 netmask 255.255.255.0 up

If you have not already done so, remember to add a default route, such as this one for a gateway of 192.168.1.1.

ip route add default via 192.168.1.1

At this point it is a good idea to test connectivity to verify the basic bridge is functional.

Once the bridge is verified to work, this is the basic traffic pattern of interest.

The green arrows are packets originating from the client and the red arrows are packets originating from the originserver. All traffic not directed to the local address will pass through the bridge. We need to break into some of thetraffic and subject it to routing so that it can be routed to ATS. This requires ebtables. The flows we want tointercept are green 1 (from client to bridge) and red 1 (origin server to bridge).

3.3. 39

http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge

http://www.tldp.org/HOWTO/BRIDGE-STP-HOWTO/preparing-the-bridge.html


3.6: Picture of traffic flow through a bridge with ATS

In this example we will intercept port 80 (HTTP) traffic. We will use the BROUTING chain because it is traversedonly for packets that originated externally and arrived on a (forwarding enabled) interface. Although it looks like thiswill intercept all port 80 traffic it will only affect the two flows described above. -j redirect marks the packetas being diverted to the bridge and not forwarded, and the DROP target puts the packets in the normal iptablesrouting so that we can use standard device tests on them1. Although this example handles only port 80, other ports arethe same except for the port value. Note also the port here is the port from the point of view of the clients and originservers, not the Traffic Server server port.

ebtables -t broute -F # Flush the table# inbound trafficebtables -t broute -A BROUTING -p IPv4 --ip-proto tcp --ip-dport 80 \

-j redirect --redirect-target DROP# returning outbound trafficebtables -t broute -A BROUTING -p IPv4 --ip-proto tcp --ip-sport 80 \

-j redirect --redirect-target DROP

Traffic Server operates at layer 3 so we need to use iptables to handle IP packets appropriately.:

iptables -t mangle -A PREROUTING -i eth1 -p tcp -m tcp --dport 80 \-j TPROXY --on-ip 0.0.0.0 --on-port 8080 --tproxy-mark 1/1

iptables -t mangle -A PREROUTING -i eth0 -p tcp -m tcp --sport 80 \-j MARK --set-mark 1/1

At this point the directionality of the interfaces matters. For the example eth1 is the inbound (client side) interface,while eth0 is the outbound (origin server side) interface. We mark both flows of packets so that we can use policyrouting on them. For inbound packets we need to use TPROXY to force acceptance of packets to foreign IP addresses.For returning outbound packets there will be a socket open bound to the foreign address, we need only force it to bedelivered locally. The value for --on-ip is 0 because the target port is listening and not bound to a specific address.The value for --on-port must match the Traffic Server server port. Otherwise its value is arbitrary. --dport and--sport specify the port from the point of view of the clients and origin servers.

Once the flows are marked we can force them to be delivered locally via the loopback interface via a policy routingtable.:

1 The --redirect-target can be omitted, but then the iptables rules would need to use --physdev instead of just -i. The actualpacket processing is identical.



ip rule add fwmark 1/1 table 1ip route add local 0.0.0.0/0 dev lo table 1

The marking used is arbitrary but it must be consistent between iptables and the routing rule. The table numbermust be in the range 1..253.

To configure Traffic Server set the following values in records.config

• :ts:cv:‘proxy.config.http.server_ports‘ value from --on-port (see below)

• :ts:cv:‘proxy.config.reverse_proxy.enabled‘ 1

• :ts:cv:‘proxy.config.url_remap.remap_required‘ 0

You may also need to set :ts:cv:‘proxy.config.cluster.ethernet_interface‘ to “br0” (the name of the bridge interfacefrom the Bridge Commands).

Additional troubleshooting

• Check to make sure that iptables is not filtering (blocking) incoming HTTP connections.

It is frequently the case that the default tables prevent incoming HTTP. You can clear all filters withthe commands:

iptables -t filter --flush FORWARDiptables -t filter --flush INPUT

That is a bit drastic and should only be used for testing / debugging. A live system will likely needsome filters in place but that is beyond the scope of this document. If this fixes the problem, thenyour filter set is too restrictive.

Note that this problem will prevent the basic bridge (without ATS) from allowing HTTP trafficthrough.

• Verify that IP packet forwarding is enabled.

You can check this with:

cat /proc/sys/net/ipv4/ip_forward

The output should be a non-zero value (usually ‘1’). If it is zero, you can set it with:

echo '1' > /proc/sys/net/ipv4/ip_forward

This can setting can be persisted by putting it in /etc/sysctl.conf:

net/ipv4/ip_forward=1

Inline on a Linux router

The routed set up presumes the set of clients are on distinct networks behind a single physical interface. For thepurposes of this example will we presume

• The clients are on network 172.28.56.0/24

• The router connects the networks 172.28.56.0/24 and 192.168.1.0/24

• Interface eth0 is on the network 192.168.1.0/24

3.3. 41


• Interface eth1 is on the network 172.28.56.0/24

• The router is already configured to route traffic correctly for the clients.

In this example we will intercept port 80 (HTTP) traffic that traverses the router. The first step is to use iptables tohandle IP packets appropriately.

# reflow client web traffic to TPROXYiptables -t mangle -A PREROUTING -i eth1 -p tcp -m tcp --dport 80 -j TPROXY \

--on-ip 0.0.0.0 --on-port 8080 --tproxy-mark 1/1# Let locally directed traffic pass through.iptables -t mangle -A PREROUTING -i eth0 --source 192.168.1.0/24 -j ACCEPTiptables -t mangle -A PREROUTING -i eth0 --destination 192.168.1.0/24 -j ACCEPT# Mark presumed return web trafficiptables -t mangle -A PREROUTING -i eth0 -p tcp -m tcp --sport 80 -j MARK --set-mark→˓1/1

We mark packets so that we can use policy routing on them. For inbound packets we use TPROXY to make it possibleto accept packets sent to foreign IP addresses. For returning outbound packets there will be a socket open bound tothe foreign address, we need only force it to be delivered locally. The value for --on-ip is 0 because the target portis listening and not bound to a specific address. The value for --on-port must match the Traffic Server server port.Otherwise its value is arbitrary. --dport and --sport specify the port from the point of view of the clients andorigin servers. The middle two lines exempt local web traffic from being marked for Traffic Server – these rules can betightened or loosened as needed. They server by matching traffic and exiting the iptables processing via ACCEPTbefore the last line is checked.

Once the flows are marked we can force them to be delivered locally via the loopback interface via a policy routingtable.

ip rule add fwmark 1/1 table 1ip route add local 0.0.0.0/0 dev lo table 1

The marking used is arbitrary but it must be consistent between iptables and the routing rule. The table numbermust be in the range 1..253.

To configure Traffic Server set the following values in records.config

proxy.config.http.server_ports STRING Default: value from --on-port

proxy.config.reverse_proxy.enabled INT Default: 1

proxy.config.url_remap.remap_required INT Default: 0

WCCP Configuration

WCCP is de-facto semi-standard used by routers to redirect network traffic to caches. It is available on most Cisco™routers although it does not appear to be officially supported by Cisco. The primary benefits of WCCP are

• If already have a router that supports WCCP inline you do not have to change your network topology.

• WCCP fails open so that if the Traffic Server machine fails, it is bypassed and users continue to have Internetaccess.

Use of WCCP only makes sense for client side transparency1 because if the clients are explicitly proxied by TrafficServer there’s no benefit to WCCP fail open, as the clients will continue to directly access the unresponsive TrafficServer host. It would be better to adjust the routing tables on the router for explicit proxying.

1 Server side transparency should also be used, but it is not as significant. In its absence, however, origin servers may see the source address ofconnections suddenly change from the Traffic Server address to client addresses, which could cause problems. Further, the primary reason for nothaving server side transparency is to hide client addresses which is defeated if the Traffic Server host fails.


http://cio.cisco.com/en/US/docs/ios/12_0t/12_0t3/feature/guide/wccp.html


Because the router serves as the inline network element, Traffic Server must run on a separate host. This host can belocated anywhere as long as Traffic Server is either on the same network segment or a GRE tunnel can be maintainedbetween the Traffic Server host and the router.

This document presumes that the router is already properly configured to handle traffic between the clients andthe origin servers. If you are not certain, verify it before attempting to configure Traffic Server with WCCP. This isalso a good state to which to revert should the configuration go badly.

Configuration overview

Setting WCCP is a three step process, first configuring the router, the Traffic Server host, and Traffic Server.

The router will not respond to WCCP protocol packets unless explicitly configured to do so. Via WCCP, therouter can be made to perform packet interception and redirection needed by Traffic Server transparency. The WCCPprotocol in effect acts as means of controlling a rough form of policy routing with positive heartbeat cutoff.

The Traffic Server host system must also be configured using iptables to accept connections on foreign addresses.This is done roughly the same way as the standard transparency use.

Finally Traffic Server itself must be configured for transparency and use of WCCP. The former is again very similar tothe standard use, while WCCP configuration is specific to WCCP and uses a separate configuration file, referred to bythe records.config file.

The primary concern for configuration in which of three basic topologies are to be used.

• Dedicated – Traffic Server traffic goes over an interface that is not used for client nor origin server traffic.

• Shared – Traffic Server traffic shares an interface with client or server traffic.

• Inside Shared – Traffic Server and client traffic share an interface.

• Outside Shared – Traffic Server and origin server traffic share an interface.

In general the dedicated topology is preferred. However, if the router has only two interfaces one of the sharedtopologies will be required2 Click the links above for more detailed configuration information on a specific topology.

Shared interface issues

A shared interface topology has additional issues compared to a dedicated topology that must be handled. Such atopology is required if the router has only two interfaces, and because of these additional issues it is normally onlyused in such cases, although nothing prevents it use even if the router has three or more interfaces.

The basic concept for a shared interface is to use a tunnel to simulate the dedicated interface case. This enablesthe packets to be distinguished at layer 3. For this reason, layer 2 redirection cannot be used because the WCCPconfiguration cannot distinguish between packets returning from the origin server and packets returning from TrafficServer as they are distinguished only by layer 2 addressing3. Fortunately the GRE tunnel used for packet forwardingand return can be used as the simulated interface for Traffic Server.

Frequently encountered problems

2 If your router has only one interface, it’s hardly a router.3 This is not fundamentally impossible, as the packets are distinct in layer

3.3. 43


MTU and fragmentation

In most cases the basic configuration using a tunnel in any topology can fail due to issues with fragmentation. Thesocket logic is unable to know that its packets will eventually be put in to a tunnel which will by its nature have asmaller MTU than the physical interface which it uses. This can lead to pathological behavior or outright failure as thepackets sent are just a little too big. It is not possible to solve easily by changing the MTU on the physical interfacebecause the tunnel interface uses that to compute its own MTU.

References

• WCCP Router Configuration Commands - IOS 12.2

WCCP Service Configuration

The service definition file is used by traffic_wccp and traffic_server directly.

The elements in the security definition file are inspired by the WCCP RFC (8/2012). There is also an older version ofthe RFC that shows up commonly in search results, WCCP (4/2001), and was the RFC reference used in the originalWCCP support for Traffic Server several years ago.

A sample service group file is included in the source tree under traffic_wccp.

Security Section

In the security section, you can define a password shared between the WCCP Client and the WCCP router. Thispassword is used to encrypt the WCCP traffic. It is optional, but highly recommended.

Attributes in this section

• option - Must be set to MD5 if you want to encrypt your WCCP traffic

• key – The same password that you set with the associated WCCP router.

Services Section

In the services section you can define a list of service groups. Each top level entry is a separate service group.

Service group attributes include

• name – A name for the service. Not used in the rest of the WCCP processing.

• description – A description of the service. Again, not used in the rest of the WCCP processing.

• id - The security group ID. It must match the service group ID that has been defined on the associated WCCProuter. This is the true service group identifier from the WCCP perspective.

• type – This defines the type of service group either “STANDARD” or “DYNAMIC”. There is one standarddefined service group, HTTP with the id of 0. The 4/2001 RFC indicates that id’s 0-50 are reserved for wellknown service groups. But more recent 8/2012 RFC indicates that values 0 through 254 are valid service id’s fordynamic services. To avoid differences with older WCCP routers, you probably want to avoid dynamic serviceID’s 0 through 50.

• priority – This is a value from 0 to 255. The higher number is a higher priority. Well known (STANDARD)services are set to a value of 240. If there are multiple service groups that could match a given packet, thehigher priority service group is applied. RFC For example, you have service group 100 defined for packets


http://en.wikipedia.org/wiki/Maximum_transmission_unit

http://www.cisco.com/en/US/docs/ios/12_2/configfun/command/reference/frf018.html

http://tools.ietf.org/html/draft-mclaggan-wccp-v2rev1-00

https://tools.ietf.org/id/draft-wilson-wrec-wccp-v2-01.txt


with destination port 80, and service group 101 defined for packets with source port 1024. For a packet withdestination port set to 80 and source port set to 1024, the priorities of the service groups would need to becompared to determine which service group applies.

• protocol – This is IP protocol number that should match. Generally this is set to 6 (TCP) or 17 (UDP).

• assignment – WCCP supports multiple WCCP clients supporting a single service group. However, the currentWCCP client implementation in Traffic Server assumes there is only a single WCCP client, and so createsassignment tables that will direct all traffic to that WCCP client. The assignment type is either hash or mask,and if it is not set, it defaults to hash. If Traffic Server ever supports more than one cache, it will likely onlysupport a balanced hash assignment. The mask/value assignment seems to be better suited to situations wherethe traffic needs to be more strongly controlled.

• primary-hash – This is the element of the packet that is used to compute the primary key. The value options aresrc_ip, dst_ip, src_port, or dst_port. This entry is a list, so multiple values can be specified. In that case, all thespecified packet attributes will be used to compute the hash bucket. In the current implementation, the primaryhash value does not matter, since the client always generates a hash table that directs all matching traffic to it.But if multiple clients are ever supported, knowledge of the local traffic distribution could be used to pick apacket attribute that will better spread traffic over the WCCP clients.

• alt-hash – The protocol supports a two level hash. This attribute is a list with the same value options as forprimary-hash. Again, since the current Traffic Server implementation only creates assignment tables to a singleclient, specifying the alt-hash values does nothing.

• ports – This is a list of port values. Up to 8 port values may be included in a service group definition.

• port-type – This attribute can have the value of src or dst. If not specified, it defaults to dst. It indicates whetherthe port values should be interpreted as source ports or destination ports.

• forward – This is a list. The list of the values can be GRE or L2. This advertises how the client wants to processWCCP packets. GRE means that the packets will be delivered in a GRE tunnel. This is the default. L2 meansthat the client is on the same network and can get traffic delivered to it from the router by L2 routing (MACaddresses).

• return – The WCCP protocol allows a WCCP client to decline a packet and return it back to the router. Thecurrent WCCP client implementation never does this. The value options are the same as for the forward attribute.

• routers – This is the list of router addresses the WCCP client communicates with. The WCCP protocols allowsfor multiple WCCP routers to be involved in a service group. The multiple router scenario has at most beenlightly tested in the Traffic Server implementation.

• proc-name – This attribute is only used by traffic_wccp. It is not used in the traffic_server WCCP support. Thisis the path to a process’ PID file. The service group is advertised to the WCCP router if the process identified inthe PID file is currently operational.

Transparent Proxying is the ability of a proxy (such as ATS) to intercept connections between clients and serverswithout being visible.

The general network structure that will be used in this documentation is shown in the following figure.

There must be a gateway device through which all network traffic passes from the client to the Internet (or externalcloud). The gateway is responsible for effectively splicing ATS in to selected streams of that traffic. Each traffic streamis split in two, with ATS terminating both sides. That is, stream green-1, red-2, is split in to the green connection andthe red connection. Note that ATS may or may not be on the gateway system, the redirected traffic can flow over othernetwork infrastructure.

Because ATS uses two connections, transparency can be set independently on the client and origin server (Internet /external cloud) side. We will define what is generally called “transparent proxy” as two aspects, inbound transparencyand outbound transparency.

3.3. 45


3.7: ATS basic traffic flow of Transparent Proxy

Inbound transparency is a proxy that is transparent to connections that are inbound to the proxy, i.e. a connection initi-ated by a client which connects to the proxy (green-1). Similarly, outbound transparency is a proxy that is transparentto connections that are outbound from the proxy, i.e. a connection initiated by the proxy to an origin server (red-2).

In most situations these two types of transparency are combined, but that is not required. Traffic Server supportstransparency independently on the two sides.

: It is critical to note that any transparency requires specialized routing and cannot be done solely by configuringATS. ATS transparency also requires support from the Linux kernel and therefore currently only works on sufficientlyrecent Linux kernels that support the following features:

• TPROXY

• POSIX capabilities

In addition the specialized routing will require using iptables and in some cases ebtables.

Standard build procedures should work for transparency support but if not consult these more detailed instructions.

Transparency is configured per server port, not globally. This is done via the configuration values:ts:cv:‘proxy.config.http.server_ports‘. In addition, :ts:cv:‘proxy.config.reverse_proxy.enabled‘ must be enabledif the client side is transparent. That should be fixed in a future patch.

In the first case use the attribute character (replacing the default ‘X’)

Attribute Transparency Style Reverse Proxy

= Full transparency: either

> Inbound (client) transparency: enabled

< Outbound (origin server) transparency: either

In the outbound transparent case clients must connect directly to ATS either through an explicit proxy mechanism orby advertising the IP address of the ATS server via DNS as the origin server address.

Some tested scenarios –

• Inline on a Linux Bridge

• Inline on a Linux router



• WCCP Configuration

Forward Proxy

The Apache Traffic Server is a general purpose proxy, configurable as both a reverse and forward proxy.

A forward proxy can be used as a central tool in your infrastructure to access the web and it may be combinedwith a cache to lower your overall bandwidth usage. Forward proxies act as a gatekeeper between client browserson your local network and all (or some, at your configuration’s discretion) web sites accessed by those clients. Theforward proxy will receive the HTTP requests, perform any filtering or request alteration rules you establish, and whenappropriate forward the request on to its destination website. The response will return through your proxy, where itmay optionally be cached and/or modified, and then returned to the original client.

There are two modes in which your forward proxy may operate:

Forward Proxy Each client must be configured explicitly to use the forward proxy. Client browsers will be aware ofthe fact they are using a proxy and will form their HTTP requests appropriately. This results in the initial HTTPcommand being issued with fully qualified URIs that contain the destination hostname:

GET http://example.com/index.php?id=123 HTTP/1.1

Transparent Proxy The use of a transparent proxy is typically done in concert with network routing rules whichredirect all outbound HTTP traffic through your proxy. Clients will behave, and form their HTTP requests, as ifthey are contacting the remote site directly, and will not be aware of the existence of a proxy server in betweenthemselves and the remote servers. HTTP requests will be generated per their usual form, with only paths in thecommand and a separate Host request header:

GET /index?id=123 HTTP/1.1Host: example.com

Apache Traffic Server may be configured to operate as both a forward and a transparent proxy simultaneously.

Proxy Configuration

Configuring basic forward proxy operation in Traffic Server is quite simple and straightforward.

1. Permit Traffic Server to process requests for hosts not explicitly configured in the remap rules, by modifying:ts:cv:‘proxy.config.url_remap.remap_required‘ in records.config:

CONFIG proxy.config.url_remap.remap_required INT 0

2. Optional: If Traffic Server will be operating strictly as a forward proxy, you will want to disable reverse proxysupport by modifying :ts:cv:‘proxy.config.reverse_proxy.enabled‘ in records.config:


You may also want to consider some of these configuration options:

• Setting :ts:cv:‘proxy.config.http.no_dns_just_forward_to_parent‘ determines which host will be used forDNS resolution.

• Proxy Authentication can be enabled or disabled with :ts:cv:‘proxy.config.http.forward.proxy_auth_to_parent‘should you also be employing a proxy cache.

• The client request header X-Forwarded-For may be toggled with :ts:cv:‘proxy.config.http.insert_squid_x_forwarded_for‘.

3.3. 47


Client Configuration

If you are operating your proxy in transparent mode, your clients should require no special proxy-related configuration.

If you are operating in explicit forward proxy mode, without automatic routing rules on your network to direct all out-bound traffic through the proxy, your client browsers will need to be directed to the proxy. This may be accomplishedin two different ways.

Clients may be configured to use the default 8080 port on your Traffic Server host as a proxy. This will result in allrequests from that client browser being issued through the single forward proxy as configured.

Security Considerations

It’s important to note that once your Apache Traffic Server is configured as a forward proxy it will indiscriminatelyaccept proxy requests from anyone. If it is reachable from the Internet, then you have configured an Open Proxy.

This is generally not desirable, as it will permit anyone to potentially use your network as the source of traffic to sitesof their choosing. To avoid this, you’ll have to make sure your proxy server is either only reachable from within yourprivate network or is secured by firewall rules that permit only those you wish to have access to the proxy.

Traffic Server Cluster

Traffic Server scales from a single node to multiple nodes that form a cluster allowing you to improve system perfor-mance and reliability.

Understanding Traffic Server Clusters

A Traffic Server cluster consists of multiple Traffic Server nodes. The nodes in a cluster share configuration infor-mation and can form a single logical cache. Traffic Server detects the addition and deletion of nodes in the clusterautomatically and can detect when a node is unavailable. Traffic Server uses its own protocol for clustering, which ismulticast for node location and heartbeat, but unicast for all data exchange within the cluster. Traffic Server has twoclustering modes:

• Management-only mode; refer to Management-Only Clustering below.

• Full-clustering mode; refer to Full Clustering

Management-Only Clustering

In management-only clustering mode, Traffic Server cluster nodes share configuration information. You can administerall the nodes at the same time. Traffic Server uses a multicast management protocol to provide a single system imageof your Traffic Server cluster. Information about cluster membership, configuration, and exceptions is shared acrossall nodes, and the traffic_manager process automatically propagates configuration changes to all the nodes.

Full Clustering

In full-clustering mode, as well as sharing configuration information, a Traffic Server cluster distributes its cacheacross its nodes into a single, virtual object store, rather than replicating the cache node by node. Traffic Server canprovide an enormous aggregate cache size and can maximize cache hit rate by storing objects only once across theentire cluster.

A fully clustered Traffic Server maps objects to specific nodes in the cluster. When a node receives a request, it checksto see if the request is a hit somewhere in the cluster. If the request is a hit on a different node, the node handling



the request obtains the object from the hit node and serves it to the client. Traffic Server uses its own communicationprotocol to obtain an object from sibling cluster nodes.

If a node fails or is shut down and removed, Traffic Server removes references to the missing node on all nodes in thecluster.

Full clustering recommends a dedicated network interface for cluster communication to achieve better performance.

Enabling Clustering Mode

Before a node is added into a cluster, please ensure that the following conditions are being met:

• All nodes are running the same operating system:

– The same distribution, e.g.: RHEL 6.6

– The same kernel, e.g.: 2.6.32-504.23.4.el6

– The same architecture, e.g.: x86_64

• All nodes have the same version of Traffic Server installed

• All nodes are composed of the same hardware

• All nodes are on the same switch or same VLAN.

Traffic Server does not apply the clustering mode change to all the nodes in the cluster. You must change the clusteringmode on each node individually. You may following these instructions:

1. Setup the same cluster name, with :ts:cv:‘proxy.config.proxy_name‘, e.g. MyCluster.

2. Set :ts:cv:‘proxy.local.cluster.type‘ to 1 to enable cluster mode. The following values of this configuration arevalid:

Value Description1 Full-Clustering mode2 Management-Only mode3 No clustering (default)

3. Configure :ts:cv:‘proxy.config.cluster.ethernet_interface‘, e.g.: eth0. This should be replacedwith the node’s real interface. We recommends a dedicated physical interface here. Refer to:ts:cv:‘proxy.local.cluster.type‘ for a full description.

4. Enable configuration changes:

traffic_ctl config reload

5. Restart traffic server:

traffic_ctl server restart

The traffic_server and traffic_manager processes will need to restart after the change of:ts:cv:‘proxy.local.cluster.type‘ and :ts:cv:‘proxy.config.cluster.ethernet_interface‘ have taken place.

Traffic Server will join the cluster in about 10 seconds. To verify the hosts in the cluster, you can run:

traffic_ctl metric get proxy.process.cluster.nodes

Cluster node status is also tracked in cluster.config in the configuration directory. This configuration is gener-ated by the system and should not be edited. It contains a list of the machines that are currently members of the cluster,for example:

3.3. 49


# 3127.1.2.3:80127.1.2.4:80127.1.2.5:80

After successfully joining a cluster, all changes of global configurations performed on any node in that cluster willtake effect on all nodes, removing the need to manually duplicate configuration changes across each node individually.

Deleting Nodes from a Cluster

To delete a node from the Traffic Server cluster, return the setting :ts:cv:‘proxy.local.cluster.type‘ to the default value3 and reload.

Common issues for Cluster setup

1. The Cluster member auto discovery is built upon multi-casting UDP, and as such is impossible to setup wheremulti-casting is not avaliable, such as AWS EC2.

2. The Cluster will depend on ports 8088, 8089, and 8086. These ports must not be blocked by any networkconfigurations or firewalls on the network used for internal cluster communication.

Performance Tuning for Busy Clusters

Beginning with version 3.2.0, Apache Traffic Server can handle multiple internal cluster connections and the numberof Cluster Threads is configurable. Each of the threads will keep one connection open to all peering cluster nodes.

Increasing Cluster Threads

In the cluster environment, the current performance of the cluster threads will consume the same cpu usage as a normalnetwork thread. It’s reasonable to keep roughly the same number of cluster threads as network threads. For example, ifyou are running a system with 10 network processing threads, you can set the number of cluster threads by modifying:ts:cv:‘proxy.config.cluster.threads‘ to 10:

traffic_ctl config set proxy.config.cluster.threads 10

Hierarchical Caching

Understanding Cache Hierarchies

A cache hierarchy consists of cache levels that communicate with each other. Traffic Server supports several types ofcache hierarchies. All cache hierarchies recognize the concept of parent and child. A parent cache is a cache higherup in the hierarchy, to which Traffic Server can forward requests. A child cache is a cache for which Traffic Server isa parent.

Traffic Server supports the following hierarchical caching options:



Parent Caching

If a Traffic Server node cannot find a requested object in its cache, then it searches a parent cache (which itself cansearch other caches) before finally retrieving the object from the origin server. You can configure a Traffic Server nodeto use multiple parent caches so that if one parent is unavailable, the other parent caches will be checked in turn untileither the request is serviced properly or no further parent caches are available and the origin server is contacted. Thisis called Parent Failover. Traffic Server supports parent caching for both HTTP and HTTPS requests.

If you do not want all requests to go to the parent cache, then simply configure Traffic Server to route certain requests(such as requests containing specific URLs) directly to the origin server. This may be achieved by setting parent proxyrules in parent.config.

The figure below illustrates a simple cache hierarchy with a Traffic Server node configured to use a parent cache. In thefollowing scenario, a client sends a request to a Traffic Server node that is a child in the cache hierarchy (because it’sconfigured to forward missed requests to a parent cache). The request is a cache miss, so Traffic Server then forwardsthe request to the parent cache where it is a cache hit. The parent sends a copy of the content to the Traffic Server,where it is cached and then served to the client. Future requests for this content can now be served directly from theTraffic Server cache (until the data is stale or expired).

3.8: Parent caching

If the request is a cache miss on the parent, then the parent retrieves the content from the origin server (or from anothercache, depending on the parent’s configuration). The parent caches the content and then sends a copy to Traffic Server(its child), where it is cached and served to the client.

3.3. 51


Interaction with Remap.config

If remap rules are required (:ts:cv:‘proxy.config.reverse_proxy.enabled‘), when a request comes in to a child node,its remap.config is evaluated before parent selection. This means that the client request is translated accordingto the remap rule, and therefore, any parent selection should be made against the remapped host name. This is trueregardless of pristine host headers (:ts:cv:‘proxy.config.url_remap.pristine_host_hdr‘) being enabled or not. Theparent node will receive the translated request (and thus needs to be configured to accept it).

Example

The client makes a request to Traffic Server for http://example.com. The origin server for the request is http://origin.example.com; the parent node is parent1.example.com, and the child node is configured as a reverse proxy.

If the child’s remap.config contains

map http://example.com http://origin.example.com

with the child’s parent.config containing

dest_domain=origin.example.com method=get parent="parent1.example.com:80 )

and parent cache (parent1.example.com) would need to have a remap.config line similar to

map http://origin.example.com http://origin.example.com

With this example, if parent1.example.com is down, the child node would automatically directly contact the origin.example.com on a cache miss.

Parent Failover

Traffic Server supports use of several parent caches. This ensures that if one parent cache is not available, anotherparent cache can service client requests.

When you configure your Traffic Server to use more than one parent cache, Traffic Server detects when a parent isnot available and sends missed requests to another parent cache. If you specify more than two parent caches, then theorder in which the parent caches are queried depends upon the parent proxy rules configured in the parent.configconfiguration file. By default, the parent caches are queried in the order they are listed in the configuration file.

Configuring Traffic Server to Use a Parent Cache

To configure Traffic Server to use one or more parent caches, you must perform the configuration adjustments detailedbelow.

: You need to configure the child cache only. Assuming the parent nodes are configured to serve the child’s originserver, no additional configuration is needed for the nodes acting as Traffic Server parent caches.

1. Enable the parent caching option by adjusting :ts:cv:‘proxy.config.http.parent_proxy_routing_enable‘ inrecords.config.

CONFIG proxy.config.http.parent_proxy_routing_enable INT 1

2. Identify the parent cache you want to use to service missed requests. To use parent failover, you must identifymore than one parent cache so that when a parent cache is unavailable, requests are sent to another parent cache.


http://example.com

http://origin.example.com

http://origin.example.com


3. Edit parent.config to set parent proxy rules which will specify the parent cache to which you want missedrequests to be forwarded.

The following example configures Traffic Server to route all requests containing the regular expression politicsand the path /viewpoint directly to the origin server (bypassing any parent hierarchies):

url_regex=politics prefix=/viewpoint go_direct=true

The following example configures Traffic Server to direct all missed requests with URLs beginning with http://host1 to the parent cache parent1. If parent1 cannot serve the requests, then requests are forwarded toparent2. Because round-robin=true, Traffic Server goes through the parent cache list in a round-robin basedon client IP address.:

dest_host=host1 scheme=http parent="parent1;parent2" round-robin=strict

Run the command traffic_ctl config reload to apply the configuration changes.

ICP Peering

The Internet Cache Protocol (ICP) is used by proxy caches to exchange information about their content. ICP querymessages ask other caches if they are storing a particular URL; ICP response messages reply with a hit or miss answer.A cache exchanges ICP messages only with specific ICP peers, which are neighboring caches that can receive ICPmessages. An ICP peer can be a sibling cache (which is at the same level in the hierarchy) or a parent cache (which isone level up in the hierarchy).

If Traffic Server has ICP caching enabled, then it sends ICP queries to its ICP peers when the HTTP request is a cachemiss. If there are no hits but parents exist, then a parent is selected using a round-robin policy. If no ICP parents exist,then Traffic Server forwards the request to its HTTP parents. If there are no HTTP parent caches established, thenTraffic Server forwards the request to the origin server.

If Traffic Server receives a hit message from an ICP peer, then Traffic Server sends the HTTP request to that peer.However, it might turn out to be a cache miss because the original HTTP request contains header information that isnot communicated by the ICP query. For example, the hit might not be the requested alternate. If an ICP hit turns outto be a miss, then Traffic Server forwards the request to either its HTTP parent caches or to the origin server.

To configure a Traffic Server node to be part of an ICP cache hierarchy, you must perform the following tasks:

• Determine if the Traffic Server can receive ICP messages only, or if it can send and receive ICP messages.

• Determine if Traffic Server can send messages directly to each ICP peer or send a single message on a specifiedmulticast channel.

• Specify the port used for ICP messages.

• Set the ICP query timeout.

• Identify the ICP peers (siblings and parents) with which Traffic Server can communicate.

To configure Traffic Server to use an ICP cache hierarchy edit the following variables in records.config file:

• :ts:cv:‘proxy.config.icp.enabled‘

• :ts:cv:‘proxy.config.icp.icp_port‘

• :ts:cv:‘proxy.config.icp.multicast_enabled‘

• :ts:cv:‘proxy.config.icp.query_timeout‘

Edit icp.config file located in the Traffic Server config directory: For each ICP peer you want to identify, enter aseparate rule in the icp.config file.

Run the command traffic_ctl config reload to apply the configuration changes.

3.3. 53


Interacting with Traffic Server

Traffic Line

Stats Over HTTP

Cache Inspector

Security

Controlling Access

Traffic Server can be configured to allow only certain clients to use the proxy cache.

1. Add a line to ip_allow.config for each IP address or range of IP addresses allowed to access Traffic Server.


SSL Termination

The Traffic Server SSL termination option enables you to secure connections in reverse proxy mode between a clientand a Traffic Server and/or Traffic Server and an origin server.

The following sections describe how to enable and configure the SSL termination option.

• Enable and configure SSL termination for client/Traffic Server connections: Client and Traffic Server Connec-tions

• Enable and configure SSL termination for Traffic Server/origin server connections: Traffic Server and OriginServer Connections

Client and Traffic Server Connections

The figure below illustrates communication between a client and Traffic Server (and between Traffic Server and anorigin server) when the SSL termination option is enabled and configured for Client/Traffic Server connections only.

The figure above depicts the following:

1. The client sends an HTTPS request for content. Traffic Server receives the request and performs the SSLhandshake to authenticate the client (depending on the authentication options configured) and determine theencryption method that will be used. If the client is allowed access, then Traffic Server checks its cache for therequested content.

2. If the request is a cache hit and the content is fresh, then Traffic Server encrypts the content and sends it to theclient. The client decrypts the content (using the method determined during the handshake) and displays it.

3. If the request is a cache miss or cached content is stale, then Traffic Server communicates with the origin servervia HTTP and obtains a plain text version of the content. Traffic Server saves the plain text version of the contentin its cache, encrypts the content, and sends it to the client. The client decrypts and displays the content.

To configure Traffic Server to use the SSL termination option for Client/Traffic Server connections, you must do thefollowing:

1. Obtain and install an SSL server certificate from a recognized certificate authority. The SSL server certificatecontains information that enables the client to authenticate Traffic Server and exchange encryption keys.



3.9: Client and Traffic Server communication using SSL termination

2. Set the port number used for SSL communication using :ts:cv:‘proxy.config.http.server_ports‘ in records.config.

3. Set the appropriate base path for your SSL certificates and private keys in records.config.

CONFIG proxy.config.ssl.server.cert.path STRING "/opt/ts/etc/ssl/certs/"CONFIG proxy.config.ssl.server.private_key.path STRING "/opt/ts/etc/ssl/keys/"

4. Add an entry to ssl_multicert.config for each certificate and key which your Traffic Server system willbe using to terminate SSL connections with clients.

ip_dest=1.2.3.4 ssl_cert_name=example.com.pemip_dest=* ssl_cert_name=default.pem

5. Optional: Configure the use of client certificates using the variable:ts:cv:‘proxy.config.ssl.client.certification_level‘ in records.config. If you configure Traffic Serverto require client certificates, then Traffic Server verifies the client certificate during the SSL handshake thatauthenticates the client. If you configure Traffic Server to not require client certificates, or if you configurecertificates to be optional and the connecting client does not present one, then access to Traffic Server ismanaged through other Traffic Server options that have been set (such as rules in ip_allow.config).

CONFIG proxy.config.ssl.client.certification_level INT 0

This variable permits one of the following values to be set:

Value Description0 Client certificates not required.1 Client certificates optional. If present, will be used to validate.2 Client certficates required, and must validate based on configured CAs.

6. Optional: Configure the use of Certification Authorities (CAs). CAs add security by verifying theidentity of the person requesting a certificate. The list of acceptable CA signers is configured with:ts:cv:‘proxy.config.ssl.CA.cert.path‘ in records.config.

CONFIG proxy.config.ssl.CA.cert.path STRING "/opt/CA/certs/private-ca.pem"

7. Run the command traffic_ctl server restart to restart Traffic Server on the local node ortraffic_ctl cluster restart to restart Traffic Server on all the nodes in a cluster.

3.5. Security 55


Traffic Server and Origin Server Connections

The figure below illustrates communication between Traffic Server and an origin server when the SSL terminationoption is enabled for Traffic Server/origin server connections.

3.10: Traffic Server and origin server communication using SSL termination

The figure above depicts the following:

1. If a client request is a cache miss or is stale, then Traffic Server sends an HTTPS request for the content to theorigin server. The origin server receives the request and performs the SSL handshake to authenticate TrafficServer and determine the encryption method to be used.

2. If Traffic Server is allowed access, then the origin server encrypts the content and sends it to Traffic Server,where it is decrypted (using the method determined during the handshake). A plain text version of the contentis saved in the cache, if Traffic Server deems the content cacheable.

3. If SSL termination is enabled for Client/Traffic Server connections, then Traffic Server re-encrypts the contentand sends it to the client via HTTPS, where it is decrypted and displayed. If SSL termination is not enabled forClient/Traffic Server connections, then Traffic Server sends the plain text version of the content to the client viaHTTP.

To configure Traffic Server to use the SSL termination option for Traffic Server and origin server connections, youmust do the following:

1. Ensure first that your origin server responds properly to SSL requests, and configure it for client certificatevalidation if you intend to use that as part of your access control scheme.

Refer to your origin server’s documentation for details. If your origin server is another Traffic Server system,then you may follow the steps outlined in Client and Traffic Server Connections for configuring the origin serverto validate client certificates.

2. Optional: Obtain and install an SSL client certificate from a recognized certificate authority, if your origin serverrequires client certificate validation for access control. Your client certificate must be signed by a CertificateAuthority recognized by your origin server.

If you are using a client certificate, you must add its location to records.config in the setting:ts:cv:‘proxy.config.ssl.client.cert.path‘ and :ts:cv:‘proxy.config.ssl.client.cert.filename‘.

CONFIG proxy.config.ssl.client.cert.path STRING "/opt/ts/etc/ssl/certs/"CONFIG proxy.config.ssl.client.cert.filename STRING "client.pem"

You must also provide the paths to the private key for this certificate, unless the key is con-tained within the same file as the certificate, using :ts:cv:‘proxy.config.ssl.client.private_key.path‘ and:ts:cv:‘proxy.config.ssl.client.private_key.filename‘.



CONFIG proxy.config.ssl.client.private_key.path STRING "/opt/ts/etc/ssl/keys/"CONFIG proxy.config.ssl.client.private_key.filename STRING "client.pem"

3. Enable or disable, per your security policy, server SSL certificate verification using:ts:cv:‘proxy.config.ssl.client.verify.server‘ in records.config.

CONFIG proxy.config.ssl.client.verify.server INT 1

4. Add the collection of authorized Certificate Authorities to the Traffic Server configura-tion in records.config using the settings :ts:cv:‘proxy.config.ssl.client.CA.cert.path‘ and:ts:cv:‘proxy.config.ssl.client.CA.cert.filename‘.

CONFIG proxy.config.ssl.client.CA.cert.path STRING "/opt/ts/etc/ssl/certs/"CONFIG proxy.config.ssl.client.CA.cert.filename STRING "CAs.pem"

5. Run the command traffic_ctl server restart to restart Traffic Server on the local node ortraffic_ctl cluster restart to restart Traffic Server on all the nodes in a cluster.

Rotating TLS Session Ticket Keys

TLS sessions can be resumed through session tickets which are encrypted with a session ticket key and stored onclients. For better security, the ticket keys can be rotated periodically, say, every 24 hours. The ticket keys are storedin a ticket key file as a reverse queue in 48-byte chunks.

1. Generate a new ticket key and push it to the beginning of the ticket key file.

2. Optional: Delete the last ticket key from the ticket key file.

3. Touch ssl_multicert.config to indicate that the SSL configuration is stale.

4. Run the command traffic_ctl config reload to apply the new ticket key.

OCSP Stapling

OCSP Stapling is an alternative approach to checking the revocation status of an SSL certificate using the OnlineCertificate Status Protocol.

Under the original OCSP implementation, clients requested a certificate’s revocation status directly from the CertificateAuthority (CA) that issued the certificate. This could cause significant load on the CA servers since they were requiredto provide a response to every client of a given certificate in real time.

Enabling OCSP Stapling instructs Traffic Server to retrieve and cache the revocation status of all configured SSLcertificates, and present them to the client when the client requests the status. Traffic Server will automatically querythe OCSP responder specified in the SSL certificate to gather the latest revocation status. Traffic Server will then cachethe results for each configured certifcate. The location of the OCSP responder is taken from the Authority InformationAccess field of the signed certificate. For example:

Authority Information Access:OCSP - URI:http://ocsp.digicert.comCA Issuers - URI:http://cacerts.digicert.com/DigiCertSHA2SecureServerCA.

→˓crt

Support for OCSP Stapling can be tested using the -status option of the OpenSSL client:

3.5. Security 57


$ openssl s_client -connect mozillalabs.com:443 -status...======================================OCSP Response Data:

OCSP Response Status: successful (0x0)Response Type: Basic OCSP ResponseVersion: 1 (0x0)

...

Details of the OCSP Stapling TLS extension can be found in RFC 6961.

To configure Traffic Server to use OCSP Stapling, edit the following variables in records.config file:

• :ts:cv:‘proxy.config.ssl.ocsp.enabled‘

• :ts:cv:‘proxy.config.ssl.ocsp.cache_timeout‘

• :ts:cv:‘proxy.config.ssl.ocsp.request_timeout‘

• :ts:cv:‘proxy.config.ssl.ocsp.update_period‘

Split DNS

The Split DNS option enables you to configure Traffic Server to use multiple DNS servers, as dictated by your securityrequirements. For example, you might configure Traffic Server to use one set of DNS servers to resolve hostnames onyour internal network, while allowing DNS servers outside the firewall to resolve hosts on the Internet. This maintainsthe security of your intranet, while continuing to provide direct access to sites outside your organization.

To configure Split DNS:

1. Specify the rules for performing DNS server selection based on the destination domain, the destination host, ora URL regular expression. These rules are located in splitdns.config.

2. Enable the Split DNS option by adjusting :ts:cv:‘proxy.config.dns.splitDNS.enabled‘ in records.config.

CONFIG proxy.config.dns.splitDNS.enabled INT 1

3. Run the command traffic_line -x to apply the configuration changes.

Cache Storage

The Traffic Server cache consists of a high-speed object database called the object store that indexes cache objectsaccording to URLs and their associated headers.

The Traffic Server Cache

The Traffic Server cache consists of a high-speed object database called the object store. The object store indexescache objects according to URLs and associated headers. This enables Traffic Server to store, retrieve, and servenot only web pages, but also parts of web pages - which provides optimum bandwidth savings. Using sophisticatedobject management, the object store can cache alternate versions of the same object (versions may differ because ofdissimilar language or encoding types). It can also efficiently store very small and very large documents, therebyminimizing wasted space. When the cache is full, Traffic Server removes stale data to ensure the most requestedobjects are kept readily available and fresh.


https://tools.ietf.org/html/rfc6961.html


Traffic Server is designed to tolerate total disk failures on any of the cache disks. If the disk fails completely, thenTraffic Server marks the entire disk as corrupt and continues using the remaining disks. An alarm is then created toindicate which disk failed. If all of the cache disks fail, then Traffic Server goes into proxy-only mode.

You can perform the following cache configuration tasks:

• Change the total amount of disk space allocated to the cache; refer to Changing Cache Capacity.

• Partition the cache by reserving cache disk space for specific protocols and origin servers/domains; refer toPartitioning the Cache.

• Delete all data in the cache; refer to Clearing the Cache.

• Override cache directives for a requested domain name, regex on a url, hostname or ip, with extra filters for time,port, method of the request, and more. ATS can be configured to never cache, always cache, ignore no-cachedirectives, etc. These are configured in cache.config.

The RAM Cache

Traffic Server maintains a small RAM cache of extremely popular objects. This RAM cache serves the most popularobjects as quickly as possible and reduces load on disks, especially during temporary traffic peaks. You can configurethe RAM cache size to suit your needs, as described in Changing the Size of the RAM Cache below.

The RAM cache supports two cache eviction algorithms, a regular LRU (Least Recently Used) and the more ad-vanced CLFUS (Clocked Least Frequently Used by Size; which balances recentness, frequency, and size to maxi-mize hit rate, similar to a most frequently used algorithm). The default is to use CLFUS, and this is controlled via:ts:cv:‘proxy.config.cache.ram_cache.algorithm‘.

Both the LRU and CLFUS RAM caches support a configuration to increase scan resistance. In a typical LRU, ifyou request all possible objects in sequence, you will effectively churn the cache on every request. The option:ts:cv:‘proxy.config.cache.ram_cache.use_seen_filter‘ can be set to add some resistance against this problem.

In addition, CLFUS also supports compressing in the RAM cache itself. This can be useful for content which is notcompressed by itself (e.g. images). This should not be confused with Content-Encoding: gzip, this feature isonly present to save space internally in the RAM cache itself. As such, it is completely transparent to the User-Agent.The RAM cache compression is enabled with the option :ts:cv:‘proxy.config.cache.ram_cache.compress‘.

Possible values are:

Value Meaning0 No compression (default)1 fastlz compression2 libz compression3 liblzma compression

Changing the Size of the RAM Cache

Traffic Server provides a dedicated RAM cache for fast retrieval of popular small objects. The default RAM cache sizeis automatically calculated based on the number and size of the cache partitions you have configured. If you’ve parti-tioned your cache according to protocol and/or hosts, then the size of the RAM cache for each partition is proportionalto the size of that partition.

You can increase the RAM cache size for better cache hit performance. However, if you increase the size of the RAMcache and observe a decrease in performance (such as increased latencies), then it’s possible that the operating systemrequires more memory for network resources. In such instances, you should return the RAM cache size to its previousvalue.

To change the RAM cache size:

3.6. Cache Storage 59


1. Stop Traffic Server.

2. Set the variable :ts:cv:‘proxy.config.cache.ram_cache.size‘ to specify the size of the RAM cache. The defaultvalue of -1 means that the RAM cache is automatically sized at approximately 1MB per gigabyte of disk.

3. Restart Traffic Server. If you increase the RAM cache to a size of 1GB or more, then restart with thetrafficserver command (refer to Start Traffic Server).

Changing Cache Capacity

You can increase or reduce the total amount of disk space allocated to the cache without clearing the content. To checkthe size of the cache (in bytes), enter the command traffic_ctl metric get proxy.process.cache.bytes_total.

Increasing Cache Capacity

To increase the total amount of disk space allocated to the cache on existing disks, or to add new disks to a TrafficServer node:


2. Add hardware, if necessary.

3. Edit storage.config to increase the amount of disk space allocated to the cache on existing disks or de-scribe the new hardware you are adding.

4. Restart Traffic Server.

Reducing Cache Capacity

To reduce the total amount of disk space allocated to the cache on an existing disk, or to remove disks from a TrafficServer node:


2. Remove hardware, if necessary.

3. Edit storage.config to reduce the amount of disk space allocated to the cache on existing disks or deletethe reference to the hardware you’re removing.


: In storage.config, a formatted or raw disk must be at least 128 MB.

Partitioning the Cache

You can manage your cache space more efficiently and restrict disk usage by creating cache volumes with differentsizes for specific protocols. You can further configure these volumes to store data from specific origin servers and/ordomains. The volume configuration must be the same on all nodes in a cluster.



Creating Cache Partitions for Specific Protocols

You can create separate volumes for your cache that vary in size to store content according to protocol. This ensuresthat a certain amount of disk space is always available for a particular protocol. Traffic Server currently supports onlythe http partition type.

To partition the cache according to protocol:

1. Enter a line in volume.config for each volume you want to create.

volume=1 scheme=http size=50%volume=2 scheme=http size=50%


: Volume definitions must be the same across all nodes in a cluster.

Making Changes to Partition Sizes and Protocols

After you’ve configured your cache volumes based on protocol, you can make changes to the configuration at anytime. Before making changes, note the following:

• You must stop Traffic Server before you change the cache volume size and protocol assignment.

• When you increase the size of a volume, the contents of the volume are not deleted. However, when you reducethe size of a volume, the contents of the volume are deleted.

• When you change the volume number, the volume is deleted and then recreated, even if the size and protocoltype remain the same.

• When you add new disks to your Traffic Server node, volume sizes specified in percentages will increase pro-portionately.

• Substantial changes to volume sizes can result in disk fragmentation, which affects performance and cache hitrate. You should clear the cache before making many changes to cache volume sizes (refer to Clearing theCache).

Partitioning the Cache According to Origin Server or Domain

After you have partitioned the cache according to size and protocol, you can assign the volumes you created to specificorigin servers and/or domains. You can assign a volume to a single origin server or to multiple origin servers. However,if a volume is assigned to multiple origin servers, then there is no guarantee on the space available in the volumes foreach origin server. Content is stored in the volumes according to popularity. In addition to assigning volumes tospecific origin servers and domains, you must assign a generic volume to store content from all origin servers anddomains that are not listed. This generic volume is also used if the partitions for a particular origin server or domainbecome corrupt. If you do not assign a generic volume, then Traffic Server will run in proxy-only mode.

: You do not need to stop Traffic Server before you assign volumes to particular hosts or domains. However, this typeof configuration is time-consuming and can cause a spike in memory usage. Therefore, it’s best to configure partitionassignment during periods of low traffic.

To partition the cache according to hostname and domain:



1. Configure the cache volumes according to size and protocol, as described in Creating Cache Partitions forSpecific Protocols.

2. Create a separate volume based on protocol for each host and domain, as well as an additional generic partitionto use for content that does not belong to these origin servers or domains. The volumes do not need to be thesame size.

3. Enter a line in the hosting.config file to allocate the volume(s) used for each origin server and/or domain.

4. Assign a generic volume to use for content that does not belong to any of the origin servers or domains listed inthe file. If all volumes for a particular origin server become corrupt, then Traffic Server will also use the genericvolume to store content for that origin server as per hosting.config.


Configuring the Cache Object Size Limit

By default, Traffic Server allows objects of any size to be cached. You can change the default behavior and specify asize limit for objects in the cache via the steps below:

1. Set :ts:cv:‘proxy.config.cache.max_doc_size‘ to specify the maximum size in bytes allowed for objects in thecache. A setting of 0 (zero) will permit cache objects to be unlimited in size.


Clearing the Cache

When you clear the cache, you remove all data from the entire cache - including data in the host database. You shouldclear the cache before performing certain cache configuration tasks such as partitioning. You cannot clear the cachewhen Traffic Server is running.

To clear the cache:

1. Stop Traffic Server (see Stop Traffic Server)

2. Enter the following command to clear the cache:

traffic_server -Cclear

The clear command deletes all data in the object store and the host database. Traffic Server does not promptyou to confirm the deletion.

3. Restart Traffic Server (see Start Traffic Server).

Removing an Object From the Cache

Traffic Server accepts the custom HTTP request method PURGE when removing a specific object from cache. If theobject is found in the cache and is successfully removed, then Traffic Server responds with a 200 OK HTTP message;otherwise, a 404 File Not Found message is returned.

: By default, the PURGE request method is only processed if received on the localhost interface.

In the following example, Traffic Server is running on the domain example.com and you want to remove the imageremove_me.jpg from cache. Because by default we do not permit PURGE requests from any other IP, we connectto the daemon via localhost:



$ curl -vX PURGE --resolve example.com:80:127.0.0.1 http://example.com/remove_me.jpg

* About to connect() to example.com port 80 (#0)

* Trying 127.0.0.1... connected

* Connected to example.com (127.0.0.1) port 80 (#0)

> PURGE /remove_me.jpg HTTP/1.1> User-Agent: curl/7.19.7> Host: example.com> Accept: */*>< HTTP/1.1 200 Ok< Date: Thu, 08 Jan 2010 20:32:07 GMT< Connection: keep-alive

The next time Traffic Server receives a request for the removed object, it will contact the origin server to retrieve anew copy, which will replace the previously cached version in Traffic Server.

This procedure only removes an object from a specific Traffic Server cache. Users may still see the old (removed)content if it was cached by intermediary caches or by the end-users’ web browser.

Inspecting the Cache

Traffic Server provides a Cache Inspector utility that enables you to view, delete, and invalidate URLs in the cache(HTTP only). The Cache Inspector utility is a powerful tool that’s capable of deleting all the objects in your cache.Therefore, make sure that only authorized administrators are allowed to access this utility through proper use of the@src_ip option in remap.config and the instructions detailed in Controlling Access.

Accessing the Cache Inspector Utility

To access the Cache Inspector utility:

1. Set :ts:cv:‘proxy.config.http_ui_enabled‘ to 1.

2. To access the cache inspector in reverse proxy mode, you must add a remap rule to remap.config to exposethe URL. This should be restricted to a limited set of hosts using the @src_ip option. To restrict access to thenetwork 172.28.56.0/24, use

map http://yourhost.com/myCI/ http://{cache} @action=allow @src_ip=172.28.56.1-→˓172.28.56.254

3. Reload the Traffic Server configuration by running traffic_ctl config reload.

4. Open your web browser and go to the the following URL:

http://yourhost/myCI/

You will now be presented with the Cache Inspector interface.

Using the Cache Inspector Utility

The Cache Inspector Utility provides several options that enable you to view and delete the contents of your cache.

Lookup URL Search for a particular URL in the cache. When Traffic Server finds the URL in the cache, it willdisplay details of the object that corresponds to the URL (e.g. header length and number of alternates). Theoption to delete the URL from the cache will be presented.



Delete URL Delete the object from the cache which corresponds to the given URL. Success or failure will be indicatedafter a delete has been attempted.

Regex Lookup Search URLs within the cache using one or more regular expressions.

Regex Delete Deletes all objects from the cache which match the provided regular expressions.

Regex Invalidate Marks any objects in the cache which match the given regular expressions as stale. Traffic Serverwill contact the relevant origin server(s) to confirm the validity and freshness of the cached object, updating thecached object if necessary.

: Only one administrator should delete and invalidate cache entries from the Cache Inspector at any point in time.Changes made by multiple administrators at the same time can lead to unpredictable results.

Plugins

Overview

One of the key features of Apache Traffic Server™ is its modularity. Features that aren’t needed in the core simplyaren’t there. This helps to provide an additional guarantee that our core can remain fast by concentrating on the thingsthat we always provide: caching and proxying.

All other functionality can be moved into plugins and by offering a consistent C API, everyone can implement theirown functionality, without having to touch the core.

Stable plugins

Plugins that are considered stable are installed by default in Traffic Server releases.

Cache URL Plugin

: This plugin is deprecated as of v6.2.0 and will be removed as of v7.0.0. It is replaced by a new Cache KeyManipulation Plugin and you should change your configurations to use the new plugin instead. Please find someexamples below.

This plugin allows you to change the cache key that is used for caching a request by using any portion of the URL viaregular expressions.

Purpose

By default Traffic Server generates keys for cache objects from the full request URL. Future requests will only be ableto use the existing cache object if they result in the same cache key. It can be the case, however, that the same contentis accessible through more than one request URL. Without any special configuration, that same content will result inmultiple cache objects based on the differing URLs.

In this scenario, it can be very beneficial to allow Traffic Server to reuse a cache object for multiple URLs. This pluginenables that behavior, by allowing the administrator to define custom patterns for generating cache keys for some, orall, of their site’s URLs. Uninteresting or irrelevant portions of request URLs may be removed, or altered, before thecache key is created using the full power of regular expressions.



Installation

This plugin is considered stable and is included with Traffic Server by default. There are no special steps necessaryfor its installation.

Configuration

1. Enable the plugin by modifying plugin.config to include the plugin:

cacheurl.so

2. Create a cacheurl.config file in the plugin directory with the url regex patterns you wish to match, usingthe following format:

<pattern> <replacement>

The <pattern> is a regular expression (PCRE) applied against the incoming request URL. The<replacement> may contain $1, $2, and so on, which will be replaced with the appropriate matchinggroup from <pattern>.

3. Reload your Traffic Server configuration with traffic_ctl config reload.

Logging

A new log file will be generated by this plugin, containing entries for each incident of an incoming URL’s cache keybeing altered, located in the Traffic Server log directory and named cacheurl.log.

Examples

While many possibilities exist, limited really only by your site’s URL scheme and the capabililties of PCRE regularexpressions, the following are examples of a few situations Traffic Server administrators may encounter.

Multiple Domains, One Cache Object

If you have multiple subdomains which serve the same file content, there may be no reason to duplicate their storage(leading to higher churn and faster potential eviction of still-fresh objects) in your Traffic Server cache. By default,however, the differing subdomains will lead to differing cache keys. To work around this, a pattern like the followingcan be used to create a single cache key which will be valid for all subdomains:

http://s[123].example.com/(.*) http://s.example.com.TSINTERNAL/$1

Now, the domains s1.example.com, s2.example.com, and s3.example.com will effectively share cacheobjects. Adding a unique suffix (TSINTERNAL in this example) to the cache key guarantees that it won’t clash witha real URL should s.example.com exist.

Converting to Cache Key Manipulation Plugin

You could do the same with Cache Key Manipulation Plugin by adding the following to the mapping rules:

3.7. Plugins 65


@plugin=cachekey.so @pparam=--capture-prefix=/s[123].example.com:.*/s.example.com.→˓TSINTERNAL/

Ignoring Some Query Parameters

If your site attaches, for example, session information to URLs, even on pages which do not include session-specificcontent and may be safely cached, you can add a pattern which strips this unnecessary information from the URLbefore generating a cache key, while still retaining important query parameters:

http://www.example.com/video\?.*?\&?(id=[0-9a-f]*).*?\&(format=[a-z]*) http://video-→˓srv.example.com.ATSINTERNAL/$1&$2



@plugin=cachekey.so @pparam=--include-params=id,format

Ignore Query String on Specific Pages

To completely ignore a query string for a specific page, it’s quite easy to simply match and drop everything from the? query string opener to the end of the URL:

http://www.example.com/some/page(?:\?|$) http://www.example.com/some/page



@plugin=cachekey.so @pparam=--remove-all-params

Configuration Remap Plugin

This plugin allows you to override configuration directives dependent on remapping rules.

Purpose

Traffic Server provides a plethora of configuration options, but specifying the values of those options in records.config is global. All requests, regardless of the cache object or its origin, will be evaluated within the same collectionof settings. Sometimes you may want Traffic Server to behave differently for portions of your cache.

Perhaps you have Negative Response Caching enabled, but you wish to greatly reduce the validity times for just oneof your origin servers while allowing the rest of your origins to have their errors cached for long durations.

Or maybe you make use of Heuristic Expiration but require different fuzz times for various objects because of thenature of their content and expected lifetimes.



Any configuration directive which is overridable can be modified on a per-map basis with this plugin. This opens upa level of flexibility in your configurations for effectively managing and caching content with varied needs, withouthaving to resort to multiple Traffic Server instances.

Installation


Configuration

Configuration of this plugin is performed alongside the actual remapping rules which trigger the desired configurationdirective changes. There are two methods available for specifying the actual directives and their modified values.

Inline Directives

In cases where you have very few remapping rules which modify directives, and they are modifying only a smallnumber of directives, you may find it easiest to simply specify those directive changes in-line with your remappingrules. This is done by specifying key = value pairs, where the key is the configuration directive name and the value isits desired setting for the remapping rule.

For example, the enable :ts:cv:‘proxy.config.url_remap.pristine_host_hdr‘ for a single map rule, you would addthe following to your remap.config:

map http://cdn.example.com/ http://origin.example.com/ \@plugin=conf_remap.so @pparam=proxy.config.url_remap.pristine_host_hdr=1

External Configuration

There may be situations in which you have many directives you wish to modify, or where multiple remapping rulesperform the same directive changes. External configurations can simplify management of these rules, and help toreduce the possibility of transcription errors, or keeping all the directive settings in sync across all the remapping rulesover time.

Instead of specifying the directives and their values in remap.config as you do with the in-line method, youplace all the affected directives in a separate text file. The location and name is entirely up to you, but we’ll use/etc/trafficserver/cdn_conf_remap.config here. The contents of this file should mirror how configuration directives arewritten in records.config:

CONFIG proxy.config.url_remap.pristine_host_hdr INT 1

Your remap.config will then contain remapping rules that point to this external file:

map http://cdn.example.com/ http://some-server.example.com \@plugin=conf_remap.so @pparam=/etc/trafficserver/cdn_conf_remap.config

Your external configuration may contain as many directives as you wish.

3.7. Plugins 67


Caveats

This plugin can only modify the values for those configuration directives which are overridable, meaning they are notfixed upon Traffic Server startup. While this generally shouldn’t prove too onerous a restriction, you should consultthe individual directives’ documentation to confirm whether they may be overridden.

Further Reading

For more information about the implementation of overridable configuration directives, you may consult the devel-oper’s documentation for TSHttpOverridableConfig.

GZip Plugin

This plugin adds compression and decompression options to both origin and cache responses.

Purpose

Not all clients can handle compressed content. Not all origin servers are configured to respond with compressedcontent when a client says it can accept it. And it’s not always necessary to make two separate requests to an origin,and track two separate cache objects, for the same content - once for a compressed version and another time for anuncompressed version.

This plugin tidies up these problems by transparently compressing or deflating origin responses, as necessary, so thatboth variants of a response are stored as alternates and the appropriate version is used for client responses, dependingon the client’s indication (via an Accept request header) of what it can support.

Additionally, this plugin adds configurability for what types of origin responses will receive this treatment, which willbe proxied and cached with default behavior, and which may be explicitly disallowed to cache both compressed anddeflated versions (because, for example, the cost of compression is known ahead of time to outweigh the space andbandwidth savings and you wish to avoid Traffic Server even testing for the possibility).

Installation


Configuration

This plugin can be used as either global plugin or remap plugin. It can be enabled globally for Traffic Server by addingthe following to your plugin.config:

gzip.so

With no further options, this will enable the following default behavior:

• Enable caching of both compressed and uncompressed versions of origin responses as alternates.

• Compress objects with text/* content types for every origin.

• Don’t hide Accept encoding headers from origin servers (for an offloading reverse proxy).

• No URLs are disallowed from compression.



• Disable flush (flush gzipped content to client).

Alternatively, a configuration may be specified (shown here using the sample configuration provided with the plugin’ssource):

gzip.so <path-to-plugin>/sample.gzip.config

This can be used as remap plugin by pointing to config file in remap rule remap.config:

@plugin=gzip.so @pparam=--config=<path-to-plugin>/sample.gzip.config

The following sections detail the options you may specify in the plugin’s configuration file. Options may be usedglobally, or may be specified on a per-site basis by preceding them with a [<site>] line, where <site> is the client-facing domain for which the options should apply.

Per site configuration for remap plugin should be ignored.

cache

When set to true, causes Traffic Server to cache both the compressed and uncompressed versions of the content asalternates. When set to false, Traffic Server will cache only the compressed or decompressed variant returned bythe origin. Enabled by default.

compressible-content-type

Provides a wildcard to match against content types, determining which are to be considered compressible. This defaultsto text/*.

disallow

Provides a wildcard pattern which will be applied to request URLs. Any which match the pattern will be considereduncompressable, and only deflated versions of the objects will be cached and returned to clients. This may be usefulfor objects which already have their own compression built-in, to avoid the expense of multiple rounds of compressionfor trivial gains.

enabled

When set to true permits objects to be compressed, and when false effectively disables the plugin in the currentcontext.

flush

Enables (true) or disables (false) flushing of compressed objects to clients.

remove-accept-encoding

When set to true this option causes the plugin to strip the request’s Accept encoding header when contacting theorigin server. Setting this option to false will leave the header intact if the client provided it.

• To ease the load on the origins.

3.7. Plugins 69


• For when the proxy parses responses, and the resulting compression and decompression is wasteful.

Examples

To establish global defaults for all site requests passing through Traffic Server, while overriding just a handful forrequests to content at www.example.com, you might create a configuration with the following options:

# Set some global options firstcache trueenabled trueremove-accept-encoding falsecompressible-content-type text/*flush false

# Now set a configuration for www.example.com[www.example.com]cache falseremove-accept-encoding truedisallow /notthis/*.jsflush true

Assuming the above options are in a file at /etc/trafficserver/gzip.config the plugin would be enabledfor Traffic Server in plugin.config as:

gzip.so /etc/trafficserver/gzip.config

Header Rewrite Plugin

This plugin allows you to modify arbitrary headers based on defined rules, for both requests and responses.

Purpose

Remapping an incoming client request to an origin server is at the heart of what we use Traffic Server for, but oftenwe need to do more to requests than just change their destination according to simple rewriting rules against the URL.

We may need to direct requests to different origins based on a client cookie.

Our origins might return error response codes which are a little too customized and we want to condense the possiblevalues to just the official codes.

We might want to strip a set of internal-use or debugging related HTTP headers from responses before sending themto clients, unless the original request had its own special header indicating that they should be retained.

Or perhaps we want to redirect certain requests differently when they come from a known group of IP addresses (say,our developers’ office network) and we have a file on our proxy caching servers called /var/developertesting.(Stranger QA methods exist.)

Maybe we want to deny access to a resource with an HTTP 403 if the client connected to Traffic Server over a particularport, used HEAD instead of GET, doesn’t list Spanish (Paraguay dialect) in their Accept-Language header, andeither the origin server replied with 304 or we randomly generate an integer between 0 and 500 and get back anythinggreater than 290.

These more complicated transformations of requests and responses (even that last one) are made possible by thisplugin.



Installation


Configuration

Header rewrite configurations, the actual conditions and operations that make up the activity performed by the plugin,are specified in external files and not in-line with the various mapping (and remapping) rules you may have configuredfor your proxy. The location of this file is arbitrary, as long as the Traffic Server processes have permissions to read it,though you may find it useful to keep it in the same location as your other proxy configuration files.

The paths given to the configuration file(s) may be absolute (leading with a / character), or they may be relative to theTraffic Server configuration directory.

There are two methods for enabling this plugin, based on whether you wish it to operate globally on every request thatpasses through your proxy, or only on some subset of the requests by enabling it only for specific mapping rules.

Enabling Globally

This plugin may be enabled globally, so that the conditions and header rewriting rules are evaluated for every requestmade to your Traffic Server instance. This is done by adding the following line to your plugin.config:

header_rewrite.so config_file_1.conf config_file_2.conf ...

You may specify multiple configuration files. Their rules will be evaluated in the order the files are listed.

Enabling Per-Mapping

Alternatively, the plugin can be enabled for specific mapping rules, by modifying the relevant entries in your remap.config:

map http://a http://b @plugin=header_rewrite.so @pparam=rules1.conf ...

As with the global method above, multiple configuration files may be listed, each with its own @pparam=<file>and their contents will be evaluated in the order the files were specified.

Rewriting Rules

Header rewriting rules consist of zero or more Conditions followed by one or more Operators. Conditions are usedto limit the requests which will be affected by the operator(s). Additionally, both conditions and operators may haveflags which modify their behavior.

A complete rule, consisting of two conditions and a single operator might look like the following:

cond %{STATUS} >399 [AND]cond %{STATUS} <500set-status 404

Which converts any 4xx HTTP status code from the origin server to a 404. A response from the origin with a status of200 would be unaffected by this rule.

3.7. Plugins 71


Conditions

Conditions are used as qualifiers, causing the associated operators to only be evaluated if the condition(s) are met.Conditions all take the following form:

cond %{<condition name>[:<argument>]} <operand> [<flags>]

Every condition begins with the literal string cond to indicate that this line is a condition, not an operator. This isfollowed by the condition name, inside curly braces and preceded by a percent sign (e.g. %{TRUE} for the conditionnamed TRUE). Some condition names take an argument. Header conditions, for example, take the name of the headerin question, and cookie conditions take the name of the cookie. For these, the condition name is followed by a colonand the argument value (e.g. %{HEADER:User-Agent} for a header condition against the User-Agent header).

The operand of a condition provides a value, pattern, or range against which to match. The format is described inCondition Operands below.

Finally, a condition may optionally have various flags associated with it. These are described in Condition Flagsbelow.

The following sections list all of the condition types currently supported. For increased clarity in their usage, theoptional [<flags>] portion of the condition is omitted from all of the examples.

ACCESS

cond %{ACCESS:<path>}

Returns true if Traffic Server was able to successfully update the access time on the file at <path>. This conditionwill return false if the file does not exist or Traffic Server cannot access it for any other reason.

CLIENT-HEADER

cond %{CLIENT-HEADER:<name>} <operand>

Value of the header <name> from the original client request (regardless of the hook context in which the rule is beingevaluated). Note that some headers may appear in an HTTP message more than once. In these cases, the value ofthe header operated on by this condition will be a comma separated string of the values from every occurrence of theheader. More details are provided in Repeated Headers below.

CLIENT-IP

cond %{CLIENT-IP} <operand>

Remote IP address, as a string, of the client connection for the current transaction.

CLIENT-URL

cond %{CLIENT-URL:<part>} <operand>

The URL of the original request. Regardless of the hook context in which the rule is evaluated, this condition willalways operate on the original, unmapped URL supplied by the client. The <part> may be specified according tothe options documented in URL Parts.



COOKIE

cond %{COOKIE:<name>} <operand>

Value of of the cookie <name>. This does not expose or match against a cookie’s expiration, the domain(s) on whichit is valid, whether it is protocol restricted, or any of the other metadata; simply the current value of the cookie aspresented by the client.

FROM-URL

cond %{FROM-URL:<part>} <operand>

In a remapping context, this condition matches against the source URL from which the remapping was generated.This condition is valid only within configurations provided through remap.config as described in Enabling Per-Mapping above.

The <part> allows the operand to match against just a component of the URL, as documented in URL Parts below.

GEO

:: cond %{GEO:<part>} <operand>

Perform a GeoIP lookup of the client-IP, using a 3rd party library and DB. Currently only the MaxMind GeoIP API issupported. The default is to do a Country lookup, but the following qualifiers are supported:

%{GEO:COUNTRY} The country code (e.g. “US”) %{GEO:COUNTRY-ISO} The country ISO code(e.g. 225) %{GEO:ASN} The AS number for the provider network (e.g. 7922) %{GEO:ASN-NAME}A descriptive string of the AS provider

These operators can be used both as conditionals, as well as values for setting headers. For example:

cond %{SEND_RESPONSE_HDR_HOOK} [AND]cond %${GEO:COUNTRY} =US

set-header ATS-Geo-Country %{GEO:COUNTRY}set-header ATS-Geo-Country-ISO %{GEO:COUNTRY-ISO}set-header ATS-Geo-ASN %{GEO:ASN}set-header ATS-Geo-ASN-NAME %{GEO:ASN-NAME}

HEADER

cond %{HEADER:<name>} <operand>

Value of the header <name> from either the original client request or the origin server’s response, depending uponthe hook context in which the rule is being evaluated. Consult Requests vs. Responses for more information on howto distinguish the two, as well as enforce that a rule is always evaluated in the desired context.

Note that some headers may appear in an HTTP message more than once. In these cases, the value of the headeroperated on by this condition will be a comma separated string of the values from every occurrence of the header.Refer to Repeated Headers for more information.

If you wish to use a client request header, regardless of hook context, you may consider using the CLIENT-HEADERcondition instead.

3.7. Plugins 73


INCOMING-PORT

cond %{INCOMING-PORT} <operand>

TCP port, as a decimal integer, on which the incoming client connection was made.

INTERNAL-TRANSACTION

cond %{INTERNAL-TRANSACTION}

Returns true if the current transaction was internally-generated by Traffic Server (usingTSHttpTxnIsInternal()). These transactions are not initiated by external client requests, but are trig-gered (often by plugins) entirely within the Traffic Server process.

METHOD

cond %{METHOD} <operand>

The HTTP method (e.g. GET, HEAD, POST, and so on) used by the client for this transaction.

NOW

cond %{NOW:<part>} <operand>

This is the current time, in the local timezone as set on the machine, typically GMC. Without any further qualifiers,this is the time in seconds since EPOCH aka Unix time. Qualifiers can be used to give various other values, such asyear, month etc.

%{NOW:YEAR} Current year (e.g. 2016)%{NOW:MONTH} Current month (0-11, 0 == January)%{NOW:DAY} Current day of the month (1-31)%{NOW:HOUR} Current hour (0-23, in the 24h system)%{NOW:MIN} Current minute (0-59}%{NOW:WEEKDAY} Current weekday (0-6, 0 == Sunday)%{NOW:YEARDAY} Current day of the year (0-365, 0 == Jan 1st)

PATH

cond %{PATH} <operand>

The path component of the transaction. This includes the leading / that immediately follows the hostname andterminates prior to the ? signifying the beginning of query parameters (or the end of the URL, whichever occurs first).

Refer to Requests vs. Responses for more information on determining the context in which the transaction’s URL isevaluated.



QUERY

cond %{QUERY} <operand>

The query parameters, if any, of the transaction. Refer to Requests vs. Responses for more information on determiningthe context in which the transaction’s URL is evaluated.

RANDOM

cond %{RANDOM:<n>} <operand>

Generates a random integer between 0 and <n>, inclusive.

STATUS

cond %{STATUS} <operand>

Numeric HTTP status code of the response.

TO-URL

cond %{TO-URL:<part>} <operand>

In a remapping context, this condition matches against the target URL to which the remapping is directed. This con-dition is valid only within configurations provided through remap.config as described in Enabling Per-Mappingabove.

The <part> allows the operand to match against just a component of the URL, as documented in URL Parts below.

TRUE / FALSE

cond %{TRUE}cond %{FALSE}

These conditions always return a true value and a false value, respectively. The true condition is implicit in any ruleswhich specify no conditions (only operators).

TXN-COUNT

cond %{TXN-COUNT} <operand>

Returns the current HTTP client session, which permits detection of requests which are sharing a client session. Sharedclient sessions occur when multiple simultaneous requests are received for the same cache object. Instead of contactingthe origin server separately for each of those client requests, one origin connection is used to fulfill all of the requestsassigned to the shared client session.

3.7. Plugins 75


URL

cond %{URL:option} <operand>

The complete URL of the current transaction. This will automatically choose the most relevant URL depending uponthe hook context in which the condition is being evaluated.

Refer to Requests vs. Responses for more information on determining the context in which the transaction’s URL isevaluated.

Condition Operands

Operands provide the means to restrict the values, provided by a condition, which will lead to that condition evaluatingas true. There are currently four types supported:

Operand Description/regex/ Matches the condition’s provided value against the regular expression.<string Matches if the value from the condition is lexically less than string.>string Matches if the value from the condition is lexically greater than string.=string Matches if the value from the condition is lexically equal to string.

The absence of an operand for conditions which accept them simply requires that a value exists (e.g. the content ofthe header is not an empty string) for the condition to be considered true.

Condition Flags

The condition flags are optional, and you can combine more than one into a comma-separated list of flags. Note thatwhitespaces are not allowed inside the brackets:

Flag DescriptionAND Indicates that both the current condition and the next must be true. This is the default behavior for all

conditions when no flags are provided.NOT Inverts the condition.OR Indicates that either the current condition or the next one must be true, as contrasted with the default

behavior from [AND].

Operators

Operators are the part of your header rewriting rules which actually modify the header content of your requests andresponses. They are always the final part of a rule, following any of the conditions which whittled down the requestsand responses to which they will be applied.

Multiple operators may be specified for a single rule, and they will be executed in the order listed. The end of the ruleis marked either by the end of the configuration file or the next appearance of a condition (whichever occurs first).

The following operators are available:

add-header

add-header <name> <value>



Adds a new <name> header line with the contents <value>. Note that this operator can produce duplicate head-ers if one of <name> already exists, or your configuration supplies multiple instances of this operator in differentrules which are all invoked. This is not an issue for headers which may safely be specified multiple times, such asSet-Cookie, but for headers which may only be specified once you may prefer to use set-header instead.

The header’s <value>may be specified as a literal string, or it may take advantage of Variable Expansion to calculatea dynamic value for the header. In contrast, set-header does not support variable expansion for the header value. If youwish to use variable expansion and avoid duplicate headers, you may consider using an rm-header operator followedby add-header.

counter

counter <name>

Increments an integer counter called <name> every time the rule is invoked. The counter is initialized at 0 if it doesnot already exist. The name you give your counter is arbitrary, though it is strongly advisable to avoid conflicts withexisting Traffic Server statistics.

This counter can be viewed at any time through the standard statistics APIs, including the Stats Over HTTP plugin.

Counters can only increment by 1 each time this operator is invoked. There is no facility to increment by otheramounts, nor is it possible to initialize the counter with any value other than 0. Additionally, the counter will resetwhenever Traffic Server is restarted.

no-op

no-op

This operator does nothing, takes no arguments, and has no side effects.

rm-header

rm-header <name>

Removes the header <name>.

set-config

set-config <name> <value>

Allows you to override the value of a Traffic Server configuration variable for the current connection. The variablesspecified by <name> must be overridable. For details on available Traffic Server configuration variables, consultthe documentation for records.config. You can read more about overridable configuration variables in thedeveloper’s documentation for TSHttpOverridableConfig.

set-conn-dscp

set-conn-dscp <value>

3.7. Plugins 77


When invoked, sets the client side DSCP value for the current transaction. The <value> should be specified as adecimal integer.

set-debug

set-debug

When invoked, this operator enables the internal transaction debugging flag (via TSHttpTxnDebugSet()), whichcauses debug messages to be printed to the appropriate logs even when the debug tag has not been enabled. Foradditional information on Traffic Server debugging statements, refer to Debug Tags in the developer’s documentation.

set-destination

set-destination <part> <value>

Modifies individual components of the remapped destination’s address. When changing the remapped destination,<part> should be used to indicate the component that is being modified (see URL Parts), and <value> will beused as its replacement. You must supply a non-zero length value, otherwise this operator will be an effective no-op(though a warning will be emitted to the logs if debugging is enabled).

set-header

set-header <name> <value>

Replaces the value of header <name> with <value>, creating the header if necessary.

Note that, unlike add-header, this operator does not currently support variable expansion. Values may only be specifiedaccording to Header Values.

set-redirect

set-redirect <code> <destination>

When invoked, sends a redirect response to the client, with HTTP status <code>, and a new location of<destination>. If the QSA flag is enabled, the original query string will be preserved and added to the newlocation automatically. This operator supports Variable Expansion for <destination>.

set-status

set-status <code>

Modifies the HTTP status code used for the response. <code> must be a valid status code. This operator will alsoupdate the reason in the HTTP response, based on the code you have chosen. If you wish to override that with yourown text, you will need to invoke the set-status-reason operator after this one.


https://en.wikipedia.org/wiki/Differentiated_services


set-status-reason

set-status-reason <reason>

Modifies the HTTP response to use <reason> as the HTTP status reason, instead of the standard string (whichdepends on the HTTP status code used).

set-timeout-out

set-timeout-out <type> <value>

Modifies the timeout values for the current transaction to <value> (specified in milliseconds). The <type> mustbe one of the following: active, inactive, connect, or dns.

skip-remap

skip-remap <value>

When invoked, and when <value> is any of 1, true, or TRUE, this operator causes Traffic Server to abort furtherrequest remapping. Any other value and the operator will effectively be a no-op.

Operator Flags

Operator flags are optional, are separated by commas when using more than one for a single operator, and must notcontain whitespace inside the brackets. For example, an operator with the L flag would be written in this manner:

set-destination HOST foo.bar.com [L]

The flags currently supported are:

Flag DescriptionL Last rule, do not continue.QSA Append the results of the rule to the query string.

Variable Expansion

Only limited variable expansion is supported in add-header. Supported substitutions are currently:

Variable Description%<proto> Protocol%<port> Port%<chi> Client IP%<cqhl> Client request length%<cqhm> Client HTTP method%<cquup> Client unmapped URI

Header Values

Setting a header with a value can take the following formats:

3.7. Plugins 79


• Any condition which extracts a value from the request.

• $N, where 0 <= N <= 9, from matching groups in a regular expression.

• A string (which can contain the numbered matches from a regular expression as described above).

• Null.

Supplying no value for a header for certain operators can lead to an effective no-op. In particular, add-header andset-header will simply short-circuit if no value has been supplied for the named header.

URL Parts

Some of the conditions and operators which use a request or response URL as their target allow for matching againstspecific components of the URL. For example, the CLIENT-URL condition can be used to test just against the queryparameters by writing it as:

cond %{CLIENT-URL:QUERY} <operand>

The URL part names which may be used for these conditions and actions are:

Part DescriptionHOST Full hostname.PATH URL substring beginning with the first / after the hostname up to, but not including, the query string.PORT Port number.QUERY URL substring from the ?, signifying the beginning of the query parameters, until the end of the URL.

Empty string if there were no quuery parameters.SCHEMEURL scheme in use (e.g. http and https).URL The complete URL.

As another example, a remap rule might use the set-destination operator to change just the hostname via:

cond %{HEADER:X-Mobile} = "foo"set-destination HOST foo.mobile.bar.com [L]

Requests vs. Responses

Both HTTP requests and responses have headers, a good number of them with the same names. When writing a ruleagainst, for example, the Connection or Via headers which are both valid in a request and a response, how canyou tell which is which, and how do you indicate to Traffic Server that the condition is specifically and exclusively fora request header or just for a response header? And how do you ensure that a header rewrite occurs against a requestbefore it is proxied?

Hook Conditions

In addition to the conditions already described above, there are a set of special hook conditions. Only one of theseconditions may be specified per ruleset, and they must be the first condition listed. Which hook condition is useddetermines the context in which the ruleset is evaluated, and whether the other conditions will default to operating onthe client request headers or the origin response (or cache response) headers.

Hook conditions are written just like the other conditions, except that none of them take any operands:

cond %{<name>}

Because hook conditions must be the first condition in a ruleset, the use of one forces the beginning of a new ruleset.



READ_RESPONSE_HDR_HOOK

Rulesets evaluated within this context will process only once the origin server response (or cached response) has beenread, but prior to Traffic Server sending that response to the client.

This is the default hook condition for all globally-configured rulesets.

READ_REQUEST_HDR_HOOK

Evaluates as soon as the client request has been read, but prior to any further processing (including contacting originservers or fetching objects from cache). Conditions and operators which adapt to matching or manipulating requestor response entities (e.g. headers) depending on their context will all operate on the request variants when using thishook, as there is no response data yet.

READ_REQUEST_PRE_REMAP_HOOK

For ruleset configurations provided via remap.config, this forces their evaluation as soon as the request has beenread, but prior to the remapping. For all context-adapting conditions and operators, matching will occur against therequest, as there is no response data available yet.

REMAP_PSEUDO_HOOK

This is the default hook condition for all rulesets configured via remapping rules in remap.config. Functionallyequivalent to READ_RESPONSE_HDR_HOOK in that rulesets will evaluate after responses from origin servers havebeen received (or the object has been retrieved from cache), but prior to sending the client response.

What sets this hook context apart is that in configuration files shared by both the global plugin.config andindividual remapping entries in remap.config, this hook condition will force the subsequent ruleset(s) to be validonly for remapped transactions.

SEND_REQUEST_HDR_HOOK

Forces evaluation of the ruleset just prior to contacting origin servers (or fetching the object from cache), but after anyremapping may have occurred.

SEND_RESPONSE_HDR_HOOK

Evaluates rulesets just prior to sending the client response, but after any cache updates may have been performed.This hook context provides a means to modify aspects of the response sent to a client, while still caching the originalversions of those attributes delivered by the origin server.

Affected Conditions

The following conditions are affected by the hook context in which they are evaluated and will adjust using request orresponse entities automatically:

• HEADER

• METHOD

3.7. Plugins 81


• PATH

• QUERY

• URL

Affected Operators

The following operators are affected by the hook context in which they are evaluated and will adjust modifying requestor response entities automatically:

• add-header

• rm-header

• set-header

Caveats

Repeated Headers

Some headers may appear more than once in a request or a response. When this occurs, all values will be collapsedinto a single comma-delimited string before the conditions see them. This avoids the problem of determining whichheader instance out of several a condition’s rule will be applied to, but it may introduce unexpected behavior in youroperands.

For example, let us assume an origin response includes a header named X-Foo which specifies a keyword of somesort. This header may appear zero or more times and we wish to construct a HEADER condition that can handle this.

Condition A

This condition will match using a direct equality operand:

cond %{HEADER:X-Foo} =bar

Condition B

This condition will match using an unanchored regular expression:

cond %{HEADER:X-Foo} /bar/

Sample Headers

Both conditions A and B will match this response:

HTTP/1.1 200 OKDate: Mon, 08 Feb 2016 18:11:30 GMTContent-Length: 1234Content-Type: text/htmlX-Foo: bar



Only condition B will match this response:

HTTP/1.1 200 OKDate: Mon, 08 Feb 2016 18:11:30 GMTContent-Length: 1234Content-Type: text/htmlX-Foo: barX-Foo: baz

That is because the HEADER condition will see the value of X-Foo as bar,baz. Condition B will still match thisbecause the regular expression, being unanchored, finds the substring bar. But condition A fails, as it is expecting thevalue to be the exact string bar, nothing more and nothing less.

Examples

Remove Origin Authentication Headers

The following ruleset removes any authentication headers from the origin response before caching it or returning it tothe client. This is accomplished by setting the hook context and then removing the cookie and basic authenticationheaders.:

cond %{READ_RESPONSE_HDR_HOOK}rm-header Set-Cookierm-header WWW-Authenticate

Count Teapots

Maintains a counter statistic, which is incremented every time an origin server has decided to be funny by returningHTTP 418:

cond %{STATUS} =418counter plugin.header_rewrite.teapots

Normalize Statuses

For client-facing purposes only (because we set the hook context to just prior to delivering the response back to theclient, but after all processing and possible cache updates have occurred), replaces all 4xx HTTP status codes from theorigin server with 404:

cond %{SEND_RESPONSE_HDR_HOOK}cond %{STATUS} >399cond %{STATUS} <500set-status 404

Remove Cache Control to Origins

Removes cache control related headers from requests before sending them to an origin server:

cond %{SEND_REQUEST_HDR_HOOK}rm-header Cache-Controlrm-header Pragma

3.7. Plugins 83


Enable Debugging Per-Request

Turns on Traffic Server debugging statements for a transaction, but only when a special header is present in the clientrequest:

cond %{READ_REQUEST_HDR_HOOK}cond %{CLIENT-HEADER:X-Debug} =supersekretset-debug

Remove Internal Headers

Removes special internal/development headers from origin responses, unless the client request included a specialdebug header:

cond %{CLIENT-HEADER:X-Debug} =keep [NOT]rm-header X-Debug-Foorm-header X-Debug-Bar

Return Original Method in Response Header

This rule copies the original HTTP method that was used by the client into a custom response header:

cond %{SEND_RESPONSE_HDR_HOOK}set-header X-Original-Method %{METHOD}

Useless Example From Purpose

Even that useless example from Purpose in the beginning of this document is possible to accomplish:

cond %{INCOMING-PORT} =8090cond %{METHOD} =HEADcond %{CLIENT-HEADER:Accept-Language} /es-py/ [NOT]cond %{STATUS} =304 [OR]cond %{RANDOM:500} >290set-status 403

Add Cache Control Headers Based on Origin Path

This rule adds cache control headers to CDN responses based matching the origin path. One provides a max age andthe other provides a “no-cache” statement to two different file paths.:

cond %{SEND_RESPONSE_HDR_HOOK}cond %{PATH} /examplepath1/add-header Cache-Control "max-age=3600" [L]cond %{SEND_RESPONSE_HDR_HOOK}cond %{PATH} /examplepath2/examplepath3/.*/add-header Cache-Control "no-cache" [L]



Health Checks Plugin

This is a simple plugin, to provide basic (but configurable) health checks. This is a server intercept plugin, and it takesone single configuration option in plugin.config, the configuration file name.

Configuration

To enable the healthchecks plugin, insert the following line in plugin.config:

healthchecks.so <healthcheck-configuration-file>

The required <healthcheck-configuration-file> may reference either an absolute or relative path to thefile containing the healthcheck configuration.

This configuration may contain one or more lines of the format:

<URI-path> <file-path> <mime> <file-exists-code> <file-missing-code>

: The URI-path can not be “/” only.

: This configuration is not reloadable.

The content of the file specified in the file-path, if any, is sent as the body of the response. The existence of thefile is sufficient to get an “OK” status. Performance wise, everything is served out of memory, and it only stats / opensfiles as necessary. However, the content of the status file is limited to 16KB, so this is not a generic static file servingplugin.

Example

This line would define a health check link available at http://www.example.com/__hc that would check if the file /var/run/ts-alive existed on the server. If the file exists, a response is built with the contents of the ts-alivefile, a mime type of text/plain and a status code of 200. If the file does not exist, a 403 response is sent:

/__hc /var/run/ts-alive text/plain 200 403

Regex Remap Plugin

This allows you to configure mapping rules based on regular expressions. This is similar to what you can accomplishusing mod_rewrite in Apache httpd, but obviously not as flexible or sophisticated (yet).

To use this plugin, configure a remap.config rule like

map http://a.com http://b.com @plugin=regex_remap.so @pparam=maps.reg

The file name parameter is always required. Unless an absolute path is specified, the file name is assumed to be a pathrelative to the Traffic Server configuration directory.

The regular expressions listed in the configuration file are evaluated sequentially. When a regular expression is pos-itively matched against a request URL, evaluation is stopped and the rewrite rule is applied. If none of the regularexpressions are a match, the default destination URL is applied (http://b.com in the example above).

3.7. Plugins 85

http://www.example.com/__hc


An optional argument (@pparam) with the string “profile” will enable profiling of this regex remap rule, e.g.

... @pparam=maps.reg @pparam=profile

Profiling is very low overhead, and the information is dumped to traffic.out, located in the log directory. Thisinformation is useful to optimize the order of your regular expression, such that the most common matches appearsearly in the file. In order to force a profile dump, you can do

$ sudo touch remap.config$ sudo traffic_ctl config reload

By default, only the path and query string of the URL are provided for the regular expressions to match. The followingoptional parameters can be used to modify the plugin instance behavior

@pparam=[no-]method [default: off]@pparam=[no-]query-string [default: on]@pparam=[no-]matrix-parameters [default: off]

If you wish to match on the HTTP method used (e.g. “GET”), you must use the option @pparam=method. e.g.

... @pparam=maps.reg @pparam=method

With this enabled, the string that you will need to match will look like

GET/path?query=bar

The methods are always all upper-case, and always followed by one single space. There is no space between themethod and the rest of the URL (or URI path).

By default, the query string is part of the string that is matched again, to turn this off use the option ‘no-query-string’,e.g.

... @pparam=maps.reg @pparam=no-query-string

Finally, you can also include the matrix parameters in the string, using the option ‘matrix-parameters’, e.g.

... @pparam=maps.reg @pparam=matrix-parameters

A typical regex would look like

^/(ogre.*)/more http://www.ogre.com/$h/$0/$1

The regular expression must not contain any white spaces!

When the regular expression is matched, only the URL path + query string is matched (without any of the optionalconfiguration options). The path will always start with a “/”. Various substitution strings are allowed on the right handside during evaluation

$0 - The entire matched string$1-9 - Regular expression groups ($1 first group etc.)$h - The original host header from the request$f - The host as used in the "from" portion of the remap rule$t - The host as used in the "to" portion of the remap rule$p - The original port number$s - The scheme (e.g. http) of the request$P - The entire path of the request$q - The query part of the request$r - The path parameters of the request (not implemented yet)



$c - The cookie string from the request$i - The client IP for this request

: The $0 substitution expands to the characters that were matched by the regular expression, not to the entire stringthat the regular expression was matched against.

You can also provide options, similar to how you configure your remap.config. The following options are available

@status=<nnn> - Force the response code to <nnn>@active_timeout=<nnn> - Active timeout (in ms)@no_activity_timeout=<nnn> - No activity timeout (in ms)@connect_timeout=<nnn> - Connect timeouts (in ms)@dns_timeout=<nnn> - Connect timeouts (in ms)

@overridable-config=<value> - see :ref:`Overridable Configurations <ts-overridable-→˓config>`

@caseless - Make regular expressions case insensitive@lowercase_substitutions - Turn on (enable) lower case substitutions

This can be useful to force a particular response for some URLs, e.g.

^/(ogre.*)/bad http://www.examle.com/ @status=404

Or, to force a 302 redirect

^/oldurl/(.*)$ http://news.example.com/new/$1 @status=302

Setting the status to 301 or 302 will force the new URL to be used as a redirect (Location:).

Stats Over HTTP Plugin

This plugin implements an HTTP interface to all Traffic Server statistics. The metrics returned are in a JSON format,for easy processing. This plugin is now part of the standard ATS build process, and should be available after install.

Enabling Stats Over HTTP

To enable this plugin, add to the plugin.config file:

stats_over_http.so

After starting Traffic Server, the JSON metrics are now available on the default URL:

http://host:port/_stats

where host and port is the hostname/IP and port number of the server.

Plugin Options

--integer-counters

3.7. Plugins 87


This option causes the plugin to emit floating point and integral metric values as JSON numbers, rather then JSONstrings. This can cause interoperability problems since integer metrics have a 64-bit unsigned range.

--wrap-counters

This option wraps 64-bit unsigned values to the 64-bit signed range. This aids interoperability with Java, since priorto the Java SE 8 release, Java did not have a 64-bit unsigned type.

You can optionally modify the path to use, and this is highly recommended in a public facing server. For example:

stats_over_http.so 81c075bc0cca1435ea899ba4ad72766b

and the URL would then be e.g.:

https://host:port/81c075bc0cca1435ea899ba4ad72766b

This is weak security at best, since the secret could possibly leak if you are careless and send it over clear text.

TCPInfo Plugin

This global plugin logs TCP metrics at various points in the HTTP processing pipeline. The TCP information isretrieved by the getsockopt(2) function using the TCP_INFO option. This is only supported on systems thatsupport the TCP_INFO option, currently Linux and BSD.

Plugin Options

The following options may be specified in plugin.config:

--hooks=NAMELIST This option specifies when TCP information should be logged. The argumentis a comma-separated list of the event names listed below. TCP information willbe sampled and logged each time the specified set of events occurs.

Event Name Triggered whensend_resp_hdr The server begins sending an HTTP response.ssn_close The TCP connection closes.ssn_start A new TCP connection is accepted.txn_close A HTTP transaction is completed.txn_start A HTTP transaction is initiated.

--log-file=NAME This specifies the base name of the file where TCP information should be logged.If this option is not specified, the name tcpinfo is used. Traffic Server willautomatically append the .log suffix.

--log-level=LEVEL The log level can be either 1 to log only the round trip time estimate, or 2 to logthe complete set of TCP information.

The following fields are logged when the log level is 1:

Field Name Descriptiontimestamp Event timestampevent Event name (one of the names listed above)client Client IP addressserver Server IP addressrtt Estimated round trip time

The following fields are logged when the log level is 2:



Field Name Descriptiontimestamp Event timestampevent Event name (one of the names listed above)client Client IP addressserver Server IP addressrtt Estimated round trip timerttvarlast_sentlast_recvsnd_cwndsnd_ssthreshrcv_ssthreshunackedsackedlostretransfackets

--sample-rate=COUNT This is the number of times per 1000 requests that the data will be logged. Apseudo-random number generator is used to determine if a request will be logged.The default value is 1000 and this option is not required to be in the configurationfile. To achieve a log rate of 1% you would set this value to 10.

Examples:

This example logs the simple TCP information to tcp-metrics.log at the start of a TCP connection and once foreach HTTP transaction thereafter:

tcpinfo.so --log-file=tcp-metrics --log-level=1 --hooks=ssn_start,txn_start

The file tcp-metrics.log will contain the following log format:

timestamp event client server rtt20140414.17h40m14s ssn_start 127.0.0.1 127.0.0.1 400020140414.17h40m14s txn_start 127.0.0.1 127.0.0.1 400020140414.17h40m16s ssn_start 127.0.0.1 127.0.0.1 400020140414.17h40m16s txn_start 127.0.0.1 127.0.0.1 400020140414.17h40m16s ssn_start 127.0.0.1 127.0.0.1 400020140414.17h40m16s txn_start 127.0.0.1 127.0.0.1 4000

Cache URL Modify the cache key used for requests by applying a regular expression to the URL.

Configuration Remap Override configuration directives on a per-rule basis.

GZip Compress or deflate cache responses.

Header Rewrite Modify requests and responses based on incoming and outgoing headers and other transaction at-tributes.

Health Checks Define service health check links.

Regex Remap Configure remapping rules using regular expressions.

Stats over HTTP Provide an HTTP interface to all Traffic Server statistics.

TCPInfo Log TCP metrics at various points of the HTTP processing pipeline.

3.7. Plugins 89


Experimental plugins

Plugins that are considered experimental are located in the plugins/experimental directory of the Traffic Server sourcetree. Experimental plugins can be compiled by passing the –enable-experimental-plugins option to configure:

$ autoconf -i$ ./configure --enable-experimental-plugins$ make

AuthProxy Plugin

There are many ways of authorizing an HTTP request. Often, this requires making IPC calls to some internal infras-tructure. AuthProxy is a plugin that takes care of the Traffic Server end of authorizing a request and delegates theauthorization decision to an external HTTP service.

This plugin can be used as either a global plugin or a remap plugin.

Note that Traffic Server optimizes latency by skipping the DNS lookup state if a document is found in the cache. Thiswill have the effect of serving the document without consulting the AuthProxy plugin. you can disable this behaviorby setting :ts:cv:‘proxy.config.http.doc_in_cache_skip_dns‘ to 0 in records.config.

Note that the authorization request will need to match a remap rule (which, as a standalone remap rule, does not needto call the AuthProxy plugin). If a second remap rule is required, by default, the authorization request will not have thesame Host header as the request from the client. It could be added using the header_rewrite plugin (set-headerHost “pristine_host.example.com”).

Plugin Options

--auth-transform=TYPE This option specifies how to route the incoming request to the authorizationservice. The transform type may be head or redirect.

If the transform type is head, then the incoming request is transformed to aHEAD request and is sent to the same destination. If the response is 200 OK,the incoming request is allowed to proceed.

If the transform type is range, then the incoming request is transformed to aRange request asking for 0 bytes. Other than that, the behavior is identical tothe head option above. This type of Range request is useful when the upstreamdestination is a cache, and it’s not able to cache HEAD requests.

If the transform type is redirect then the incoming request is sent to the au-thorization service designated by the –auth-host and –auth-port parameters. Ifthe response is 200 OK, the incoming request is allowed to proceed.

When the authorization service responds with a status other than 200 OK, thatresponse is returned to the client as the response to the incoming request. Thisallows mechanisms such as HTTP basic authentication to work correctly. Notethat the body of the authorization service response is not returned to the client.

--auth-host=HOST The name or address of the authorization host. This is only used by theredirect transform.

--auth-port=PORT The TCP port of the authorization host. This is only used by the redirecttransform.


https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;a=tree;f=plugins/experimental;hb=HEAD


--force-cacheability If this options is set, the plugin will allow Traffic Server to cache the re-sult of authorized requests. In the normal case, requests with authoriza-tion headers are nor cacheable, but this flag allows that by setting the:ts:cv:‘proxy.config.http.cache.ignore_authentication‘ option on the request.

Examples

In this example, the authentication is performed by converting the incoming HTTP request to a HEAD request andsending that to the origin server origin.internal.com:

map http://cache.example.com http://origin.internal.com/ \@plugin=authproxy.so @pparam=--auth-transform=head

map http://origin.internal.com http://origin.internal.com/

In this example, the request is directed to a local authentication server that authorizes the request based on internalpolicy rules:

map http://cache.example.com http://origin.internal.com/ \@plugin=authproxy.so @pparam=--auth-transform=redirect @pparam=--auth-host=127.0.0.

→˓1 @pparam=--auth-port=9000

map http://origin.internal.com/ http://origin.internal.com/ \@plugin=authproxy.so @pparam=--auth-transform=redirect @pparam=--auth-host=127.0.0.

→˓1 @pparam=--auth-port=9000

AWS S3 Authentication plugin

This is a plugin for Apache Traffic Server that provides support for the Amazon S3 authentication features. This isuseful if you for example want to use S3 as your origin server, yet want to avoid direct user access to the content.

Using the plugin

There are three configuration options for this plugin:

--access_key <key>--secret_key <key>--virtual_host--config <config file>

Using the first two in a remap rule would be e.g.:

... @plugin=s3_auth @pparam=--access_key @pparam=my-key \@pparam=--secret_key @pparam=my-secret \@pparam=--virtual_host

Alternatively, you can store the access key and secret in an external configuration file, and point the remap rule(s) toit:

... @plugin=s3_auth @pparam=–config @pparam=s3.config

Where s3.config would look like:

3.7. Plugins 91


# AWS S3 authenticationaccess_key=my-keysecret_key=my-secretvirtual_host=yes

For more details on the S3 auth, see:

http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html

ToDo

This is a pretty barebone start for the S3 services, it is missing a number of features:

• It does not do UTF8 encoding (as required)

• It only implements the v2 authentication mechanism. For details on v4, see

http://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html

• It does not deal with canonicalization of AMZ headers.

• It does not handle POST requests (but do we need to ?)

• It does not incorporate query parameters.

Contributions to any of these would be appreciated.

Background Fetch Plugin

This is a plugin for Apache Traffic Server that allows you to proactively fetch content from Origin in a way that it willfill the object into cache. This is particularly useful when all (or most) of your client requests are of the byte-Rangetype. The underlying problem being that Traffic Server is not able to cache request / responses with byte ranges.

Using the plugin

This plugin functions as either a global or per remap plugin, and it takes an optional argument for specifying a configfile with inclusion or exclusion criteria. The config file can be specified both via an absolute path or via a relative pathto the install dir

To activate the plugin in global mode, in plugin.config, simply add:

background_fetch.so --config <config-file>

To activate the plugin in per remap mode, in remap.config, simply append the below to the specific remap line:

@plugin=background_fetch.so @pparam=<config-file>

Functionality

Examining the responses from origin, we decide to trigger a background fetch of the original (Client) request underthese conditions:

• The request is a GET request (we only support these right now)

• The response is a 206 response


http://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html


• The original client request, and the Origin server response, is clearly indicating that the response is cacheable.This uses the new API c:func:TSHttpTxnIsCacheable(), which also implies honoring current Traffic Serverconfigurations.

Once deemed a good candidate to performance a background fetch, we’ll replay the original client request through theTraffic Server proxy again, except this time eliminating the Range header. This is transparent to the original clientrequest, which continues as normal.

Only one background fetch per URL is ever performed, making sure we do not accidentally put pressure on the originservers.

The plugin now supports a config file that can specify exclusion or inclusion of background fetch based on any arbitraryheader or client-ip:

background_fetch.so --config <config-file>

The contents of the config-file could be as below:

include User-Agent ABCDEFexclude User-Agent *exclude Content-Type textexclude X-Foo-Bar textexclude Content-Length <1000

The plugin also now supports per remap activation. To activate the plugin for a given remap, add the below on theremap line:

@plugin=background_fetch.so @pparam=<config-file>

Future additions

• Limiting the background fetches to content of certain sizes

Balancer Plugin

The balancer balances requests across multiple origin servers. To use this plugin, configure it in a remap.configrule, specifying a balancing policy and a set of origin servers. For example:

map http://foo.com http://foo.com \@plugin=balancer.so @pparam=--policy=hash,url @pparam=one.bar.com @pparam=two.bar.

→˓com

The replacement URL in the mapping rule is not used. The argument to the --policy option is a comma-separated list of keywords. The first keyword is the name of a balancing policy. The subsequent keywords are used torefine the requested policy.

The remaining plugin arguments are balancer targets. Typically, these will be the host names of origin servers thatrequests should be balanced across. The target name may contain a colon-separated port number.

Hash Balancing Policy

The hash balancing policy performs a consistent hash across the set of origins. This minimizes the number of hashentries that must be moved when the set of origin servers changes. An optional list of hash fields follows the hashkeyword. Each specified hash field is hashed to select an outbound origin server.

3.7. Plugins 93


The following fields can be supplied to the hash:

key The request cache key. Note that the cache key will only be set if you have already chained a plugin that sets acustom cache key.

url The request URL. This is the default hash field that is used if no other fields are specified.

srcaddr The source IP address of the request.

dstaddr The destination IP address of the request.

Round Robin Balancing Policy

The roundrobin balancing policy simply allocates requests to origin servers in order. Over time, the number ofrequests received by each origin should be approximately the same.

Health Checking

The balancer plugin does not check the health of the origin servers, however the plugin is fully reloadable so healthchecking is usualy simple to implement. Most production environments already have mechanisms to check servicehealth. It is recommended that you write a simple script to monitor this information and rewrite remap.configwhen appropriate. Running traffic_ctl config reload will reload the balancer plugin with the new setof origin servers.

Buffer Upload Plugin

The Buffer Upload plugin offers the following features

Installation

Configuration can be explicitly specified as a parameter in plugin.config

buffer_upload.so /FOOBAR/upload.conf

Memory buffering (buffer the entire POST data in IOBuffer before connecting to OS)

Memory buffer size is configured with “mem_buffer_size” in config file. Default and minimum value is 32K. You canincrease it in the config file. If the size of a request is larger than the “mem_buffer_size” value specifiied in the configfile, then the upload proxy feature will be disabled for this particular request

Disk buffering (buffer the entire POST data on disk before connecting to OS)

1. Disk async IO is used. AIO api call only involves certain amount of threads. The number of threads is configurablein plugin’s config file (default is 4)

2. Directories and files are generated on disk . Base directory is /FOOBAR/var/buffer_upload_tmp/ (configurable inconfig file). Number of subdirectories is 64 (configurable in config file). Filename are randomly generated. Files willbe removed when the entire data have been sent out to OS . At startup time, dangling files are removed (left on diskdue to transaction interruption or traffic server crash)

3. Default chunk size when reading from disk is 16K, configurable in config file



Trigger POST buffering on certain URLs

1. Certain URLs will be provided in a plain text file (one URL each line)

2. Specify filename in config file by “url_list_file”

3. max length of each URL is 4096 (configurable in config file)

4. use exact match, don’t support regex for now

Other Features

1. Default buffering mode is disk aio buffering mode. To turn off disk buffering, add a “use_disk_buffer 0” line inconfig file

2. All request headers inlcuding cookies plus the entire POST data will be buffered (either in memory or on disk)

Configuration File

sample config file

use_disk_buffer 1convert_url 1chunk_size 1024url_list_file /tmp/url_list.confmax_url_length 10000base_dir /tmp/test1subdir_num 100thread_num 10mem_buffer_size 40000

Cache Key Manipulation Plugin

Description

This plugin allows some common cache key manipulations based on various HTTP request components. It can

• sort query parameters to prevent query parameters reordereding from being a cache miss

• ignore specific query parameters from the cache key by name or regular expression

• ignore all query parameters from the cache key

• only use specific query parameters in the cache key by name or regular expression

• include headers or cookies by name

• capture values from the User-Agent header.

• classify request using User-Agent and a list of regular expressions

• capture and replace strings from the URI and include them in the cache key

• do more - please find more examples below.

3.7. Plugins 95


Cache key structure and related plugin parameters

hierarchical part query------------------------------------------------------------------------------------------------------------| Prefix | User-Agent | Headers | Cookies | Path | Query→˓ || section | section | section | section | section | section→˓ || (default) | (optional) | (optional) | (optional) | (default) | (default)→˓ |------------------------------------------------------

• The cache key set by the cachekey plugin can be considered as devided into several sections.

• Every section is manupulated separately by the related plugin parameters (more info in each section descriptionbelow).

• “User-Agent”, “Headers” and “Cookies” sections are optional and will be missing from the cache key if norelated plugin parameters are used.

• “Prefix”, “Path” and “Query” sections always have default values even if no related plugin parameters are used.

• All cachekey plugin parameters are optional and if missing some of the cache key sections will be missing (theoptional sections) or their values will be left to their defaults.

“Prefix” section

Optional components | --------------------------------------(included in this order) | | --static-prefix | --capture-prefix | --capture-prefix-→˓uri |

| --------------------------------------Default values if no | | /host/port→˓ |optional components | ---------------------------------------configured |

• --static-prefix=<value> (default: empty string) - if specified and not an empty string the <value>will be added to the cache key.

• --capture-prefix=<capture_definition> (default: empty string) - if specified and not empty thenstrings are captured from host:port based on the <capture_definition> and are added to the cachekey.

• --capture-prefix-uri=<capture_definition> (default: empty string) - if specified and notempty then strings are captured from the entire URI based on the <capture_definition> and are addedto the cache key.

• If any of the “Prefix” related plugin parameters are used together in the plugin configuration they are added tothe cache key in the order shown in the diagram.

“User-Agent” section

Optional components | -----------------(included in this order) | | --ua-class | --ua-capture |

| -----------------



Default values if no | | (empty) |optional components | ------------------configured |

• User-Agent classification

– --ua-whitelist=<classname>:<filename> (default: empty string) - loads a regex pat-terns list from a file <filename>, the patterns are matched against the User-Agent header and ifmatched <classname> is added it to the key.

– --ua-blacklist=<classname>:<filename> (default: empty string) - loads a regex pat-terns list from a file <filename>, the patterns are matched against the User-Agent header and ifnot matched <classname> is added it to the key.

– Multiple --ua-whitelist and --ua-blacklist can be used and the result will be defined bytheir order in the plugin configuration.

• User-Agent regex capturing and replacement

– --ua-capture=<capture_definition> (default: empty string) - if specified and not emptythen strings are captured from the User-Agent header based on <capture_definition> (seebelow) and are added to the cache key.

• If any User-Agent classigication and regex capturing and replacement plugin parameters are used togetherthey are added to the cache key in the order shown in the diagram.

“Headers” section

Optional components | -------------| | --include-headers || -------------

Default values if no | | (empty) |optional components | -------------configured |

• --include-headers (default: empty list) - comma separated list of headers to be added to the cache key.The list of headers defined by --include-headers are always sorted before adding them to the cache key.

“Cookies” section

Optional components | -------------| | --include-cookies || -------------

Default values if no | | (empty) |optional components | -------------configured |

• --include-cookies (default: empty list) - comma separated list of cookies to be added to the cache key.The list of cookies defined by --include-cookies are always sorted before adding them to the cache key.

“Path” section

3.7. Plugins 97


Optional components | ------------------------(included in this order) | | --path-capture-uri | --path-capture |

| ------------------------Default values if no | | URI path |optional components | -------------------------configured |

• if no path related plugin parameters are used, the URI path string is included in the cache key.

• --capture-path=<capture_definition> (default: empty string) - if specified and not empty thenstrings are captured from URI path based on the <capture_definition> and are added to the cache key.

• --capture-path-uri=<capture_definition> (default: empty string) - if specified and not emptythen strings are captured from the entire URI based on the <capture_definition> and are added to thecache key.

“Query” section

• If no query related plugin parameters are used, the query string is included in the cache key.

• --exclude-params (default: empty list) - comma-separated list of query params to be black-listed in thecache key. If the list is empty then no black-list is applied (no query parameters will be excluded from the cachekey). The exclude list overrides the include list.

• --include-params (default: empty list) - comma-separated list of query params to be white-listed in thecache key. If the list is empty then no white-list is applied (all query parameters will be included in the cachekey).

• --include-match-params (default: empty list) - regular expression matching query parameter nameswhich will be white-listed in the cache key.

• --exclude-match-params (default: empty list) - regular expression matching query parameter nameswhich will be black-listed in the cache key.

• --remove-all-params (boolean:true|false, 0|1, yes|no, default: false) - if equals true thenall query parameters are removed (the whole query string) and all other URI query parameter related settings (ifused) will have no effect.

• --sort-params (boolean:true|false, 0|1, yes|no, default: false) - if equals true then all queryparameters are sorted in an increasing case-sensitive order

All parameters are optional, and if not used, their default values are as mentioned below. Boolean values default tofalse and the rest default to an empty list. Examples of each parameter’s usage can be found below.

<capture_definition>

• <capture_definition> can be in the following formats

– <regex> - <regex> defines regex capturing groups, up to 10 captured strings based on <regex>will be added to the cache key.

– /<regex>/<replacement>/ - <regex> defines regex capturing groups, <replacement>defines a pattern where the captured strings referenced with $0 ... $9 will be substituted and theresult will be added to the cache key.



Detailed examples and troubleshooting

| hierarchical part→˓ queryHTTP request | ------------------------------------------------------components | URI host and port HTTP headers and cookies URI→˓path URI query

| --------------------------------------------------Sample 1 | /www.example.com/80/popular/Mozilla/5.0/H1:v1/H2:v2/C1=v1;C2=v2/path/→˓to/data?a=1&b=2&c=3Sample 2 | /nice_custom_prefix/popular/Mozilla/5.0/H1:v1/H2:v2/C1=v1;C2=v2/path/→˓to/data?a=1&b=2&c=3

| -------------------------------------------Cache Key | host:port or UA-class UA-captures headers cookies→˓path querycomponents | custom prefix replacement

The following is an example of how the above sample keys were generated (Sample 1 and Sample 2).

Traffic server configuration

$ cat etc/trafficserver/remap.configmap http://www.example.com http://www.origin.com \

@plugin=cachekey.so \@pparam=--ua-whitelist=popular:popular_agents.config \@pparam=--ua-capture=(Mozilla\/[^\s]*).* \@pparam=--include-headers=H1,H2 \@pparam=--include-cookies=C1,C2 \@pparam=--include-params=a,b,c \@pparam=--sort-params=true

$ cat etc/trafficserver/popular_agents.config^Mozilla.*^Twitter.*^Facebo.*

$ cat etc/trafficserver/plugin.configxdebug.so

HTTP request

$ curl 'http://www.example.com/path/to/data?c=3&a=1&b=2&x=1&y=2&z=3' \-v -x 127.0.0.1:8080 -o /dev/null -s \-H "H1: v1" \-H "H2: v2" \-H "Cookie: C1=v1; C2=v2" \-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.75.

→˓14 (KHTML, like Gecko) Version/7.0.3 Safari/7046A194A' \-H 'X-Debug: X-Cache-Key'

* About to connect() to proxy 127.0.0.1 port 8080 (#0)

* Trying 127.0.0.1... connected

* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)> GET http://www.example.com/path/to/data?c=3&a=1&b=2&x=1&y=2&z=3 HTTP/1.1> Host: www.example.com> Accept: */*> Proxy-Connection: Keep-Alive> H1: v1> H2: v2

3.7. Plugins 99


> Cookie: C1=v1; C2=v2> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.75.14→˓(KHTML, like Gecko) Version/7.0.3 Safari/7046A194A> X-Debug: X-Cache-Key>< HTTP/1.1 200 OK< Server: ATS/6.1.0< Date: Thu, 19 Nov 2015 23:17:58 GMT< Content-type: application/json< Age: 0< Transfer-Encoding: chunked< Proxy-Connection: keep-alive< X-Cache-Key: /www.example.com/80/popular/Mozilla/5.0/H1:v1/H2:v2/C1=v1;C2=v2/path/→˓to/data?a=1&b=2&c=3<{ [data not shown]

* Connection #0 to host 127.0.0.1 left intact

* Closing connection #0

The response header X-Cache-Key header contains the cache key:

/www.example.com/80/popular/Mozilla/5.0/H1:v1/H2:v2/C1=v1;C2=v2/path/to/data?a=1&b=2&→˓c=3

The xdebug.so plugin and X-Debug request header are used just to demonstrate basic cache key troubleshooting.

If we add --static-prefix=nice_custom_prefix to the remap rule then the cache key would look like thefollowing:

/nice_custom_prefix/popular/Mozilla/5.0/H1:v1/H2:v2/C1=v1;C2=v2/path/to/data?a=1&b=2&→˓c=3

Usage examples

URI query parameters

Ignore the query string (all query parameters)

The following added to the remap rule will ignore the query, removing it from the cache key.

@plugin=cachekey.so @pparam=--remove-all-params=true

Cache key normalization by sorting the query parameters

The following will normalize the cache key by sorting the query parameters.

@plugin=cachekey.so @pparam=--sort-params=true

If the URI has the following query string c=1&a=1&b=2&x=1&k=1&u=1&y=1 the cache key will usea=1&b=2&c=1&k=1&u=1&x=1&y=1



Ignore (exclude) certain query parameters

The following will make sure query parameters a and b will not be used when constructing the cache key.

@plugin=cachekey.so @pparam=--exclude-params=a,b

If the URI has the following query string c=1&a=1&b=2&x=1&k=1&u=1&y=1 the cache key will usec=1&x=1&k=1&u=1&y=1

Ignore (exclude) certain query parameters from the cache key by using regular expression (PCRE)

The following will make sure query parameters a and b will not be used when constructing the cache key.

@plugin=cachekey.so @pparam=--exclude-match-params=(a|b)

If the URI has the following query string c=1&a=1&b=2&x=1&k=1&u=1&y=1 the cache key will usec=1&x=1&k=1&u=1&y=1

Include only certain query parameters

The following will make sure only query parameters a and c will be used when constructing the cache key and the restwill be ignored.

@plugin=cachekey.so @pparam=--include-params=a,c

If the URI has the following query string c=1&a=1&b=2&x=1&k=1&u=1&y=1 the cache key will use c=1&a=1

Include only certain query parameters by using regular expression (PCRE)

The following will make sure only query parameters a and c will be used when constructing the cache key and therest will be ignored.

@plugin=cachekey.so @pparam=--include-match-params=(a|c)

If the URI has the following query string c=1&a=1&b=2&x=1&k=1&u=1&y=1 the cache key will use c=1&a=1

White-list + black-list certain parameters using multiple parameters in the same remap rule.

If the plugin is used with the following plugin parameters in the remap rule:

@plugin=cachekey.so \@pparam=--exclude-params=x \@pparam=--exclude-params=y \@pparam=--exclude-params=z \@pparam=--include-params=y,c \@pparam=--include-params=x,b

and if the URI has the following query string c=1&a=1&b=2&x=1&k=1&u=1&y=1 the cache key will usec=1&b=1

3.7. Plugins 101


White-list + black-list certain parameters using multiple parameters in the same remap rule andregular expressions (PCRE).


@plugin=cachekey.so \@pparam=--exclude-match-params=x \@pparam=--exclude-match-params=y \@pparam=--exclude-match-params=z \@pparam=--include-match-params=(y|c) \@pparam=--include-match-params=(x|b)


Mixing –include-params, –exclude-params, –include-match-param and –exclude-match-param


@plugin=cachekey.so \@pparam=--exclude-params=x \@pparam=--exclude-match-params=y \@pparam=--exclude-match-params=z \@pparam=--include-params=y,c \@pparam=--include-match-params=(x|b)


HTTP Headers

Include certain headers in the cache key

The following headers HeaderA and HeaderB will be used when constructing the cache key and the rest will beignored.

@plugin=cachekey.so @pparam=--include-headers=HeaderA,HeaderB

HTTP Cookies

Include certain cookies in the cache key

The following headers CookieA and CookieB will be used when constructing the cache key and the rest will beignored.

@plugin=cachekey.so @pparam=--include-headers=CookieA,CookieB

Prefix (host, port, capture and replace from URI)



Replacing host:port with a static cache key prefix

If the plugin is used with the following plugin parameter in the remap rule.

@plugin=cachekey.so @pparam=--static-prefix=static_prefix

the cache key will be prefixed with /static_prefix instead of host:port when --static-prefix is notused.

Capturing from the host:port and adding it to the prefix section


@plugin=cachekey.so \@pparam=--capture-prefix=(test_prefix).*:([^\s\/$]*)

the cache key will be prefixed with /test_prefix/80 instead of test_prefix_371.example.com:80when --capture-prefix is not used.

Capturing from the entire URI and adding it to the prefix section


@plugin=cachekey.so \@pparam=--capture-prefix-uri=/(test_prefix).*:.*(object).*$/$1_$2/

and if the request URI is the following

http://test_prefix_123.example.com/path/to/object?a=1&b=2&c=3

the the cache key will be prefixed with /test_prefix_object instead of test_prefix_123.example.com:80 when --capture-prefix-uri is not used.

Combining prefix plugin parameters, i.e. –static-prefix and –capture-prefix

If the plugin is used with the following plugin parameters in the remap rule.

@plugin=cachekey.so \@pparam=--capture-prefix=(test_prefix).*:([^\s\/$]*) \@pparam=--static-prefix=static_prefix

the cache key will be prefixed with /static_prefix/test_prefix/80 instead of test_prefix_371.example.com:80 when neither --capture-prefix nor --static-prefix are used.

Path, capture and replace from the path or entire URI

Capture and replace groups from path for the “Path” section


3.7. Plugins 103


@plugin=cachekey.so \@pparam=--capture-path=/.*(object).*/const_path_$1/

and the request URI is the following

http://test_path_123.example.com/path/to/object?a=1&b=2&c=3

then the cache key will have /const_path_object in the path section of the cache key instead of /path/to/object when neither --capture-path nor --capture-path-uri are used.

Capture and replace groups from whole URI for the “Path” section


@plugin=cachekey.so \@pparam=--capture-path-uri=/(test_path).*(object).*/$1_$2/



the the cache key will have /test_path_object in the path section of the cache key instead of /path/to/object when neither --capture-path nor --capture-path-uri are used.

Combining path plugin parameters –capture-path and –capture-path-uri

If the plugin is used with the following plugin parameters in the remap rule.

@plugin=cachekey.so \@pparam=--capture-path=/.*(object).*/const_path_$1/ \@pparam=--capture-path-uri=/(test_path).*(object).*/$1_$2/



the the cache key will have /test_path_object/const_path_object in the path section of the cache keyinstead of /path/to/object when neither --capture-path nor --capture-path-uri are used.

User-Agent capturing, replacement and classification

Let us say we have a request with User-Agent header:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3)AppleWebKit/537.75.14 (KHTML, like Gecko)Version/7.0.3 Safari/7046A194A

Capture PCRE groups from User-Agent header

If the plugin is used with the following plugin parameter:



@plugin=cachekey.so \@pparam=--ua-capture=(Mozilla\/[^\s]*).*(AppleWebKit\/[^\s]*)

then Mozilla/5.0 and AppleWebKit/537.75.14 will be used when constructing the key.

Capture and replace groups from User-Agent header


@plugin=cachekey.so \@pparam=--ua-capture=/(Mozilla\/[^\s]*).*(AppleWebKit\/[^\s]*)/$1_$2/

then Mozilla/5.0_AppleWebKit/537.75.14 will be used when constructing the key.

User-Agent white-list classifier


@plugin=cachekey.so \@pparam=--ua-whitelist=browser:browser_agents.config

and if browser_agents.config contains:

^Mozilla.*^Twitter.*^Facebo.*

then browser will be used when constructing the key.

User-Agent black-list classifier


@plugin=cachekey.so \@pparam=--ua-blacklist=browser:tool_agents.config

and if tool_agents.config contains:

^PHP.*^Python.*^curl.*

then browser will be used when constructing the key.

Cache Promote Plugin

The cache_promote plugin provides a means to control when an object should be allowed to enter the cache. This isorthogonal from normal Cache-Control directives, providing a different set of policies to apply. The typical use casefor this plugin is when you have a very large data set, where you want to avoid churning the ATS cache for the longtail content.

All configuration is done via remap.config, and the following options are available:

3.7. Plugins 105


--policy The promotion policy. lru and chance are supported

--sample The sampling rate for the request to be considered

For the lru plugin, the following options are also available:

--hits The minimum number of hits before promotion

--buckets The size (entries) of the LRU

These two options combined with your usage patterns will control how likely a URL is to become promoted to enterthe cache.

Examples

map http://cdn.example.com/ http://some-server.example.com @plugin=cache_promote.so@pparam=–policy=chance –sample=10%

map http://cdn.example.com/ http://some-server.example.com @plugin=cache_promote.so@pparam=–policy=lru @pparam=–hits=10 @pparam=–buckets=10000

Note that the –sample option is available for all policies, and when used to reduce pressure on the under heavy load.

Combo Handler Plugin

This plugin provides an intelligent way to combine multiple URLs into a single URL, and have Apache Traffic Servercombine the components into one response. This is useful for example to create URLs that combine multiple CSS orJavascript files into one.

Installation

This plugin is only built if the configure option

--enable-experimental-plugins

is given at build time. Note that this plugin is built and installed in combination with the ESI module, since they sharecommon code.

Configuration

The arguments in the plugin.config line in order represent

1. The path that should triggers combo handler (defaults to “admin/v1/combo”)

2. The name of the key used for signature verification (disabled by default)

A “-” can be supplied as a value for any of these arguments to request default value be applied.

Also, just like the original combohandler, this plugin generates URLs of the form http://localhost/<dir>/<file-path>. <dir> here defaults to l unless specified by the file path in the query parameter using a colon. Forexample:

http://combo.com/admin/v1/combo?filepath1&dir1:filepath2&filepath3

Will result in these three pages being fetched:


http://cdn.example.com/

http://some-server.example.com

http://cdn.example.com/

http://some-server.example.com


http://localhost/l/filepath1http://localhost/dir1/filepath2http://localhost/l/filepath3

Remap rules have to be specified to map the above URLs to desired content servers.

The plugin also supports a prefix parameter. Common parts of successive file paths can be extracted and specifiedseparately using a ‘p’ query parameter. Successive file path parameters are appended to this prefix to create completefile paths. The prefix will remain active until changed or cleared (set to an empty string). For example, the query

"/file1&p=/path1/&file2&file3&p=&/file4&p=/dir:path2/&file5&file6"

results in these file paths being “reconstructed”:

/file1/path1/file2/path1/file3/file4/dir:path2/file5/dir:path2/file6

Epic Plugin

The Epic plugin emits Traffic Server metrics in a format that is consumed by the Epic Network Monitoring System. Itis a global plugin that is installed by adding it to the plugin.config file.

Plugin Options

--directory=DIR Specify the directory the plugin will write sample files to. The default is /usr/local/epic/cache/eapi.

--period=SECS Specify the sample period in seconds. The default is to write samples every 30seconds.

Caveats

The Traffic Server metrics system does not store the semantics of metrics, so it is not possible to programmaticallydetermine whether a metrics can be rate converted. The plugin contains a static list of metrics that should not be rateconverted (gauges in Epic terminilogy).

Escalate Plugin

The Escalate plugin allows Traffic Server to try an alternate origin when the origin server in the remap rule is eitherunavailable or returns specific HTTP error codes. Some services call this failover or fail-action.

Plugin Configuration

The escalate plugin is a remap plugin (not global) and takes a parameter with two delimitated fields:comma-separated-error-codes:secondary-origin-server. For instance,

@pparam=401,404,410,502:second-origin.example.com

3.7. Plugins 107

https://code.google.com/p/epicnms/

would have Traffic Server send a cache miss to second-origin.example.com when the origin server in theremap rule returns a 401, 404, 410, or 502 error code.

@pparam=–pristine This option sends the “pristine” Host: header (eg, the Host: header that the client sent) to theescalated request.

Installation



is given at build time.

Example

With this line in remap.config

map cdn.example.com origin.example.com@plugin=escalate.so @pparam=401,404,410,502:second-origin.example.com @pparam=--

→˓pristine

Traffic Server would accept a request for cdn.example.com and, on a cache miss, proxy the request to origin.example.com. If the response code from that server is a 401, 404, 410, or 502, then Traffic Server would proxy therequest to second-origin.example.com, using a Host: header of cdn.example.com.

ESI Plugin

This plugin implements the ESI specification.

Specification

Supportted ESI tags:

esi:includeesi:removeesi:commentesi:varsesi:chooseesi:whenesi:otherwiseesi:tryesi:attemptesi:except

extended ESI tags: esi:special-include

Supported variables:

$(HTTP_HOST)$(HTTP_REFERER)$(HTTP_ACCEPT_LANGUAGE{name})


$(HTTP_COOKIE{name}) or $(HTTP_COOKIE{name;subkey})$(QUERY_STRING{name})$(HTTP_HEADER{hdr_name})

Note: the name is the key name such as “username”, “id” etc. For cookie support sub-name or sub-key, the for-mat is: name;subkey, such as “l;u”, “l;t” etc. e.g. such cookie string: l=u=test&t=1350952328, the value of$(HTTP_COOKIE{“l;u”}) is test and the value of $(HTTP_COOKIE{“l;t”}) is 1350952328

Compile and Installation



is given at build time. Note that this plugin is built and installed in combination with the combo handler module, sincethey share common code.

Enabling ESI

1. First we need to set up /usr/local/etc/trafficserver/plugin.config and make sure the following line is present.

esi.so

2. There are four options you can add to the above.

• “–private-response” will add private cache control and expires header to the processed ESI document.

• “–packed-node-support” will enable the support for using packed node, which will improve the performance ofparsing cached ESI document.

• “–disable-gzip-output” will disable gzipped output, which will NOT gzip the output anyway.

• “–first-byte-flush” will enable the first byte flush feature, which will flush content to users as soon as the entireESI document is received and parsed without all ESI includes fetched (the flushing will stop at the ESI includemarkup till that include is fetched).

3. We need a mapping for origin server response that contains the ESI markup. Assume that the ATS server isabc.com. And your origin server is xyz.com and the response containing ESI markup is http://xyz.com/esi.php.We will need the following line in /usr/local/etc/trafficserver/remap.config

map http://abc.com/esi.php http://xyz.com/esi.php

4. Your response should contain ESI markup and a response header of ‘X-Esi: 1’. e.g. using PHP,

<?php header('X-Esi: 1'); ?><html><body>Hello, <esi:include src="http://abc.com/date.php"/></body></html>

5. You will need a mapping for the src of the ESI include in remap.config if it is not already present.

map http://abc.com/date.php http://xyz.com/date.php

3.7. Plugins 109

http://xyz.com/esi.php


Or if both your ESI response and the ESI include comes from the same origin server, you can have the following linein remap.config instead to replace separate map rules for date.php and esi.php

map http://abc.com/ http://xyz.com/

6. Here is a sample PHP for date.php

<?phpheader ("Cache-control: no-cache");echo date('l jS \of F Y h:i:s A');?>

Useful Note

1. You can provide proper cache control header and the ESI response and ESI include response can be cachedseparately. It is extremely useful for rendering page with multiple modules. The page layout can be a ESIresponse with multiple ESI include include, each for different module. The page layour ESI response can becached and each individual ESI include can also be cached with different duration.

2. You might want to compile the code without using ESI_PACKED_NODE_SUPPORT because it may not workin some corner cases

Differences from Spec - http://www.w3.org/TR/esi-lang

1. <esi:include> does not support “alt” and “onerror” attributes

2. <esi:inline> is not supported

3. You cannot have <esi:try> inside another <esi:try>

4. HTTP_USER_AGENT variable is not supported

5. HTTP_COOKIE supports fetching for sub-key

6. HTTP_HEADER supports accessing request headers as variables

Generator Plugin

The Generator allows testing of synthetic workloads by generating HTTP responses of various sizes. The size andcacheability of the response is specified by the first two coomponents of the requested URL path. This plugin onlysupports the GET and HEAD HTTP methods.

Path component Description12

cache or nocache. If cache is specifed, the Gener-ator plugin will respond with Cache-Control head-ers marking the response as cacheable for 24 hours. In-tegral number of bytes to return in the response.

Path components after the first 2 are ignored. This means that the trailing path components can be manipulated tocreate unique URLs following any covenient convention.

The Generator plugin publishes the following metrics:

generator.response_bytes: The total number of bytes emitted

generator.response_count: The number of HTTP responses generated by the plugin



Examples:

The most common way to use the Generator plugin is to configure it as a remap plugin in remap.config:

map http://workload.example.com/ http://127.0.0.1/ \@plugin=generator.so

Notice that although the remap target is never contacted because the Generator plugin intercepts the request and actsas the origin server, it must be syntactically valid and resolvable in DNS.

A 10 byte, cacheable object can then be generated:

$ curl -o /dev/null -x 127.0.0.1:8080 http://workload.example.com/cache/10/→˓caf1fc92332b3a3c8cb8b3826b6a1658

The Generator plugin can return responses as large as you like:

$ curl -o /dev/null -x 127.0.0.1:8080 http://workload.example.com/cache/$((10 * 1024→˓* 1024))/$RANDOM

GeoIP ACLs Plugin

This is a simple ATS plugin for denying (or allowing) requests based on the source IP geo-location. Currently onlythe Maxmind APIs are supported, but we’d be happy to other other (open) APIs if you let us know. This plugin comeswith the standard distribution of Apache Traffic Server, and should be installed as part of the normal build process.

Configuration

Once installed, there are three primary use cases, which we will discuss in details. Note that in all configurations, thefirst plugin parameter must specify what the matches should be applied to. Currently, only one rule set is supported,for Country ISO codes. This is specified with a parameter of

@pparam=country

Future additions to this plugin could include other regions, such as city, state, continent etc.

The three typical use cases are as follows:

1. Per remap configurations, applicable to the entire remap rule. This is useful when you can partition your contentso that entire prefix paths should be filtered. For example, lets assume that http://example.com/music is restrictedto US customers only, and everything else is world wide accessible. In remap.config, you would have somethinglike

map http://example.com/music http://music.example.com \@plugin=geoip_acl.so @pparam=country @pparam=allow @pparam=US

map http://example.com http://other.example.com

2. If you can not partition the data with a path prefix, you can specify a separate regex mapping filter. Theremap.config file might then look like

map http://example.com http://music.example.com \@plugin=geoip_acl.so @pparam=country \@pparam=regex::/etc/music.regex

where music.regex is a format with PCRE (perl compatible) regular expressions, and unique rules for match. E.g.:

3.7. Plugins 111

http://example.com/music


.*\.mp3 allow US

.*\.ogg deny US

Note that the default in the case of no matches on the regular expressions is to “allow” the request. This can beoverriden, see next use case.

3. You can also combine 1) and 2), and provide defaults in the remap.config configuration, which then applies forthe cases where no regular expressions matches at all. This would be useful to override the default which is toallow all requests that don’t match. For example

map http://example.com http://music.example.com \@plugin=geoip_acl.so @pparam=country @pparam=allow @pparam= US@pparam=regex::/etc/music.regex

This tells the plugin that in the situation where there is no matching regular expression, only allow requests originatingfrom the US.

Finally, there’s one additional parameter option that can be used

@pparam=html::/some/path.html

This will override the default reponse body for the denied responses with a custom piece of HTML. This can be usefulto explain to your users why they are getting denied access to a particular piece of content. This configuration can beused with any of the use cases described above.

HIPES Plugin

This is a remap plugin used in the HIPES system.

Configuration

urlp:<name> Default: url Name of the query parameter for the service URL

path:<path> Default: / Path to use for the service URL

ssl Default: no Use SSL when connecting to the service

service Service server, host[:port]

server Default: hipes.yimg.com Name of HIPES server, host[:port]

active_timeout The active connection timeout in ms

no_activity_timeout The no activity timeout in ms

connect_timeout The connect timeout in ms

dns_timeout The DNS timeout

The timeout options override the server defaults (from `records.config), and only apply to the connection to thespecific “service” host.

Notes on HIPES

HTTP Pipes (aka HIPES and pronounced “Hippies”) allows data services to be pipelined together, as illustrated by theexample below.



Example

1. ATS is run on port 80 and apache HTTP web server is run on port 8080 on localhost (127.0.0.1)

2. The HIPES plugin is used in remap.config

map http://127.0.0.1/svc_case http://nosuchhost @plugin=hipes.so→˓@pparam=service:127.0.0.1:8080 @pparam=path:svc_case.php @pparam=server:127.0.0.→˓1map http://127.0.0.1/svc_reverse http://nosuchhost @plugin=hipes.so→˓@pparam=service:127.0.0.1:8080 @pparam=path:svc_reverse.php @pparam=server:127.→˓0.0.1map http://127.0.0.1/test.txt http://127.0.0.1:8080/test.txt

3. The plugin remaps the incoming URL such as

http://127.0.0.1/svc_reverse/svc_case;case=u/test.txt

to the following

http://127.0.0.1:8080/svc_reverse?url=http%3A%2F%2F127.0.0.1%2Fsvc_case%3Bcase%3Du→˓%2Ftest.txt

4. The service svc_reverse.php fetches the url from the ATS again and the plugin remaps the URL

http://127.0.0.1/svc_case;case=u/test.txt

to this URL

http://127.0.0.1:8080/svc_case.php?case=u&url=http%3A%2F%2F127.0.0.1%2Ftest.txt

5. In this example, the service svc_case.php fetches and transforms the response of http://127.0.0.1/test.txt (which ATS proxies the request to a local file) to upper case. And subsequently the servicesvc_reverse.php receives the response and reverse the order before the response is sent back to the client byATS.

Notes on reducing traffic

There can be significant overhead using HIPES because the data can traverse through ATS many times. Caching canbe important to reduce traffic to services/through ATS and can be achieved via a proper Cache-Control headerreturned by the services. Another way to reduce traffic through ATS is to have ATS to return 302 redirects to url forthe requests made by service, instead of proxying the requests to that url. However, the service must then be able tofollow the redirect. The down side is that we cannot use ATS to cache intermediate results. Below is an example ofusing redirect.

Modification to above example to reduce traffic using redirect

1. The service svc_reverse.php is modified to add a header of X-HIPES-Redirect: 2 to the request madeagainst url.

2. HIPES plugin will instruct ATS to return a redirect response to this url

http://127.0.0.1:8080/svc_case.php?case=u&url=http%3A%2F%2F127.0.0.1%2Ftest.txt

for the following request

3.7. Plugins 113


http://127.0.0.1/svc_case;case=u/test.txt

3. The service svc_reverse.php is also modified to follow the redirect. Thus the response of the service ofsvc_case.php will not pass through ATS and will pass to svc_reverse.php service instead.

Memcache Plugin

This plugin implements the memcache protocol for cache contents.

Installation

Add the following line to plugin.config:

tsmemcache.so 11211

In this case, the plugin will use the default behaviour:

• Listen for memcache traffic on the default memcache port 11211

• Write memcache data to the cache in the background (without waiting)

Configuration

Alternatively, a configuration can also be specified:

tsmemcache.so <path-to-plugin>/sample.tsmemcache.config

After modifying plugin.config, restart traffic server (sudo traffic_ctl server restart) the configuration is also re-readwhen a management update is given (sudo traffic_ctl config reload)

Options

Flags and options are:

port: A single port to listen on for memcache protocol commands.

Options in the code: TSMEMCACHE_WRITE_SYNC whether or not to wait for the write to complete.

Metalink Plugin

The Metalink plugin implements the Metalink download description format in order to try not to download the samefile twice. This improves cache efficiency and speeds up users’ downloads.

It takes standard headers and knowledge about objects in the cache and potentially rewrites those headers so that aclient will use a URL that’s already cached instead of one that isn’t. The headers are specified in RFC 6249 (Met-alink/HTTP: Mirrors and Hashes) and RFC 3230 (Instance Digests in HTTP) and are sent by various download redi-rectors or content distribution networks.

A lot of download sites distribute the same files from many different mirrors and users don’t know which mirrors arealready cached. These sites often present users with a simple download button, but the button doesn’t predictablyaccess the same mirror, or a mirror that’s already cached. To users it seems like the download works sometimes (takesseconds) and not others (takes hours), which is frustrating.


http://en.wikipedia.org/wiki/Metalink


An extreme example of this happens when users share a limited, possibly unreliable internet connection, as is commonin parts of Africa for example.

How it Works

When the plugin sees a response with a Location: ... header and a Digest: SHA-256=... header, itchecks if the URL in the Location header is already cached. If it isn’t, then it tries to find a URL that is cached touse instead. It looks in the cache for some object that matches the digest in the Digest header and if it succeeds,then it rewites the Location header with that object’s URL.

This way a client should get sent to a URL that’s already cached and won’t download the file again.

Installation

The Metalink plugin is a global plugin. Enable it by adding metalink.so to your plugin.config file. Thereare no options.

Implementation Status

The plugin implements the TS_HTTP_SEND_RESPONSE_HDR_HOOK hook to check and potentially rewrite theLocation and Digest headers after responses are cached. It doesn’t do it before they’re cached because thecontents of the cache can change after responses are cached. It uses TSCacheRead() to check if the URL in theLocation header is already cached. In future, the plugin should also check if the URL is fresh or not.

The plugin implements the TS_HTTP_READ_RESPONSE_HDR_HOOK hook and a null transformation to com-pute the SHA-256 digest for content as it’s added to the cache. It uses SHA256_Init(), SHA256_Update(), andSHA256_Final() from OpenSSL to compute the digest, then it uses TSCacheWrite() to associate the digest withthe request URL. This adds a new cache object where the key is the digest and the object is the request URL.

To check if the cache already contains content that matches a digest, the plugin must call TSCacheRead() with thedigest as the key, read the URL stored in the resultant object, and then call TSCacheRead() again with this URL asthe key. This is probably inefficient and should be improved.

An early version of the plugin scanned Link: <...>; rel=duplicate headers. If the URL in theLocation: ... header wasn’t already cached, it scanned Link: <...>; rel=duplicate headers fora URL that was. The Digest: SHA-256=... header is superior because it will find content that already existsin the cache in every case that a Link: <...>; rel=duplicate header would, plus in cases where the URLis not listed among the Link: <...>; rel=duplicate headers, maybe because the content was downloadedfrom a URL not participating in the content distribution network, or maybe because there are too many mirrors to listin Link: <...>; rel=duplicate headers.

The Digest: SHA-256=... header is also more efficient than Link: <...>; rel=duplicate headersbecause it involves a constant number of cache lookups. RFC 6249 requires a Digest: SHA-256=... header orLink: <...>; rel=duplicate headers MUST be ignored:

If Instance Digests are not provided by the Metalink servers, the Link header fields pertaining to thisspecification MUST be ignored.

Metalinks contain whole file hashes as described in Section 6, and MUST include SHA-256, as specifiedin [FIPS-180-3].

3.7. Plugins 115


MP4 Plugin

This module provides streaming media server support for MP4 files. User can send a HTTP request to the server withstart argument which is measured in seconds, and the server will respond with the stream such that its start positioncorresponds to the requested time, for example:

http://v.foo.com/dota2.mp4?start=290.12

This allows performing a random seeking at any time. We can use flash player, vlc, mplayer, firefox or chrome to playthe streaming media.

This plugin can be used as a remap plugin. We can write this in remap.config:

map http://v.foo.com/ http://v.internal.com/ @plugin=mp4.so

Note

This plugin requires that the moov box in the mp4 file should be ahead of mdat box. It is not a good idea to cache alarge mp4 file, many video sites will cut a large video file into many small mp4 files, and each small mp4 file will beless than 80M(bytes), it will be a reasonable choice.

MySQL Remap Plugin

This is a basic plugin for doing dynamic “remaps” from a database. It essentially rewrites the incoming request’s Hostheader / origin server connection to one retrieved from a database.

The generic proxying setup is the following:

UA ----> Traffic Server ----> Origin Server

Without the plugin a request like:

GET /path/to/something HTTP/1.1Host: original.host.com

Ends up requesting http://original.host.com/path/to/something

With this plugin enabled, you can easily change that to anywhere you desire. Imagine the many possibilities....

We have benchmarked the plugin with ab at about 9200 requests/sec (1.7k object) on a commodity hardware with alocal setup of both, MySQL and Traffic Server local. Real performance is likely to be substantially higher, up to theMySQL’s max queries / second.

Installation






Configuration

Import the default schema to a database you create:

mysql -u root -p -e "CREATE DATABASE mysql_remap;" # create a new databasemysql -u root -p mysql_remap < schema/import.sql # import the provided schema

insert some interesting values in mysql_remap.hostname & mysql_remap.map

Traffic Server plugin configuration is done inside a global configuration file: /etc/trafficserver/plugin.config:

mysql_remap.so /etc/trafficserver/mysql_remap.ini

The INI file should contain the following values:

[mysql_remap]mysql_host = localhost #defaultmysql_port = 3306 #defaultmysql_username = remap_usermysql_password =mysql_database = mysql_remap #default

To debug errors, start trafficserver manually using:

traffic_server -T "mysql_remap"

And resolve any errors or warnings displayed.

Signed URL Plugin

This plugin checks a signature query string on a URL and rejects (HTTP 403) or redirects (HTTP 302) when thecheck fails. The signature is based on a secret key that both a signing portal and the Traffic Server cache share. Thealgorithm for the signature may be either MD5 or SHA1. When the signature check passes, the query string of therequest is stripped and continues to process as if there were no query string at all.

Installation

To make this plugin available, you must either enable experimental plugins when building Traffic Server:

./configure --enable-experimental-plugins

Or use tsxs to compile the plugin against your current Traffic Server build. To do this, you must ensure that:

1. Development packages for Traffic Server are installed.

2. The tsxs binary is in your path.

3. The version of this plugin you are building, and the version of Traffic Server against which you are building itare compatible.

Once those conditions are satisfied, enter the source directory for the plugin and perform the following:

make -f Makefile.tsxsmake -f Makefile.tsxs install

3.7. Plugins 117


Configuration

Configuring URL signature verification within Traffic Server using this plugin is a two step process. First, you mustgenerate a configuration file containing the list of valid signing keys. Secondly, you must indicate to Traffic Serverwhich URLs require valid signatures.

Generating Keys

This plugin comes with two Perl scripts which assist in generating signatures. For Traffic Server to verify URLsignatures, it must have the relevant keys. Using the provided genkeys script, you can generate a suitable configurationfile:

./genkeys.pl > url_sig.config

The resulting file will look something like the following, with the actual keys differing (as they are generated randomlyeach time the script is run):

key0 = YwG7iAxDo6Gaa38KJOceV4nsxiAJZ3DSkey1 = nLE3SZKRgaNM9hLz_HnIvrCw_GtTUJT1key2 = YicZbmr6KlxfxPTJ3p9vYhARdPQ9WJYZkey3 = DTV4Tcn046eM9BzJMeYrYpm3kbqOtBs7key4 = C1r6R6MINoQd5YSH25fU66tuRhhz3fs_key5 = l4dxe6YEpYbJtyiOmX5mafhwKImC5kejkey6 = ekKNHXu9_oOC5eqIGJVxV0vI9FYvKVyakey7 = BrjibTmpTTuhMHqphkQAuCWA0Zg97WQBkey8 = rEtWLb1jcYoq9VG8Z8TKgX4GxBuro20Jkey9 = mrP_6ibDBG4iYpfDB6W8yn3ZyGmdwc6Mkey10 = tbzoTTGZXPLcvpswCQCYz1DAIZcAOGyXkey11 = lWsn6gUeSEW79Fk2kwKVfzhVG87EXLnakey12 = Riox0SmGtBWsrieLUHVWtpj18STM4MP1key13 = kBsn332B7yG3HdcV7Tw51pkvHod7_84lkey14 = hYI4GUoIlZRf0AyuXkT3QLvBMEoFxkmakey15 = EIgJKwIR0LU9CUeTUdVtjMgGmxeCIbdgerror_url = 403

This file should be placed in your Traffic Server etc directory, with permissions and ownership set such that only theTraffic Server processes may read it.

: The configuration file contains the full set of secret keys which Traffic Server will be using to verify incomingrequests, and as such should be treated with as much care as any other file in your infrastructure containing keys, passphrases, and other sensitive data. Unauthorized access to the contents of this file will allow others to spoof requestsfrom your signing portal, thus defeating the entire purpose of using a signing portal in the first place.

Requiring Signatures on URLs

To require a valid signature, verified by a key from the list you generated earlier, modify your remap.configconfiguration to include this plugin for any rules you wish it to affect.

Two parameters for each remap rule are required:

@plugin=url_sig.so @pparam=<config file>



The first simply enables this plugin for the rule. The second specifies the location of the configuration file containingyour signing keys.

For example, if we wanted to restrict all paths under a /download directory on our website foo.com we mighthave a remap line like this:

map http://foo.com/download/ http://origin.server.tld/download/ \@plugin=url_sig.so @pparam=url_sig.config

Signing a URL

Signing a URL is solely the responsibility of your signing portal service. This requires that whatever application runsthat service must also have a list of your signing keys (generated earlier in Configuration and stored in the url_sig.config file in your Traffic Server configuration directory). How your signing portal application is informed about,or stores, these keys is up to you, but it is critical that the keyN index numbers are matched to the same keys.

Signing is performed by adding several query parameters to a URL, before redirecting the client. The parameters’values are all generated by your signing portal application, and then a hash is calculated by your portal, using theentire URL just constructed, and attached as the final query parameter.

: Ordering is important when adding the query parameters and generating the signature. The signature itself is a hash,using your chosen algorithm, of the entire URL to which you are about to redirect the client, up to and including theS= substring indicating the signature parameter.

The following list details all the query parameters you must add to the URL you will hand back to the client forredirection.

Client IP The IP address of the client being redirected. This must be their IP as it will appear to your Traffic Servercache:

C=<ip address>

Expiration The time at which this signature will no longer be valid, expressed in seconds since epoch (and thus inUTC):

E=<seconds since epoch expiration>

Algorithm The hash algorithm which your signing portal application has elected to use for this signature:

A=<algorithm number>

The only supported values at this time are:

Value Algorithm1 HMAC_SHA12 HMAC_MD5

Key Index The key number, from your plugin configuration, which was used for this signature. See Configurationfor generating these keys and determining the index number of each:

K=<key number>

Parts Configures which components of the URL to use for signature verification. The value of this paramerts is astring of ones and zeroes, each enabling or disabling the use of a URL part for signatures. The URL scheme(e.g. http://) is never part of the signature. The first number of this parameter’s value indicates whether to

3.7. Plugins 119


include the FQDN, and all remaining numbers determine whether to use the directory parts of the URL. If thereare more directories in the URL than there are numbers in this parameter, the last number is repeated as manytimes as necessary:

P=<parts specification>

Examples include:

Value Effect1 Use the FQDN and all directory parts for signature verification.01 Ignore the FQDN, but verify using all directory parts.0110 Ignore the FQDN, and use only the first two directory parts, skipping the remainder, for signatures.110 Use the FQDN and first directory for signature verification, but ignore the remainder of the path.

Signature The actual signature hash:

S=<signature hash>

The hash should be calculated in accordance with the parts specification you have declared in the P= queryparameter, which if you have chosen any value other than 1 may require additional URL parsing be performedby your signing portal.

Additionally, all query parameters up to and including the S= substring for this parameter must be included, andmust be in the same order as they are returned to the client for redirection. For obvious reasons, the value of thisparameter is not included in the source string being hashed.

As a simple example, if we are about to redirect a client to the URL https://foo.com/downloads/expensive-app.exe with signature verification enabled, then we will compute a signature on the followingstring:

foo.com/downloads/expensive-app.exe?C=1.2.3.4&E=1453846938&A=1&K=2&P=1&S=

And, assuming that key2 from our secrets file matches our example in Configuration, then our signature be-comes:

8c5cfa440458233452ee9b5b570063a0e71827f2

Which is then appended to the URL for redirection as the value of the S parameter.

For an example implementation of signing which may be adapted for your own portal, refer to the file sign.plincluded with the source code of this plugin.

Edge Cache Debugging

To include debugging output for this plugin in your Traffic Server logs, adjust the values for:ts:cv:‘proxy.config.diags.debug.enabled‘ and :ts:cv:‘proxy.config.diags.debug.tags‘ in your records.configas so:

CONFIG proxy.config.diags.debug.enabled INT 1CONFIG proxy.config.diags.debug.tags STRING url_sig

Once updated, issue a traffic_ctl config reload to make the settings active.

Example

1. Enable experimental plugins when building Traffic Server:




2. Generate a secrets configuration for Traffic Server (replacing the output location with something appropriate toyour Traffic Server installation):

genkeys.pl > /usr/local/trafficserver/etc/trafficserver/url_sig.config

3. Verify that your configuration looks like the following, with actual key values altered:

key0 = YwG7iAxDo6Gaa38KJOceV4nsxiAJZ3DSkey1 = nLE3SZKRgaNM9hLz_HnIvrCw_GtTUJT1key2 = YicZbmr6KlxfxPTJ3p9vYhARdPQ9WJYZkey3 = DTV4Tcn046eM9BzJMeYrYpm3kbqOtBs7key4 = C1r6R6MINoQd5YSH25fU66tuRhhz3fs_key5 = l4dxe6YEpYbJtyiOmX5mafhwKImC5kejkey6 = ekKNHXu9_oOC5eqIGJVxV0vI9FYvKVyakey7 = BrjibTmpTTuhMHqphkQAuCWA0Zg97WQBkey8 = rEtWLb1jcYoq9VG8Z8TKgX4GxBuro20Jkey9 = mrP_6ibDBG4iYpfDB6W8yn3ZyGmdwc6Mkey10 = tbzoTTGZXPLcvpswCQCYz1DAIZcAOGyXkey11 = lWsn6gUeSEW79Fk2kwKVfzhVG87EXLnakey12 = Riox0SmGtBWsrieLUHVWtpj18STM4MP1key13 = kBsn332B7yG3HdcV7Tw51pkvHod7_84lkey14 = hYI4GUoIlZRf0AyuXkT3QLvBMEoFxkmakey15 = EIgJKwIR0LU9CUeTUdVtjMgGmxeCIbdgerror_url = 403

4. Enable signature verification for a remap rule in Traffic Server by modifying remap.config (here we willjust remap to Google’s homepage for demonstrative purposes):

map http://test-remap.domain.com/download/foo http://google.com \@plugin=url_sig.so @pparam=url_sig.config

5. Reload your Traffic Server configuration to ensure the changes are active:


6. Attempt to access the now-protected URL without a valid signature. This will fail, and that is a good thing, asit demonstrates that Traffic Server now rejects any requests to paths matching that rule which do not include avalid signature.:

$ curl -vs -H'Host: test-remap.domain.com' http://localhost:8080/download/foo

* Adding handle: conn: 0x200f8a0

* Adding handle: send: 0

* Adding handle: recv: 0

* Curl_addHandleToPipeline: length: 1

* - Conn 0 (0x200f8a0) send_pipe: 1, recv_pipe: 0

* About to connect() to localhost port 8080 (#0)

* Trying 127.0.0.1...

* Connected to localhost (127.0.0.1) port 8080 (#0)> GET /download/foo HTTP/1.1> User-Agent: curl/7.32.0> Accept: */*> Host: test-remap.domain.com>< HTTP/1.1 403 Forbidden< Date: Tue, 15 Apr 2014 22:57:32 GMT

3.7. Plugins 121


< Connection: close

* Server ATS/5.0.0 is not blacklisted< Server: ATS/5.0.0< Cache-Control: no-store< Content-Type: text/plain< Content-Language: en< Content-Length: 21<

* Closing connection 0Authorization Denied$

7. Generate a signed URL using the included sign.pl script:

sign.pl --url http://test-remap.domain.com/download/foo \--useparts 1 --algorithm 1 --duration 60 --keyindex 3 \--key DTV4Tcn046eM9BzJMeYrYpm3kbqOtBs7

8. Now access the protected URL with a valid signature:

$ curl -s -o /dev/null -v --max-redirs 0 -H 'Host: test-remap.domain.com' \'http://test-remap.domain.com/download/foo?E=1453848506&A=1&K=3&P=1&

→˓S=7aea86592de3e9c1b05771b2538a30956c6f10a3'

* Adding handle: conn: 0xef0a90

* Adding handle: send: 0

* Adding handle: recv: 0

* Curl_addHandleToPipeline: length: 1

* - Conn 0 (0xef0a90) send_pipe: 1, recv_pipe: 0

* About to connect() to localhost port 8080 (#0)

* Trying 127.0.0.1...

* Connected to localhost (127.0.0.1) port 8080 (#0)> GET /download/foo?E=1397603088&A=1&K=3&P=1&→˓S=28d822f68ac7265db61a8441e0877a98fe1007cc HTTP/1.1> User-Agent: curl/7.32.0> Accept: */*> Host: test-remap.domain.com>< HTTP/1.1 200 OK< Location: http://www.google.com/< Content-Type: text/html; charset=UTF-8< Date: Tue, 15 Apr 2014 23:04:36 GMT< Expires: Thu, 15 May 2014 23:04:36 GMT< Cache-Control: public, max-age=2592000

* Server ATS/5.0.0 is not blacklisted< Server: ATS/5.0.0< Content-Length: 219< X-XSS-Protection: 1; mode=block< X-Frame-Options: SAMEORIGIN< Alternate-Protocol: 80:quic< Age: 0< Connection: keep-alive<{ [data not shown]

* Connection #0 to host localhost left intact



SSL Headers Plugin

The sslheaders plugins injects SSL session information into HTTP request headers. It can operate as a globalplugin or as a remap plugin.

Plugin Options

The following options may be specified when loading the plugin in plugin.config or remap.config:

--attach=WHICH This option specifies which HTTP request the SSL headers are attached to.

client causes the headers to be injected into the client request. This is primarilyuseful if another plugin should inspect then. server is the default and injectsthe headers into the origin server request. both injects the headers into both theclient request and the origin server request.

A list of KEY=VALUE pairs follows any options. The KEY names the HTTP header to inject, and VALUE names theSSL session field.

SSL session field Descriptionclient.certificate The client certificate in PEM formatclient.subject The client certificate subject DNclient.issuer The client certificate issuer DNclient.serial The client certificate serial number in hexadecimal formatclient.signature The client certificate signature in hexadecimal formatclient.notbefore The client certificate validity start timeclient.notafter The client certificate validity end timeserver.certificate The server certificate in PEM formatserver.subject The server certificate subject DNserver.issuer The server certificate issuer DNserver.serial The server certificate serial number in hexadecimal formatserver.signature The server certificate signature in hexadecimal formatserver.notbefore The server certificate validity start timeserver.notafter The server certificate validity end time

The client.certificate and server.certificate fields emit the corresponding certificate in PEM format, with newline char-acters replaced by spaces.

If the sslheaders plugin activtes on non-SSL connections, it will delete all the configured HTTP header names sothat malicious clients cannot inject misleading information. If any of the SSL fields expand to an empty string, thoseheaders are also deleted.

Examples:

In this example, the origin server is interested in the subject of the server certificate that was used to accept a clientconnetion. We can apply the sslheaders plugin to a generic remap rule to provide this information. The remap.config configuration would be:

regex_map https://*.example.com/ http://origin.example.com/ \@plugin=sslheaders.so @pparam=SSL-Server=server.subject

In this example, we have set :ts:cv:‘proxy.config.ssl.client.certification_level‘ to request SSL client certificates. Wecan then configure sslheaders to populate the client certificate subject globally by adding it to plugin.config:

3.7. Plugins 123


sslheaders.so SSL-Client-ID=client.subject SSL-Client-NotBefore=client.notbefore SSL-→˓Client-NotAfter-client.notafter

Stale While Revalidate Plugin

refresh content asynchronously while serving stale data

TS Lua Plugin

This module embeds Lua, via the standard Lua 5.1 interpreter, into Apache Traffic Server™. With this module, wecan implement ATS plugin by writing Lua script instead of C code. Lua code executed using this module can be 100%non-blocking because the powerful Lua coroutines have been integrated into the ATS event model.

Synopsis

test_hdr.lua

function send_response()ts.client_response.header['Rhost'] = ts.ctx['rhost']return 0

end

function do_remap()local req_host = ts.client_request.header.Hostts.ctx['rhost'] = string.reverse(req_host)ts.hook(TS_LUA_HOOK_SEND_RESPONSE_HDR, send_response)return 0

end

sethost.lua

local HOSTNAME = ''function __init__(argtb)

if (#argtb) < 1 thenprint(argtb[0], 'hostname parameter required!!')return -1

endHOSTNAME = argtb[1]

end

function do_remap()ts.client_request.header['Host'] = HOSTNAMEreturn 0

end

Installation






Configuration

This module acts as remap plugin of Traffic Server, so we should realize ‘do_remap’ function in each lua script. Wecan write this in remap.config:

map http://a.tbcdn.cn/ http://inner.tbcdn.cn/ @plugin=/XXX/tslua.so @pparam=/XXX/test_→˓hdr.lua

Sometimes we want to receive parameters and process them in the script, we should realize ‘__init__’ function in thelua script, and we can write this in remap.config:

map http://a.x.cn/ http://b.x.cn/ @plugin=/X/tslua.so @pparam=/X/sethost.lua→˓@pparam=a.st.cn

This module can also act as a global plugin of Traffic Server. In this case we should provide one of these functions ineach lua script:

• ‘do_global_txn_start’

• ‘do_global_txn_close’

• ‘do_global_os_dns’

• ‘do_global_pre_remap’

• ‘do_global_post_remap’

• ‘do_global_read_request’

• ‘do_global_send_request’

• ‘do_global_read_response’

• ‘do_global_send_response’

• ‘do_global_cache_lookup_complete’

• ‘do_global_read_cache’

• ‘do_global_select_alt’

We can write this in plugin.config:

tslua.so /etc/trafficserver/script/test_global_hdr.lua

TS API for Lua

Introduction

The API is exposed to Lua in the form of one standard packages ts. This package is in the default global scope andis always available within lua script. This package can be introduced into Lua like this:

ts.say('Hello World')ts.sleep(10)

TOP

3.7. Plugins 125


ts.now

syntax: val = ts.now()

context: global

description: This function returns the time since the Epoch (00:00:00 UTC, January 1, 1970), measured in seconds.It includes milliseconds as the decimal part.

Here is an example:

local nt = ts.now() -- 1395221053.123

TOP

ts.debug

syntax: ts.debug(MESSAGE)

context: global

description: Log the MESSAGE to traffic.out if debug is enabled.

Here is an example:

ts.debug('I am in do_remap now.')

The debug tag is ts_lua and we should write this in records.config:

CONFIG proxy.config.diags.debug.tags STRING ts_lua

TOP

Remap status constants

context: do_remap

TS_LUA_REMAP_NO_REMAP (0)TS_LUA_REMAP_DID_REMAP (1)TS_LUA_REMAP_NO_REMAP_STOP (2)TS_LUA_REMAP_DID_REMAP_STOP (3)TS_LUA_REMAP_ERROR (-1)

These constants are usually used as return value of do_remap function.

TOP

ts.remap.get_to_url_host

syntax: ts.remap.get_to_url_host()

context: do_remap

description: retrieve the “to” host of the remap rule

Here is an example:



function do_remap()local to_host = ts.remap.get_to_url_host()ts.debug(to_host)return 0

end

TOP

ts.remap.get_to_url_port

syntax: ts.remap.get_to_url_port()

context: do_remap

description: retrieve the “to” port of the remap rule

TOP

ts.remap.get_to_url_scheme

syntax: ts.remap.get_to_url_scheme()

context: do_remap

description: retrieve the “to” scheme of the remap rule

TOP

ts.remap.get_to_uri

syntax: ts.remap.get_to_uri()

context: do_remap

description: retrieve the “to” path of the remap rule

TOP

ts.remap.get_to_url

syntax: ts.remap.get_to_url()

context: do_remap

description: retrieve the “to” url of the remap rule

TOP

ts.remap.get_from_url_host

syntax: ts.remap.get_from_url_host()

context: do_remap

description: retrieve the “from” host of the remap rule

TOP

3.7. Plugins 127


ts.remap.get_from_url_port

syntax: ts.remap.get_from_url_port()

context: do_remap

description: retrieve the “from” port of the remap rule

TOP

ts.remap.get_from_url_scheme

syntax: ts.remap.get_from_url_scheme()

context: do_remap

description: retrieve the “from” scheme of the remap rule

TOP

ts.remap.get_from_uri

syntax: ts.remap.get_from_uri()

context: do_remap

description: retrieve the “from” path of the remap rule

TOP

ts.remap.get_from_url

syntax: ts.remap.get_from_url()

context: do_remap

description: retrieve the “from” url of the remap rule

TOP

ts.hook

syntax: ts.hook(HOOK_POINT, FUNCTION)

context: global or do_remap or do_global_* or later

description: Hooks are points in http transaction processing where we can step in and do some work. FUNCTIONwill be called when the http transaction steps in to HOOK_POINT.

Here is an example

function send_response()s.client_response.header['SHE'] = 'belief'

end

function do_remap()ts.hook(TS_LUA_HOOK_SEND_RESPONSE_HDR, send_response)

end



Then the client will get the response like this:

HTTP/1.1 200 OKContent-Type: text/htmlServer: ATS/3.2.0SHE: beliefConnection: Keep-Alive...

You can create global hook as well

function do_some_work()ts.debug('do_some_work')return 0

end

ts.hook(TS_LUA_HOOK_READ_REQUEST_HDR, do_some_work)

Or you can do it this way

ts.hook(TS_LUA_HOOK_READ_REQUEST_HDR,function()

ts.debug('do_some_work')return 0

end)

Also the return value of the function will control how the transaction will be re-enabled. Return value of 0 willcause the transaction to be re-enabled normally (TS_EVENT_HTTP_CONTINUE). Return value of 1 will be usingTS_EVENT_HTTP_ERROR instead.

TOP

Hook point constants

TS_LUA_HOOK_OS_DNSTS_LUA_HOOK_PRE_REMAPTS_LUA_HOOK_READ_CACHE_HDRTS_LUA_HOOK_SELECT_ALTTS_LUA_HOOK_TXN_CLOSETS_LUA_HOOK_POST_REMAPTS_LUA_HOOK_CACHE_LOOKUP_COMPLETETS_LUA_HOOK_READ_REQUEST_HDRTS_LUA_HOOK_SEND_REQUEST_HDRTS_LUA_HOOK_READ_RESPONSE_HDRTS_LUA_HOOK_SEND_RESPONSE_HDRTS_LUA_REQUEST_TRANSFORMTS_LUA_RESPONSE_TRANSFORM

These constants are usually used in ts.hook method call.

Additional Information:

3.7. Plugins 129


Hook Point Lua Hook Pointconstant

Hook function beregistered withindo_remap() via ts.hook()?

Hook function beregistered within globalcontext via ts.hook()?

TS_HTTP_TXN_START_HOOKTS_LUA_HOOK_TXN_STARTNO YESTS_HTTP_READ_REQUEST_HDR_HOOKTS_LUA_HOOK_READ_REQUEST_HDRNO YESTS_HTTP_PRE_REMAP_HOOKTS_LUA_HOOK_PRE_REMAPNO YESTS_HTTP_POST_REMAP_HOOKTS_LUA_HOOK_POST_REMAPYES YESTS_HTTP_SELECT_ALT_HOOKTS_LUA_HOOK_SELECT_ALTNO NOTS_HTTP_READ_CACHE_HDR_HOOKTS_LUA_HOOK_READ_CACHE_HDRYES YESTS_HTTP_OS_DNS_HOOKTS_LUA_HOOK_OS_DNSYES YESTS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK

TS_LUA_HOOK_CACHE_LOOKUP_COMPLETE

YES YES

TS_HTTP_SEND_REQUEST_HDR_HOOK

TS_LUA_HOOK_SEND_REQUEST_HDR

YES YES

TS_HTTP_READ_RESPONSE_HDR_HOOK

TS_LUA_HOOK_READ_RESPONSE_HDR

YES YES

TS_HTTP_SEND_RESPONSE_HDR_HOOK

TS_LUA_HOOK_SEND_RESPONSE_HDR

YES YES

TS_HTTP _RE-QUEST_TRANSFORM_HOOK

TS_LUA_REQUEST_TRANSFORMYES YES

TS_HTTP _RE-SPONSE_TRANSFORM_HOOK

TS_LUA_RESPONSE_TRANSFORMYES YES

TS_HTTP_TXN_CLOSE_HOOKTS_LUA_HOOK_TXN_CLOSEYES YES

TOP

ts.ctx

syntax: ts.ctx[KEY] = VALUE

syntax: VALUE = ts.ctx[KEY]

context: do_remap or do_global_* or later

description: This table can be used to store per-request Lua context data and has a life time identical to the currentrequest.

Here is an example:

function send_response()ts.client_response.header['F-Header'] = ts.ctx['hdr']return 0

end

function do_remap()ts.ctx['hdr'] = 'foo'ts.hook(TS_LUA_HOOK_SEND_RESPONSE_HDR, send_response)return 0

end

Then the client will get the response like this:

HTTP/1.1 200 OKContent-Type: text/htmlServer: ATS/3.2.0F-Header: foo



Connection: Keep-Alive...

TOP

ts.client_request.get_method

syntax: ts.client_request.get_method()


description: This function can be used to retrieve the current client request’s method name. String like “GET” or“POST” is returned.

TOP

ts.client_request.set_method

syntax: ts.client_request.set_method()

context: do_remap or do_global_*

description: This function can be used to override the current client request’s method with METHOD_NAME.

ts.client_request.get_version

syntax: ver = ts.client_request.get_version()


description: Return the http version string of the client request.

Current possible values are 1.0, 1.1, and 0.9.

TOP

ts.client_request.set_version

syntax: ts.client_request.set_version(VERSION_STR)


description: Set the http version of the client request with the VERSION_STR

ts.client_request.set_version('1.0')

TOP

ts.client_request.get_uri

syntax: ts.client_request.get_uri()

context: do_remap or later

description: This function can be used to retrieve the client request’s path.

3.7. Plugins 131


Here is an example:

function do_remap()local uri = ts.client_request.get_uri()print(uri)

end

Then GET /st?a=1 will yield the output:

/st

TOP

ts.client_request.set_uri

syntax: ts.client_request.set_uri(PATH)


description: This function can be used to override the client request’s path.

The PATH argument must be a Lua string and starts with /

TOP

ts.client_request.get_uri_args

syntax: ts.client_request.get_uri_args()


description: This function can be used to retrieve the client request’s query string.

Here is an example:

function do_remap()local query = ts.client_request.get_uri_args()print(query)

end

Then GET /st?a=1&b=2 will yield the output:

a=1&b=2

TOP

ts.client_request.set_uri_args

syntax: ts.client_request.set_uri_args(QUERY_STRING)


description: This function can be used to override the client request’s query string.

ts.client_request.set_uri_args('n=6&p=7')

TOP



ts.client_request.get_uri_params

syntax: ts.client_request.get_uri_params()


description: This function can be used to retrieve the client request’s parameter string.

Here is an example:

function do_remap()local query = ts.client_request.get_uri_params()print(query)

end

Then GET /st;a=1 will yield the output:

a=1

TOP

ts.client_request.set_uri_params

syntax: ts.client_request.set_uri_params(PARAMETER_STRING)


description: This function can be used to override the client request’s parameter string.

ts.client_request.set_uri_params('n=6')

TOP

ts.client_request.get_url

syntax: ts.client_request.get_url()


description: This function can be used to retrieve the whole client request’s url.

Here is an example:

function do_remap()local url = ts.client_request.get_url()print(url)

end

Then GET /st?a=1&b=2 HTTP/1.1\r\nHost: a.tbcdn.cn\r\n... will yield the output:

http://a.tbcdn.cn/st?a=1&b=2

TOP

ts.client_request.header.HEADER

syntax: ts.client_request.header.HEADER = VALUE

3.7. Plugins 133


syntax: ts.client_request.header[HEADER] = VALUE

syntax: VALUE = ts.client_request.header.HEADER


description: Set, add to, clear or get the current client request’s HEADER.

Here is an example:

function do_remap()local ua = ts.client_request.header['User-Agent']print(ua)ts.client_request.header['Host'] = 'a.tbcdn.cn'

end

Then GET /st HTTP/1.1\r\nHost: b.tb.cn\r\nUser-Agent: Mozilla/5.0\r\n... willyield the output:

Mozilla/5.0

TOP

ts.client_request.get_headers

syntax: ts.client_request.get_headers()


description: Returns a Lua table holding all the headers for the current client request.

Here is an example:

function do_remap()hdrs = ts.client_request.get_headers()for k, v in pairs(hdrs) do

print(k..': '..v)end

end

Then GET /st HTTP/1.1\r\nHost: b.tb.cn\r\nUser-Aget: Mozilla/5.0\r\nAccept:

*/* will yield the output

Host: b.tb.cnUser-Agent: Mozilla/5.0Accept: */*

TOP

ts.client_request.client_addr.get_addr

syntax: ts.client_request.client_addr.get_addr()


description: This function can be used to get socket address of the client.

The ts.client_request.client_addr.get_addr function returns three values, ip is a string, port and family is number.

Here is an example:



function do_remap()ip, port, family = ts.client_request.client_addr.get_addr()print(ip) -- 192.168.231.17print(port) -- 17786print(family) -- 2(AF_INET)return 0

end

TOP

ts.client_request.client_addr.get_incoming_port

syntax: ts.client_request.client_addr.get_incoming_port()


description: This function can be used to get incoming port of the request.

The ts.client_request.client_addr.get_incoming_port function returns incoming port as number.

Here is an example:

function do_global_read_request()port = ts.client_request.client_addr.get_incoming_port()print(port) -- 80

end

TOP

ts.client_request.get_url_host

syntax: host = ts.client_request.get_url_host()


description: Return the host field of the request url.

Here is an example:

function do_remap()local url_host = ts.client_request.get_url_host()print(url_host)

end

Then GET /liuyurou.txt HTTP/1.1\r\nHost: 192.168.231.129:8080\r\n... will yield theoutput:

192.168.231.129

TOP

ts.client_request.set_url_host

syntax: ts.client_request.set_url_host(str)


3.7. Plugins 135


description: Set host field of the request url with str. This function is used to change the address of the originserver, and we should return TS_LUA_REMAP_DID_REMAP(_STOP) in do_remap.

Here is an example:

function do_remap()ts.client_request.set_url_host('192.168.231.130')ts.client_request.set_url_port(80)ts.client_request.set_url_scheme('http')return TS_LUA_REMAP_DID_REMAP

end

remap.config like this:

map http://192.168.231.129:8080/ http://192.168.231.129:9999/

Then server request will connect to 192.168.231.130:80

TOP

ts.client_request.get_url_port

syntax: port = ts.client_request.get_url_port()


description: Returns the port field of the request url as a Lua number.

Here is an example:

function do_remap()local url_port = ts.client_request.get_url_port()print(url_port)

end

Then Then GET /liuyurou.txt HTTP/1.1\r\nHost: 192.168.231.129:8080\r\n... will yieldthe output:

8080

TOP

ts.client_request.set_url_port

syntax: ts.client_request.set_url_port(NUMBER)


description: Set port field of the request url with NUMBER. This function is used to change the address of the originserver, and we should return TS_LUA_REMAP_DID_REMAP(_STOP) in do_remap.

TOP

ts.client_request.get_url_scheme

syntax: scheme = ts.client_request.get_url_scheme()




description: Return the scheme field of the request url.

Here is an example:

function do_remap()local url_scheme = ts.client_request.get_url_scheme()print(url_scheme)

end


http

TOP

ts.client_request.set_url_scheme

syntax: ts.client_request.set_url_scheme(str)


description: Set scheme field of the request url with str. This function is used to change the scheme of the serverrequest, and we should return TS_LUA_REMAP_DID_REMAP(_STOP) in do_remap.

TOP

ts.http.set_cache_url

syntax: ts.http.set_cache_url(KEY_URL)


description: This function can be used to modify the cache key for the client request.

Here is an example:

function do_remap()ts.http.set_cache_url('http://a.tbcdn.cn/foo')return 0

end

TOP

ts.http.get_cache_lookup_url

syntax: ts.http.get_cache_lookup_url()

context: do_global_cache_lookup_complete

description: This function can be used to get the cache lookup url for the client request.

Here is an example

function cache_lookup()ts.http.set_cache_lookup_url('http://bad.com/bad.html')local cache = ts.http.get_cache_lookup_url()ts.debug(cache)

3.7. Plugins 137


end

function do_remap()ts.hook(TS_LUA_HOOK_CACHE_LOOKUP_COMPLETE, cache_lookup)return 0

end

TOP

ts.http.set_cache_lookup_url

syntax: ts.http.set_cache_lookup_url()

context: do_global_cache_lookup_complete

description: This function can be used to set the cache lookup url for the client request.

TOP

ts.http.set_server_resp_no_store

syntax: ts.http.set_server_resp_no_store(status)

context: do_global_read_response

description: This function can be used to signal ATS to not store the response in cache

Here is an example:

function do_global_read_response()ts.http.set_server_resp_no_store(1)return 0

end

TOP

ts.http.set_resp

syntax: ts.http.set_resp(CODE, BODY)


description: This function can be used to set response for the client with the CODE status and BODY string.

Here is an example:

function do_remap()ts.http.set_resp(403, "Document access failed :)\n")return 0

end

We will get the response like this:

HTTP/1.1 403 ForbiddenDate: Thu, 20 Mar 2014 06:12:43 GMTConnection: closeServer: ATS/5.0.0



Cache-Control: no-storeContent-Type: text/htmlContent-Language: enContent-Length: 27

Document access failed :)

TOP

ts.http.get_cache_lookup_status

syntax: ts.http.get_cache_lookup_status()

context: function @ TS_LUA_HOOK_CACHE_LOOKUP_COMPLETE hook point

description: This function can be used to get cache lookup status.

Here is an example:

function cache_lookup()local cache_status = ts.http.get_cache_lookup_status()if cache_status == TS_LUA_CACHE_LOOKUP_HIT_FRESH then

print('hit')else

print('not hit')end

end


end

TOP

ts.http.set_cache_lookup_status

syntax: ts.http.set_cache_lookup_status()

context: function after TS_LUA_HOOK_CACHE_LOOKUP_COMPLETE hook point

description: This function can be used to set cache lookup status.

Here is an example:


print('hit')else

print('not hit')endts.http.set_cache_lookup_status(TS_LUA_CACHE_LOOKUP_MISS)

end

function do_remap()ts.hook(TS_LUA_HOOK_CACHE_LOOKUP_COMPLETE, cache_lookup)

3.7. Plugins 139


return 0end

TOP

Http cache lookup status constants

context: global

TS_LUA_CACHE_LOOKUP_MISS (0)TS_LUA_CACHE_LOOKUP_HIT_STALE (1)TS_LUA_CACHE_LOOKUP_HIT_FRESH (2)TS_LUA_CACHE_LOOKUP_SKIPPED (3)

TOP

ts.cached_response.get_status

syntax: status = ts.cached_response.get_status()

context: function @ TS_LUA_HOOK_CACHE_LOOKUP_COMPLETE hook point or later

description: This function can be used to retrieve the status code of the cached response. A Lua number will bereturned.

Here is an example:


code = ts.cached_response.get_status()print(code) -- 200

endend


end

TOP

ts.cached_response.get_version

syntax: ver = ts.cached_response.get_version()


description: Return the http version string of the cached response.


TOP



ts.cached_response.header.HEADER

syntax: VALUE = ts.cached_response.header.HEADER

syntax: VALUE = ts.cached_response.header[HEADER]


description: get the current cached response’s HEADER.

Here is an example:

function cache_lookup()local status = ts.http.get_cache_lookup_status()if status == TS_LUA_CACHE_LOOKUP_HIT_FRESH then

local ct = ts.cached_response.header['Content-Type']print(ct) -- text/plain

endend


end

TOP

ts.cached_response.get_headers

syntax: ts.cached_response.get_headers()


description: Returns a Lua table holding all the headers for the current cached response.

Here is an example:

function cache_lookup()local status = ts.http.get_cache_lookup_status()if status == TS_LUA_CACHE_LOOKUP_HIT_FRESH then

hdrs = ts.cached_response.get_headers()for k, v in pairs(hdrs) do


endend


end

We will get the output:

Connection: keep-aliveContent-Type: text/plainDate: Thu, 20 Mar 2014 06:12:20 GMTCache-Control: max-age=864000Last-Modified: Sun, 19 May 2013 13:22:01 GMT

3.7. Plugins 141


Accept-Ranges: bytesContent-Length: 15Server: ATS/5.0.0

TOP

ts.server_request.get_uri

syntax: ts.server_request.get_uri()

context: function @ TS_LUA_HOOK_SEND_REQUEST_HDR hook point or later

description: This function can be used to retrieve the server request’s path.

Here is an example:

function send_request()local uri = ts.server_request.get_uri()print(uri)

end

function do_remap()ts.hook(TS_LUA_HOOK_SEND_REQUEST_HDR, send_request)return 0

end

Then GET /am.txt?a=1 will yield the output:

/am.txt

TOP

ts.server_request.set_uri

syntax: ts.server_request.set_uri(PATH)

context: function @ TS_LUA_HOOK_SEND_REQUEST_HDR hook point

description: This function can be used to override the server request’s path.

The PATH argument must be a Lua string and starts with /

TOP

ts.server_request.get_uri_args

syntax: ts.server_request.get_uri_args()


description: This function can be used to retrieve the server request’s query string.

Here is an example:

function send_request()local query = ts.server_request.get_uri_args()print(query)

end




end

Then GET /st?a=1&b=2 will yield the output:

a=1&b=2

TOP

ts.server_request.set_uri_args

syntax: ts.server_request.set_uri_args(QUERY_STRING)


description: This function can be used to override the server request’s query string.

ts.server_request.set_uri_args('n=6&p=7')

TOP

ts.server_request.get_uri_params

syntax: ts.server_request.get_uri_params()


description: This function can be used to retrieve the server request’s parameter string.

Here is an example:

function send_request()local query = ts.server_request.get_uri_params()print(query)

end


end

Then GET /st;a=1 will yield the output:

a=1

TOP

ts.server_request.set_uri_params

syntax: ts.server_request.set_uri_params(PARAMETER_STRING)


description: This function can be used to override the server request’s parameter string.

3.7. Plugins 143


ts.server_request.set_uri_params('n=6')

TOP

ts.server_request.header.HEADER

syntax: ts.server_request.header.HEADER = VALUE

syntax: ts.server_request.header[HEADER] = VALUE

syntax: VALUE = ts.server_request.header.HEADER


description: Set, add to, clear or get the current server request’s HEADER.

Here is an example:

function send_request()local ua = ts.server_request.header['User-Agent']print(ua)ts.server_request.header['Accept-Encoding'] = 'gzip'

end


end

Then GET /st HTTP/1.1\r\nHost: b.tb.cn\r\nUser-Agent: Mozilla/5.0\r\n... willyield the output:

Mozilla/5.0

TOP

ts.server_request.get_headers

syntax: ts.server_request.get_headers()


description: Returns a Lua table holding all the headers for the current server request.

Here is an example:

function send_request()hdrs = ts.cached_response.get_headers()for k, v in pairs(hdrs) do


end


end




Host: b.tb.cnUser-Agent: curl/7.19.7Accept: */*

TOP

ts.server_request.server_addr.get_addr

syntax: ts.server_request.server_addr.get_addr()


description: This function can be used to get socket address of the origin server.

The ts.server_request.server_addr.get_addr function returns three values, ip is a string, port and family is number.

Here is an example:

function do_global_send_request()ip, port, family = ts.server_request.server_addr.get_addr()print(ip) -- 192.168.231.17print(port) -- 80print(family) -- 2(AF_INET)

end

TOP

ts.server_request.server_addr.get_ip

syntax: ts.server_request.server_addr.get_ip()


description: This function can be used to get ip address of the origin server.

The ts.server_request.server_addr.get_ip function returns ip as a string.

Here is an example:

function do_global_send_request()ip = ts.server_request.server_addr.get_ip()print(ip) -- 192.168.231.17

end

TOP

ts.server_request.server_addr.get_port

syntax: ts.server_request.server_addr.get_port()


description: This function can be used to get port of the origin server.

The ts.server_request.server_addr.get_port function returns port as number.

Here is an example:

3.7. Plugins 145


function do_global_send_request()port = ts.server_request.server_addr.get_port()print(port) -- 80

end

TOP

ts.server_request.server_addr.get_outgoing_port

syntax: ts.server_request.server_addr.get_outgoing_port()


description: This function can be used to get outgoing port to the origin server.

The ts.server_request.server_addr.get_outgoing_port function returns outgoing port as number.

Here is an example:

function do_global_send_request()port = ts.server_request.server_addr.get_outgoing_port()print(port) -- 50880

end

TOP

ts.server_request.get_url_host

syntax: host = ts.server_request.get_url_host()


description: Return the host field of the request url.

Here is an example:

function send_request()local url_host = ts.server_request.get_url_host()print(url_host)

end


end

Then GET http://abc.com/p2/a.txt HTTP/1.1 will yield the output:

abc.com

TOP

ts.server_request.set_url_host

syntax: ts.server_request.set_url_host(str)




description: Set host field of the request url with str. This function is used to change the host name in the GETrequest to next tier

Here is an example:

function send_request()ts.server_request.set_url_host("")ts.server_request.set_url_scheme("")

end


end

The GET request like this:

+++++++++ Proxy's Request +++++++++- State Machine Id: 5593GET http://origin.com/dir1/a.txt HTTP/1.1User-Agent: curl/7.29.0Host: abc.comAccept: /Client-ip: 135.xx.xx.xxX-Forwarded-For: 135.xx.xx.xx

Will be changed to:

+++++++++ Proxy's Request +++++++++- State Machine Id: 5593GET /dir1/a.txt HTTP/1.1User-Agent: curl/7.29.0Host: abc.comAccept: /Client-ip: 135.xx.xx.xxX-Forwarded-For: 135.xx.xx.xx

TOP

ts.server_request.get_url_scheme

syntax: scheme = ts.server_request.get_url_scheme()


description: Return the scheme field of the request url.

Here is an example:

function send_request()local url_scheme = ts.server_request.get_url_scheme()print(url_host)

end


end

3.7. Plugins 147



http

TOP

ts.server_request.set_url_scheme

syntax: ts.server_request.set_url_scheme(str)


description: Set scheme field of the request url with str. This function is used to change the scheme of the serverrequest.

TOP

ts.server_response.get_status

syntax: status = ts.server_response.get_status()

context: function @ TS_LUA_HOOK_READ_RESPONSE_HDR hook point or later

description: This function can be used to retrieve the status code of the origin server’s response. A Lua number willbe returned.

Here is an example:

function read_response()local code = ts.server_response.get_status()print(code) -- 200

end

function do_remap()ts.hook(TS_LUA_HOOK_READ_RESPONSE_HDR, read_response)return 0

end

TOP‘

ts.server_response.set_status

syntax: ts.server_response.set_status(NUMBER)

context: function @ TS_LUA_HOOK_READ_RESPONSE_HDR hook point

description: This function can be used to set the status code of the origin server’s response.

Here is an example:

function read_response()ts.server_response.set_status(404)

end

function do_remap()ts.hook(TS_LUA_HOOK_READ_RESPONSE_HDR, read_response)



return 0end

TOP‘

ts.server_response.get_version

syntax: ver = ts.server_response.get_version()

context: function @ TS_LUA_HOOK_READ_RESPONSE_HDR hook point or later.

description: Return the http version string of the server response.


TOP

ts.server_response.set_version

syntax: ts.server_response.set_version(VERSION_STR)

context: function @ TS_LUA_HOOK_READ_RESPONSE_HDR hook point

description: Set the http version of the server response with the VERSION_STR

ts.server_response.set_version('1.0')

TOP

ts.server_response.header.HEADER

syntax: ts.server_response.header.HEADER = VALUE

syntax: ts.server_response.header[HEADER] = VALUE

syntax: VALUE = ts.server_response.header.HEADER

context: function @ TS_LUA_HOOK_READ_RESPONSE_HDR hook point or later.

description: Set, add to, clear or get the current server response’s HEADER.

Here is an example:

function read_response()local ct = ts.server_response.header['Content-Type']print(ct)ts.server_response.header['Cache-Control'] = 'max-age=14400'

end


end


text/html

TOP‘

3.7. Plugins 149


ts.server_response.get_headers

syntax: ts.server_response.get_headers()

context: function @ TS_LUA_HOOK_READ_RESPONSE_HDR hook point or later

description: Returns a Lua table holding all the headers for the current server response.

Here is an example:

function read_response()hdrs = ts.server_response.get_headers()for k, v in pairs(hdrs) do


end


end


Server: nginx/1.5.9Date: Tue, 18 Mar 2014 10:12:25 GMTContent-Type: text/htmlContent-Length: 555Last-Modified: Mon, 19 Aug 2013 14:25:55 GMTConnection: keep-aliveETag: "52122af3-22b"Cache-Control: max-age=14400Accept-Ranges: bytes

TOP

ts.client_response.get_status

syntax: status = ts.client_response.get_status()

context: function @ TS_LUA_HOOK_SEND_RESPONSE_HDR hook point

description: This function can be used to retrieve the status code of the response to the client. A Lua number will bereturned.

Here is an example:

function send_response()local code = ts.client_response.get_status()print(code) -- 200

end

function do_remap()ts.hook(TS_LUA_HOOK_SEND_RESPONSE_HDR, send_response)return 0

end

TOP



ts.client_response.set_status

syntax: ts.client_response.set_status(NUMBER)


description: This function can be used to set the status code of the response to the client.

Here is an example:

function send_response()ts.client_response.set_status(404)

end


end

TOP

ts.client_response.get_version

syntax: ver = ts.client_response.get_version()

context: function @ TS_LUA_HOOK_SEND_RESPONSE_HDR hook point.

description: Return the http version string of the response to the client.


TOP

ts.client_response.set_version

syntax: ts.client_response.set_version(VERSION_STR)


description: Set the http version of the response to the client with the VERSION_STR

ts.client_response.set_version('1.0')

TOP

ts.client_response.header.HEADER

syntax: ts.client_response.header.HEADER = VALUE

syntax: ts.client_response.header[HEADER] = VALUE

syntax: VALUE = ts.client_response.header.HEADER


description: Set, add to, clear or get the current client response’s HEADER.

Here is an example:

3.7. Plugins 151


function send_response()local ct = ts.client_response.header['Content-Type']print(ct)ts.client_response.header['Cache-Control'] = 'max-age=3600'

end


end


text/html

TOP

ts.client_response.get_headers

syntax: ts.client_response.get_headers()


description: Returns a Lua table holding all the headers for the current client response.

Here is an example:

function send_response()hdrs = ts.client_response.get_headers()for k, v in pairs(hdrs) do


end


end


Server: ATS/5.0.0Date: Tue, 18 Mar 2014 10:12:25 GMTContent-Type: text/htmlTransfer-Encoding: chunkedLast-Modified: Mon, 19 Aug 2013 14:25:55 GMTConnection: keep-aliveCache-Control: max-age=14400Age: 2641Accept-Ranges: bytes

TOP

ts.client_response.set_error_resp

syntax: ts.client_response.set_error_resp(CODE, BODY)




description: This function can be used to set the error response to the client.

With this function we can jump to send error response to the client if exception exists, meanwhile we should return -1from the function where exception raises.

Here is an example:

function send_response()ts.client_response.set_error_resp(404, 'bad luck :(\n')

end

function cache_lookup()return -1

end

function do_remap()ts.hook(TS_LUA_HOOK_CACHE_LOOKUP_COMPLETE, cache_lookup)ts.hook(TS_LUA_HOOK_SEND_RESPONSE_HDR, send_response)return 0

end


HTTP/1.1 404 Not FoundDate: Tue, 18 Mar 2014 11:16:00 GMTConnection: keep-aliveServer: ATS/5.0.0Content-Length: 12

bad luck :(

TOP

Number constants

context: global

TS_LUA_INT64_MAX (9223372036854775808)TS_LUA_INT64_MIN (-9223372036854775808L)

These constants are usually used in transform handler.

TOP

ts.http.resp_cache_transformed

syntax: ts.http.resp_cache_transformed(BOOL)


description: This function can be used to tell trafficserver whether to cache the transformed data.

Here is an example:

3.7. Plugins 153


function upper_transform(data, eos)return string.upper(data), eos

end

function do_remap()ts.hook(TS_LUA_RESPONSE_TRANSFORM, upper_transform)ts.http.resp_cache_transformed(0)return 0

end

This function is usually called after we hook TS_LUA_RESPONSE_TRANSFORM.

TOP

ts.http.resp_cache_untransformed

syntax: ts.http.resp_cache_untransformed(BOOL)


description: This function can be used to tell trafficserver whether to cache the untransformed data.

Here is an example:

function upper_transform(data, eos)return string.upper(data), eos

end

function do_remap()ts.hook(TS_LUA_RESPONSE_TRANSFORM, upper_transform)ts.http.resp_cache_untransformed(1)return 0

end

This function is usually called after we hook TS_LUA_RESPONSE_TRANSFORM.

TOP

ts.http.resp_transform.get_upstream_bytes

syntax: ts.http.resp_transform.get_upstream_bytes()

context: transform handler

description: This function can be used to retrive the total bytes to be received from the upstream. If we got chunkedresponse body from origin server, TS_LUA_INT64_MAX will be returned.

Here is an example:

local APPEND_DATA = 'TAIL\n'

function append_transform(data, eos)if ts.ctx['len_set'] == nil then

local sz = ts.http.resp_transform.get_upstream_bytes()if sz ~= TS_LUA_INT64_MAX then

ts.http.resp_transform.set_downstream_bytes(sz + string.len(APPEND_DATA))end



ts.ctx['len_set'] = trueend

if eos == 1 thenreturn data .. APPEND_DATA, eos

elsereturn data, eos

endend

function do_remap()ts.hook(TS_LUA_RESPONSE_TRANSFORM, append_transform)ts.http.resp_cache_transformed(0)ts.http.resp_cache_untransformed(1)return 0

end

TOP

ts.http.resp_transform.set_downstream_bytes

syntax: ts.http.resp_transform.set_downstream_bytes(NUMBER)

context: transform handler

description: This function can be used to set the total bytes to be sent to the downstream.

Sometimes we want to set Content-Length header in client_response, and this function should be called before anyreal data is returned from the transform handler.

TOP

ts.http.skip_remapping_set

syntax: ts.http.skip_remapping_set(BOOL)

context: do_global_read_request

description: This function can be used to tell trafficserver to skip doing remapping

Here is an example:

function do_global_read_request()ts.http.skip_remapping_set(1);ts.client_request.header['Host'] = 'www.yahoo.com'return 0

end

This function is usually called in do_global_read_request function

TOP

ts.http.is_internal_request

syntax: ts.http.is_internal_request()


3.7. Plugins 155


description: This function can be used to tell is a request is internal or not

Here is an example:

function do_global_read_request()local internal = ts.http.is_internal_request()ts.debug(internal)return 0

end

TOP

ts.http.transaction_count

syntax: ts.http.transaction_count()


description: This function returns the number of transaction in this connection

Here is an example

function do_remap()local count = ts.http.transaction_count()ts.debug(tostring(count))return 0

end

TOP

ts.add_package_path

syntax: ts.add_package_path(lua-style-path-str)

context: init stage of the lua script

description: Adds the Lua module search path used by scripts.

The path string is in standard Lua path form.

Here is an example:

ts.add_package_path('/home/a/test/lua/pac/?.lua')local nt = require("nt")function do_remap()

print(nt.t9(7979))return 0

end

TOP

ts.add_package_cpath

syntax: ts.add_package_cpath(lua-style-cpath-str)

context: init stage of the lua script

description: Adds the Lua C-module search path used by scripts.



The cpath string is in standard Lua cpath form.

Here is an example:

ts.add_package_cpath('/home/a/test/c/module/?.so')local ma = require("ma")function do_remap()

print(ma.ft())return 0

end

TOP

ts.md5

syntax: digest = ts.md5(str)

context: global

description: Returns the hexadecimal representation of the MD5 digest of the str argument.

Here is an example:

function do_remap()uri = ts.client_request.get_uri()print(uri) -- /fooprint(ts.md5(uri)) -- 1effb2475fcfba4f9e8b8a1dbc8f3caf

end

TOP

ts.md5_bin

syntax: digest = ts.md5_bin(str)

context: global

description: Returns the binary form of the MD5 digest of the str argument.

Here is an example:

function do_remap()uri = ts.client_request.get_uri()bin = ts.md5_bin(uri)

end

TOP

ts.sha1

syntax: digest = ts.sha1(str)

context: global

description: Returns the hexadecimal representation of the SHA-1 digest of the str argument.

Here is an example:

3.7. Plugins 157


function do_remap()uri = ts.client_request.get_uri()print(uri) -- /fooprint(ts.sha1(uri)) -- 6dbd548cc03e44b8b44b6e68e56255ce4273ae49

end

TOP

ts.sha1_bin

syntax: digest = ts.sha1_bin(str)

context: global

description: Returns the binary form of the SHA-1 digest of the str argument.

Here is an example:

function do_remap()uri = ts.client_request.get_uri()bin = ts.sha1_bin(uri)

end

TOP

ts.base64_encode

syntax: value = ts.base64_encode(str)

context: global

description: Returns the base64 encoding of the str argument.

Here is an example:

function do_remap()uri = ts.client_request.get_uri()value = ts.base64_encode(uri)

end

TOP

ts.base64_decode

syntax: value = ts.base64_decode(str)

context: global

description: Returns the base64 decoding of the str argument.

Here is an example:

function do_remap()uri = ts.client_request.get_uri()encoded_value = ts.base64_encode(uri)decoded_value = ts.base64_decode(encoded_value)

end



TOP

ts.escape_uri

syntax: value = ts.escape_uri(str)

context: global

description: Returns the uri-escaped value of the str argument.

Here is an example:

function do_remap()test = '/some value/'value = ts.escape_uri(test)

end

TOP

ts.unescape_uri

syntax: value = ts.unescape_uri(str)

context: global

description: Returns the uri-unescaped value of the str argument.

Here is an example:

function do_remap()test = '/some value/'escaped_value = ts.escape_uri(test)unescaped_value = ts.unescape_uri(escaped_value)

end

TOP

ts.fetch

syntax: res = ts.fetch(url, table?)

context: after do_remap

description: Issues a synchronous but still non-block http request with the url and the optional table.

Returns a Lua table with serveral slots (res.status, res.header, res.body, and res.truncated).

res.status holds the response status code.

res.header holds the response header table.

res.body holds the response body which may be truncated, you need to check res.truncated to see if the data istruncated.

Here is a basic example:

3.7. Plugins 159


function post_remap()local url = string.format('http://%s/foo.txt', ts.ctx['host'])local res = ts.fetch(url)if res.status == 200 then

print(res.body)end

end

function do_remap()local inner = ts.http.is_internal_request()if inner ~= 0 then

return 0endlocal host = ts.client_request.header['Host']ts.ctx['host'] = hostts.hook(TS_LUA_HOOK_POST_REMAP, post_remap)

end

We can set the optional table with serveral members:

header holds the request header table.

method holds the request method. The default method is ‘GET’.

cliaddr holds the request client address in ip:port form. The default cliaddr is ‘127.0.0.1:33333’

Issuing a post request:

res = ts.fetch('http://xx.com/foo', {method = 'POST', body = 'hello world'})

TOP

ts.fetch_multi

syntax: vec = ts.fetch_multi({{url, table?}, {url, table?}, ...})


Just like ts.fetch, but supports multiple http requests running in parallel.

This function will fetch all the urls specified by the input table and return a table which contain all the results in thesame order.

Here is an example:

local vec = ts.fetch_multi({{'http://xx.com/slayer'},{'http://xx.com/am', {cliaddr = '192.168.1.19:35423'}},{'http://xx.com/naga', {method = 'POST', body = 'hello world'}},

})

for i = 1, #(vec) doprint(vec[i].status)

end

TOP



ts.http.intercept

syntax: ts.http.intercept(FUNCTION, param1?, param2?, ...)


description: Intercepts the client request and processes it in FUNCTION with optional params.

We should construct the response for the client request, and the request will not be processed by other modules, likehostdb, cache, origin server...

Intercept FUNCTION will be executed in a new lua_thread, so we can delivery optional params from old lua_threadto new lua_thread if needed.

Here is an example:

require 'os'

function send_data(dstr)local nt = os.time()..' Zheng.\n'local resp = 'HTTP/1.0 200 OK\r\n' ..

'Server: ATS/3.2.0\r\n' ..'Content-Type: text/plain\r\n' ..'Content-Length: ' .. string.format('%d', string.len(nt)) .. '\r\n'

→˓..'Last-Modified: ' .. os.date("%a, %d %b %Y %H:%M:%S GMT", os.

→˓time()) .. '\r\n' ..'Connection: keep-alive\r\n' ..'Cache-Control: max-age=7200\r\n' ..'Accept-Ranges: bytes\r\n\r\n' ..nt

print(dstr)ts.say(resp)

end

function do_remap()ts.http.intercept(send_data, 'hello world')return 0

end

Then we will get the response like this:

HTTP/1.1 200 OKServer: ATS/5.0.0Content-Type: text/plainContent-Length: 18Last-Modified: Tue, 18 Mar 2014 08:23:12 GMTCache-Control: max-age=7200Accept-Ranges: bytesDate: Tue, 18 Mar 2014 12:23:12 GMTAge: 0Connection: keep-alive

1395145392 Zheng.

TOP

3.7. Plugins 161


ts.http.server_intercept

syntax: ts.http.server_intercept(FUNCTION, param1?, param2?, ...)


description: Intercepts the server request and acts as the origin server.

Just like ts.http.intercept, but this function will intercept the server request, and we can acts as the origin server inFUNCTION.

Here is an example:

require 'os'

function process_combo(host)local url1 = string.format('http://%s/css/1.css', host)local url2 = string.format('http://%s/css/2.css', host)local url3 = string.format('http://%s/css/3.css', host)

local hdr = {['Host'] = host,['User-Agent'] = 'blur blur',

}

local ct = {header = hdr,method = 'GET'

}

local arr = ts.fetch_multi({

{url1, ct},{url2, ct},{url3, ct},

})

local ctype = arr[1].header['Content-Type']local body = arr[1].body .. arr[2].body .. arr[3].body

local resp = 'HTTP/1.1 200 OK\r\n' ..'Server: ATS/5.2.0\r\n' ..'Last-Modified: ' .. os.date("%a, %d %b %Y %H:%M:%S GMT", os.

→˓time()) .. '\r\n' ..'Cache-Control: max-age=7200\r\n' ..'Accept-Ranges: bytes\r\n' ..'Content-Type: ' .. ctype .. '\r\n' ..'Content-Length: ' .. string.format('%d', string.len(body)) ..

→˓'\r\n\r\n' ..body

ts.say(resp)end

function do_remap()local inner = ts.http.is_internal_request()if inner ~= 0 then

return 0end



local h = ts.client_request.header['Host']ts.http.server_intercept(process_combo, h)

end

TOP

ts.say

syntax: ts.say(data)

context: intercept or server_intercept

description: Write response to ATS within intercept or server_intercept.

TOP

ts.flush

syntax: ts.flush()

context: intercept or server_intercept

description: Flushes the output to ATS within intercept or server_intercept.

In synchronous mode, the function will not return until all output data has been written into the system send buffer.Note that using the Lua coroutine mechanism means that this function does not block the ATS event loop even in thesynchronous mode.

Here is an example:

require 'os'

function send_data()ss = 'wo ai yu ye hua\n'local resp = 'HTTP/1.0 200 OK\r\n' ..

'Server: ATS/3.2.0\r\n' ..'Content-Type: text/plain\r\n' ..'Content-Length: ' .. string.format('%d', 5*string.len(ss)) .. '\r\n

→˓' ..'Last-Modified: ' .. os.date("%a, %d %b %Y %H:%M:%S GMT", os.

→˓time()) .. '\r\n' ..'Connection: keep-alive\r\n' ..'Cache-Control: max-age=7200\r\n' ..'Accept-Ranges: bytes\r\n\r\n'

ts.say(resp)for i=1, 5 do

ts.say(ss)ts.flush()

endend

function do_remap()ts.http.intercept(send_data)return 0

end

3.7. Plugins 163



HTTP/1.1 200 OKServer: ATS/5.0.0Content-Type: text/plainContent-Length: 80Last-Modified: Tue, 18 Mar 2014 08:38:29 GMTCache-Control: max-age=7200Accept-Ranges: bytesDate: Tue, 18 Mar 2014 12:38:29 GMTAge: 0Connection: keep-alive

wo ai yu ye huawo ai yu ye huawo ai yu ye huawo ai yu ye huawo ai yu ye hua

TOP

ts.sleep

syntax: ts.sleep(sec)


description: Sleeps for the specified seconds without blocking.

Behind the scene, this method makes use of the ATS event model.

Here is an example:

function send_response()ts.sleep(3)

end

function read_response()ts.sleep(3)

end

function do_remap()ts.hook(TS_LUA_HOOK_READ_RESPONSE_HDR, read_response)ts.hook(TS_LUA_HOOK_SEND_RESPONSE_HDR, send_response)

end

TOP

ts.schedule

syntax: ts.schedule(THREAD_TYPE, sec, FUNCTION, param1?, param2?, ...)


description: Schedule function to be run after specified seconds without blocking.

Behind the scene, this method makes use of the ATS event model.

Here is an example:



function schedule()ts.debug('test schedule starts')

end

function cache_lookup()ts.debug('cache-lookup')ts.schedule(TS_LUA_THREAD_POOL_NET, 0, schedule)return 0

end

function do_remap()ts.hook(TS_LUA_HOOK_CACHE_LOOKUP_COMPLETE, cache_lookup)

end

TOP

ts.http.config_int_get

syntax: val = ts.http.config_int_get(CONFIG)

context: do_remap or do_global_* or later.

description: Configuration option which has a int value can be retrieved with this function.

val = ts.http.config_int_get(TS_LUA_CONFIG_HTTP_CACHE_HTTP)

TOP

ts.http.config_int_set

syntax: ts.http.config_int_set(CONFIG, NUMBER)


description: This function can be used to overwrite the configuration options.

Here is an example:

function do_remap()ts.http.config_int_set(TS_LUA_CONFIG_HTTP_CACHE_HTTP, 0) -- bypass the cache

→˓processorreturn 0

end

TOP

ts.http.config_float_get

syntax: val = ts.http.config_float_get(CONFIG)


description: Configuration option which has a float value can be retrieved with this function.

TOP

3.7. Plugins 165


ts.http.config_float_set

syntax: ts.http.config_float_set(CONFIG, NUMBER)



TOP

ts.http.config_string_get

syntax: val = ts.http.config_string_get(CONFIG)


description: Configuration option which has a string value can be retrieved with this function.

TOP

ts.http.config_string_set

syntax: ts.http.config_string_set(CONFIG, NUMBER)



TOP

Http config constants


TS_LUA_CONFIG_URL_REMAP_PRISTINE_HOST_HDRTS_LUA_CONFIG_HTTP_CHUNKING_ENABLEDTS_LUA_CONFIG_HTTP_NEGATIVE_CACHING_ENABLEDTS_LUA_CONFIG_HTTP_NEGATIVE_CACHING_LIFETIMETS_LUA_CONFIG_HTTP_CACHE_WHEN_TO_REVALIDATETS_LUA_CONFIG_HTTP_KEEP_ALIVE_ENABLED_INTS_LUA_CONFIG_HTTP_KEEP_ALIVE_ENABLED_OUTTS_LUA_CONFIG_HTTP_KEEP_ALIVE_POST_OUTTS_LUA_CONFIG_HTTP_SERVER_SESSION_SHARING_MATCHTS_LUA_CONFIG_NET_SOCK_RECV_BUFFER_SIZE_OUTTS_LUA_CONFIG_NET_SOCK_SEND_BUFFER_SIZE_OUTTS_LUA_CONFIG_NET_SOCK_OPTION_FLAG_OUTTS_LUA_CONFIG_HTTP_FORWARD_PROXY_AUTH_TO_PARENTTS_LUA_CONFIG_HTTP_ANONYMIZE_REMOVE_FROMTS_LUA_CONFIG_HTTP_ANONYMIZE_REMOVE_REFERERTS_LUA_CONFIG_HTTP_ANONYMIZE_REMOVE_USER_AGENTTS_LUA_CONFIG_HTTP_ANONYMIZE_REMOVE_COOKIETS_LUA_CONFIG_HTTP_ANONYMIZE_REMOVE_CLIENT_IPTS_LUA_CONFIG_HTTP_ANONYMIZE_INSERT_CLIENT_IPTS_LUA_CONFIG_HTTP_RESPONSE_SERVER_ENABLEDTS_LUA_CONFIG_HTTP_INSERT_SQUID_X_FORWARDED_FORTS_LUA_CONFIG_HTTP_SERVER_TCP_INIT_CWNDTS_LUA_CONFIG_HTTP_SEND_HTTP11_REQUESTS



TS_LUA_CONFIG_HTTP_CACHE_HTTPTS_LUA_CONFIG_HTTP_CACHE_CLUSTER_CACHE_LOCALTS_LUA_CONFIG_HTTP_CACHE_IGNORE_CLIENT_NO_CACHETS_LUA_CONFIG_HTTP_CACHE_IGNORE_CLIENT_CC_MAX_AGETS_LUA_CONFIG_HTTP_CACHE_IMS_ON_CLIENT_NO_CACHETS_LUA_CONFIG_HTTP_CACHE_IGNORE_SERVER_NO_CACHETS_LUA_CONFIG_HTTP_CACHE_CACHE_RESPONSES_TO_COOKIESTS_LUA_CONFIG_HTTP_CACHE_IGNORE_AUTHENTICATIONTS_LUA_CONFIG_HTTP_CACHE_CACHE_URLS_THAT_LOOK_DYNAMICTS_LUA_CONFIG_HTTP_CACHE_REQUIRED_HEADERSTS_LUA_CONFIG_HTTP_INSERT_REQUEST_VIA_STRTS_LUA_CONFIG_HTTP_INSERT_RESPONSE_VIA_STRTS_LUA_CONFIG_HTTP_CACHE_HEURISTIC_MIN_LIFETIMETS_LUA_CONFIG_HTTP_CACHE_HEURISTIC_MAX_LIFETIMETS_LUA_CONFIG_HTTP_CACHE_GUARANTEED_MIN_LIFETIMETS_LUA_CONFIG_HTTP_CACHE_GUARANTEED_MAX_LIFETIMETS_LUA_CONFIG_HTTP_CACHE_MAX_STALE_AGETS_LUA_CONFIG_HTTP_KEEP_ALIVE_NO_ACTIVITY_TIMEOUT_INTS_LUA_CONFIG_HTTP_KEEP_ALIVE_NO_ACTIVITY_TIMEOUT_OUTTS_LUA_CONFIG_HTTP_TRANSACTION_NO_ACTIVITY_TIMEOUT_INTS_LUA_CONFIG_HTTP_TRANSACTION_NO_ACTIVITY_TIMEOUT_OUTTS_LUA_CONFIG_HTTP_TRANSACTION_ACTIVE_TIMEOUT_OUTTS_LUA_CONFIG_HTTP_ORIGIN_MAX_CONNECTIONSTS_LUA_CONFIG_HTTP_CONNECT_ATTEMPTS_MAX_RETRIESTS_LUA_CONFIG_HTTP_CONNECT_ATTEMPTS_MAX_RETRIES_DEAD_SERVERTS_LUA_CONFIG_HTTP_CONNECT_ATTEMPTS_RR_RETRIESTS_LUA_CONFIG_HTTP_CONNECT_ATTEMPTS_TIMEOUTTS_LUA_CONFIG_HTTP_POST_CONNECT_ATTEMPTS_TIMEOUTTS_LUA_CONFIG_HTTP_DOWN_SERVER_CACHE_TIMETS_LUA_CONFIG_HTTP_DOWN_SERVER_ABORT_THRESHOLDTS_LUA_CONFIG_HTTP_CACHE_FUZZ_TIMETS_LUA_CONFIG_HTTP_CACHE_FUZZ_MIN_TIMETS_LUA_CONFIG_HTTP_DOC_IN_CACHE_SKIP_DNSTS_LUA_CONFIG_HTTP_BACKGROUND_FILL_ACTIVE_TIMEOUTTS_LUA_CONFIG_HTTP_RESPONSE_SERVER_STRTS_LUA_CONFIG_HTTP_CACHE_HEURISTIC_LM_FACTORTS_LUA_CONFIG_HTTP_CACHE_FUZZ_PROBABILITYTS_LUA_CONFIG_HTTP_BACKGROUND_FILL_COMPLETED_THRESHOLDTS_LUA_CONFIG_NET_SOCK_PACKET_MARK_OUTTS_LUA_CONFIG_NET_SOCK_PACKET_TOS_OUTTS_LUA_CONFIG_HTTP_INSERT_AGE_IN_RESPONSETS_LUA_CONFIG_HTTP_CHUNKING_SIZETS_LUA_CONFIG_HTTP_FLOW_CONTROL_ENABLEDTS_LUA_CONFIG_HTTP_FLOW_CONTROL_LOW_WATER_MARKTS_LUA_CONFIG_HTTP_FLOW_CONTROL_HIGH_WATER_MARKTS_LUA_CONFIG_HTTP_CACHE_RANGE_LOOKUPTS_LUA_CONFIG_HTTP_NORMALIZE_AE_GZIPTS_LUA_CONFIG_HTTP_DEFAULT_BUFFER_SIZETS_LUA_CONFIG_HTTP_DEFAULT_BUFFER_WATER_MARKTS_LUA_CONFIG_HTTP_REQUEST_HEADER_MAX_SIZETS_LUA_CONFIG_HTTP_RESPONSE_HEADER_MAX_SIZETS_LUA_CONFIG_HTTP_NEGATIVE_REVALIDATING_ENABLEDTS_LUA_CONFIG_HTTP_NEGATIVE_REVALIDATING_LIFETIMETS_LUA_CONFIG_HTTP_ACCEPT_ENCODING_FILTER_ENABLEDTS_LUA_CONFIG_SSL_HSTS_MAX_AGETS_LUA_CONFIG_SSL_HSTS_INCLUDE_SUBDOMAINSTS_LUA_CONFIG_HTTP_CACHE_OPEN_READ_RETRY_TIMETS_LUA_CONFIG_HTTP_CACHE_MAX_OPEN_READ_RETRIES

3.7. Plugins 167


TS_LUA_CONFIG_HTTP_CACHE_RANGE_WRITETS_LUA_CONFIG_HTTP_POST_CHECK_CONTENT_LENGTH_ENABLEDTS_LUA_CONFIG_HTTP_GLOBAL_USER_AGENT_HEADERTS_LUA_CONFIG_HTTP_AUTH_SERVER_SESSION_PRIVATETS_LUA_CONFIG_HTTP_SLOW_LOG_THRESHOLDTS_LUA_CONFIG_HTTP_CACHE_GENERATIONTS_LUA_CONFIG_BODY_FACTORY_TEMPLATE_BASETS_LUA_CONFIG_HTTP_CACHE_OPEN_WRITE_FAIL_ACTIONTS_LUA_CONFIG_HTTP_ENABLE_REDIRECTIONTS_LUA_CONFIG_HTTP_NUMBER_OF_REDIRECTIONSTS_LUA_CONFIG_HTTP_CACHE_MAX_OPEN_WRITE_RETRIESTS_LUA_CONFIG_LAST_ENTRY

TOP

ts.http.timeout_set

syntax: ts.http.timeout_set(CONFIG, NUMBER)


description: This function can be used to overwrite the timeout settings.

Here is an example:

function do_remap()ts.http.timeout_set(TS_LUA_TIMEOUT_DNS, 30) -- 30 secondsreturn 0

end

TOP

Timeout constants


TS_LUA_TIMEOUT_ACTIVETS_LUA_TIMEOUT_DNSTS_LUA_TIMEOUT_CONNECTTS_LUA_TIMEOUT_NO_ACTIVITY

TOP

ts.http.client_packet_mark_set

syntax: ts.http.client_packet_mark_set(NUMBER)


description: This function can be used to set packet mark for client connection.

Here is an example:



function do_remap()ts.http.client_packet_mark_set(0)return 0

end

TOP

ts.http.server_packet_mark_set

syntax: ts.http.server_packet_mark_set(NUMBER)


description: This function can be used to set packet mark for server connection.

TOP

ts.http.client_packet_tos_set

syntax: ts.http.client_packet_tos_set(NUMBER)


description: This function can be used to set packet tos for client connection.

TOP

ts.http.server_packet_tos_set

syntax: ts.http.server_packet_tos_set(NUMBER)


description: This function can be used to set packet tos for server connection.

TOP

ts.http.client_packet_dscp_set

syntax: ts.http.client_packet_dscp_set(NUMBER)


description: This function can be used to set packet dscp for client connection.

TOP

ts.http.server_packet_dscp_set

syntax: ts.http.server_packet_dscp_set(NUMBER)


description: This function can be used to set packet dscp for server connection.

TOP

3.7. Plugins 169


ts.http.enable_redirect

syntax: ts.http.enable_redirect(NUMBER)


decription: This function can be used to make transaction follow redirect

Here is an example:

function do_remap()ts.http.enable_redirect(1)return 0

end

TOP

ts.http.set_debug

syntax: ts.http.set_debug(NUMBER)


decription: This function can be used to enable debug log for the transaction

Here is an example:

function do_remap()ts.http.set_debug(1)return 0

end

TOP

ts.http.cntl_get

syntax: val = ts.http.cntl_get(CNTL_TYPE)


description: This function can be used to retireve the value of control channel.

val = ts.http.cntl_get(TS_LUA_HTTP_CNTL_GET_LOGGING_MODE)

TOP

ts.http.cntl_set

syntax: ts.http.cntl_set(CNTL_TYPE, BOOL)


description: This function can be used to set the value of control channel.

Here is an example:



function do_remap()ts.http.cntl_set(TS_LUA_HTTP_CNTL_SET_LOGGING_MODE, 0) -- do not log the

→˓requestreturn 0

end

TOP

Http control channel constants


TS_LUA_HTTP_CNTL_GET_LOGGING_MODETS_LUA_HTTP_CNTL_SET_LOGGING_MODETS_LUA_HTTP_CNTL_GET_INTERCEPT_RETRY_MODETS_LUA_HTTP_CNTL_SET_INTERCEPT_RETRY_MODE

TOP

ts.http.milestone_get

syntax: val = ts.http.milestone_get(MILESTONE_TYPE)


description: This function can be used to retireve the various milestone times. They are how long the transactiontook to traverse portions of the HTTP state machine. Each milestone value is a fractional number of seconds since thebeginning of the transaction.

val = ts.http.milestone_get(TS_LUA_MILESTONE_SM_START)

TOP

Milestone constants


TS_LUA_MILESTONE_UA_BEGINTS_LUA_MILESTONE_UA_FIRST_READTS_LUA_MILESTONE_UA_READ_HEADER_DONETS_LUA_MILESTONE_UA_BEGIN_WRITETS_LUA_MILESTONE_UA_CLOSETS_LUA_MILESTONE_SERVER_FIRST_CONNECTTS_LUA_MILESTONE_SERVER_CONNECTTS_LUA_MILESTONE_SERVER_CONNECT_ENDTS_LUA_MILESTONE_SERVER_BEGIN_WRITETS_LUA_MILESTONE_SERVER_FIRST_READTS_LUA_MILESTONE_SERVER_READ_HEADER_DONETS_LUA_MILESTONE_SERVER_CLOSETS_LUA_MILESTONE_CACHE_OPEN_READ_BEGINTS_LUA_MILESTONE_CACHE_OPEN_READ_ENDTS_LUA_MILESTONE_CACHE_OPEN_WRITE_BEGINTS_LUA_MILESTONE_CACHE_OPEN_WRITE_ENDTS_LUA_MILESTONE_DNS_LOOKUP_BEGIN

3.7. Plugins 171


TS_LUA_MILESTONE_DNS_LOOKUP_ENDTS_LUA_MILESTONE_SM_STARTTS_LUA_MILESTONE_SM_FINISHTS_LUA_MILESTONE_PLUGIN_ACTIVETS_LUA_MILESTONE_PLUGIN_TOTAL

TOP

ts.mgmt.get_counter

syntax: val = ts.mgmt.get_counter(RECORD_NAME)


description: This function can be used to retrieve the record value which has a counter type.

n = ts.mgmt.get_counter('proxy.process.http.incoming_requests')

TOP

ts.mgmt.get_int

syntax: val = ts.mgmt.get_int(RECORD_NAME)


description: This function can be used to retrieve the record value which has a int type.

TOP

ts.mgmt.get_float

syntax: val = ts.mgmt.get_float(RECORD_NAME)


description: This function can be used to retrieve the record value which has a float type.

TOP

ts.mgmt.get_string

syntax: val = ts.mgmt.get_string(RECORD_NAME)


description: This function can be used to retrieve the record value which has a string type.

name = ts.mgmt.get_string('proxy.config.product_name')

TOP



ts.stat_create

syntax: val = ts.stat_create(STAT_NAME, RECORDDATA_TYPE, PERSISTENT, SYNC)

context: global

description: This function can be used to create a statistics record given the name, data type, persistent requirement,and sync requirement. A statistics record table will be created with 4 functions to increment, decrement, get and setthe value.

stat:increment(value)stat:decrement(value)v = stat:get_value()stat:set_value(value)

Here is an example.

local test_stat;

function __init__(args)test_stat = ts.stat_create("test_stat",

TS_LUA_RECORDDATATYPE_INT,TS_LUA_STAT_PERSISTENT,TS_LUA_STAT_SYNC_COUNT)

end

function do_global_read_request()local value = test_stat:get_value()ts.debug(value)test_stat:increment(1)return 0

end

TOP

ts.stat_find

syntax: val = ts.stat_create(STAT_NAME)

context: global

description: This function can be used to find a statistics record given the name. A statistics record table will bereturned with 4 functions to increment, decrement, get and set the value. That is similar to ts.stat_create()

TOP

Todo

• ts.cache_xxx

• protocol

Currently when we use ts_lua as a global plugin, each global hook is using a separate lua state for the same transaction.This can be wasteful. Also the state cannot be reused for the same transaction across the global hooks. The alternativewill be to use a TXN_START hook to create a lua state first and then add each global hook in the lua script as transac-tion hook instead. But this will have problem down the road when we need to have multiple plugins to work together

3.7. Plugins 173


in some proper orderings. In the future, we should consider different approach, such as creating and maintaining thelua state in the ATS core.

TOP

Notes on Unit Testing Lua scripts for ATS Lua Plugin

Follow the steps below to use busted framework to run some unit tests on sample scripts and modules

• Build and install lua 5.1.5 using the source code from here - http://www.lua.org/ftp/lua-5.1.tar.gz

• Build and install luarocks 2.2.2 from here - https://github.com/keplerproject/luarocks/wiki/Download

• Run “sudo luarocks install busted”

• Run “sudo luarocks install luacov”

• “cd trafficserver/plugins/experimental/ts_lua/ci”

• Run “busted -c module_test.lua; luacov”. It will produce “luacov.report.out” containing the code coverage for“module.lua”

• Run “busted -c script_test.lua; luacov”. It will produce “luacov.report.out” containing the code coverage for“script.lua”

Reference for further information

• Busted - http://olivinelabs.com/busted/

• Specifications for asserts/mocks/stubs/etc inside busted framework: https://github.com/Olivine-Labs/luassert/tree/master/spec

• luacov - https://luarocks.org/modules/hisham/luacov

TOP

More docs

• https://github.com/portl4t/ts-lua

TOP

Webp Transform Plugin

This plugin converts jpeg and png images and transforms them into webp format. All response with content-type‘image/jpeg’ or ‘image/png’ will go through the transform. Content-type is changed to ‘image/webp’ on successfultransformation.

Installation

Add the following line to plugin.config:

webp_transform.so


http://www.lua.org/ftp/lua-5.1.tar.gz

https://github.com/keplerproject/luarocks/wiki/Download

http://olivinelabs.com/busted/

https://github.com/Olivine-Labs/luassert/tree/master/spec

https://github.com/Olivine-Labs/luassert/tree/master/spec

https://luarocks.org/modules/hisham/luacov

https://github.com/portl4t/ts-lua


Note

This plugin only supports jpeg and png and requires Magick++ from ImageMagick. Other image formats can easilybe supported.

XDebug Plugin

The XDebug plugin allows HTTP clients to debug the operation of the Traffic Server cache using the default X-Debugheader. The plugin is triggered by the presence of the X-Debug or the configured header in the client request. Thecontents of this header should be the names of the debug headers that are desired in the response. The XDebug pluginwill remove the X-Debug header from the client request and inject the desired headers into the client response.

XDebug is a global plugin. It is installed by adding it to the plugin.config file. It currently takes a single, optionalconfiguration option, --header. E.g.

–header=ATS-My-Debug

This overrides the default X-Debug header name.

Debugging Headers

The XDebug plugin is able to generate the following debugging headers:

Via If the Via header is requested, the XDebug plugin sets the :ts:cv:‘proxy.config.http.insert_response_via_str‘configuration variable to 3 for the request.

Diags If the Diags header is requested, the XDebug plugin enables the transaction specific diagnostics for the trans-action. This also requires that :ts:cv:‘proxy.config.diags.debug.enabled‘ is set to 1.

X-Cache-Key The X-Cache-Key header contains the URL that identifies the HTTP object in the Traffic Servercache. This header is particularly useful if a custom cache key is being used.

X-Cache The X-Cache header contains the results of any cache lookup.

Value Descriptionnone No cache lookup was attempted.miss The object was not found in the cache.hit-stale The object was found in the cache, but it was stale.hit-fresh The object was fresh in the cache.skipped The cache lookup was skipped.

X-Cache-Generation The cache generation ID for this transaction, as specified by the:ts:cv:‘proxy.config.http.cache.generation‘ configuration variable.

X-Milestones The X-Milestones header contains detailed information about how long the transactiontook to traverse portions of the HTTP state machine. The timing information is obtained from theTSHttpTxnMilestoneGet() API. Each milestone value is a fractional number of seconds since the be-ginning of the transaction.

Collapsed Forwarding Plugin

This is a plugin for Apache Traffic Server that allows to achieve effective connection collapse by blocking all but oneof the multiple concurrent requests for the same object from going to the Origin.

3.7. Plugins 175


Installation

To make this plugin available, you must either enable experimental plugins when building Traffic Server:


Or use tsxs to compile the plugin against your current Traffic Server build. To do this, you must ensure that:

1. Development packages for Traffic Server are installed.

2. The tsxs binary is in your path.

3. The version of this plugin you are building, and the version of Traffic Server against which you are building itare compatible.

Once those conditions are satisfied, enter the source directory for the plugin and perform the following:

make -f Makefile.tsxsmake -f Makefile.tsxs install

Using the plugin

This plugin functions as a per remap plugin, and it takes two optional arguments for specifying the delay betweensuccessive retries and a max number of retries.

To activate the plugin in per remap mode, in remap.config, simply append the below to the specific remap line:

@plugin=collapsed_forwarding.so @pparam=--delay=<delay> @pparam=--retries=<retries>

Functionality

Traffic Server plugin to allow collapsed forwarding of concurrent requests for the same object. This plugin is based onopen_write_fail_action feature, which detects cache open write failure on a cache miss and returns a 502 error alongwith a special @-header indicating the reason for 502 error. The plugin acts on the error by using an internal redirectfollow back to itself, essentially blocking the request until a response arrives, at which point, relies on read-while-writer feature to start downloading the object to all waiting clients. The following config parameters are assumed tobe set for this plugin to work:

:ts:cv:`proxy.config.http.cache.open_write_fail_action` 1:ts:cv:`proxy.config.cache.enable_read_while_writer` 1:ts:cv:`proxy.config.http.redirection_enabled` 1:ts:cv:`proxy.config.http.number_of_redirections` 10:ts:cv:`proxy.config.http.redirect_use_orig_cache_key` 1:ts:cv:`proxy.config.http.background_fill_active_timeout` 0:ts:cv:`proxy.config.http.background_fill_completed_threshold` 0

Additionally, given that collapsed forwarding works based on cache write lock failure detection, the plugin requirescache to be enabled and ready. On a restart, Traffic Server typically takes a few seconds to initialize the cachedepending on the cache size and number of dirents. While the cache is not ready yet, collapsed forwarding can notdetect the write lock contention and so can not work. The setting :ts:cv:‘proxy.config.http.wait_for_cache‘ may beenabled which allows blocking incoming connections from being accepted until cache is ready.



Description

Traffic Server has been affected severely by the Thundering Herd problem caused by its inability to do effectiveconnection collapse of multiple concurrent requests for the same segment. This is especially critical when TrafficServer is used as a solution to use cases such as delivering a large scale video live streaming. This problem results in aspecific behavior where multiple number of requests for the same file are leaked upstream to the Origin layer chokingthe upstream bandwidth due to the duplicated large file downloads or process intensive file at the Origin layer. Thisultimately can cause stability problems on the origin layer disrupting the overall network performance.

Traffic Server supports several kind of connection collapse mechanisms including Read-While-Writer (RWW), Stale-While-Revalidate (SWR) etc each very effective dealing with a majority of the use cases that can result in the Thun-dering herd problem.

For a large scale Video Streaming scenario, there’s a combination of a large number of revalidations (e.g.media playlists) and cache misses (e.g. media segments) that occur for the same file. Traffic Server’sRWW works great in collapsing the concurrent requests in such a scenario, however, as described in_admin-configuration-reducing-origin-requests, Traffic Server’s implementation of RWW has asignificant limitation, which restricts its ability to invoke RWW only when the response headers are already received.This means that any number of concurrent requests for the same file that are received before the response headersarrive are leaked upstream, which can result in a severe Thundering herd problem, depending on the network latencies(which impact the TTFB for the response headers) at a given instant of time.

To address this limitation, Traffic Server supports a few Cache tuning solutions, such as Open Read Retry, and anew feature called Open Write Fail action from 6.0. To understand how these approaches work, it is important tounderstand the high level flow of how Traffic Server handles a GET request.

On receiving a HTTP GET request, Traffic Server generates the cache key (basically, a hash of the request URL) andlooks up for the directory entry (dirent) using the generated index. On a cache miss, the lookup fails and Traffic Serverthen tries to just get a write lock for the cache object and proceeds to the origin to download the object. On the Otherhand, if the lookup is successful, meaning, the dirent exists for the generated cache key, Traffic Server tries to obtaina read lock on the cache object to be able to serve it from the cache. If the read lock is not successful (possibly, dueto the fact that the object’s being written to at that same instant and the response headers are not in the cache yet),Traffic Server then moves to the next step of trying to obtain an exclusive write lock. If the write lock is already heldexclusively by another request (transaction), the attempt fails and at this point Traffic Server simply disables the cacheon that transaction and downloads the object in a proxy-only mode:

1). Cache Lookup (lookup for the dirent using the request URL as cache key).1.1). If lookup fails (cache miss), goto (3).1.2). If lookup succeeds, try to obtain a read lock, goto (2).

2). Open Cache Read (try to obtain read lock)2.1). If read lock succeeds, serve from cache, goto (4).2.2). If read lock fails, goto (3).

3). Open Cache Write (try to obtain write lock).3.1). If write lock succeeds, download the object into cache and to the client in

→˓parallel3.2). If write lock fails, disable cache, and download to the client in a proxy-

→˓only mode.4). Done

As can be seen above, if a majority of concurrent requests arrive before response headers are received, they hit (2.2)and (3.2) above. Open Read Retry can help to repeat (2) after a configured delay on 2.2, thereby increasing the chancesfor obtaining a read lock and being able to serve from the cache.

However, the Open Read Retry can not help with the concurrent requests that hit (1.1) above, jumping to (3) directly.Only one such request will be able to obtain the exclusive write lock and all other requests are leaked upstream. Thisis where, the recently developed Traffic Server feature Open Write Fail Action will help. The feature detects the writelock failure and can return a stale copy for a Cache Revalidation or a 5xx status code for a Cache Miss with a special

3.7. Plugins 177


internal header <@Ats-Internal> that allows a TS plugin to take other special actions depending on the use-case.

collapsed_forwarding plugin catches that error in SEND_RESPONSE_HDR_HOOK and performs an internal3xx Redirect back to the same host, the configured number of times with the configured amount of delay betweenconsecutive retries, allowing to be able to initiate RWW, whenever the response headers are received for the requestthat was allowed to go to the Origin.

More details are available at

https://docs.trafficserver.apache.org/en/6.0.x/admin/http-proxy-caching.en.html#reducing-origin-server-requests-avoiding-the-thundering-herd

AuthProxy Delegates the authorization decision of a request to an external HTTP service.

AWS S3 Authentication Support for Amazon S3 authentication features.

Background Fetch Proactively fetch content from Origin in a way that it will fill the object into cache.

Balancer Balances requests across multiple origin servers.

Buffer Upload Buffers POST data before connecting to the Origin server.

Cache Key Manipulation Allows some common cache key manipulations based on various HTTP request elements.

Cache Promote Provides additional control over when an object should be allowed into the cache.

Combo Handler Provides an intelligent way to combine multiple URLs into a single URL, and have Apache TrafficServer combine the components into one response.

Epic Emits Traffic Server metrics in a format that is consumed by the Epic Network Monitoring System.

Escalate Escalate: when the origin returns specific status codes, retry the request at a secondary origin (failover/fail-action)

ESI Implements the Edge Side Includes (ESI) specification.

Generator Generate arbitrary response data.

GeoIP ACL Deny or allow requests based on the source IP geo-location.

HIPES Adds support for HTTP Pipes.

Memcache Implements the memcache protocol for cache contents.

Metalink Implements the Metalink download description format in order to try not to download the same file twice.

MP4 MP4 streaming media.

MySQL Remap Allows dynamic remaps from a MySQL database.

Signed URLs Adds support for verifying URL signatures for incoming requests to either deny or redirect access.

SSL Headers Populate request headers with SSL session information.

Stale While Revalidate Refresh content asynchronously while serving stale data.

TS Lua Allows plugins to be written in Lua instead of C code.

WebP Transform Converts jpeg and png images to webp format.

XDebug Allows HTTP clients to debug the operation of the Traffic Server cache using the X-Debug header.

Collapsed-Forwarding Allows to Collapse multiple Concurrent requests by downloading once from the Origin andserving all clients in parallel.


https://docs.trafficserver.apache.org/en/6.0.x/admin/http-proxy-caching.en.html#reducing-origin-server-requests-avoiding-the-thundering-herd


Event and Error Monitoring

Logging

Traffic Server generates log files that contain information about every request it receives and every error it detects.This chapter will examine the various log features, the configuration formats and also examine the various pre-definedlog formats that are available.

Understanding Traffic Server Log Files

Traffic Server records information about every transaction (or request) it processes and every error it detects in logfiles. Traffic Server keeps three types of log files:

• Error log files record information about why a particular transaction was in error.

• Event log files (also called access log files) record information about the state of each transaction Traffic Serverprocesses.

• System log files record system information, including messages about the state of Traffic Server and er-rors/warnings it produces. This kind of information might include a note that event log files were rolled, awarning that cluster communication timed out, or an error indicating that Traffic Server was restarted.

All system information messages are logged with the system-wide logging facility syslog under the dae-mon facility. The syslog.conf(5) configuration file (stored in the /etc directory) specifies where thesemessages are logged. A typical location is /var/log/messages (Linux).

The syslog(8) process works on a system-wide basis, so it serves as the single repository for messages fromall Traffic Server processes (including traffic_server, traffic_manager, and traffic_cop).

System information logs observe a static format. Each log entry in the log contains information about the dateand time the error was logged, the hostname of the Traffic Server that reported the error, and a description of theerror or warning.

Refer to Error Messages for a list of the messages logged by Traffic Server.

By default, Traffic Server creates both error and event log files and records system information in sys-tem log files. You can disable event logging and/or error logging by setting the configuration variable:ts:cv:‘proxy.config.log.logging_enabled‘ in records.config to one of the following values:

Value Description0 Disable both event and error logging.1 Enable error logging only.2 Enable event logging only.3 Enable both event and error logging.

By analyzing the log files, you can determine how many people use the Traffic Server cache, how much informationeach person requested, what pages are most popular, and so on. Traffic Server supports several standard log fileformats, such as Squid and Netscape, as well as user-defined custom formats. You can analyze the standard formatlog files with off-the-shelf analysis packages. To help with log file analysis, you can separate log files so they containinformation specific to protocol or hosts. You can also configure Traffic Server to roll log files automatically at specificintervals during the day or when they reach a certain size.

Managing Logs

Traffic Server enables you to control where event log files are located and how much space they can consume. Addi-tionally, you can specify how to handle low disk space in the logging directory.

3.8. Event and Error Monitoring 179


Choosing the Logging Directory

By default, Traffic Server writes all event log files in the logs directory located in the directory where you in-stalled Traffic Server. To change this location, adjust the value of :ts:cv:‘proxy.config.log.logfile_dir‘ in records.config. You will need to either restart Traffic Server or run the command traffic_line -x for changes to takeeffect.

Controlling Logging Space

Traffic Server enables you to control the amount of disk space that the logging directory can consume. This allowsthe system to operate smoothly within a specified space window for a long period of time. After you establish a spacelimit, Traffic Server continues to monitor the space in the logging directory. When the free space dwindles to theheadroom limit, it enters a low space state and takes the following actions:

• If the autodelete option (discussed in Rolling Logs) is enabled, then Traffic Server identifies previously-rolledlog files (log files with the .old extension). It starts deleting files one by one, beginning with the oldest file,until it emerges from the low state. Traffic Server logs a record of all deleted files in the system error log.

• If the autodelete option is disabled or there are not enough old log files to delete for the system to emergefrom its low space state, then Traffic Server issues a warning and continues logging until space is exhausted.When available space is consumed, event logging stops. Traffic Server resumes event logging when enoughspace becomes available for it to exit the low space state. To make space available, either explicitly increase thelogging space limit or remove files from the logging directory manually.

You can run a cron(8) script in conjunction with Traffic Server to automatically remove old log files from thelogging directory before Traffic Server enters the low space state. Relocate the old log files to a temporary partition,where you can run a variety of log analysis scripts. Following analysis, either compress the logs and move to anarchive location, or simply delete them.

Setting Log File Management Options

To set log management options, follow the steps below:

1. In the records.config file, edit the following variables

• :ts:cv:‘proxy.config.log.max_space_mb_for_logs‘

• :ts:cv:‘proxy.config.log.max_space_mb_headroom‘


Rolling Logs

Traffic Server provides automatic log file rolling. At specific intervals during the day or when log files reach a certainsize, Traffic Server closes its current set of log files and opens new log files. Depending on the amount of traffic yourservers are exposed to, you may find that increasing the frequency of log rolling is beneficial, or even necessary, tomaintain manageable log file sets. Traffic Server nodes processing moderately high levels of traffic may want to startby rolling logs every six hours, and adjusting from there.

Log file rolling offers the following benefits:

• It defines an consistent interval over which log analysis can be performed.

• It keeps any single log file from becoming too large and helps to keep the logging system within the specifiedspace limits.



• It provides an easy way to identify files that are no longer being used so that an automated script can clean thelogging directory and run log analysis programs.

Rolled Log Filename Format

Traffic Server provides a consistent naming scheme for rolled log files that enables you to easily identify log files.When Traffic Server rolls a log file, it saves and closes the old file before it starts a new file. Traffic Server renamesthe old file to include the following information:

• The format of the file (such as squid.log).

• The hostname of the Traffic Server that generated the log file.

• Two timestamps separated by a hyphen (-). The first timestamp is a lower bound for the timestamp of the firstrecord in the log file. The lower bound is the time when the new buffer for log records is created. Under lowload, the first timestamp in the filename can be different from the timestamp of the first entry. Under normalload, the first timestamp in the filename and the timestamp of the first entry are similar. The second timestampis an upper bound for the timestamp of the last record in the log file (this is normally the rolling time).

• The suffix .old, which makes it easy for automated scripts to find rolled log files.

Timestamps have the following format:

%Y%M%D.%Hh%Mm%Ss-%Y%M%D.%Hh%Mm%Ss

The following table describes the format:

Format Description Sample%Y The year in four-digit format. 2000%M The month in two-digit format, from 01-12. 07%D The day in two-digit format, from 01-31. 19%H The hour in two-digit format, from 00-23. 21%M The minute in two-digit format, from 00-59. 52%S The second in two-digit format, from 00-59. 36

The following is an example of a rolled log filename:

squid.log.mymachine.20110912.12h00m00s-20000913.12h00m00s.old

The logging system buffers log records before writing them to disk. When a log file is rolled, the log buffer might bepartially full. If it is, then the first entry in the new log file will have a timestamp earlier than the time of rolling. Whenthe new log file is rolled, its first timestamp will be a lower bound for the timestamp of the first entry.

For example, suppose logs are rolled every three hours, and the first rolled log file is:


If the lower bound for the first entry in the log buffer at 3:00:00 is 2:59:47, then the next log file will have the followingtimestamp when rolled:


The contents of a log file are always between the two timestamps. Log files do not contain overlapping entries, even ifsuccessive timestamps appear to overlap.



Rolling Intervals

Log files are rolled at specific intervals relative to a given hour of the day. Three options may be used to control whenlog files are rolled:

• A file size threshold, which will prevent any individual log from growing too large.

• The offset hour, which is an hour between 0 (midnight) and 23.

• The rolling interval.

Both the offset hour and the rolling interval determine when log file rolling starts. Rolling occurs every rolling intervaland at the offset hour. For example, if the rolling interval is six hours and the offset hour is 0 (midnight), then the logswill roll at midnight (00:00), 06:00, 12:00, and 18:00 each day. If the rolling interval is 12 hours and the offset hour is3, then logs will roll at 03:00 and 15:00 each day.

To set log file rolling options and/or configure Traffic Server to roll log files when they reach a certain size, adjust thefollowing settings in records.config:

1. Enable log rolling with :ts:cv:‘proxy.config.log.rolling_enabled‘.

CONFIG proxy.config.log.rolling_enabled INT 1

2. Configure the upper limit on log file size with :ts:cv:‘proxy.config.log.rolling_size_mb‘.

CONFIG proxy.config.log.rolling_size_mb INT 1024

3. Set the offset hour with :ts:cv:‘proxy.config.log.rolling_offset_hr‘.

CONFIG proxy.config.log.rolling_offset_hr INT 0

4. Set the interval (in seconds) with :ts:cv:‘proxy.config.log.rolling_interval_sec‘.

CONFIG proxy.config.log.rolling_interval_sec INT 21600


You can fine-tune log file rolling settings for a custom log file in the LogObject specification in logs_xml.config.The custom log file uses the rolling settings in its LogObject, which override the default settings you specify in TrafficManager or records.config described above.

Separating Logs by Origin

The default log_hosts.config file is located in the Traffic Server config directory. To record HTTP transac-tions for different origin servers in separate log files, you must specify the hostname of each origin server on a separateline in log_hosts.config. For example, if you specify the keyword sports, then Traffic Server records allHTTP transactions from sports.yahoo.com and www.foxsports.com in a log file called squid-sports.log (if the Squid format is enabled).

: If Traffic Server is clustered and you enable log file collation, then you should use the same log_hosts.configfile on every Traffic Server node in the cluster.

To edit the log hosts list:

1. Enter the hostname of each origin server on a separate line in log_hosts.config.



webserver1webserver2webserver3

2. Run the command traffic_ctl config reload to apply the changes.

Log Formats

This document provides a reference for all the different logging formats Traffic Server supports. Rather than justreading about those formats, you may also want to try our online event log builder for an interactive way of buildingand understanding log formats.

Binary or ASCII?

You can configure Traffic Server to create event log files in either of the following:

ASCII These files are human-readable and can be processed using standard, off-the-shelf log analysis tools. However,Traffic Server must perform additional processing to create the files in ASCII, which mildly impacts systemoverhead. ASCII files also tend to be larger than the equivalent binary files. By default, ASCII log files have a.log filename extension.

Binary These files generate lower system overhead and generally occupy less space on the disk than ASCII files(depending on the type of information being logged). However, you must use a converter application before youcan read or analyze binary files via standard tools. By default, binary log files use a .blog filename extension.

While binary log files typically require less disk space, there are exceptions.

For example, the value 0 (zero) requires only one byte to store in ASCII, but requires four bytes when stored as abinary integer. Conversely, if you define a custom format that logs IP addresses, then a binary log file would onlyrequire four bytes of storage per 32-bit address. However, the same IP address stored in dot notation would requirearound 15 characters (bytes) in an ASCII log file.

It is wise to consider the type of data that will be logged before you select ASCII or binary for your log files, if yourdecision is being driven by storage space concerns. For example, you might try logging to both formats simultaneouslyfor a representative period of time and compare the storage requirements of the two logs to determine whether one orthe other provides any measurable savings.

Defining Log Objects

To perform any logging at all on your Traffic Server nodes, you must have at least one LogObject defined inlogs_xml.config. These definitions configure what logs will be created, the format they will use (covered inthe sections Standard Formats and Custom Formats), any filters which may be applied to events before logging them,and when or how the logs will be rolled.

Log Filters

LogFilter objects, configured in logs_xml.config allow you to create filters, which may be applied to LogObjectdefinitions, limiting the types of entries which will be included in the log output. This may be useful if your TrafficServer nodes receive many events which you have no need to log or analyze.


http://trafficserver.apache.org/logbuilder/


Standard Formats

The standard log formats include Squid, Netscape Common, Netscape extended, and Netscape Extended-2. Thestandard log file formats can be analyzed with a wide variety of off-the-shelf log-analysis packages. You should useone of the standard event log formats unless you need information that these formats do not provide.

These formats may be used by enabling the :ts:cv:‘proxy.config.log.custom_logs_enabled‘ setting in records.config and adding appropriate entries to logs_xml.config.

Squid

The following figure shows a sample log entry in a Squid log file.

3.11: Sample log entry in squid.log

Field Sym-bol

Description

1 cqtq The client request timestamp in Squid format. The time of the client request in seconds sinceJanuary 1, 1970 UTC (with millisecond resolution).

2 ttms The time Traffic Server spent processing the client request. The number of milliseconds betweenthe time the client established the connection with Traffic Server and the time Traffic Server sentthe last byte of the response back to the client.

3 chi The IP address of the client’s host machine.4 crc/psscThe cache result code; how the cache responded to the request: HIT, MISS, and so on. Cache

result codes are described in Cache Result Codes. The proxy response status code (HTTP responsestatus code from Traffic Server to client).

5 psql The length of the Traffic Server response to the client in bytes, including headers and content.6 cqhm The client request method: GET, POST, and so on.7 cauc The client request canonical URL; blanks and other characters that might not be parsed by log

analysis tools are replaced by escape sequences. The escape sequence is a percentage sign followedby the ASCII code number of the replaced character in hex.

8 caun The username of the authenticated client. A hyphen (-) means that no authentication was required.9 phr/pqsnThe proxy hierarchy route. The route Traffic Server used to retrieve the object.10 psct The proxy response content type. The object content type taken from the Traffic Server response

header.

Netscape Common

The following figure shows a sample log entry in a Netscape Common log file.



3.12: Sample log entry in common.log

Field Symbol Description1 chi The IP address of the client’s host machine.2 – This hyphen (-) is always present in Netscape log entries.3 caun The authenticated client username. A hyphen (-) means no authentication was required.4 cqtd The date and time of the client request, enclosed in brackets.5 cqtx The request line, enclosed in quotes.6 pssc The proxy response status code (HTTP reply code).7 pscl The length of the Traffic Server response to the client in bytes.

Netscape Extended

The following figure shows a sample log entry in a Netscape Extended log file.

3.13: Sample log entry in extended.log

In addition to field 1-7 from the Netscape Common log format above, the Extended format also adds the followingfields:

Field Sym-bol

Description

8 sssc The origin server response status code.9 sshl The server response transfer length; the body length in the origin server response to Traffic Server,

in bytes.10 cqbl The client request transfer length; the body length in the client request to Traffic Server, in bytes.11 pqbl The proxy request transfer length; the body length in the Traffic Server request to the origin server.12 cqhl The client request header length; the header length in the client request to Traffic Server.13 pshl The proxy response header length; the header length in the Traffic Server response to the client.14 pqhl The proxy request header length; the header length in Traffic Server request to the origin server.15 sshl The server response header length; the header length in the origin server response to Traffic Server.16 tts The time Traffic Server spent processing the client request; the number of seconds between the time

that the client established the connection with Traffic Server and the time that Traffic Server sent thelast byte of the response back to the client.



Netscape Extended2

The following figure shows a sample log entry in a Netscape Extended2 log file.

3.14: Sample log entry in extended2.log

In addition to field 1-16 from the log formats above, the Extended2 format also adds the following fields:

Field Sym-bol

Description

17 phr The proxy hierarchy route; the route Traffic Server used to retrieve the object.18 cfsc The client finish status code: FIN if the client request completed successfully or INTR if the client

request was interrupted.19 pfsc The proxy finish status code: FIN if the Traffic Server request to the origin server completed

successfully or INTR if the request was interrupted.20 crc The cache result code; how the Traffic Server cache responded to the request: HIT, MISS, and so

on. Cache result codes are listed in Cache Result Codes.

Custom Formats

Defining a Format

Custom logging formats in Traffic Server are defined by editing logs_xml.config and adding new LogFormatentries for each format you wish to define. The syntax is fairly simple: every LogFormat element should containat least two child elements (additional elements are used for features such as log summarization and are coveredelsewhere):

• A <Name> which contains an arbitrary string (using only the allowed characters: [a-z0-9]) naming yourcustom format.

• A <Format> which defines the fields that will populate each entry in the custom logs, as well as the order inwhich they appear.

A very simple example format, which contains only the timestamp of when the event began and the canonical URL ofthe request, and named myformat would be written as follows:

<LogFormat><Name = "myformat"/><Format = "%<cqtq> %<cauc>"/>

</LogFormat>

You may include as many custom field codes as you wish. The full list of codes available can be found in CustomLogging Fields. You may also include any literal characters in your format. For example, if we wished to separate thetimestamp and canonical URL in our customer format above with a slash instead of a space, or even a slash surroundedby spaces, we could do so by just adding the desired characters to the format string:



%<cqtq> / %<cauc>

You may define as many custom formats as you wish. To apply changes to custom formats, you will need to run thecommand traffic_line -x after saving your changes to logs_xml.config.

Custom Logging Fields

The following list describes Traffic Server custom logging fields.

{HTTP header field name}cqh Logs the information in the requested field of the client request HTTPheader. For example, %<{Accept-Language}cqh> logs the Accept-Language: field in client requestheaders.

: ecqh is the URL-encoded version of this map

{HTTP header field name}pqh Logs the information in the requested field of the proxy request HTTPheader. For example, %<{Authorization}pqh> logs the Authorization: field in proxy request head-ers.

: epqh is the URL-encoded version of this map

{HTTP header field name}psh Logs the information in the requested field of the proxy response HTTPheader. For example, %<{Retry-After}psh> logs the Retry-After: field in proxy response headers.

: epsh is the URL-encoded version of this map

{HTTP header field name}ssh Logs the information in the requested field of the server response HTTPheader. For example, %<{Age}ssh> logs the Age: field in server response headers.

: essh is the URL-encoded version of this map

{HTTP header field name}cssh Logs the information in the requested field of the cached server responseHTTP header. For example, %<{Age}cssh> logs the Age: field in the cached server response headers.

: ecssh is the URL-encoded version of this map

caun The client authenticated username; result of the RFC931/ident lookup of the client username.

cfsc The client finish status code; specifies whether the client request to Traffic Server was successfully completed(FIN) or interrupted (INTR).

chi The IP address of the client’s host machine.

chih The IP address of the client’s host machine in hexadecimal.

hii This is the incoming (interface) IP address for Traffic Server, in otherwords this is the IP address the clientconnected to.

hiih The the incoming (interface) IP address in hexadecimal.

chp The port number of the client’s host machine.



cqbl The client request transfer length; the body length in the client request to Traffic Server (in bytes).

cqhl The client request header length; the header length in the client request to Traffic Server.

cqhm The HTTP method in the client request to Traffic Server: GET, POST, and so on (subset of cqtx).

cqhv The client request HTTP version.

cqpv The client request protocol and version.

cqtd The client request timestamp. Specifies the date of the client request in the format yyyy-mm-dd, where yyyy isthe 4-digit year, mm is the 2-digit month, and dd is the 2-digit day.

cqtn The client request timestamp; date and time of the client’s request (in the Netscape timestamp format).

cqtq The time of the client request since January 1, 1970 UTC (epoch), with millisecond resolution.

cqts The time of the client request since January 1, 1970 UTC (EPOCH), with second resolution.

cqtt The client request timestamp. The time of the client request in the format hh:mm:ss, where hh is the two-digithour in 24-hour format, mm is the two-digit minutes value, and ss is the 2-digit seconds value (for example,16:01:19).

cqtr The TCP reused status; indicates if this client request went through an already established connection.

cqssl The SSL client request status indicates if this client connection is over SSL.

cqssr The SSL session ticket reused status; indicates if this request hit the SSL session ticket and avoided a full SSLhandshake.

cqssv The SSL version used to communicate with the client.

cqssc The cipher used by Traffic Server to communicate with the client over SSL.

cqtx The full HTTP client request text, minus headers; for example,

GET http://www.company.com HTTP/1.0

In reverse proxy mode, Traffic Server logs the rewritten/mapped URL (according to the rules in remap.config), _not_ the pristine/unmapped URL.

cqu The universal resource identifier (URI) of the request from client to Traffic Server (subset of cqtx ).

In reverse proxy mode, Traffic Server logs the rewritten/mapped URL (according to the rules in remap.config), _not_ the pristine/unmapped URL.

cquc The client request canonical URL. This differs from cqu in that blanks (and other characters that might notbe parsed by log analysis tools) are replaced by escape sequences. The escape sequence is a percentage signfollowed by the ASCII code number in hex.

See cquuc.

cqup The client request URL path; specifies the argument portion of the URL (everything after the host). Forexample, if the URL is http://www.company.com/images/x.gif, then this field displays /images/x.gif

See cquup.

cqus The client request URL scheme.

cquuc The client request unmapped URL canonical. This field records a URL before it is remapped (reverse proxymode).

cquup The client request unmapped URL path. This field records a URL path before it is remapped (reverse proxymode).



cquuh The client request unmapped URL host. This field records a URL’s host before it is remapped (reverse proxymode).

cluc The cache lookup URL, or cache key, for the client request. This URL is canonicalized as well.

crat The Retry-After time in seconds, if specified by the origin server.

crc The cache result code; specifies how the cache responded to the request (HIT, MISS, and so on).

chm The cache hit-miss status, specifying which level of the cache this was served out of. This is useful for exampleto show whether it was a RAM cache vs disk cache hit. Future versions of the cache will support more levels,but right now it only supports RAM (HIT_RAM) vs rotational disk (HIT_DISK).

csscl The cached response length (in bytes) from origin server to Traffic Server.

csshl The cached header length in the origin server response to Traffic Server (in bytes).

csshv The cached server response HTTP version (1.0, 1.1, etc.).

csssc The cached HTTP response status code from origin server to Traffic Server.

cwr The cache write result (-, WL_MISS, INTR`, ERR or FIN)

cwtr The cache write transform result

fsiz The size of the file (n bytes) as seen by the origin server.

{Milestone field name}ms The timestamp in milliseconds of a specific milestone for this request. seeTSHttpTxnMilestoneGet() for milestone names.

{Milestone field name1-Milestone field name2}msdms The difference in milliseconds of be-tween two milestones. see TSHttpTxnMilestoneGet() for milestone names.

pfsc The proxy finish status code; specifies whether the Traffic Server request to the origin server was successfullycompleted (FIN), interrupted (INTR) or timed out (TIMEOUT).

phn The hostname of the Traffic Server that generated the log entry in collated log files.

phi The IP of the Traffic Server that generated the log entry in collated log files.

phr The proxy hierarchy route; the route Traffic Server used to retrieve the object.

php The TCP port number that Traffic Server served this request from.

piid The plugin ID for the transaction. This is set for plugin driven transactions viaTSHttpConnectWithPluginId().

pitag The plugin tag for the transaction. This is set for plugin driven transactions viaTSHttpConnectWithPluginId().

pqbl The proxy request transfer length; the body length in Traffic Server’s request to the origin server.

pqhl The proxy request header length; the header length in Traffic Server’s request to the origin server.

pqsi The proxy request server IP address (0 on cache hits and parent-ip for requests to parent proxies).

pqsn The proxy request server name; the name of the server that fulfilled the request.

pscl The length of the Traffic Server response to the client (in bytes).

psct The content type of the document from server response header: (for example, img/gif ).

pshl The header length in Traffic Server‘s response to the client.

psql The proxy response transfer length in Squid format (includes header and content length).

pssc The HTTP response status code from Traffic Server to the client.

pqssl Indicates whether the connection from Traffic Server to the origin was over SSL or not.



sca The number of attempts in the transaction Traffic Server tries to connect to the origin server.

shi The IP address resolved from the DNS name lookup of the host in the request. For hosts with multiple IPaddresses, this field records the IP address resolved from that particular DNS lookup.

This can be misleading for cached documents. For example: if the first request was a cache miss and came fromIP1 for server S and the second request for server S resolved to IP2 but came from the cache, then the log entryfor the second request will show IP2.

shn The hostname of the origin server.

sscl The response length (in bytes) from origin server to Traffic Server.

sshl The header length (in bytes) in the origin server response to Traffic Server.

sshv The server response HTTP version (1.0, 1.1, etc.).

sssc The HTTP response status code from origin server to Traffic Server.

stms The time spent accessing the origin (in milliseconds); the time is measured from the time the connection withthe origin is established to the time the connection is closed.

stmsh Same as stms but in hexadecimal.

stmsf The time Traffic Server spends accessing the origin as a fractional number of seconds. That is, the time isformated as a floating-point number, instead of an integer as in stms.

For example: if the time is 1500 milliseconds, then this field displays 1.5 while the stms field displays 1500and the sts field displays 1.

sts The time Traffic Server spends accessing the origin, in seconds.

sstc The number of transactions between Traffic Server and the origin server from a single server session. A valuegreater than 0 indicates connection reuse.

ttms The time Traffic Server spends processing the client request; the number of milliseconds between the timethe client establishes the connection with Traffic Server and the time Traffic Server sends the last byte of theresponse back to the client.

ttmsh Same as ttms but in hexadecimal.

ttmsf The time Traffic Server spends processing the client request as a fractional number of seconds. Time isspecified in millisecond resolution; however, instead of formatting the output as an integer (as with ttms), thedisplay is formatted as a floating-point number representing a fractional number of seconds.

For example: if the time is 1500 milliseconds, then this field displays 1.5 while the ttms field displays 1500and the tts field displays 1.

tts The time Traffic Server spends processing the client request; the number of seconds between the time at whichthe client establishes the connection with Traffic Server and the time at which Traffic Server sends the last byteof the response back to the client.

Custom Field Cross-Reference

The following sections illustrate the correspondence between Traffic Server custom logging fields and standard loggingfields for the Squid and Netscape formats.

Squid

The following is a list of the Squid logging fields and the corresponding logging field symbols.



Squid Field Symbolstime cqtselapsed ttmsclient chiaction/code crc/psscsize psqlmethod cqhmurl cqucident caunhierarchy/from phr/pqsncontent psct

This is the equivalent XML configuration for the log above:

<LogFormat><Name = "squid"/><Format = "%<cqtq> %<ttms> %<chi> %<crc>/%<pssc> %<psql> %<cqhm> %<cquc>

%<caun> %<phr>/%<pqsn> %<psct>"/></LogFormat>

Netscape Common

The following is a list of the Netscape Common logging fields and the corresponding Traffic Server logging fieldsymbols.

Netscape Common Field Symbolshost chiusr caun[time] [cqtn]“req” “cqtx“s1 psscc1 pscl


<LogFormat><Name = "common"/><Format = "%<chi> - %<caun> [%<cqtn>] \"%<cqtx>\" %<pssc> %<pscl>"/>

</LogFormat>

Netscape Extended

The following table lists the Netscape Extended logging fields and the corresponding Traffic Server logging fieldsymbols.



Netscape Extended Field Symbolshost chiusr caun[time] [cqtn]“req” “cqtx“s1 psscc1 pscls2 ssscc2 ssclb1 cqblb2 pqblh1 cqhlh2 pshlh3 pqhlh4 sshlxt tts


<LogFormat><Name = "extended"/><Format = "%<chi> - %<caun> [%<cqtn>] \"%<cqtx>\" %<pssc> %<pscl>

%<sssc> %<sscl> %<cqbl> %<pqbl> %<cqhl> %<pshl> %<pqhl> %<sshl> %<tts>"/></LogFormat>

Netscape Extended-2

The following is a list of the Netscape Extended-2 logging fields and the corresponding Traffic Server logging fieldsymbols.

Netscape Extended-2 Field Symbolshost chiusr caun[time] [cqtn]"req" "cqtx"s1 psscc1 pscls2 ssscc2 ssclb1 cqblb2 pqblh1 cqhlh2 pshlh3 pqhlh4 sshlxt ttsroute phrpfs cfscss pfsccrc crc




<LogFormat><Name = "extended2"/><Format = "%<chi> - %<caun> [%<cqtn>] \"%<cqtx>\" %<pssc> %<pscl>

%<sssc> %<sscl> %<cqbl> %<pqbl> %<cqhl> %<pshl> %<pqhl> %<sshl> %<tts> %→˓<phr> %<cfsc> %<pfsc> %<crc>"/></LogFormat>

Log Field Slicing

It is sometimes desirable to slice a log field to limit the length of a given log field’s output.

Log Field slicing can be specified as below:

%<field[start:end]>%<{field}container[start:end]>

Omitting the slice notation defaults to the entire log field.

Slice notation only applies to a log field that is of type string and can not be applied to IPs or timestamp which areconverted to string from integer.

The below slice specifiers are allowed.

[start:end] Log field value from start through end-1

[start:] Log field value from start through the rest of the string

[:end] Log field value from the beginning through end-1

[:] Default - entire Log field

Some examples below

'%<cqup>' // the whole characters of <cqup>.'%<cqup>[:]' // the whole characters of <cqup>.'%<cqup[0:30]>' // the first 30 characters of <cqup>.'%<cqup[-10:]>' // the last 10 characters of <cqup>.'%<cqup[:-5]>' // everything except the last 5 characters of <cqup>.

Cache Result Codes

The following table describes the cache result codes in Squid and Netscape log files.

TCP_HIT A valid copy of the requested object was in the cache and Traffic Server sent the object to the client.

TCP_MISS The requested object was not in cache, so Traffic Server retrieved the object from the origin server (or aparent proxy) and sent it to the client.

TCP_REFRESH_HIT The object was in the cache, but it was stale. Traffic Server made an if-modified-sincerequest to the origin server and the origin server sent a 304 not-modified response. Traffic Server sent thecached object to the client.

TCP_REF_FAIL_HIT The object was in the cache but was stale. Traffic Server made an if-modified-sincerequest to the origin server but the server did not respond. Traffic Server sent the cached object to the client.

TCP_REFRESH_MISS The object was in the cache but was stale. Traffic Server made an if-modified-sincerequest to the origin server and the server returned a new object. Traffic Server served the new object to theclient.



TCP_CLIENT_REFRESH The client issued a request with a no-cache header. Traffic Server obtained the re-quested object from the origin server and sent a copy to the client. Traffic Server deleted the previous copy ofthe object from cache.

TCP_IMS_HIT The client issued an if-modified-since request and the object was in cache and fresher thanthe IMS date, or an if-modified-since request to the origin server revealed the cached object was fresh.Traffic Server served the cached object to the client.

TCP_IMS_MISS The client issued an if-modified-since request and the object was either not in cache orwas stale in cache. Traffic Server sent an if-modified-since request to the origin server and receivedthe new object. Traffic Server sent the updated object to the client.

TCP_SWAPFAIL The object was in the cache but could not be accessed. The client did not receive the object.

ERR_CLIENT_ABORT The client disconnected before the complete object was sent.

ERR_CONNECT_FAIL Traffic Server could not reach the origin server.

ERR_DNS_FAIL The Domain Name Server (DNS) could not resolve the origin server name, or no DNS could bereached.

ERR_INVALID_REQ The client HTTP request was invalid. (Traffic Server forwards requests with unknown methodsto the origin server.)

ERR_READ_TIMEOUT The origin server did not respond to Traffic Server‘s request within the timeout interval.

ERR_PROXY_DENIED Client service was denied.

ERR_UNKNOWN The client connected, but subsequently disconnected without sending a request.

Summary Logs

Due to the speed and efficiency of Traffic Server, a heavily-loaded node will generate many events and the event logscan quickly grow to very large sizes. Using SQL-like aggregate operators, you can configure Traffic Server to createsummary log files that summarize a set of log entries over a specified period of time. This can significantly reduce thesize of the log files generated.

To generate a summary log file, create a LogFormat object in the XML-based logging configuration logs_xml.config using the SQL-like aggregate operators below. You can apply each of these operators to specific fields, overa specified interval.

• COUNT

• SUM

• AVERAGE

• FIRST

• LAST

To create a summary log file format:

1. Define the format of the log file in logs_xml.config as follows:

<LogFormat><Name = "summary"/><Format = "%<operator(field)> : %<operator(field)>"/><Interval = "n"/>

</LogFormat>



Where operator is one of the five aggregate operators (COUNT, SUM, AVERAGE, FIRST, LAST); field isthe logging field you want to aggregate; and n is the interval (in seconds) between summary log entries.

You can specify more than one operator in the format line. For more information, refer to logs_xml.config.

2. Run the command traffic_line -x to apply configuration changes .

The following example format generates one entry every 10 seconds. Each entry contains the timestamp of the lastentry of the interval, a count of the number of entries seen within that 10-second interval, and the sum of all bytes sentto the client:

<LogFormat><Name = "summary"/><Format = "%<LAST(cqts)> : %<COUNT(*)> : %<SUM(psql)>"/><Interval = "10"/>

</LogFormat>

: You cannot create a format specification that contains both aggregate operators and regular fields. For example, thefollowing specification would be invalid:

<Format = "%<LAST(cqts)> : %<COUNT(*)> : %<SUM(psql)> : %<cqu>"/>

Log Collation

You can use the Traffic Server log file collation feature to collect all logged information in one place. Log collationenables you to analyze a set of Traffic Server clustered nodes as a whole (rather than as individual nodes) and to use alarge disk that might only be located on one of the nodes in the cluster.

Traffic Server collates log files by using one or more nodes as log collation servers and all remaining nodes as logcollation clients. When a Traffic Server node generates a buffer of event log entries, it first determines if it is thecollation server or a collation client. The collation server node writes all log buffers to its local disk, just as it wouldif log collation was not enabled. Log collation servers can be standalone or they can be part of a node running TrafficServer.

The collation client nodes prepare their log buffers for transfer across the network and send the buffers to the logcollation server. When the log collation server receives a log buffer from a client, it writes it to its own log file as if itwas generated locally. For a visual representation of this, see the figure below.

If log clients cannot contact their log collation server, then they write their log buffers to their local disks, into orphanlog files. Orphaned log files require manual collation.

: Log collation can have an impact on network performance. Because all nodes are forwarding their log data buffersto the single collation server, a bottleneck can occur. In addition, collated log files contain timestamp information foreach entry, but entries in the files do not appear in strict chronological order. You may want to sort collated log filesbefore doing analysis.

Server Configuration

To configure a Traffic Server node to be a collation server, perform the following configuration adjustments inrecords.config:



3.15: Log collation

1. Set :ts:cv:‘proxy.local.log.collation_mode‘ to 1 to indicate this node will be a server.

CONFIG proxy.local.log.collation_mode INT 1

2. Configure the port on which the server will listen to incoming collation transfers from clients, using:ts:cv:‘proxy.config.log.collation_port‘. If omitted, this defaults to port 8085.

CONFIG proxy.config.log.collation_port INT 8085

3. Configure the shared secret used by collation clients to authenticate their sessions, using:ts:cv:‘proxy.config.log.collation_secret‘.

CONFIG proxy.config.log.collation_secret STRING "seekrit"


: If you modify the collation_port or secret after connections between the collation server and collationclients have been established, then you must restart Traffic Server on all nodes.

Standalone Collator

If you do not want the log collation server to be a Traffic Server node, then you can install and configure a standalonecollator (SAC) that will dedicate more of its power to collecting, processing, and writing log files.

To install and configure a standalone collator:

1. Configure your Traffic Server nodes as log collation clients based on the instructions in Client Configuration.



2. Copy the traffic_sac binary from the Traffic Server bin directory, and place in a suitable location on thesystem that will act as the standalone collator.

3. Copy the libtsutil.so libraries from the Traffic Server lib directory to the machine serving as the stan-dalone collator.

4. Create a directory called config in the directory that contains the traffic_sac binary.

5. Create a directory called internal in the config directory you created above. This directory is used inter-nally by the standalone collator to store lock files.

6. Copy the records.config file from a Traffic Server node configured to be a log collation client to theconfig directory you created on the standalone collator.

The records.config file contains the log collation secret and the port you specified when configuringTraffic Server nodes to be collation clients. The collation port and secret must be the same for all collationclients and servers.

7. Edit :ts:cv:‘proxy.config.log.logfile_dir‘ in records.config to specify a location on your standalone col-lator where the collected log files should be stored.

CONFIG proxy.config.log.logfile_dir STRING "/var/log/trafficserver/"

8. Enter the following command to start the standalone collator process:

traffic_sac -c config

You will likely want to configure this program to run at server startup, as well as configure a service monitor in theevent the process terminates abnormally. Please consult your operating system’s documentation for how to achievethis.

Client Configuration

To configure a Traffic Server node to be a collation client, follow the steps below. If you modify thecollation_port or secret after connections between the collation clients and the collation server have beenestablished, then you must restart Traffic Server.

1. In the records.config file, edit the following variables:

• :ts:cv:‘proxy.local.log.collation_mode‘: 2 to configure this node as log collation client and sen standardformatted log entries to the collation server. For XML-based formatted log entries, see logs_xml.config file.

• :ts:cv:‘proxy.config.log.collation_host‘

• :ts:cv:‘proxy.config.log.collation_port‘

• :ts:cv:‘proxy.config.log.collation_secret‘

• :ts:cv:‘proxy.config.log.collation_host_tagged‘

• :ts:cv:‘proxy.config.log.max_space_mb_for_orphan_logs‘


Collating Custom Logs

If you use custom event log files, then you must edit logs_xml.config, in addition to configuring a collationserver and collation clients.



To collate custom event log files:

1. On each collation client, edit logs_xml.config and add the CollationHosts attribute to the LogObjectspecification:

<LogObject><Format = "squid"/><Filename = "squid"/><CollationHosts="ipaddress:port"/>

</LogObject>

Where ipaddress is the hostname or IP address of the collation server to which all log entries (for this object)are forwarded, and port is the port number for communication between the collation server and collationclients.

2. Run the command traffic_line -L to restart Traffic Server on the local node or traffic_line -M torestart Traffic Server on all the nodes in a cluster.

ASCII Log Pipes

In addition to ASCII and BINARY file modes for custom log formats, Traffic Server can output log entries inASCII_PIPE mode. This mode writes the log entries to a UNIX named pipe (a buffer in memory). Other pro-cesses may read from this named pipe using standard I/O functions.

The advantage of this mode is that Traffic Server does not need to write the entries to disk, which frees disk spaceand bandwidth for other tasks. When the buffer is full, Traffic Server drops log entries and issues an error messageindicating how many entries were dropped. Because Traffic Server only writes complete log entries to the pipe, onlyfull records are dropped.

Event Log Builder

If you need any assistance building your event log, you can try out our online log builder. This is a work in progress,so any comments, critique or suggestions are most welcome.

Traffic Manager Alarms

Traffic Server signals an alarm when it detects a problem. For example, the space allocated to event logs could be fullor Traffic Server may not be able to write to a configuration file.

Email Alarms

To configure Traffic Server to send an email to a specific address whenever an alarm occurs, follow the steps below:

1. Set :ts:cv:‘proxy.config.alarm_email‘ in records.config to the email address you want to receive alarmnotifications.

CONFIG proxy.config.alarm_email STRING "[email protected]"



http://trafficserver.apache.org/logbuilder/


Using a Script File for Alarms

Alarm messages are built into Traffic Server and cannot be changed. However, you can write a script file to execute cer-tain actions when an alarm is signaled. Traffic Server provides a sample script file named example_alarm_bin.sh in the bin directory which can serve as the basis for your custom alarm scripts.

Viewing Statistics from Traffic Line

You can use the Traffic Line command-line interface to view statistics about Traffic Server performance and webtraffic. In addition to viewing statistics, you can also configure, stop, and restart the Traffic Server system. Foradditional information, refer to Configure Traffic Server Using Traffic Line and traffic_line. You can viewspecific information about a Traffic Server node or cluster by specifying the variable that corresponds to the statisticyou want to see.

To view a statistic, enter the following command::

traffic_ctl metric get VARIABLE

where variable is the variable representing the information you want to view. For a list of variables you can specify,refer to Traffic Server Metrics.

For example, the following command displays the document hit rate for the Traffic Server node::

traffic_ctl metric get proxy.node.cache_hit_ratio

If the Traffic Server bin directory is not in your path, then prepend the Traffic Line command with ./ (for example:traffic_ctl metric get VARIABLE).

Statistics

Statistics provide a live view into the current state of Traffic Server and many of the plugins which may be usd with it.They can, and generally should be, used as a core piece in any performance and operational monitoring infrastructureas they expose a much greater detail about the workings of Traffic Server than can reliably be obtained through anyother means.

Accessing Statistics

There are currently two methods provided with Traffic Server to view statistics: Traffic Line and Stats Over HTTP.

Traffic Line

The command line utility traffic_line offers a text based interface for viewing Traffic Server statistics. Invoca-tion is simple and requires knowing the specific name of the statistic you wish you view:

traffic_line -r <statistic name>

You may notice that this is the same utility, and argument, used for viewing configuration variables of a runningTraffic Server instance. Unlike configuration variables, you cannot modify a statistic value with the traffic_lineprogram.

This utility is enabled and built by default, and will be located in the bin/ subdirectory of your Traffic Serverinstallation. There are no required changes to your configuration to allow traffic_line to function, however itmay only be run by users with permissions to access the Traffic Server Unix socket. This will typically limit use to



root as well as the system user you have configured Traffic Server to run under or any other system users which sharethe same group as you have configured Traffic Server to use.

Stats Over HTTP

Traffic Server includes a stable plugin, Stats Over HTTP Plugin, which provides HTTP access to all Traffic Serverstatistcs. The plugin returns a JSON object with all statistics and their current values. It is not possible to return asubset of the statistics. The plugin must be enabled before you may use it.

Enabling Stats Over HTTP

To enable the Stats Over HTTP Plugin plugin, you must add the following to your plugin.config:

stats_over_http.so

Once the plugin is enabled and Traffic Server has reloaded, you can test that it is working properly by issuing a simpleHTTP request with curl. Assuming your Traffic Server installation is using the default interface and port bindings,running the following command on the same host as Traffic Server should now work:

curl http://localhost:8080/_stats

You should be presented with an HTTP response containing a single JSON object which lists all the available statisticsand their current values. If you have configured Traffic Server to only listen on a specific interface, or to use a differentport, you may need to adjust the URL in the command above.

If you wish to have the stats made available at a non-default path, then that path should be given as the sole argumentto the plugin, as so:

stats_over_http.so 81c075bc0cca1435ea899ba4ad72766b

The above plugin.config entry will result in your Traffic Server statistics being located at /81c075bc0cca1435ea899ba4ad72766b on any host and port on which you have your Traffic Server instancelistening.

Statistics Security and Privacy

Simply changing the path at which your statistics are available should be considered very weak security. While cacheobjects themselves cannot be accessed through the plugin’s JSON output, and no modifications to the configuration oroperation of Traffic Server may be made through the plugin, the statistics may reveal much more about your network’straffic and architecture than you wish to be publicly available.

A better method is to use an ACL Filter in remap.config to restrict access to clients. For instance, if your TrafficServer host resides on a private network in the 10.1.1.0/24 IPv4 address space listening on the address 10.1.1.10,separate from its public interface(s) used to serve client requests, you could add the following remap configuration:

map http://10.1.1.10/_stats http://10.1.1.10/_stats @action=deny @src_ip=0.0.0.0-255.→˓255.255.255map http://10.1.1.10/_stats http://10.1.1.10/_stats @action=allow @src_ip=10.1.1.0-10.→˓1.1.255

The above configuration sets the default policy for the entirety of IPv4 address space to deny, but then exempts the10.1.1.0/24 network by permitting their requests to be processed by Traffic Server. If your monitoring infrastructuremakes use of locally-installed data collection agents, you may even wish to restrict access to the Stats Over HTTPplugin to all but localhost.



Core Statistics

Core statistics are available regardless of which plugins are enabled. This section breaks them down into categoriesbased on the general Traffic Server function or component to which they’re related.

General

Logging

Cache

Cache Volume

Each configured cache volume receives a complete set of statistics from Traffic Server. They are differentiated fromeach other by the incrementing number that is part of the statistic name. The volume number begins at 0 and incrementsby 1 for each additional volume.

The statistics are documented in this section using the default volume number in a configuration with only one cachevolume: 0.

Hierarchical Cache

HostDB

The HostDB component of Traffic Server manages origin server name resolution and host health checks. The statisticsdocumented here can help to ensure that your Traffic Server instances are not spending an unreasonable amount oftiming resolving your origin servers’ hostnames prior to object revalidation or retrieval.

DNS

Cluster

The statistics documented in this section relate to Traffic Server clusters, which are synchronized sets of Traffic Serverinstances running across multiple hosts. For more information on configuring and using the clustering feature, pleaserefer to the Traffic Server Cluster chapter of the Administrator’s Guide.

Origin Server

Network I/O

SSL/TLS

SSL Cipher

The following statistics provide individual counters for the number of client requests which were satisfied using thegiven SSL cipher.



HTTP Transaction

HTTP Response Code

The statistics documented in this section provide counters for the number of HTTP responses delivered to user agentsby your Traffic Server instance, grouped by the HTTP status code of the response. Please refer to the HTTP StatusCodes appendix for more details on what each status code means.

HTTP Request Method

The statistics documented in this section provide counters for the number of incoming requests to the Traffic Serverinstance, broken down by the HTTP method of the request.

HTTP Connection

HTTP Document Size

HTTP Header

Bandwidth and Transfer

Congestion

SOCKS

WebSocket

Miscellaneous

Plugin Specific Statistics

Monitoring

Using Built-in Tools

Integrating Third-Party Tools

There are many monitoring and alerting systems available, too many for us to hope to cover every possible optionhere. We can, however, attempt to document using some of the more common options.

Some of these third party monitoring applications and services are able to tap into the extensive list of Traffic Serverstatistics, others make use of Traffic Server log files, and yet others are aimed at simple health check reporting. An ex-tensive service monitoring configuration may take advantage of more than one service to play to their complementarystrengths. Which tool, or combination of tools, is right for your infrastructure will likely vary.



Circonus

Circonus is a commercial systems monitoring and analytics product, available in hosted (SaaS) and on-site configura-tions.

While it does not currently offer a module specific to Apache Traffic Server™ statistics, it is very easy to add moni-toring by combining the Traffic Server plugin Stats Over HTTP with Circonus’s JSON metrics collector.

Setup

The following steps assume that you already have an active Circonus account on their hosted service, or that youalready have an on-site installation. If you do not, that must be set up first. Trial accounts are available and will sufficefor this guide.

Enable Stats Over HTTP

Follow the directions in the Stats Over HTTP Plugin chapter to configure the plugin. At a bare minimum, you willneed to add the plugin to plugin.config. For this guide, we will assume the following entry:

stats_over_http.so _statistics

This enables the plugin and makes Traffic Server statistics available at the path /_statistics. For productionenvironments, you are strongly encouraged to use a different (and obfuscated) path for your statistics since they revealmany internal details of your caching proxy. The plugin documentation provides more details.

Create Circonus Check

For these steps, we will assume that Traffic Server is listening on port 80 at the domain trafficserver.example.com. You should adjust these to match your environment.

1. Begin the new check creation process from within your Circonus account by clicking the New Check button nearthe top-right of the checks screen.

2. For the check type, you should select JSON under the Custom list, and then choose the Pull type. The brokeryou choose to use is entirely up to you and will depend largely on whether you are using on-site Circonus or thehosted service, as well as the geographic location of your Traffic Server instance(s).


http://www.circonus.com/


3. Click on Configure Check to proceed.

4. Click on the Show Advanced Configuration link to expand the check options. Enter your Traffic Server instance’sexternally-accessible domain for Host and make sure to set URI to the path at which the statistics are available.Adjust any of the other options to match your environment if necessary (for this guide, only Host and URI willneed to be entered).



5. Click on Test Check to proceed.

6. If all is well, you should see a long list (several hundred entries) of Traffic Server statistics, with their associatedvalues. If you do not, verify that Traffic Server is running and use curl to manually fetch the statistics datafrom your server.



7. You may want to limit the metrics you actually track from Traffic Server, since so many are made available. Ifso, simply uncheck those you aren’t interested in from the list. If you only want to track a few metrics, clickSelect None and then check the ones you want. You can filter the results to easily find metrics in the list.

8. Save your new check.

9. Verify the collected metrics in Circonus by opening your new check and clicking on Metrics Grid for anoverview visualization of all the data being collected.

Congratulations! You’re now ready to begin setting up alerts, visualizations, and dashboards for your Traffic Serverstatistics. You can repeat the above steps for any additional Traffic Server instances you have running.

Further Reading

• Official Circonus Website


http://www.circonus.com/


Logstash

Logstash is a powerful, open source, unstructured data processing program that can accept text data from many differ-ent sources (directly over TCP/UDP, via Unix sockets, or by reading in files from disk for example), in many differentformats and transform those inputs into structured, searchable documents.

One of the most common use cases is to take text based system and application logs, extract individual fields (e.g. hostnames, error codes, timing data, and so on), and make the data available in Elasticsearch for searching and reporting.

In this guide, we will cover the basics of getting your Traffic Server log data into Logstash. Going the next step andbuilding fancy Kibana dashboards on top of that is currently left as an exercise for the reader.

Traffic Server Log Formats

Traffic Server provides a very flexible set of logging outputs. Almost any format can be constructed. The full range ofoptions is covered in the Logging chapter.

This guide will walk you through using the appropriate filters in Logstash for the common logging formats in TrafficServer. If you have constructed your own custom log formats, you will need to build upon these examples and refer tothe Logstash documentation to produce custom filters capable of parsing your own formats.

Logstash Input

For the on-disk logs produced by Traffic Server, you will want to use Logstash’s file input plugin. Note that yourlogs must be in ASCII format, not binary, for the plugin to work.

Assuming that your Traffic Server event logs are named access-<rotationtimestamp>.log and stored at/var/log/trafficserver/, the following Logstash input configuration should work:

input {file {path => /var/log/trafficserver/access-*.log

}}

Logstash provides some additional tweaking options, which are explained in the file plugin documentation but theabove provides the bare minimum required to have Logstash read log data from local disks.

Logstash Filters

The grok filter in Logstash allows you to completely tailor the parsing of your source data and extract as many or asfew fields as you like.

Some patterns are already built and can be used very easily. If you have built custom log formats for Traffic Server,you may need to write your own patterns, however.

Squid Compatible

The Squid log format includes, unsurprisingly, a few useful fields for proxy servers. Using the following grok patternwill extract this information from your Traffic Server logs if you employ the Squid compatible log format:


https://www.elastic.co/products/logstash

https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html

https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html


filter {grok {match => { "message" => "%{NUMBER:timestamp} %{NUMBER:timetoserve} %

→˓{IPORHOST:clientip} %{WORD:cachecode}/%{NUMBER:response} %{NUMBER:bytes} %→˓{WORD:verb} %{NOTSPACE:request} %{USER:auth} %{NOTSPACE:route} %{DATA:contenttype}"→˓}}date {match => [ "timestamp", "UNIX" ]

}}

The resulting structured document will contain the following fields:

Field Descriptiontimes-tamp

Date and time of the client request.

time-toserve

Time, in seconds, from initial client connection to Traffic Server until the last byte has been sent backto client from Traffic Server.

clientip Client IP address or hostname.cachecode Cache Result Codes.response HTTP response status code sent by Traffic Server to the client.bytes Length, in bytes, of the Traffic Server response to the client, including headers.verb HTTP method (e.g. GET, POST, etc.) of the client request.request URL specified by the client request.auth Authentication username supplied by the client, if present.route Proxy hierarchy route; the route used by Traffic Server to retrieve the cache object.content-type

Content type of the response.

Netscape Common

If your Traffic Server instance is already outputting Netscape Common format logs, then Logstash’sCOMMONAPACHELOG pattern will handle your logs out of the box. Add the following filter block to your Logstashconfiguration:

filter {grok {match => { "message" => "%{COMMONAPACHELOG}" }

}}

This will produce a structured document for each log entry with the following fields:



Field Descriptionclientip Client IP address or hostname.ident Always a literal - character for Traffic Server logs.auth The authentication username for the client request. A - means no authentication was required (or

supplied).timestamp The date and time of the client request.verb HTTP method used for the request (e.g. GET, POST, etc.).request URL specified by the client request.httpver-sion

HTTP version (e.g. 1.1) used by the client.

rawre-quest

See note below.

response HTTP status code used for Traffic Server response (not the origin’s response code).bytes Length of Traffic Server response to client, in bytes.

: rawrequest is populated when the usual "<verb> <request> http/<httpversion>" pattern was notmatched. In that event, those three fields will be missing from the document, and instead rawrequest will have theoriginal string.

Netscape Extended

The following pattern adds to Common Apache to support the additional fields found in Netscape Extended:

filter {grok {match => { "message" => "%{COMMONAPACHELOG} %{NUMBER:originstatus} %

→˓{NUMBER:originrespbytes} %{NUMBER:clientreqbytes} %{NUMBER:proxyreqbytes} %→˓{NUMBER:clienthdrbytes} %{NUMBER:proxyresphdrbytes} %{NUMBER:proxyreqhdrbytes} %→˓{NUMBER:originhdrbytes} %{NUMBER:timetoserve}" }}

}

Because this starts out with the COMMONAPACHELOG pattern, you will get all of the fields mentioned in NetscapeCommon above, as well as the following:

Field Descriptionoriginstatus HTTP status code returned by origin server.originresp-bytes

Body length, in bytes, of origin’s response to Traffic Server.

clientre-qbytes

Body length, in bytes, of client request to Traffic Server.

proxyre-qbytes

Body length, in bytes, of Traffic Server request to origin.

clienthdr-bytes

Header length, in bytes, of client request to Traffic Server.

proxyresph-drbytes

Header length, in bytes, of Traffic Server response to client.

proxyreqhdr-bytes

Header length, in bytes, of Traffic Server request to origin.

originhdr-bytes

Header length, in bytes, of origin’s response to Traffic Server.

timetoserve Time, in seconds, from initial client connection to Traffic Server until the last byte has been sentback to client from Traffic Server.



Further Reading

• Logstash Documentation

• Grok Patterns

• Elasticsearch Documentation

Error Messages

The following table lists messages that can appear in system log files. This list is not exhaustive; it simply describescommon warning messages that can occur and which might require your attention.

Fatal Process Messages

Accept port is not between 1 and 65535. Please check configuration The port speci-fied in records.config that accepts incoming HTTP requests is not valid.

Self loop is detected in parent proxy configuration The name and port of the parent proxymatch that of Traffic Server. This creates a loop when Traffic Server attempts to send the request to the parentproxy.

Process Warnings

<Logfile> error: error_number Generic logging error.

Bad cluster major version range <version1-version2> for node <IP address> connect failedIncompatible software versions causing a problem.

Connect by disallowed client <IP address>, closing The specified client is not allowed to con-nect to Traffic Server; the client IP address is not listed in the ip_allow.config file.

Could not rename log <filename> to <rolled filename> System error when renaming log fileduring roll.

Did <this amount> of backup; still to do <remaining amount> Congestion is approaching.

Different clustering minor versions <version1, version2> for node <IP address> continuingIncompatible software versions are causing a problem.

Log format symbol <symbol name> not found Custom log format references a field symbol that doesnot exist. Refer to Log Formats.

Missing field for field marker Error reading a log buffer.

Unable to open log file <filename>, errno=<error number> Cannot open the log file.

Error accessing disk <disk name> Traffic Server might have a cache read problem. You might need toreplace the disk.

Too many errors accessing disk <disk name>: declaring disk bad Traffic Server is notusing the cache disk because it encountered too many errors. The disk might be corrupt and might have tobe replaced.

No cache disks specified in storage.config file: cache disabled The Traffic Serverstorage.config file does not list any cache disks; Traffic Server is running in proxy-only mode. Youmust add the disks you want to use for the cache to storage.config.


https://www.elastic.co/guide/en/logstash/current/index.html

https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns

https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html


Alarm Messages

[Rollback::Rollback] Config file is read-only: <filename> Go to the Traffic Serverconfig directory and check the indicated file permissions; change if necessary.

[Rollback::Rollback] Unable to read or write config file <filename> Go to the Traf-fic Server config directory and make sure the indicated file exists. Check permissions and modify if necessary.

[Traffic Manager] Configuration File Update Failed: <error number> Go to the Traf-fic Server config directory and check the indicated file permissions; change if necessary.

[Traffic Manager] Mgmt <==>Proxy conn. closed An informational message to inform you that thetraffic_server process is down.

Access logging suspended - configured space allocation exhausted. The space allo-cated to the event log files is full; you must either increase the space or delete some log files so that accesslogging to continue. To prevent this error, consider rolling log files more frequently and enabling the autodeletefeature.

Access logging suspended - no more space on the logging partition. The entire parti-tion containing the event logs is full; you must delete or move some log files to enable access logging tocontinue. To prevent this error, consider rolling log files more frequently and enabling the autodelete feature.

Created zero length place holder for config file <filename> Go to the Traffic Serverconfig directory and check the indicated file. If it is indeed zero in length, then use a backup copy of theconfiguration file.

Traffic Server could not open logfile <filename> Check permissions for the indicated file andthe logging directory.

Traffic Server failed to parse line <line number> of the logging config file <filename>Check your custom log configuration file; there could be syntax errors. Refer to Custom Logging Fields forcorrect custom log format fields.

vip_config binary is not setuid root, manager will be unable to enable virtual ip addressesThe traffic_manager process is not able to set virtual IP addresses. You must setuid root for thevip_config file in the Traffic Server bin directory.

HTML Messages Sent to Clients

Traffic Server returns detailed error messages to client browsers when there are problems with the HTTP transactionsrequested by the browser. These Traffic Server response messages correspond to standard HTTP response codes, butprovide more information. A list of the more frequently encountered HTTP response codes is provided in HTTP StatusCodes.

The error messages can be customized. The actual response is generated from a template. These templates are storedin files which means the errors responses can be customized by modifying these files. The default directory forthe template files is PREFIX/body_factory/default but this can be changed by the configuration variable:ts:cv:‘proxy.config.body_factory.template_sets_dir‘. All files in this directory are added to a lookup table which isconsulted when the error message is generated. The name used for lookup is by default that listed in the following table.It can be overridden by :ts:cv:‘proxy.config.body_factory.template_base‘ which, if set, is a string that is prependedto the search name along with an underscore. For example, if the default lookup name is cache#read_error thenby default the response will be generated from the template in the file named cache#read_error. If the templatebase name were set to “apache” then the lookup would look for a file named apache_cache#read_error in thetemplate table. This can be used to switch out error message sets or, because this variable is overridable, to select anerror message set based on data in the transaction.

The text for an error message is processed as if it were a custom logging format which enables customization by valuespresent in the transaction for which the error occurred.



The following table lists the hard-coded Traffic Server HTTP messages, with corresponding HTTP response codes andcustomizable files.

Access Denied 403 You are not allowed to access the document at location URL. access#denied

Cache Read Error 500 Error reading from cache; please retry request. cache#read_error

Connection Timed Out 504 Too much time has elapsed since the server has sent data.timeout#inactivity

Content Length Required 400 Could not process this request because Content-Length was not speci-fied. request#no_content_length

Cycle Detected 400 Your request is prohibited because it would cause an HTTP proxy cycle.request#cycle_detected

Forbidden 403 <port number> is not an allowed port for SSL connections (you have made a request for asecure SSL connection to a forbidden port number). access#ssl_forbidden

Host Header Required 400 An attempt was made to transparently proxy your request, but this attempt failedbecause your browser did not send an HTTP Host header. Manually configure your browser to use http://<proxy name>:<proxy port> as the HTTP proxy. Alternatively, end users can upgrade to a browserthat supports the HTTP Host header field. interception#no_host

Host Header Required 400 Because your browser did not send a Host HTTP header field, the virtual hostbeing requested could not be determined. To access the website correctly, you must upgrade to a browser thatsupports the HTTP Host header field. request#no_host

HTTP Version Not Supported 505 The origin server <server name> is using an unsupported version ofthe HTTP protocol. response#bad_version

Invalid Content Length 400 Could not process this request because the specified Content-Length wasinvalid (less than 0).. request#invalid_content_length

Invalid HTTP Request 400 Could not process this <client request> HTTP method request for URL.request#syntax_error

Invalid HTTP Response 502 The host <server name> did not return the document URL correctly.response#bad_response

Malformed Server Response 502 The host <server name> did not return the document URL correctly.response#bad_response

Malformed Server Response Status 502 The host <server name> did not return the document URLcorrectly. response#bad_response

Maximum Transaction Time exceeded 504 Too much time has elapsed while transmitting document URL.timeout#activity

No Response Header From Server 502 The host <server name> did not return the document URL cor-rectly. response#bad_response

Not Cached 504 This document was not available in the cache, and you (the client) only accept cached copies.cache#not_in_cache

Not Found on Accelerator 404 The request for URL on host <server name> was not found. Check thelocation and try again. urlrouting#no_mapping

NULL 502 The host <hostname> did not return the document URL correctly. response#bad_response

Proxy Authentication Required 407 Please log in with username and password.access#proxy_auth_required



Server Hangup 502 The server <hostname> closed the connection before the transaction was completed.connect#hangup

Temporarily Moved 302 The document you requested, URL, has moved to a new location. The new location is<new URL>. redirect#moved_temporarily

Transcoding Not Available 406 Unable to provide the document URL in the format requested by yourbrowser. transcoding#unsupported

Tunnel Connection Failed 502 Could not connect to the server <hostname>.connect#failed_connect

Unknown Error 502 The host <hostname> did not return the document URL correctly.response#bad_response

Unknown Host 500 Unable to locate the server named <hostname>; the server does not have a DNS entry.Perhaps there is a misspelling in the server name or the server no longer exists; double-check the name and tryagain. connect#dns_failed

Unsupported URL Scheme 400 Cannot perform your request for the document URL because the protocolscheme is unknown. request#scheme_unsupported

Configuring Traffic Server

Traffic Server provides several options for configuring the system.

Configure Traffic Server Using Traffic Line

Traffic Line enables you to quickly and easily change your Traffic Server configuration via command-line interface.

View Configuration Options in Traffic Line

To view a configuration setting, enter the following command:

traffic_ctl config get VARIABLE

where var is the variable associated with the configuration option. For a list of variables, refer to ConfigurationVariables.

Change Configuration Options in Traffic Line

To change the value of a configuration setting, enter the following command:

traffic_config set VARIABLE VALUE

where var is the variable associated with the configuration option and value is the value you want to use. For a list ofthe variables, see Configuration Variables.

Configure Traffic Server Using Configuration Files

As an alternative to using Traffic Line, you can change Traffic Server configuration options by manually editingspecific variables in records.config.

3.9. Configuring Traffic Server 213


Traffic Server must reread the configuration files for any changes to take effect. This is done with traffic_ctlconfig reload. Some configuration changes require a full restart of Traffic Server.

The following is a sample portion of records.config:

3.16: Sample records.config file

In addition to records.config, Traffic Server provides other configuration files that are used to configure specificfeatures. You can manually edit all configuration files as described in Configuration Files.

Performance Tuning

Apache Traffic Server™ in its default configuration should perform suitably for running the included regression testsuite, but will need special attention to both its own configuration and the environment in which it runs to performoptimally for production usage.

There are numerous options and strategies for tuning the performance of Traffic Server and we attempt to document asmany of them as possible in the sections below. Because Traffic Server offers enough flexibility to be useful for manycaching and proxying scenarios, which tuning strategies will be most effective for any given use case may differ, aswell as the specific values for various configuration options.



Before You Start

One of the most important aspects of any attempt to optimize the performance of a Traffic Server installation is theability to measure that installation’s performance; both prior to and after any changes are made. To that end, it isstrongly recommended that you establish some means to monitor and record a variety of performance metrics: requestand response speed, latency, and throughput; memory and CPU utilization; and storage I/O operations.

Attempts to tune a system without being able to compare the impact of changes made will at best result in haphazard,feel good results that may end up having no real world impact on your customers’ experiences, and at worst mayeven result in lower performance than before you started. Additionally, in the all too common situation of budgetconstraints, having proper measurements of existing performance will greatly ease the process of focusing on thoseindividual components that, should they require hardware expenditures or larger investments of employee time, havethe highest potential gains relative to their cost.

Building Traffic Server

While the default compilation settings for Traffic Server will produce a set of binaries capable of serving most cachingand proxying needs, there are some build options worth considering in specific environments.

Hardware Tuning

As with any other server software, efficient allocation of hardware resources will have a significant impact on TrafficServer performance.

CPU Selection

Apache Traffic Server™ uses a hybrid event-driven engine and multi-threaded processing model for handling incomingrequests. As such, it is highly scalable and makes efficient use of modern, multicore processor architectures.

Memory Allocation

Though Traffic Server stores cached content within an on-disk host database, the entire Cache Directory is alwaysmaintained in memory during server operation. Additionally, most operating systems will maintain disk caches withinsystem memory. It is also possible, and commonly advisable, to maintain an in-memory cache of frequently accessedcontent.

The memory footprint of the Traffic Server process is largely fixed at the time of server startup. Your Traffic Serversystems will need at least enough memory to satisfy basic operating system requirements, as well as capacity for thecache directory, and any memory cache you wish to use. The default settings allocate roughly 10 megabytes of RAMcache for every gigabyte of disk cache storage, though this setting can be adjusted manually in records.configusing the setting :ts:cv:‘proxy.config.cache.ram_cache.size‘. Traffic Server will, under the default configuration,adjust this automatically if your system does not have enough physical memory to accomodate the aforementionedtarget.

Aside from the cost of physical memory, and necessary supporting hardware to make use of large amounts of RAM,there is little downside to increasing the memory allocation of your cache servers. You will see, however, no benefitfrom sizing your memory allocation larger than the sum of your content (and index overhead).

Disk Storage

Except in cases where your entire cache may fit into system memory, your cache nodes will eventually need to interactwith their disks. While a more detailed discussion of storage stratification is covered in Cache Partitioning below, very

3.10. Performance Tuning 215


briefly you may be able to realize gains in performance by separating more frequently accessed content onto fasterdisks (PCIe SSDs, for instance) while maintaining the bulk of your on-disk cache objects, which may not receive thesame high volume of requests, on lower-cost mechanical drives.

Operating System Tuning

Apache Traffic Server™ is supported on a variety of operating systems, and as a result the tuning strategies availableat the OS level will vary depending upon your chosen platform.

General Recommendations

TCP Keep Alive

TCP Congestion Control Settings

Ephemeral and Reserved Ports

Jumbo Frames

Linux

FreeBSD

OmniOS / illumos

Mac OS X

Traffic Server Tuning

Traffic Server itself, of course, has many options you may want to consider adjusting to achieve optimal performancein your environment. Many of these settings are recorded in records.config and may be adjusted with thetraffic_ctl config set command line utility while the server is operating.

CPU and Thread Optimization

Thread Scaling

By default, Traffic Server creates 1.5 threads per CPU core on the host system. This may be adjusted with the followingsettings in records.config:

• :ts:cv:‘proxy.config.exec_thread.autoconfig‘

• :ts:cv:‘proxy.config.exec_thread.autoconfig.scale‘

• :ts:cv:‘proxy.config.exec_thread.limit‘



Thread Affinity

On multi-socket servers, such as Intel architectures with NUMA, you can adjust the thread affinity configuration totake advantage of cache pipelines and faster memory access, as well as preventing possibly costly thread migrationsacross sockets. This is adjusted with :ts:cv:‘proxy.config.exec_thread.affinity‘ in records.config.

CONFIG proxy.config.exec_thread.affinity INT 1

Thread Stack Size

:ts:cv:‘proxy.config.thread.default.stacksize‘

Polling Timeout

If you are experiencing unusually or unacceptably high CPU utilization during idle workloads, you may consideradjusting the polling timeout with :ts:cv:‘proxy.config.net.poll_timeout‘:

CONFIG proxy.config.net.poll_timeout INT 60

Memory Optimization

:ts:cv:‘proxy.config.thread.default.stacksize‘ :ts:cv:‘proxy.config.cache.ram_cache.size‘

Disk Storage Optimization

:ts:cv:‘proxy.config.cache.force_sector_size‘ :ts:cv:‘proxy.config.cache.max_doc_size‘:ts:cv:‘proxy.config.cache.target_fragment_size‘

Cache Partitioning

Network Tuning

:ts:cv:‘proxy.config.net.connections_throttle‘

Error responses from origins are conistent and costly

If error responses are costly for your origin server to generate, you may elect to have Traffic Server cache theseresponses for a period of time. The default behavior is to consider all of these responses to be uncacheable, which willlead to every client request to result in an origin request.

This behavior is controlled by both enabling the feature via :ts:cv:‘proxy.config.http.negative_caching_enabled‘and setting the cache time (in seconds) with :ts:cv:‘proxy.config.http.negative_caching_lifetime‘.

CONFIG proxy.config.http.negative_caching_enabled INT 1CONFIG proxy.config.http.negative_caching_lifetime INT 10

3.10. Performance Tuning 217


SSL-Specific Options

:ts:cv:‘proxy.config.ssl.max_record_size‘ :ts:cv:‘proxy.config.ssl.session_cache‘:ts:cv:‘proxy.config.ssl.session_cache.size‘

Thread Types

Logging Configuration

Plugin Tuning

Common Scenarios and Pitfalls

While environments vary widely and Traffic Server is useful in a great number of different situations, there are at leastsome recurring elements that may be used as shortcuts to identifying problem areas, or realizing easier performancegains.

Configuration Files

cache.config

cache.config ( /usr/local/etc/trafficserver/ ) Traffic Server web . :

• Not to cache objects from specific IP addresses

• How long to pin particular objects in the cache

• How long to consider cached objects as fresh

• Whether to ignore no-cache directives from the server

: After you modify the cache.config file, navigate to the Traffic Server bin directory; then run thetraffic_ctl config reload command to apply changes. When you apply the changes to a node in a cluster,Traffic Server automatically applies the changes to all other nodes in the cluster.

Format

Each line in the cache.config file contains a caching rule. Traffic Server recognizes three space-delimited tags:

primary_destination=value secondary_specifier=value action=value

You can use more than one secondary specifier in a rule. However, you cannot repeat a secondary specifier. Thefollowing list shows the possible primary destinations with allowed values.

dest_domain A requested domain name. Traffic Server matches the domain name of the destination from the URLin the request.

dest_host A requested hostname. Traffic Server matches the hostname of the destination from the URL in therequest.

dest_ip A requested IP address. Traffic Server matches the IP address of the destination in the request.

url_regex A regular expression (regex) to be found in a URL.



The secondary specifiers are optional in the cache.config file. The following list shows possible secondaryspecifiers with allowed values.

port A requested URL port.

scheme A request URL protocol: http or https.

prefix A prefix in the path part of a URL.

suffix A file suffix in the URL.

method A request URL method: get, put, post, trace.

time A time range, such as 08:00-14:00.

src_ip A client IP address.

internal A boolean value, true or false, specifying if the rule should match (or not match) a transactionoriginating from an internal API. This is useful to differentiate transaction originating from an ATS plugin.

The following list shows possible actions and their allowed values.

action One of the following values:

• never-cache configures Traffic Server to never cache specified objects.

• ignore-no-cache configures Traffic Server to ignore all Cache-Control: no-cache headers.

• ignore-client-no-cache configures Traffic Server to ignore Cache-Control: no-cacheheaders from client requests.

• ignore-server-no-cache configures Traffic Server to ignore Cache-Control: no-cacheheaders from origin server responses.

• cluster-cache-local configures the cluster cache to allow for this content to be stored locally onevery cluster node.

cache-responses-to-cookies Change the style of caching with regard to cookies. This effectively overridesthe configuration parameter :ts:cv:‘proxy.config.http.cache.cache_responses_to_cookies‘ and uses the samevalues with the same semantics. The override happens only for requests that match.

pin-in-cache Preserves objects in cache, preventing them from being overwritten. Does not affect objects thatare determined not to be cacheable. This setting can have performance issues, and severely affect the cache. Forinstance, if the primary destination matches all objects, once the cache is full, no new objects could get writtenas nothing would be evicted. Similarly, for each cache-miss, each object would incur extra checks to determineif the object it would replace could be overwritten.

The value is the amount of time you want to keep the object(s) in the cache. The following time formats areallowed:

• d for days; for example: 2d

• h for hours; for example: 10h

• m for minutes; for example: 5m

• s for seconds; for example: 20s

• mixed units; for example: 1h15m20s

revalidate For objects that are in cache, overrides the the amount of time the object(s) are to be considered fresh.Use the same time formats as pin-in-cache.

ttl-in-cache Forces object(s) to become cached, as if they had a Cache-Control: max-age:<time> header. Canbe overruled by requests with cookies. The value is the amount of time object(s) are to be kept in the cache,regardless of Cache-Control response headers. Use the same time formats as pin-in-cache and revalidate.

3.11. Configuration Files 219


Examples

The following example configures Traffic Server to revalidate gif and jpeg objects in the domain mydomain.comevery 6 hours, and all other objects in mydomain.com every hour. The rules are applied in the order listed.

dest_domain=mydomain.com suffix=gif revalidate=6hdest_domain=mydomain.com suffix=jpeg revalidate=6hdest_domain=mydomain.com revalidate=1h

Force a specific regex to be in cache between 7-11pm of the server’s time for 26hours.

url_regex=example.com/articles/popular.* time=19:00-23:00 ttl-in-cache=1d2h

Prevent objects from being evicted from cache:

url_regex=example.com/game/.* pin-in-cache=1h

congestion.config

The congestion.config file (by default, located in /usr/local/etc/trafficserver/) enables youto configure Traffic Server to stop forwarding HTTP requests to origin servers when they become congested, andthen send the client a message to retry the congested origin server later. After you modify the congestion.config file, navigate to the Traffic Server bin directory; then run the traffic_ctl config reload com-mand to apply changes. When you apply the changes to a node in a cluster, Traffic Server automatically applies thechanges to all other nodes in the cluster. Traffic Server uses the congestion.config file only if you enable the:ts:cv:‘proxy.config.http.congestion_control.enabled‘ option.

You can create rules in the congestion.config file to specify:

• Which origin servers Traffic Server tracks for congestion.

• The timeouts Traffic Server uses, depending on whether a server is congested.

• The page Traffic Server sends to the client when a server becomes congested.

• If Traffic Server tracks the origin servers per IP address or per hostname.

Format

Each line in congestion.configmust follow the format below. Traffic Server applies the rules in the order listed,starting at the top of the file. Traffic Server recognizes three space-delimited tags:


The following list shows possible primary destinations with allowed values.

dest_domain A requested domain name.

dest_host A requested hostname.

dest_ip A requested IP address.

url_regex A regular expression (regex) to be found in a URL.

The secondary specifiers are optional in the congestion.config file. The following list shows possible secondary spec-ifiers with allowed values. You can use more than one secondary specifier in a rule; however, you cannot repeat asecondary specifier.

port A requested URL port or range of ports.




The following list shows the possible tags and their allowed values.

max_connection_failures Default: 5 The maximum number of connection failures allowed within the failwindow described below before Traffic Server marks the origin server as congested.

fail_window Default: 120 seconds. The time period during which the maximum number of connection failurescan occur before Traffic Server marks the origin server as congested.

proxy_retry_interval Default: 10 seconds. The number of seconds that Traffic Server waits before contact-ing a congested origin server again.

client_wait_interval Default: 300 seconds. The number of seconds that the client is advised to wait beforeretrying the congested origin server.

wait_interval_alpha Default: 30 seconds The upper limit for a random number that is added to the waitinterval.

live_os_conn_timeout Default: 60 seconds. The connection timeout to the live (uncongested) origin server.If a client stops a request before the timeout occurs, then Traffic Server does not record a connection failure.

live_os_conn_retries Default: 2 The maximum number of retries allowed to the live (uncongested) originserver.

dead_os_conn_timeout Default: 15 seconds. The connection timeout to the congested origin server.

dead_os_conn_retries Default: 1 The maximum number of retries allowed to the congested origin server.

max_connection Default: -1 The maximum number of connections allowed from Traffic Server to the originserver.

error_page Default: "congestion#retryAfter" The error page sent to the client when a server is con-gested. You must enclose the value in quotes;

congestion_scheme Default: "per_ip" Specifies if Traffic Server applies the rule on a per-host("per_host") or per-IP basis ("per_ip"). You must enclose the value in quotes.

For example: if the server www.host1.com has two IP addresses and you use the tag value "per_ip", theneach IP address has its own number of connection failures and is marked as congested independently. If you usethe tag value "per_host" and the server www.host1.com is marked as congested, then both IP addressesare marked as congested.

Examples

The following congestion.config rule configures Traffic Server to stop forwarding requests to the server www.host.com on port 80 (HTTP traffic) if the server is congested, according to the timeouts specified. Traffic Serveruses the default tag values because no tag has been specified.

dest_host=www.host.com port=80

You can use one or more tags in a rule, but each tag must have one value only. If you specify no tags in the rule, thenTraffic Server uses the default values.

You can override any of the default tag values by adding configuration variables at the end of records.config asfollows:

CONFIG proxy.config.http.congestion_control.default.tag INT|STRING value

where tag is one of the tags described in the list under congestion.config and value is the value you want touse.



For example:

CONFIG proxy.config.http.congestion_control.default.congestion_scheme STRING per_host

: Rules in the congestion.config file override the following variables in the records.config file:

proxy.config.http.connect_attempts_max_retriesproxy.config.http.connect_attempts_max_retries_dead_serverproxy.config.http.connect_attempts_rr_retriesproxy.config.http.connect_attempts_timeoutproxy.config.http.down_server.cache_timeproxy.config.http.down_server.abort_threshold

hosting.config

The hosting.config file (by default, located in /usr/local/etc/trafficserver/) you to assign cachevolumes to specific origin servers and/or domains so that you can manage cache space efficiently and restrict diskusage. For step-by-step instructions on partitioning the cache according to origin servers and/or domains, refer toPartitioning the Cache.

Before you can assign cache volumes to specific origin servers and/or domains, you must first partition your cacheaccording to size and protocol in the volume.config file.

After you modify hosting.config, navigate to the Traffic Server bin directory and run traffic_ctl configreload to apply your changes.

When you apply the changes to a node in a cluster, Traffic Server automatically applies the changes to all other nodesin the cluster.

: The volume.config configuration must be the same on all nodes in a cluster.

Format

Each line in the hosting.config file must have one of the following formats:

hostname=HOST volume=NUMBERSdomain=DOMAIN volume=NUMBERS

where HOST is the fully-qualified hostname of the origin server whose content you want to store on a particularvolume (for example, www.myhost.com); DOMAIN is the domain whose content you want to store on a particularpartition(for example, mydomain.com); and NUMBERS is a comma-separated list of the partitions on which youwant to store the content that belongs to the origin server or domain listed. The partition numbers must be validnumbers listed in the file:volume.config.

Note: To allocate more than one partition to an origin server or domain, you must enter the partitions in a comma-separated list on one line, as shown in the example below. The hosting.config file cannot contain multipleentries for the same origin server or domain.



Generic Partition

When configuring the hosting.config file, you must assign a generic volume to use for content that does notbelong to any of the origin servers or domains listed. If all volumes for a particular origin server become corrupt,Traffic Server will also use the generic volume to store content for that origin server.

The generic volume must have the following format:

hostname=* volume=NUMBERS

where NUMBERS is a comma-separated list of generic volumes.

Examples

The following example configures Traffic Server to store content from the domain mydomain.com in volume 1 andcontent from www.myhost.com in volume 2. Traffic Server stores content from all other origin servers in volumes3 and 4.

domain=mydomain.com volume=1hostname=www.myhost.com volume=2hostname=* volume=3,4

icp.config

The icp.config file defines ICP peers (parent and sibling caches).

: After you modify the icp.config file, navigate to the Traffic Server bin directory and run the traffic_ctlconfig reload command to apply the changes. When you apply the changes to a node in a cluster, Traffic Serverautomatically applies the changes to all other nodes in the cluster.

Format

Each line in the icp.config file contains the name and configuration information for a single ICP peer in the followingformat:

host : host_IP : ctype : proxy_port : icp_port : MC_on : MC_IP : MC_TTL :

Each field is described in the following list.

host The hostname of the ICP peer.

This field is optional; if you do not specify the hostname of the ICP peer, you must specify the IP address.

host_IP The IP address of the ICP peer.

This field is optional; if you do not specify the IP address of the ICP peer, you must specify the hostname.

ctype Use the following options:

• 1 to indicate an ICP parent cache

• 2 to indicate an ICP sibling cache

• 3 to indicate an ICP local cache

proxy_port The port number of the TCP port used by the ICP peer for proxy communication.



icp_port The port number of the UDP port used by the ICP peer for ICP communication.

MC_on Enable or disable MultiCast:

• 0 if multicast is disabled

• 1 if multicast is enabled

MC_ip The MultiCast IP address.

MC_ttl The multicast time to live. Use the following options:

• 1 if IP multicast datagrams will not be forwarded beyond a single subnetwork

• 2 to allow delivery of IP multicast datagrams to more than one subnet (if there are one or more multicastrouters attached to the first hop subnet).

Examples

The following example configuration is for three nodes: the local host, one parent, and one sibling.

localhost:0.0.0.0:3:8080:3130:0:0.0.0.0:1host1:123.12.1.23:1:8080:3131:0:0.0.0.0:1host2:123.12.1.24:2:8080:3131:0:0.0.0.0:1

ip_allow.config

The ip_allow.config file controls client access to the Traffic Server proxy cache. You can specify ranges of IPaddresses that are allowed to use the Traffic Server as a web proxy cache. After you modify the ip_allow.configfile, navigate to the Traffic Server bin directory and run the traffic_ctl config reload command to applychanges. When you apply the changes to a node in a cluster, Traffic Server automatically applies the changes to allother nodes in the cluster.

Format

Each line in the ip_allow.config file must have the following format:

src_ip=<range of IP addresses> action=<action> [method=<list of methods separated by→˓'|'>]

where src_ip is the IP address or range of IP addresses of the client(s). The action ip_allow enables the specifiedclient(s) to access the Traffic Server proxy cache, and ip_deny denies the specified client(s) to access the TrafficServer proxy cache. Multiple method keywords can be specified (method=GET method=HEAD), or multiple methodscan be separated by an ‘|’ (method=GET|HEAD). The method keyword is optional and it is defaulted to ALL. Thissupports ANY string as the HTTP method, meaning no validation is done to check wether it is a valid HTTP method.This allows you to create filters for any method that your origin may require, this is especially useful if you use newermethods that aren’t know to trafficserver (such as PROPFIND) or if your origin uses an http-ish protocol.

By default, the ip_allow.config file contains the following lines, which allows all methods to localhost to accessthe Traffic Server proxy cache and denies PUSH, PURGE and DELETE to all IPs (note this allows all other methodsto all IPs):

src_ip=127.0.0.1 action=ip_allow method=ALLsrc_ip=::1 action=ip_allow method=ALLsrc_ip=0.0.0.0-255.255.255.255 action=ip_deny→˓method=PUSH|PURGE|DELETEsrc_ip=::-ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff action=ip_deny→˓method=PUSH|PURGE|DELETE



Examples

The following example enables all clients to access the Traffic Server proxy cache:

src_ip=0.0.0.0-255.255.255.255 action=ip_allow

The following example allows all clients on a specific subnet to access the Traffic Server proxy cache:

src_ip=123.12.3.000-123.12.3.123 action=ip_allow

The following example denies all clients on a specific subnet to access the Traffic Server proxy cache:

src_ip=123.45.6.0-123.45.6.123 action=ip_deny

log_hosts.config

To record HTTP transactions for different origin servers in separate log files, you must list each origin server hostnamein the log_hosts.config file.

You should use the same log_hosts.config file on every Traffic Server node in your cluster. After you modifythe log_hosts.config file, run the traffic_ctl config reload command to apply the changes. Whenyou apply the changes to a node in a cluster, Traffic Server automatically applies the changes to all other nodes in thecluster.

Format

Each line in the log_hosts.config file has the following format:

hostname

where hostname is the hostname of the origin server.

: You can specify keywords in the log_hosts.config file to record all transactions from origin servers with thespecified keyword in their names in a separate log file. See the example below.

Examples

The following example configures Traffic Server to create separate log files containing all HTTP transactions for theorigin servers webserver1, webserver2, and webserver3:

webserver1webserver2webserver3

The following example records all HTTP transactions from origin servers that contain sports in their names. Forexample: sports.yahoo.com and www.foxsports.com in a log file called squid-sport.log (the Squidformat is enabled):



sports

logs_xml.config

The logs_xml.config file defines the custom log file formats, filters, and processing options. The format of thisfile is modeled after XML, the Extensible Markup Language.

Format

The logs_xml.config file contains the specifications below:

• LogFormat specifies the fields to be gathered from each protocol event access.

• LogFilter specifies the filters that are used to include or exclude certain entries being logged based on thevalue of a field within that entry.

• LogObject specifies an object that contains a particular format, a local filename, filters, and collation servers.

The logs_xml.config file ignores extra white space, blank lines, and all comments.

LogFormat

The following list shows LogFormat specifications.

<Name = "valid_format_name"/> Required Valid format names include any name except squid, common,extended, or extended2, which are pre-defined formats. There is no default for this tag. The format objectneeds to be above the the LogObject object.

<Format = "valid_format_specification"/> Required A valid format specification is a printf-stylestring describing each log entry when formatted for ASCII output.

The printf-style could accept Oct/Hex escape representation:

• \abc is Oct escape sequence, a,b,c should be one of [0-9], and (a*8^2 + b*8 + c) should be greater than0 and less than 255.

• \xab is Hex escape sequence, a,b should be one of [0-9, a-f, A-F], and (a*16 + b) should be greater than0 and less than 255.

Use %< field > as a placeholder for valid field names. For more information, refer to Custom Logging Fields.

The specified field can be one of the following types:

Simple. For example: %<cqu> A field within a container, such as an HTTP header or a statistic. Fields of thistype have the syntax:

%<{ field } container>

Aggregates, such as COUNT, SUM, AVG, FIRST, LAST. Fields of this type have the syntax: %<operator (field )>

: You cannot create a format specification that contains both aggregate operators and regular fields.

<Interval = "aggregate_interval_secs"/> Optional Use this tag when the format contains aggregateoperators. The value “aggregate_interval_secs” represents the number of seconds between individualaggregate values being produced.



The valid set of aggregate operators are:

• COUNT

• SUM

• AVG

• FIRST

• LAST

LogFilter

The following list shows the LogFilter specifications.

<Name = "valid_filter_name"/> Required All filters must be uniquely named.

<Condition = "valid_log_field valid_operator valid_comparison_value"/>

Required This field contains the following elements:

valid_log_field - the field that will be compared against the given value. For more information,refer to Custom Field Cross-Reference.

valid_operator_field - any one of the following: MATCH, CASE_INSENSITIVE_MATCH,CONTAIN, CASE_INSENSITIVE_CONTAIN.

• MATCH is true if the field and value are identical (case-sensitive).

• CASE_INSENSITIVE_MATCH is similar to MATCH, except that it is case-insensitive.

• CONTAIN is true if the field contains the value (the value is a substring of the field).

• CASE_INSENSITIVE_CONTAIN is a case-insensitive version of CONTAIN.

valid_comparison_value - any string or integer matching the field type. For integer values,all of the operators are equivalent and mean that the field must be equal to the specified value.

For IP address fields, this can be a list of IP addresses and include ranges. A range is an IP address, followed bya dash ‘-‘, and then another IP address of the same family. For instance, the 10/8 network can be representedby 10.0.0.0-10.255.255.255. Currently network specifiers are not supported.

: There are no negative comparison operators. If you want to specify a negative condition, then use the Action fieldto REJECT the record.

<Action = "valid_action_field"/> Required: ACCEPT or REJECT or WIPE_FIELD_VALUE. AC-CEPT or REJECT instructs Traffic Server to either accept or reject records that satisfy the filter condition.WIPE_FIELD_VALUE wipes out the values of the query params in the url fields specified in the Condition.

NOTES: 1. WIPE_FIELD_VALUE action is only applied to the parameters in the query part.

2. Multiple parameters can be listed in a single WIPE_FIELD_VALUE filter

3. If the same parameter appears more than once in the query part , only the value of the first occurance iswiped

LogObject

The following list shows the LogObject specifications.



<Format = "valid_format_name"/> Required Valid format names include the predefined logging formats:squid, common, extended, and extended2, as well as any previously-defined custom log formats. Thereis no default for this tag. The format object needs to be above the the LogObject object.

<Filename = "file_name"/> Required The filename to which the given log file is written on the local filesystem or on a remote collation server. No local log file will be created if you fail to specify this tag. Allfilenames are relative to the default logging directory.

If the name does not contain an extension (for example, squid), then the extension .log is automaticallyappended to it for ASCII logs and .blog for binary logs (refer to Mode = “valid_logging_mode”).

If you do not want an extension to be added, then end the filename with a single (.) dot (for example: squid.).

<Mode = "valid_logging_mode"/> Optional Valid logging modes include ascii , binary , andascii_pipe . The default is ascii .

• Use ascii to create event log files in human-readable form (plain ASCII).

• Use binary to create event log files in binary format. Binary log files generate lower system over-head and occupy less space on the disk (depending on the information being logged). You must use thetraffic_logcat utility to translate binary log files to ASCII format before you can read them.

• Use ascii_pipe to write log entries to a UNIX named pipe (a buffer in memory). Other processes canthen read the data using standard I/O functions. The advantage of using this option is that Traffic Serverdoes not have to write to disk, which frees disk space and bandwidth for other tasks. In addition, writingto a pipe does not stop when logging space is exhausted because the pipe does not use disk space.

If you are using a collation server, then the log is written to a pipe on the collation server. A local pipe iscreated even before a transaction is processed, so you can see the pipe right after Traffic Server starts. Pipes ona collation server, however, are created when Traffic Server starts.

<Filters = "list_of_valid_filter_names"/> Optional A comma-separated list of names of anypreviously-defined log filters. If more than one filter is specified, then all filters must accept a record for therecord to be logged. The filters need to be above the LogObject object.

<Protocols = "list_of_valid_protocols"/> Optional A comma-separated list of the protocols thisobject should log. Valid protocol names for this release are HTTP (FTP is deprecated).

<ServerHosts = "list_of_valid_servers"/> Optional A comma-separated list of valid host-names.This tag indicates that only entries from the named servers will be included in the file.

<CollationHosts = "list_of_valid_hostnames:port|failover hosts"/> Optional Acomma-separated list of collation servers (with pipe delimited failover servers) to which all log entries (forthis object) are forwarded. Collation servers can be specified by name or IP address. Specify the collationport with a colon after the name. For example, in host1:5000|failhostA:5000|failhostB:6000,host2:6000 logs would be sent to host1 and host2, with failhostA and failhostB acting as failover hosts forhost1. When host1 disconnects, logs would be sent to failhostA. If failhostA disconnects, log entries would besent to failhostB until host1 or failhostA comes back. Logs would also be sent to host2.

<Header = "header"/> Optional The header text you want the log files to contain. The header text appears atthe beginning of the log file, just before the first record.

<RollingEnabled = "truth value"/> Optional Enables or disables log file rolling for the LogObject.This setting overrides the value for the :ts:cv:‘proxy.config.log.rolling_enabled‘ variable in the records.config file. Set truth value to one of the following values:

• 0 to disable rolling for this particular LogObject.

• 1 to roll log files at specific intervals during the day (you specify time intervals with theRollingIntervalSec and RollingOffsetHr fields).



• 2 to roll log files when they reach a certain size (you specify the size with the RollingSizeMb field).

• 3 to roll log files at specific intervals during the day or when they reach a certain size (whichever occursfirst).

• 4 to roll log files at specific intervals during the day when log files reach a specific size (at a specified timeif the file is of the specified size).

<RollingIntervalSec = "seconds"/> Optional The seconds between log file rolling for the LogObject;enables you to specify different rolling intervals for different LogObjects.

This setting overrides the value for :ts:cv:‘proxy.config.log.rolling_interval_sec‘ in the records.configfile.

<RollingOffsetHr = "hour"/> Optional Specifies an hour (from 0 to 23) at which rolling is guaranteed toalign. Rolling might start before then, but a rolled file will be produced only at that time. The impact of thissetting is only noticeable if the rolling interval is larger than one hour. This setting overrides the configurationsetting for :ts:cv:‘proxy.config.log.rolling_offset_hr‘ in the records.config file.

<RollingSizeMb = "size_in_MB"/> Optional The size at which log files are rolled.

Examples

The following is an example of a LogFormat specification that collects information using three common fields:

<LogFormat><Name="minimal"/><Format = "%<chi> : %<cqu> : %<pssc>"/>

</LogFormat>

The following is an example of a LogFormat specification that uses aggregate operators:

<LogFormat><Name = "summary"/><Format = "%<LAST(cqts)> : %<COUNT(*)> : %<SUM(psql)>"/><Interval = "10"/>

</LogFormat>

The following is an example of a LogFilter that will cause only REFRESH_HIT entries to be logged:

<LogFilter><Name = "only_refresh_hits"/><Action = "ACCEPT"/><Condition = "%<pssc> MATCH REFRESH_HIT"/>

</LogFilter>

: When specifying the field in the filter condition, you can omit the %<>. This means that the filter below is equivalentto the example directly above:

<LogFilter><Name = "only_refresh_hits"/><Action = "ACCEPT"/><Condition = "pssc MATCH REFRESH_HIT"/>

</LogFilter>

The following is an example of a LogFilter that will cause the value of passwd field be wiped in cquc



<LogFilter><Name = "wipe_password"/><Condition = "cquc CONTAIN passwd"/><Action = "WIPE_FIELD_VALUE"/>

</LogFilter>

The following is an example of a LogObject specification that creates a local log file for the minimal format definedearlier. The log filename will be minimal.log because this is an ASCII log file (the default).:

<LogObject><Format = "minimal"/><Filename = "minimal"/>

</LogObject>

The following is an example of a LogObject specification that includes only HTTP requests served by hosts in thedomain company.com or by the specific server server.somewhere.com. Log entries are sent to port 4000 ofthe collation host logs.company.com and to port 5000 of the collation host 209.131.52.129.

<LogObject><Format = "minimal"/><Filename = "minimal"/><ServerHosts = "company.com,server.somewhere.com"/><Protocols = "http"/><CollationHosts = "logs.company.com:4000,209.131.52.129:5000"/>

</LogObject>

WELF

Traffic Server supports WELF (WebTrends Enhanced Log Format) so you can analyze Traffic Server log fileswith WebTrends reporting tools. A predefined <LogFormat> that is compatible with WELF is provided in thelogs_xml.config file (shown below). To create a WELF format log file, create a <LogObject> that uses thispredefined format.

<LogFormat><Name = "welf"/><Format = "id=firewall time=\"%<cqtd> %<cqtt>\" fw=%<phn> pri=6

proto=%<cqus> duration=%<ttmsf> sent=%<psql> rcvd=%<cqhl>src=%<chi> dst=%<shi> dstname=%<shn> user=%<caun> op=%<cqhm>arg=\"%<cqup>\" result=%<pssc> ref=\"%<{Referer}cqh>\"agent=\"%<{user-agent}cqh>\" cache=%<crc>"/>

</LogFormat>

parent.config

The parent.config file identifies the parent proxies used in an cache hierarchy. Use this file to perform thefollowing configuration:

• Set up parent cache hierarchies, with multiple parents and parent failover

• Configure selected URL requests to bypass parent proxies

Traffic Server uses the parent.config file only when the parent caching option is enabled (refer to ConfiguringTraffic Server to Use a Parent Cache).



After you modify the parent.config file, run the traffic_ctl config reload command to apply yourchanges. When you apply the changes to one node in a cluster, Traffic Server automatically applies the changes to allother nodes in the cluster.

Format

Each line in the parent.config file must contain a parent caching rule. Traffic Server recognizes three space-delimited tags:


The following list shows the possible primary destinations and their allowed values.

dest_domain A requested domain name, and its subdomains.

dest_host A requested hostname.

dest_ip A requested IP address or range of IP addresses separated by a dash (-).

url_regex A regular expression (regex) to be found in a URL

The secondary specifiers are optional in the parent.config file. The following list shows the possible secondaryspecifiers and their allowed values.

port A requested URL port.

scheme A request URL protocol: http or https.


suffix A file suffix in the URL.

method A request URL method. It can be one of the following:

• get

• post

• put

• trace

time A time range, such as 08:00-14:00, during which the parent cache is used to serve requests.

src_ip A client IP address.

internal A boolean value, true or false, specifying if the rule should match (or not match) a transactionoriginating from an internal API. This is useful to differentiate transaction originating from an ATS plugin.

The following list shows the possible actions and their allowed values.

parent An ordered list of parent servers. If the request cannot be handled by the last parent server in the list, thenit will be routed to the origin server. You can specify either a hostname or an IP address, but; you must specifythe port number.

secondary_parent An optional ordered list of secondary parent servers. This optional list may only be usedwhen round_robin is set to consistent_hash. If the request cannot be handled by a parent server fromthe parent list, then the request will be re-tried from a server found in this list using a consistent hash of theurl.

parent_is_proxy One of the following values:

• true - This is the default. The list of parents and secondary parents are proxy cache servers.



• false - The list of parents and secondary parents are the origin servers go_direct flag is ignoredand origins are selected using the specified round_robin algorithm. The FQDN is removed fromthe http request line.

parent_retry If parent_is_proxy is false, then you may configure parent_retry for one of the follow-ing values:

• simple_retry - If the parent origin server returns a 404 response on a request a new parent is selectedand the request is retried. The number of retries is controlled by max_simple_retries which is setto 1 by default.

• unavailable_server_retry - If the parent returns a 503 response or if the reponse matches a listof http 5xx responses defined in unavailable_server_retry_responses, the currently selectedparent is marked down and a new parent is selected to retry the request. The number of retries is controlledby max_unavailable_server_retries which is set to 1 by default.

• both - This enables both simple_retry and unavailable_server_retry as described above.

unavailable_server_retry_responses If parent_is_proxy is false and parent_retry is set toeither unavailable_server_retry or both, this parameter is a comma separated list of http 5xx re-sponse codes that will invoke the unavailable_server_retry described in the parent_retry sec-tion. By default, unavailable_server_retry_responses is set to 503.

max_simple_retries By default the value for max_simple_retries is 1. It may be set to any value in therange 1 to 5. If parent_is_proxy is false and parent_retry is set to simple_retry or both a 404reponse from a parent origin server will cause the request to be retried using a new parent at most 1 to 5 timesas configured by max_simple_retries.

max_unavailable_server_retries By default the value for max_unavailable_server_retries is1. It may be set to any value in the range 1 to 5. If parent_is_proxy is false and parent_retry is setto unavailable_server_retries or both a 503 reponse by default or any http 5xx response listed inthe list unavailable_server_retry_responses from a parent origin server will cause the request tobe retried using a new parent after first marking the current parent down. The request will be retried at most 1to 5 times as configured by max_unavailable_server_retries.

round_robin One of the following values:

• true - Traffic Server goes through the parent cache list in a round robin-based on client IP address.

• strict - Traffic Server machines serve requests strictly in turn. For example: machine proxy1 servesthe first request, proxy2 serves the second request, and so on.

• false - Round robin selection does not occur.

• consistent_hash - consistent hash of the url so that one parent is chosen for a given url. If a parentis down, the traffic that would go to the down parent is rehashed amongst the remaining parents. Theother traffic is unaffected. Once the downed parent becomes available, the traffic distribution returns to thepre-down state.

go_direct One of the following values:

• true - requests bypass parent hierarchies and go directly to the origin server.

• false - requests do not bypass parent hierarchies.

qstring One of the following values:

• consider - Use the query string when finding a parent.

• ignore - Do not consider the query string when finding a parent.



Examples

The following rule configures a parent cache hierarchy consisting of Traffic Server (which is the child) and two parents,p1.x.com and p2.x.com. Traffic Server forwards the requests it cannot serve to the parent servers p1.x.comand p2.x.com in a round-robin fashion:

round_robin=truedest_domain=. method=get parent="p1.x.com:8080; p2.y.com:8080" round_robin=trueround_robin=consistent_hashdest_domain=. method=get parent="p1.x.com:8080|1.0; p2.y.com:8080|2.0" round_→˓robin=consistent_hash

The following rule configures Traffic Server to route all requests containing the regular expression politics andthe path /viewpoint directly to the origin server (bypassing any parent hierarchies): url_regex=politicsprefix=/viewpoint go_direct=true

Every line in the parent.config file must contain either a parent= or go_direct= directive.

plugin.config

Description

The plugin.config file controls run-time loadable plugins available to Traffic Server, as well as their configura-tion. Plugins listed in this file are referred to as global plugins because they are always loaded and have global effect.This is in contrast to plugins specified in remap.config, whose effects are limited to the specific mapping rules towhich they are applied.

Each configuration line consists of a path to an .so file. This path can either be absolute, or relative to the plugindirectory (usually /usr/local/libexec/trafficserver). Failure to load a plugin is fatal, and will causeTraffic Server to abort. In general, it is not possible to know whether it is safe for the service to run without aparticular plugin, since plugins can have arbitrary effects on caching and authorization policies.

: Plugins should only be listed once. The order in which the plugins are listed is also the order in which they arechained for request processing.

An option list of whitespace-separated arguments may follow the plugin name. These are passed as an argument vec-tor to the plugin’s initialization function, TSPluginInit(). Arguments that begin with the $ character designateTraffic Server configuration variables. These arguments will be replaced with the value of the corresponding configu-ration variable before the plugin is loaded. When using configuration variable expansion, note that most Traffic Serverconfiguration can be changed. If a plugin requires the current value, it must obtain that using the management API.

Examples

# Comments start with a '#' and continue to the end of the line# Blank lines are ignored## test-plugin.so arg1 arg2 arg3#plugins/iwx/iwx.soplugins/abuse/abuse.so etc/trafficserver/abuse.configplugins/icx/icx.so etc/trafficserver/icx.config $proxy.config.http.connect_attempts_→˓timeout



See Also

TSAPI(3ts), TSPluginInit(3ts), remap.config(5)

records.config

The records.config file (by default, located in /usr/local/etc/trafficserver/) is a list of config-urable variables used by the Traffic Server software. Many of the variables in the records.config file are setautomatically when you set configuration options in Traffic Line. After you modify the records.config file, runthe command traffic_ctl config reload to apply the changes. When you apply changes to one node in acluster, Traffic Server automatically applies the changes to all other nodes in the cluster.

Format

Each variable has the following format:

SCOPE variable_name DATATYPE variable_value

where

SCOPE is related to clustering and is either CONFIG (all members of the cluster) or LOCAL (only the local machine)

DATATYPE is one of INT (integer), STRING (string), FLOAT (floating point).

A variable marked as Deprecated is still functional but should be avoided as it may be removed in a future releasewithout warning.

A variable marked as Reloadable can be updated via the command:


A variable marked as Overridable can be changed on a per-remap basis using plugins (like the ConfigurationRemap Plugin).

INT type configurations are expressed as any normal integer, e.g. 32768. They can also be expressed using morehuman readable values using standard prefixes, e.g. 32K. The following prefixes are supported for all INT typeconfigurations

• K Kilobytes (1024 bytes)

• M Megabytes (1024^2 or 1,048,576 bytes)

• G Gigabytes (1024^3 or 1,073,741,824 bytes)

• T Terabytes (1024^4 or 1,099,511,627,776 bytes)

: Traffic Server currently writes back configurations to disk periodically, and when doing so, will not preserve theprefixes.

Examples

In the following example, the variable ‘proxy.config.proxy_name‘_ is a STRING datatype with the valuemy_server. This means that the name of the Traffic Server proxy is my_server.



CONFIG proxy.config.proxy_name STRING my_server

If the server name should be that_server the line would be

CONFIG proxy.config.proxy_name STRING that_server

In the following example, the variable proxy.config.arm.enabled is a yes/no flag. A value of 0 (zero) disablesthe option; a value of 1 enables the option.

CONFIG proxy.config.arm.enabled INT 0

In the following example, the variable sets the cluster startup timeout to 10 seconds.

CONFIG proxy.config.cluster.startup_timeout INT 10

The last examples configures a 64GB RAM cache, using a human readable prefix.

CONFIG proxy.config.cache.ram_cache.size INT 64G

Environment Overrides

Every records.config configuration variable can be overridden by a corresponding environment variable. Thiscan be useful in situations where you need a static records.config but still want to tweak one or two settings.The override variable is formed by converting the records.config variable name to upper case, and replacing anydot separators with an underscore.

Overriding a variable from the environment is permanent and will not be affected by future configuration changesmade in records.config or applied with traffic_ctl.

For example, we could override the ‘proxy.config.product_company‘_ variable like this:

$ PROXY_CONFIG_PRODUCT_COMPANY=example traffic_cop &$ traffic_ctl config get proxy.config.product_company

Configuration Variables

The following list describes the configuration variables available in the records.config file.

System Variables

A value of 0 means no signal will be sent.

Value Effect0 assign threads to machine1 assign threads to NUMA nodes [default]2 assign threads to sockets3 assign threads to cores4 assign threads to processing units

: This option only has an affect when Traffic Server has been compiled with --enable-hwloc.



Metrics Variables

Network

Unless explicitly specified in ‘proxy.config.http.server_ports‘_ the server port will be bound to one of these ad-dresses, selected by IP address family. The built in default is any address. This is used if no address for a family isspecified. This setting is useful if most or all server ports should be bound to the same address.

: This is ignored for inbound transparent server ports because they must be able to accept connections on arbitrary IPaddresses.

Example

Set the global default for IPv4 to 192.168.101.18 and leave the global default for IPv6 as any address.:

LOCAL proxy.local.incoming_ip_to_bind STRING 192.168.101.18

Example

Set the global default for IPv4 to 191.68.101.18 and the global default for IPv6 tofc07:192:168:101::17.:LOCAL proxy.local.incoming_ip_to_bind STRING 192.168.101.18 [fc07:192:168:101::17]

: This is ignored for outbound transparent ports as the local outbound address will be the same as the client localaddress.

Example

Set the default local outbound IP address for IPv4 connections to 192.168.101.18.:LOCAL proxy.local.outgoing_ip_to_bind STRING 192.168.101.18

Example

Set the default local outbound IP address to 192.168.101.17 for IPv4 and fc07:192:168:101::17 forIPv6.:LOCAL proxy.local.outgoing_ip_to_bind STRING 192.168.101.17 [fc07:192:168:101::17]



Cluster

Value Effect1 full-clustering mode2 management-only mode3 no clustering

The network interface to be used for cluster communication. This has to be identical on all members of a clsuter.ToDo: Is that reasonable ?? Should this be local”

Local Manager

This setting specifies whether the management API should be restricted to root processes. If this is set to 0, then onplatforms that support passing process credentials, non-root processes will be allowed to make read-only managementAPI calls. Any management API calls that modify server state (eg. setting a configuration variable) will still berestricted to root processes.

This setting is not reloadable, since it is must be applied when program:traffic_manager initializes.

Process Manager

Alarm Configuration

HTTP Engine

This is a list, separated by space or comma, of port descriptors. Each descriptor is a sequence of keywords and valuesseparated by colons. Not all keywords have values, those that do are specifically noted. Keywords with values can havean optional ‘=’ character separating the keyword and value. The case of keywords is ignored. The order of keywordsis irrelevant but unspecified results may occur if incompatible options are used (noted below). Options without valuesare idempotent. Options with values use the last (right most) value specified, except for ip-out as detailed later.

Quick reference chart.

Name Note Definitionnumber Required The local port.blind Blind (CONNECT) port.compress N/I Compressed. Not implemented.ipv4 Default Bind to IPv4 address family.ipv6 Bind to IPv6 address family.ip-in Value Local inbound IP address.ip-out Value Local outbound IP address.ip-resolve Value IP address resolution style.proto Value List of supported session protocols.ssl SSL terminated.tr-full Fully transparent (inbound and outbound)tr-in Inbound transparent.tr-out Outbound transparent.tr-pass Pass through enabled.

number Local IP port to bind. This is the port to which ATS clients will connect.

blind Accept only the CONNECT method on this port.

Not compatible with: tr-in, ssl.

compress Compress the connection. Retained only by inertia, should be considered “not implemented”.



ipv4 Use IPv4. This is the default and is included primarily for completeness. This forced if the ip-in option isused with an IPv4 address.

ipv6 Use IPv6. This is forced if the ip-in option is used with an IPv6 address.

ssl Require SSL termination for inbound connections. SSL must be configured for this option to provide a functionalserver port.

Not compatible with: blind.

proto Specify the session level protocols supported. These should be separated by semi-colons. For TLS proxy portsthe default value is all available protocols. For non-TLS proxy ports the default is HTTP only.

tr-full Fully transparent. This is a convenience option and is identical to specifying both tr-in and tr-out.

Not compatible with: Any option not compatible with tr-in or tr-out.

tr-in Inbound transparent. The proxy port will accept connections to any IP address on the port. To have IPv6 inboundtransparent you must use this and the ipv6 option. This overrides :ts:cv:‘proxy.local.incoming_ip_to_bind‘for this port.

Not compatible with: ip-in, blind

tr-out Outbound transparent. If ATS connects to an origin server for a transaction on this port, it will use the client’saddress as its local address. This overrides :ts:cv:‘proxy.local.outgoing_ip_to_bind‘ for this port.

Not compatible with: ip-out, ip-resolve

tr-pass Transparent pass through. This option is useful only for inbound transparent proxy ports. If the parsingof the expected HTTP header fails, then the transaction is switched to a blind tunnel instead of generating anerror response to the client. It effectively enables :ts:cv:‘proxy.config.http.use_client_target_addr‘ for thetransaction as there is no other place to obtain the origin server address.

ip-in Set the local IP address for the port. This is the address to which clients will connect. This forces the IP addressfamily for the port. The ipv4 or ipv6 can be used but it is optional and is an error for it to disagree with theIP address family of this value. An IPv6 address must be enclosed in square brackets. If this option is omitted:ts:cv:‘proxy.local.incoming_ip_to_bind‘ is used.

Not compatible with: tr-in.

ip-out Set the local IP address for outbound connections. This is the address used by ATS locally when it connectsto an origin server for transactions on this port. If this is omitted :ts:cv:‘proxy.local.outgoing_ip_to_bind‘ isused.

This option can used multiple times, once for each IP address family. The address used is selected by the IPaddress family of the origin server address.

Not compatible with: tr-out.

ip-resolve Set the :ts:cv:‘host resolution style <proxy.config.hostdb.ip_resolve>‘ for transactions on this proxyport.

Not compatible with: tr-out - this option requires a value of client;none which is forced and should notbe explicitly specified.

Example

Listen on port 80 on any address for IPv4 and IPv6.:

80 80:ipv6



Example

Listen transparently on any IPv4 address on port 8080, and transparently on port 8080 on local addressfc01:10:10:1::1 (which implies ipv6).:

IPv4:tr-FULL:8080 TR-full:IP-in=[fc02:10:10:1::1]:8080

Example

Listen on port 8080 for IPv6, fully transparent. Set up an SSL port on 443. These ports will use the IP address from:ts:cv:‘proxy.local.incoming_ip_to_bind‘. Listen on IP address 192.168.17.1, port 80, IPv4, and connect toorigin servers using the local address 10.10.10.1 for IPv4 and fc01:10:10:1::1 for IPv6.:

8080:ipv6:tr-full 443:ssl ip-in=192.168.17.1:80:ip-out=[fc01:10:10:1::1]:ip-out=10.→˓10.10.1

Example

Listen on port 9090 for TSL enabled HTTP/2 or HTTP connections, accept no other session protocols.:

9090:proto=http2;http:ssl

Traffic Server allows tunnels only to the specified ports. Supports both wildcards (‘*’) and ranges (“0-1023”).

: These are the ports on the origin server, not Traffic Server :ts:cv:‘proxy ports <proxy.config.http.server_ports>‘.

Value Effect0 Do not modify / set this via header1 Update the via, with normal verbosity2 Update the via, with higher verbosity3 Update the via, with highest verbosity

: The Via header string can be decoded with the Via Decoder Ring.

Value Effect0 Do not modify / set this via header1 Update the via, with normal verbosity2 Update the via, with higher verbosity3 Update the via, with highest verbosity

: The Via header string can be decoded with the Via Decoder Ring.

Parent Proxy Configuration

HTTP Connection Timeouts

The value of 0 specifies that there is no timeout.


http://trafficserver.apache.org/tools/via



The default value of 0 specifies that there is no timeout.

HTTP Redirection

Origin Server Connect Attempts

Congestion Control

Negative Response Caching

Proxy User Variables

Security

Cache Control

RAM Cache

Heuristic Expiration

: These fuzzing options are marked as deprecated as of v6.2.0, and will be removed for v7.0.0. Instead, we recommendlooking at the new :ts:cv:‘proxy-config-http-cache-open-write-fail-action‘ configuration and the features aroundthundering heard avoidance (see cache-basics for details).

Dynamic Content & Content Negotiation

For example: if you specify User-agent, then Traffic Server caches all the different user-agent versions of docu-ments it encounters.

Customizable User Response Pages

DNS

: If the variable :ts:cv:‘proxy.config.http.enable_url_expandomatic‘ is set to 1 (the default value), then you do nothave to add www. and .com to this list because Traffic Server automatically tries www. and .com after trying thevalues you’ve specified.

HostDB

: For values above 200000, you must increase :ts:cv:‘proxy.config.hostdb.storage_size‘ by at least 44 bytes perentry.



When this and :ts:cv:‘proxy.config.hostdb.timed_round_robin‘ are both disabled (set to 0), Traffic Server al-ways uses the same origin server for the same client, for as long as the origin server is available. Oth-erwise if this is set then IP address is rotated on every request. This setting takes precedence over:ts:cv:‘proxy.config.hostdb.timed_round_robin‘.

When this and :ts:cv:‘proxy.config.hostdb.strict_round_robin‘ are both disabled (set to 0), Traffic Server alwaysuses the same origin server for the same client, for as long as the origin server is available. Otherwise if this is set to:arg:‘N‘ the IP address is rotated if more than :arg:‘N‘ seconds have past since the first time the current address wasused.

If this is set (non-empty) then the file is presumed to be a hosts file in the standard host file format. It is read and theentries there added to the HostDB. The file is periodically checked for a more recent modification date in which caseit is reloaded. The interval is set by the value :ts:cv:‘proxy.config.hostdb.host_file.interval‘.

While not technically reloadable, the value is read every time the file is to be checked so that if changed the new valuewill be used on the next check and the file will be treated as modified.

The file is checked every this many seconds to see if it has changed. If so the HostDB is updated with the new valuesin the file.

This is an ordered list of keywords separated by semicolons that specify how a host name is to be resolved to an IPaddress. The keywords are case insensitive.

Keyword Meaningipv4 Resolve to an IPv4 address.ipv6 Resolve to an IPv6 address.client Resolve to the same family as the client IP address.none Stop resolving.

The order of the keywords is critical. When a host name needs to be resolved it is resolved in same order as thekeywords. If a resolution fails, the next option in the list is tried. The keyword none means to give up resolutionentirely. The keyword list has a maximum length of three keywords, more are never needed. By default there is animplicit ipv4;ipv6 attached to the end of the string unless the keyword none appears.

Example

Use the incoming client family, then try IPv4 and IPv6.

client;ipv4;ipv6

Because of the implicit resolution this can also be expressed as just

client

Example

Resolve only to IPv4.

ipv4;none

Example

Resolve only to the same family as the client (do not permit cross family transactions).

client;none


http://tools.ietf.org/html/rfc1123#page-13


This value is a global default that can be overridden by :ts:cv:‘proxy.config.http.server_ports‘.

: This style is used as a convenience for the administrator. During a resolution the resolution order will be one family,then possibly the other. This is determined by changing client to ipv4 or ipv6 based on the client IP address andthen removing duplicates.

: This option has no effect on outbound transparent connections The local IP address used in the connection to theorigin server is determined by the client, which forces the IP address family of the address used for the origin server.In effect, outbound transparent connections always use a resolution style of “client”.

Logging Configuration

: All files in the logging directory contribute to the space used, even if they are not log files. In collationclient mode, if there is no local disk logging, or :ts:cv:‘proxy.config.log.max_space_mb_for_orphan_logs‘is set to a higher value than :ts:cv:‘proxy.config.log.max_space_mb_for_logs‘, TS will take:ts:cv:‘proxy.config.log.max_space_mb_for_orphan_logs‘ for maximum allowed log space.

: When max_space_mb_for_orphan_logs is take as the maximum allowed log space in thelogging system, the same rule apply to proxy.config.log.max_space_mb_for_logs also apply toproxy.config.log.max_space_mb_for_orphan_logs, ie: All files in the logging directory contribute to the spaceused, even if they are not log files. you may need to consider this when you enable full remote logging, and bump tothe same size as proxy.config.log.max_space_mb_for_logs.

: The following defaults (with the format‘‘CONFIG proxy.config.log.[format].[default]‘‘ have been removed fromrecords.config. They can be accessed by editing logs_xml.config.

• log_enabled INT 1 Enables (1) or disables (0) the file format.

• log_is_ascii INT 0

The log file type:

– 1 = ASCII

– 0 = binary

• log_name STRING [format] The filename (ex. squid log).

• log_header STRING NULL The file header text (ex. squid log).

The format can be either squid (Squid), common (Netscape Common), extended (Netscape Extended), orextended2 (Netscape Extended2).



Value Effect0 Log collation is disabled.1 This host is a log collation server.2 This host is a collation client and sends entries using standard formats to the collation server.3 This host is a collation client and sends entries using the traditional custom formats to the collation server.4 This host is a collation client and sends entries that use both the standard and traditional custom formats

to the collation server.

For information on sending XML-based custom formats to the collation server, refer to Log Formats and logs_xml.config.

: Although Traffic Server supports traditional custom logging, you should use the more versatile XML-based customformats.

Diagnostic Logging Configuration

Example

To log debug diagnostics to both syslog and diags.log:

CONFIG proxy.config.diags.output.debug STRING SL

Tag Subsytem usagedns DNS query resolutionhttp_hdrs Logs the headers for HTTP requests and responsesprivileges Privilege elevationssl TLS termination and certificate processing

Traffic Server plugins will typically log debug messages using the TSDebug() API, passing the pluginname as the debug tag.

Reverse Proxy

URL Remap Rules

SSL Termination

Client-Related Configuration

OCSP Stapling Configuration

ICP Configuration

HTTP/2 Configuration

SPDY Configuration

Plug-in Configuration



SOCKS Processor

Sockets

Value Effect0 Decouple inbound connections and cache initialization. Connections will be accepted as soon as possible

and Traffic Server will run regardless of the results of cache initialization.1 Do not accept inbound connections until cache initialization has finished. Traffic server will run regardless

of the results of cache initialization.2 Do not accept inbound connections until cache initialization has finished and been sufficiently successful

that cache is enabled. This means at least one cache span is usable. If there are no spans instorage.config or none of the spans can be successfully parsed and initialized then Traffic Server willshut down.

3 Do not accept inbound connections until cache initialization has finished and been completely successful.This requires at least one cache span in storage.config and that every span specified is valid andsuccessfully initialized. Any error will cause Traffic Server to shut down.

remap.config

The remap.config file (by default, located in /usr/local/etc/trafficserver/) contains mapping rulesthat Traffic Server uses to perform the following actions:

• Map URL requests for a specific origin server to the appropriate location on Traffic Server when Traffic Serveracts as a reverse proxy for that particular origin server

• Reverse-map server location headers so that when origin servers respond to a request with a location header thatredirects the client to another location, the clients do not bypass Traffic Server

• Redirect HTTP requests permanently or temporarily without Traffic Server having to contact any origin servers

Refer to Reverse Proxy and HTTP Redirects, for information about redirecting HTTP requests and using reverse proxy.

After you modify the remap.config run the traffic_ctl config reload to apply the changes. When youapply the changes to one node in a cluster, Traffic Server automatically applies the changes to all other nodes in thecluster.

Format

Each line in the remap.config file must contain a mapping rule. Empty lines, or lines starting with # are ignored.Each line can be broken up into multiple lines for better readability by using \ as continuation marker.

Traffic Server recognizes three space-delimited fields: type, target, and replacement. The following listdescribes the format of each field.

type Enter one of the following:

• map –translates an incoming request URL to the appropriate origin server URL.

• map_with_recv_port –exactly like ‘map’ except that it uses the port at which the request was receivedto perform the mapping instead of the port present in the request. The regex qualifier can also be used forthis type. When present, ‘map_with_recv_port’ mappings are checked first. If there is a match, then it ischosen without evaluating the “regular” forward mapping rules.

• map_with_referer – extended version of ‘map’, which can be used to activate “deep linking protec-tion”, where target URLs are only accessible when the Referer header is set to a URL that is allowed tolink to the target.



• reverse_map –translates the URL in origin server redirect responses to point to the Traffic Server.

• redirect –redirects HTTP requests permanently without having to contact the origin server. Permanentredirects notify the browser of the URL change (by returning an HTTP status code 301) so that the browsercan update bookmarks.

• redirect_temporary –redirects HTTP requests temporarily without having to contact the originserver. Temporary redirects notify the browser of the URL change for the current request only (by re-turning an HTTP status code 307).

target Enter the origin (“from”) URL. You can enter up to four components:

scheme://host:port/path_prefix

where scheme is http, https, ws or wss.

replacement Enter the origin (“from”) URL. You can enter up to four components:

scheme://host:port/path_prefix

where scheme is http, https, ws or wss.

Precedence

Remap rules are not processed top-down, but based on an internal priority. Once these rules are executed we pick thefirst match based on configuration file parse order.

1. map_with_recv_port and `regex_map_with_recv_port`

2. map and regex_map and reverse_map

3. redirect and redirect_temporary

4. regex_redirect and regex_redirect_temporary

Match-All

A map rule with a single / acts as a wildcard, it will match any request. This should be use with care, and certainlyonly once at the end of the remap.config file. E.g.

map / http://all.example.com

Examples

The following section shows example mapping rules in the remap.config file.

Reverse Proxy Mapping Rules

The following example shows a map rule that does not specify a path prefix in the target or replacement:

map http://www.x.com/ http://server.hoster.com/reverse_map http://server.hoster.com/ http://www.x.com/



This rule results in the following translations:

Client Request Translated Requesthttp://www.x.com/Widgets/index.html

http://server.hoster.com/Widgets/index.html

http://www.x.com/cgi/form/submit.sh?arg=true

http://server.hoster.com/cgi/form/submit.sh?arg=true

The following example shows a map rule with path prefixes specified in the target:

map http://www.y.com/marketing/ http://marketing.y.com/reverse_map http://marketing.y.com/ http://www.y.com/marketing/map http://www.y.com/sales/ http://sales.y.com/reverse_map http://sales.y.com/ http://www.y.com/sales/map http://www.y.com/engineering/ http://engineering.y.com/reverse_map http://engineering.y.com/ http://www.y.com/engineering/map http://www.y.com/stuff/ http://info.y.com/reverse_map http://info.y.com/ http://www.y.com/stuff/

These rules result in the following translations:

Client Request Translated Requesthttp://www.y.com/marketing/projects/manhattan/specs.html

http://marketing.y.com/projects/manhattan/specs.html

http://www.y.com/stuff/marketing/projects/boston/specs.html

http://info.y.com/marketing/projects/boston/specs.html

http://www.y.com/engineering/marketing/requirements.html

http://engineering.y.com/marketing/requirements.html

The following example shows that the order of the rules matters:

map http://www.g.com/ http://external.g.com/reverse_map http://external.g.com/ http://www.g.com/map http://www.g.com/stuff/ http://stuff.g.com/reverse_map http://stuff.g.com/ http://www.g.com/stuff/

These rules result in the following translation.

Client Request Translated Requesthttp://www.g.com/stuff/a.gif http://external.g.com/stuff/a.gif

In the above examples, the second rule is never applied because all URLs that match the second rule also match thefirst rule. The first rule takes precedence because it appears earlier in the remap.config file.

The following example shows a mapping with a path prefix specified in the target and replacement:

map http://www.h.com/a/b/ http://server.h.com/customers/x/yreverse_map http://server.h.com/customers/x/y/ http://www.h.com/a/b/

This rule results in the following translation.

Client Request Translated Requesthttp://www.h.com/a/b/c/d/doc.html

http://server.h.com/customers/x/y/c/d/doc.html

http://www.h.com/a/index.html Translation fails

The following example shows reverse-map rules:

map http://www.x.com/ http://server.hoster.com/x/reverse_map http://server.hoster.com/x/ http://www.x.com/



These rules result in the following translations.

Client Request Translated Requesthttp://www.x.com/Widgets http://server.hoster.com/x/Widgets

Client Request Origin Server Header Translated Requesthttp://www.x.com/Widgets

http://server.hoster.com/x/Widgets/

http://www.x.com/Widgets/

When acting as a reverse proxy for multiple servers, Traffic Server is unable to route to URLsfrom older browsers that do not send the Host: header. As a solution, set the variable:ts:cv:‘proxy.config.header.parse.no_host_url_redirect‘ in the records.config file to the URL to whichTraffic Server will redirect requests without host headers.

Redirect Mapping Rules

The following rule permanently redirects all HTTP requests for www.company.com to www.company2.com:

redirect http://www.company.com/ http://www.company2.com/

The following rule temporarily redirects all HTTP requests for www.company1.com to www.company2.com:

redirect_temporary http://www.company1.com/ http://www.company2.com/

Regular Expression (regex) Remap Support

Regular expressions can be specified in remapping rules, with the limitations below:

• Only the host field can contain a regex; the scheme, port, and other fields cannot. For path manipulationvia regexes, use the regex_remap plugin.

• The number of capturing subpatterns is limited to 9. This means that $0 through $9 can be used as subtractionplaceholders ($0 will be the entire input string).

• The number of substitutions in the expansion string is limited to 10.

• There is no regex_ equivalent to reverse_remap, so when using regex_remap you should make surethe reverse path is clear by setting (:ts:cv:‘proxy.config.url_remap.pristine_host_hdr‘)

Examples

regex_map http://x([0-9]+).z.com/ http://real-x$1.z.com/regex_redirect http://old.(.*).z.com http://new.$1.z.com

map_with_referer

the format of is the following:

map_with_referer client-URL origin-server-URL redirect-URL regex1 [regex2 ...]

‘redirect-URL’ is a redirection URL specified according to RFC 2616 and can contain special formatting instructionsfor run-time modifications of the resulting redirection URL. All regexes Perl compatible regular expressions, whichdescribes the content of the “Referer” header which must be verified. In case an actual request does not have “Referer”header or it does not match with referer regular expression, the HTTP request will be redirected to ‘redirect-URL’.



At least one regular expressions must be specified in order to activate ‘deep linking protection’. There are limitationsfor the number of referer regular expression strings - 2048. In order to enable the ‘deep linking protection’ feature inTraffic Server, configure records.config with:

CONFIG proxy.config.http.referer_filter INT 1

In order to enable run-time formatting for redirect URL, configure:

CONFIG proxy.config.http.referer_format_redirect INT 1

When run-time formatting for redirect-URL was enabled the following format symbols can be used:

%r - to substitute original "Referer" header string%f - to substitute client-URL from 'map_with_referer' record%t - to substitute origin-server-URL from 'map_with_referer' record%o - to substitute request URL to origin server, which was created a

the result of a mapping operation

Note: There is a special referer type “~*” that can be used in order to specify that the Referer header is optional in therequest. If “~*” referer was used in map_with_referer mapping, only requests with Referer header will be verified forvalidity. If the “~” symbol was specified before referer regular expression, it means that the request with a matchingreferer header will be redirected to redirectURL. It can be used to create a so-called negative referer list. If “*” wasused as a referer regular expression - all referers are allowed. Various combinations of “*” and “~” in a referer list canbe used to create different filtering rules.

map_with_referer Examples

map_with_referer http://y.foo.bar.com/x/yy/ http://foo.bar.com/x/yy/ http://games.→˓bar.com/new_games .*\.bar\.com www.bar-friends.com

Explanation: Referer header must be in the request, only ”.*.bar.com” and “www.bar-friends.com” are allowed.

map_with_referer http://y.foo.bar.com/x/yy/ http://foo.bar.com/x/yy/ http://games.→˓bar.com/new_games * ~.*\.evil\.com

Explanation: Referer header must be in the request but all referers are allowed except ”.*.evil.com”.

map_with_referer http://y.foo.bar.com/x/yy/ http://foo.bar.com/x/yy/ http://games.→˓bar.com/error ~* * ~.*\.evil\.com

Explanation: Referer header is optional. However, if Referer header exists, only request from ”.*.evil.com” will beredirected to redirect-URL.

Plugin Chaining

Plugins can be configured to be evaluated in a specific order, passing the results from one in to the next (unless a pluginreturns 0, then the “chain” is broken).

Examples

map http://url/path http://url/path \@plugin=/etc/traffic_server/config/plugins/plugin1.so @pparam=1 @pparam=2 \@plugin=/etc/traffic_server/config/plugins/plugin2.so @pparam=3



will pass “1” and “2” to plugin1.so and “3” to plugin2.so.

This will pass “1” and “2” to plugin1.so and “3” to plugin2.so

Acl Filters

Acl filters can be created to control access of specific remap lines. The markup is very similar to that of ip_allow.config, with slight changes to accomodate remap markup

Examples

map http://foo.example.com/neverpost http://foo.example.com/neverpost @action=deny→˓@method=postmap http://foo.example.com/onlypost http://foo.example.com/onlypost @action=allow→˓@method=post

map http://foo.example.com/ http://foo.example.com/ @action=deny @src_ip=1.2.3.4map http://foo.example.com/ http://foo.example.com/ @action=allow @src_ip=127.0.0.1

map http://foo.example.com/ http://foo.example.com/ @action=allow @src_ip=10.5.2.1→˓@in_ip=72.209.23.4

map http://foo.example.com/ http://foo.example.com/ @action=allow @src_ip=127.0.0.1→˓@method=post @method=get @method=head

Note that these Acl filters will return a 403 response if the resource is restricted.

The difference between @src_ip and @in_ip is that the @src_ip is the client ip and the in_ip is the ip addressthe client is connecting to (the incoming address).

Named Filters

Named filters can be created and applied to blocks of mappings using the .definefilter, .activatefilter,and .deactivatefilter directives. Named filters must be defined using .definefilter before being used.Once defined, .activatefilter can used to activate a filter for all mappings that follow until deactivated with.deactivatefilter.

The @internal operator can be used to filter on whether a request is generated by Traffic Server itself, usually by aplugin. This operator is helpful for remapping internal requests without allowing access to external users. By defaultboth internal and external requests are allowed.

Examples

.definefilter disable_delete_purge @action=deny @method=delete @method=purge

.definefilter local_only @action=allow @src_ip=192.168.0.1-192.168.0.254 @src_ip=10.0.→˓0.1-10.0.0.254

.activatefilter disable_delete_purge

map http://foo.example.com/ http://bar.example.com/

.activatefilter local_onlymap http://www.example.com/admin http://internal.example.com/admin



.deactivatefilter local_only

map http://www.example.com/ http://internal.example.com/map http://auth.example.com/ http://auth.internal.example.com/ @action=allow @internal

The filter disable_delete_purge will be applied to all of the mapping rules. (It is activated before any mappings and isnever deactivated.) The filter local_only will only be applied to the second mapping.

Including Additional Remap Files

The .include directive allows mapping rules to be spread across multiple files. The argument to the .includedirective is a list of file names to be parsed for additional mapping rules. Unless the names are absolute paths, they areresolved relative to the Traffic Server configuration directory.

The effect of the .include directive is as if the contents of the listed files is included in the parent and parsingrestarted at the point of inclusion. This means that and filters named in the included files are global in scope, and thatadditional .include directives are allowed.

: Included remap files are not currently tracked by the configuration subsystem. Changes to included remap fileswill not be noticed by online configuration changes applied by traffic_ctl config reload unless remap.config has also changed.

Examples

In this example, a top-level remap.config file simply references additional mapping rules files

.include filters.config

.include one.example.com.config two.example.com.config

The file filters.config contains

.definefilter deny_purge @action=deny @method=purge

.definefilter allow_purge @action=allow @method=purge

The file one.example.com.config contains:

.activatefilter deny_purgemap http://one.example.com http://origin-one.example.com.deactivatefilter deny_purge

The file two.example.com.config contains:

.activatefilter allow_purgemap http://two.example.com http://origin-two.example.com.deactivatefilter dallowpurge

splitdns.config

The splitdns.config file enables you to specify the DNS server that Traffic Server should use for resolving hostsunder specific conditions. For more information, refer to Split DNS.

To specify a DNS server, you must supply the following information in each active line within the file:



• A primary destination specifier in the form of a destination domain, a destination host, or a URL regular expres-sion

• A set of server directives, listing one or more DNS servers with corresponding port numbers

You can also include the following optional information with each DNS server specification:

• A default domain for resolving hosts

• A search list specifying the domain search order when multiple domains are specified

After you modify the splitdns.config file, run the traffic_ctl config reload command to apply thechanges. When you apply changes to a node in a cluster, Traffic Server automatically applies the changes to all othernodes in the cluster.

Format

Each line in the splitdns.config file uses one of the following formats:

dest_domain=dest_domain | dest_host | url_regex named=dns_server def_domain=def_→˓domain search_list=search_list

The following list describes each field.

dest_domain A valid domain name. This specifies that DNS server selection will be based on the destinationdomain. You can prefix the domain with an exclamation mark (!) to indicate the NOT logical operator.

dest_host A valid hostname. This specifies that DNS server selection will be based on the destination host. Youcan prefix the host with an exclamation mark (!) to indicate the NOT logical operator.

url_regex A valid URL regular expression. This specifies that DNS server selection will be based on a regularexpression.

named This is a required directive. It identifies the DNS server that Traffic Server should use with the given destina-tion specifier. You can specify a port using a colon (:). If you do not specify a port, then 53 is used. Specifymultiple DNS servers with spaces or semicolons (;) as separators.

You must specify the domains with IP addresses in CIDR (“dot”) notation.

def_domain A valid domain name. This optional directive specifies the default domain name to use for resolvinghosts. Only one entry is allowed. If you do not provide the default domain, the system determines its value from/etc/resolv.conf

search_list A list of domains separated by spaces or semicolons (;). This specifies the domain search order. Ifyou do not provide the search list, the system determines the value from resolv.conf(5)

Examples

Consider the following DNS server selection specifications:

dest_domain=internal.company.com named=255.255.255.255:212 255.255.255.254 def_→˓domain=company.com search_list=company.com company1.comdest_domain=!internal.company.com named=255.255.255.253

Now consider the following two requests:

http://minstar.internal.company.com



This request matches the first line and therefore selects DNS server 255.255.255.255 on port 212. All resolverrequests use company.com as the default domain, and company.com and company1.com as the set of domainsto search first.

http://www.microsoft.com

This request matches the second line. Therefore, Traffic Server selects DNS server 255.255.255.253. Because nodef_domain or search_list was supplied, Traffic Server retrieves this information from resolv.conf(5)

ssl_multicert.config

The ssl_multicert.config file lets you configure Traffic Server to use multiple SSL server certificates toterminate the SSL sessions. If you have a Traffic Server system with more than one IP address assigned to it, then youcan assign a different SSL certificate to be served when a client requests a particular IP address or host name.

At configuration time, certificates are parsed to extract the certificate subject and all the DNS subject alternative names.A certificate will be presented for connections requesting any of the hostnames found in the certificate. Wildcard namesare supported, but only of the form *.domain.com, ie. where * is the leftmost domain component.

Changes to ssl_multicert.config can be applied to a running Traffic Server using traffic_ctl configreload.

Format

Each ssl_multicert.config line consists of a sequence of key=value fields that specify how Traffic Servershould use a particular SSL certificate.

ssl_cert_name=FILENAME[,FILENAME ...] The name of the file containing the TLS certificate. FILENAME islocated relative to the directory specified by the :ts:cv:‘proxy.config.ssl.server.cert.path‘ configuration vari-able. It may also include the intermediate CA certificates, sorted from leaf to root. At a minimum, the file mustinclude a leaf certificate.

When running with OpenSSL 1.0.2 or later, this directive can be used to configure the intermediate CA chain ona per-certificate basis. Multiple chain files are separated by comma character. For example, it is possible ableto configure a ECDSA certificate chain and a RSA certificate chain and serve them simultaneously, allowingOpenSSL to determine which certificate would be used when the TLS session cipher suites are negotiated. Notethat the leaf certs in FILENAME1 and FILENAME2 must have the same subjects and alternate names. The firstcertificate is used to to match the client’s SNI request.

You can also configure multiple leaf certificates in a same chain with OpenSSL 1.0.1.

This is the only field that is required to be present.

dest_ip=ADDRESS (optional) The IP (v4 or v6) address that the certificate should be presented on. This is now onlyused as a fallback in the case that the TLS SubjectNameIndication extension is not supported. If ADDRESS is*, the corresponding certificate will be used as the global default fallback if no other match can be made. Theaddress may contain a port specifier, in which case the corresponding certificate will only match for connectionsaccepted on the specified port. IPv6 addresses must be enclosed by square brackets if they have a port, eg,[::1]:80. Care should be taken to make each ADDRESS unique.

ssl_key_name=FILENAME (optional) The name of the file containing the private key for this certificate. If the keyis contained in the certificate file, this field can be omitted, otherwise FILENAME is resolved relative to the:ts:cv:‘proxy.config.ssl.server.private_key.path‘ configuration variable.

ssl_ca_name=FILENAME (optional) If the certificate is issued by an authority that is not in the system CA bundle,additional certificates may be needed to validate the certificate chain. FILENAME is resolved relative to the:ts:cv:‘proxy.config.ssl.CA.cert.path‘ configuration variable.


http://en.wikipedia.org/wiki/SubjectAltName


ssl_ticket_enabled=1|0 (optional) Enable RFC 5077 stateless TLS session tickets. To support this, OpenSSL shouldbe upgraded to version 0.9.8f or higher. This option must be set to 0 to disable session ticket support.

ticket_key_name=FILENAME (optional) The name of session ticket key file which contains a secret for encrypt-ing and decrypting TLS session tickets. If FILENAME is not an absolute path, it is resolved relative to the:ts:cv:‘proxy.config.ssl.server.cert.path‘ configuration variable. This option has no effect if session tick-ets are disabled by the ssl_ticket_enabled option. The contents of the key file should be 48 random(ASCII) bytes. One way to generate this would be to run head -c48 /dev/urandom | openssl enc-base64 | head -c48 > file.ticket.

Session ticket support is enabled by default. If neither of the ssl_ticket_enabled andticket_key_name options are specified, and internal session ticket key is generated. This key will bedifferent each time Traffic Server is started.

ssl_key_dialog=builtin|”exec:/path/to/program [args]” (optional) Method used to provide a pass phrase for en-crypted private keys. If the pass phrase is incorrect, SSL negotiation for this dest_ip will fail for clients whoattempt to connect. Two options are supported: builtin and exec:

builtin - Requests pass phrase via stdin/stdout. User will be provided the ssl_cert_name andbe prompted for the pass phrase. Useful for debugging.

exec: - Executes program /path/to/program and passes args, if specified, to the program andreads the output from stdout for the pass phrase. If args are provided then the entire exec:string must be quoted with “” (see examples). Arguments with white space are supported bysingle quoting (‘). The intent is that this program runs a security check to ensure that the systemis not compromised by an attacker before providing the pass phrase.

Certificate Selection

Traffic Server attempts two certificate selections during SSL connection setup. An initial selection is made when aTCP connection is accepted. This selection examines the IP address and port that the client is connecting to andchooses the best certificate from the those that have a dest_ip specification. If no matching certificates are found,a default certificate is chosen. The final certificate selection is made during the SSL handshake. At this point, theclient may use Server Name Indication to request a specific hostname. Traffic Server will use this request to select acertificate with a matching subject or subject alternative name. Failing that, a wildcard certificate match is attempted.If no match can be made, the initial certificate selection remains in force.

In all cases, Traffic Server attempts to select the most specific match. An address specification that contains a portnumber will take precedence over a specification that does not contain a port number. A specific certificate subjectwill take precedence over a wildcard certificate. In the case of multiple matching certificates the first match will bereturned to non-SNI capable clients.

Examples

The following example configures Traffic Server to use the SSL certificate server.pem for all requests to the IPaddress 111.11.11.1 and the SSL certificate server1.pem for all requests to the IP address 11.1.1.1. Connectionsfrom all other IP addresses are terminated with the default.pem certificate. Since the private key is included in thecertificate files, no private key name is specified.

dest_ip=111.11.11.1 ssl_cert_name=server.pemdest_ip=11.1.1.1 ssl_cert_name=server1.pemdest_ip=* ssl_cert_name=default.pem

The following example configures Traffic Server to use the ECDSA certificate chain ecdsa.pem or RSA certificatechain rsa.pem for all requests.


http://en.wikipedia.org/wiki/Server_Name_Indication


dest_ip=* ssl_cert_name=ecdsa.pem,rsa.pem

The following example configures Traffic Server to use the ECDSA certificate chain ecdsa.pem or RSA certificatechain rsa.pem for all requests, the public key and private key are in separate PEM files. Note that the number offiles in ssl_key_name must match the files in ssl_cert_name, and they should be presented in the same order.

dest_ip=* ssl_cert_name=ecdsa_pub.pem,rsa_pub.pem ssl_key_name=ecdsa_private.pem,rsa_→˓private.pem

The following example configures Traffic Server to use the SSL certificate server.pem and the private keyserverKey.pem for all requests to port 8443 on IP address 111.11.11.1. The general.pem certificate is used forserver name matches.

dest_ip=111.11.11.1:8443 ssl_cert_name=server.pem ssl_key_name=serverKey.pem ssl_cert_→˓name=general.pem

The following example configures Traffic Server to use the SSL certificate server.pem for all requests to the IPaddress 111.11.11.1. Session tickets are enabled with a persistent ticket key.

dest_ip=111.11.11.1 ssl_cert_name=server.pem ssl_ticket_enabled=1 ticket_key_→˓name=ticket.key

The following example configures Traffic Server to use the SSL certificate server.pem and disable session ticketsfor all requests to the IP address 111.11.11.1.

dest_ip=111.11.11.1 ssl_cert_name=server.pem ssl_ticket_enabled=0

The following examples configure Traffic Server to use the SSL certificate server.pemwhich includes an encryptedprivate key. The external program /usr/bin/mypass will be called on startup with one parameter (foo) in the firstexample, and with two parameters (foo) and (ba r) in the second example, the program (mypass) will return the passphrase to decrypt the keys.

ssl_cert_name=server1.pem ssl_key_dialog="exec:/usr/bin/mypass foo"ssl_cert_name=server2.pem ssl_key_dialog="exec:/usr/bin/mypass foo 'ba r'"

storage.config

The storage.config file (by default, located in /usr/local/etc/trafficserver/) lists all the files,directories, and/or hard disk partitions that make up the Traffic Server cache. After you modify the storage.config file the new settings will not be effective until Traffic Server is restarted.

Format

The format of the storage.config file is a series of lines of the form

pathname size [ volume=number ] [ id=string ]

where :arg:‘pathname‘ is the name of a partition, directory or file, :arg:‘size‘ is the size of the named partition, direc-tory or file (in bytes), and :arg:‘volume‘ is the volume number used in the files volume.config and hosting.config. :arg:‘id‘ is used for seeding the Assignment Table. You must specify a size for directories; size is optionalfor files and raw partitions. :arg:‘volume‘ and arg:seed are optional.



: The :arg:‘volume‘ option is independent of the :arg:‘seed‘ option and either can be used with or without the other,and their ordering on the line is irrelevant.

: If the :arg:‘id‘ option is used every use must have a unique value for :arg:‘string‘.

You can use any partition of any size. For best performance:

• Use raw disk partitions.

• For each disk, make all partitions the same size.

• For each node, use the same number of partitions on all disks.

• Group similar kinds of storage into different volumes. For example split out SSD’s or RAM drives into theirown volume.

Specify pathnames according to your operating system requirements. See the following examples. In the storage.config file, a formatted or raw disk must be at least 128 MB.

When using raw disk or partitions, you should make sure the :ts:cv:‘Traffic Server user<proxy.config.admin.user_id>‘ used by the Traffic Server process has read and write privileges on the rawdisk device or partition. One good practice is to make sure the device file is set with ‘g+rw’ and the Traffic Serveruser is in the group which owns the device file. However, some operating systems have stronger requirements - seethe following examples for more information.

As with standard records.config integers, human readable prefixes are also supported. They include

• K Kilobytes (1024 bytes)

• M Megabytes (1024^2 or 1,048,576 bytes)

• G Gigabytes (1024^3 or 1,073,741,824 bytes)

• T Terabytes (1024^4 or 1,099,511,627,776 bytes)

Assignment Table

Each storage element defined in storage.config is divided in to stripes. The assignment table maps from anobject URL to a specific stripe. The table is initialized based on a pseudo-random process which is seeded by hashinga string for each stripe. This string is composed of a base string, an offset (the start of the stripe on the storage element),and the length of the stripe. By default the path for the storage is used as the base string. This ensures that each stripehas a unique string for the assignment hash. This does make the assignment table very sensitive to the path for thestorage elements and changing even one can have a cascading effect which will effectively clear most of the cache.This can be problem when drives fail and a system reboot causes the path names to change.

The :arg:‘id‘ option can be used to create a fixed string that an administrator can use to keep the assignment tableconsistent by maintaing the mapping from physical device to base string even in the presence of hardware changes andfailures.

Examples

The following basic example shows 128 MB of cache storage in the /big_dir directory:

/big_dir 134217728



You can use the . symbol for the current directory. Here is an example for 64 MB of cache storage in the currentdirectory:

. 134217728

As an alternative, using the human readable prefixes, you can express a 64GB cache file with:

/really_big_dir 64G

: When using on-filesystem cache disk storage, you can only have one such directory specified. This will be addressin a future version.

Solaris Example

The following example is for the Solaris operating system:

/dev/rdsk/c0t0d0s5/dev/rdsk/c0t0d1s5

: Size is optional. If not specified, the entire partition is used.

Linux Example

: Rather than refer to disk devices like /dev/sda, /dev/sdb, etc., modern Linux supports alternative symlinkednames for disk devices in the /dev/disk directory structure. As noted for the Assignment Table the path used forthe disk can effect the cache if it changes. This can be ameloriated in some cases by using one of the alternate pathsin via /dev/disk. Note that if the by-id or by-path style is used, replacing a failed drive will cause that path tochange because the new drive will have a different physical ID or path. The original hash string can be kept by adding:arg:‘id‘ or :arg:‘path‘ with the original path to the storage line.

If this is not sufficient then the :arg:‘id‘ or :arg:‘path‘ argument should be used to create a more permanent assign-ment table. An example would be:

/dev/sde id=cache.disk.0/dev/sdg id=cache.disk.1

The following example will use an entire raw disk in the Linux operating system:

/dev/disk/by-id/[DiskA_ID] volume=1/dev/disk/by-path/[DiskB_Path] volume=2

In order to make sure traffic_server will have access to this disk you can use udev(7) to persistently set theright permissions. The following rules are targeted for an Ubuntu system, and stored in /etc/udev/rules.d/51-cache-disk.rules:

# Assign DiskA and DiskB to the tserver group# make the assignment final, no later changes allowed to the group!SUBSYSTEM=="block", KERNEL=="sd[ef]", GROUP:="tserver"


https://wiki.archlinux.org/index.php/persistent_block_device_naming#by-id_and_by-path

https://wiki.archlinux.org/index.php/persistent_block_device_naming#by-id_and_by-path


In order to apply these settings, trigger a reload with udevadm(8)::

udevadm trigger --subsystem-match=block

FreeBSD Example

Starting with 5.1 FreeBSD dropped support for explicit raw devices. All devices on FreeBSD can be accessed rawnow.

The following example will use an entire raw disk in the FreeBSD operating system:

/dev/ada1/dev/ada2

In order to make sure traffic_server will have access to this disk you can use devfs(8) to persistently set theright permissions. The following rules are stored in devfs.conf(5):

# Assign /dev/ada1 and /dev/ada2 to the tserver userown ada[12] tserver:tserver

volume.config

The volume.config file enables you to manage your cache space more efficiently and restrict disk usage bycreating cache volumes of different sizes for specific protocols. You can further configure these volumes to store datafrom certain origin servers and/or domains in the hosting.config file.

: The volume configuration must be the same on all nodes in a cluster. You must stop Traffic Server before youchange the cache volume size and protocol assignment. For step-by-step instructions about partitioning the cache,refer to Partitioning the Cache.

Format

For each volume you want to create, enter a line with the following format:

volume=volume_number scheme=protocol_type size=volume_size

where volume_number is a number between 1 and 255 (the maximum number of volumes is 255) andprotocol_type is http. Traffic Server supports http for HTTP volume types; volume_size is the amount ofcache space allocated to the volume. This value can be either a percentage of the total cache space or an absolute value.The absolute value must be a multiple of 128 MB, where 128 MB is the smallest value. If you specify a percentage,then the size is rounded down to the closest multiple of 128 MB.

Each volume is striped across several disks to achieve parallel I/O. For example: if there are four disks, then a 1-GBvolume will have 256 MB on each disk (assuming each disk has enough free space available). If you do not allocateall the disk space in the cache, then the extra disk space is not used. You can use the extra space later to create newvolumes without deleting and clearing the existing volumes.

Examples

The following example partitions the cache evenly between HTTP and HTTPS requests:



volume=1 scheme=http size=50%volume=2 scheme=https size=50%

Audience

This section of the Apache Traffic Server™ Manual is aimed at those individuals looking to install and configureTraffic Server for use as a caching proxy server. The section assumes basic knowledge of the command line andsystem administration tasks. Detailed steps are provided on how to build and configure the software, so even if youhave never used a compiler or looked at a Makefile before, you will find everything you need in the following chaptersto get you up and running with your own instance of Traffic Server. Or so is our hope.

Should you encounter problems, or find any of the instructions unclear or lacking, please seek help through one of theavenues listed in Other Resources.


CHAPTER 4

Developer’s Guide

This documentation is a work in progress. It was originally written for a previous, commercially-available versionof Traffic Server that supported different operating systems and more functions than the current version. As a result,some of the sections may refer to functionality that no longer exists.

If you find any such issues, you may want to submit a bug or a patch. We also have a Wiki page explaining how tocreate useful bug reports. We encourage everyone to file tickets, early and often. Looking for existing, duplicate bugsis encouraged, but not required.

Introduction

This guide has the following basic components:

• Introduction and overview.

• Tutorials about writing specific kinds of plugins: HTTP header-based plugins, content transformation plugins,and protocol plugins.

• Guides about specific interfaces.

• Reference material.

If you’re new to writing Traffic Server plugins, Getting Started should be your starting point. Header-Based PluginExamples provides details about plugins that work on HTTP headers, while HTTP Transformations explains howto write a plugin that transforms or scans the body of an HTTP response. New Protocol Plugins provides essentialinformation if you want to support your own protocol on Traffic Server.

For a reference to the C API functions and types that your plugin will use, refer to the API Reference.

Below is a section-by-section breakdown of this guide:

Getting Started How to compile and load plugins. Walks through a simple “hello world” example; explains how toinitialize and register plugins. Basic structures that all plugins use: events, continuations, and how to hook onto Traffic Server processes. Detailed explication of a sample blacklisting plugin.

259

https://issues.apache.org/jira/secure/CreateIssue!default.jspa?pid=12310963

https://cwiki.apache.org/confluence/display/TS/Filing+useful+bug+reports


Query Remap Plugin This chapter demonstrates on a practical example how you can exploit the Traffic Server remapAPI for your plugins.

Header-Based Plugin Examples Detailed explanation about writing plugins that work on HTTP headers; discussessample blacklisting and basic authorization plugins.

HTTP Transformations Detailed explanation of the null-transform example; also discusses VConnections, VIOs,and IO buffers.

New Protocol Plugins Detailed explanation of a sample protocol plugin that supports a synthetic protocol. DiscussesVConnections and mutexes, as well as the new NetConnection, DNS lookup, logging, and cache APIs.

The remaining sections comprise the API function reference and are organized by function type:

Plugin Interfaces Details error-writing and tracing functions, thread functions, and Traffic Server API versions of themalloc and fopen families. The Traffic Server API versions overcome various C library limitations.

Hooks and Transactions Functions in this chapter hook your plugin to Traffic Server HTTP processes.

HTTP Headers Contains instructions for implementing performance enhancements for all plugins that manipulateHTTP headers. These functions examine and modify HTTP headers, MIME headers, URLs, and the marshalbuffers that contain header information. If you are working with headers, then be sure to read this chapter.

Mutexes

Continuations Continuations provide the basic callback mechanism and data abstractions used in Traffic Server.


Actions Describes how to use TSActions and the TSDNSLookup API.

IO Describes how to use the Traffic Server IO interfaces: TSVConnection, TSVIO, TSIOBuffer,TSNetVConnection, the Cache API.

Plugin Management These functions enable you to set up a configuration interface for plugins, access installed pluginfiles, and set up plugin licensing.

Adding Statistics These functions add statistics to your plugin.

API Functions Traffic Server API Function Documentation.

Release Process

Managing a release is easiest in an environment that is as clean as possible. For this reason, cloning the code base into a new directory for the release process is recommended.

Requirements

• A system for git and building.

• A cryptographic key that has been signed by at least two other PMC members. This should be preferentiallyassociated with your apache.org email address but that is not required.

Release Candidate

The first step in a release is making a release candidate. This is distributed to the community for validation before theactual release.

260 Chapter 4. Developer’s Guide


Document

Gather up information about the changes for the release. The CHANGES file is a good starting point. You may alsowant to check the commits since the last release. The primary purpose of this is to generate a list of the importantchanges since the last release.

Create or update a page on the Wiki for the release. If it is a major or minor release it should have its own page. Usethe previous release page as a template. Point releases should get a section at the end of the corresponding releasepage.

Write an announcement for the release. This will contain much of the same information that is on the Wiki page butmore concisely. Check the mailing list archives for examples to use as a base.

Build

1. Go to the top level source directory.

2. Check the version in configure.ac. There are two values near the top that need to be set, TS_VERSION_Sand TS_VERSION_N. These are the release version number in different encodings.

3. Check the variable RC in the top level Makefile.am. This should be the point release value. This needs to bechanged for every release candidate. The first release candidate is 0 (zero).

4. Execute the following commands to make the distribution files.

autoreconf -i./configuremake rel-candidate

These steps will create the distribution files and sign them using your key. Expect to be prompted twice for yourpassphrase unless you use an ssh key agent. If you have multiple keys you will need to set the default appropriatelybeforehand, as no option will be provided to select the signing key. The files should have names that start withtrafficserver-X.Y.Z-rcA.tar.bz2 where X.Y.Z is the version and A is the release candidate counter.There should be four such files, one with no extension and three others with the extensions asc, md5, and sha1. Thiswill also create a signed git tag of the form X.Y.Z-rcA.

Distribute

The release candidate files should be uploaded to some public storage. Your personal storage on people.apach.org isa reasonable location to use.

Send the release candiate announcement to the users and dev mailinging lists, noting that it is a release candidate andproviding a link to the distribution files you uploaded. This announcement should also call for a vote on the candidate,generally with a 72 hours time limit.

If the voting was successful (at least three “+1” votes and no “-1” votes), proceed to Official Release. Otherwise,repeat the Release Candidate process.

Official Release

Build the distribution files with the command

make release

4.2. Release Process 261

http://mail-archives.apache.org/mod_mbox/trafficserver-dev/


Be sure to not have changed anything since the release candidate was built so the checksums are identical. This willcreate a signed git tag of the form X.Y.Z and produce the distribution files. Push the tag to the ASF repository withthe command

git push origin X.Y.Z

This presumes origin is the name for the ASF remote repository which is correct if you originally clone from theASF repository.

The distribution files must be added to an SVN repository. This can be accessed with the command:

svn co https://dist.apache.org/repos/dist/release/trafficserver <local-directory>

All four of the distribution files go here. If you are making a point release then you should also remove the distributionfiles for the previous release. Allow 24 hours for the files to be distributed through the ASF infrastructure.

The Traffic Server website must be updated. This is an SVN repository which you can access with

svn co https://svn.apache.org/repos/asf/trafficserver/site/trunk <local-directory>

The files of interest are in the content directory.

index.html This is the front page. The places to edit here are any security announcements at the top and the“News” section.

downloads.en.mdtext Update the downloads page to point to the new download objects.

After making changes, commit them and then run

publish.pl trafficserver <apache-id>

On the people.apache.org host.

If needed, update the Wiki page for the release to point at the release distribution files.

Update the announcement, if needed, to refer to the release distribution files and remove the comments concerning therelease candidate. This announcement should be sent to the users and dev mailing lists. It should also be sent to theASF announcement list, which must be done using an apache.org email address.

Finally, update various files after the release:

• The STATUS file for master and for the release branch to include this version.

• The CHANGES file to have a header for the next version.

• configure.ac to be set to the next version.

• In the top level Makefile.am change RC to have the value 0.

Contributing to Traffic Server

Using Vagrant to Test Traffic Server

The Apache Traffic Server™ project’s official repository includes a Vagrantfile, intended to ease the process of cre-ating environments suitable for building and testing Traffic Server, where all the necessary dependencies are installedautomatically for a variety of operating systems and common distribution releases.



Vagrant is a tool for building complete development environments. With an easy-to-use workflowand focus on automation, Vagrant lowers development environment setup time, increases develop-ment/production parity, and makes the “works on my machine” excuse a relic of the past.

—VagrantUp website

Vagrant can be used in combination with any of the popular configurtion management and automation tools, such asChef, Puppet, Ansible, and more. The Vagrantfile included in the Traffic Server repository happens to make use ofPuppet.

Installing Vagrant and Dependencies

VirtualBox

The virtualization software VirtualBox is required to create and run the virtual machines created by the includedproject Vagrantfile.

VirtualBox can be obtained by free from the official website, and many distributions provide their own packages aswell. No special configuration of the software is required.

Vagrant

A fairly recent version of Vagrant is necessary to use the included Vagrantfile. While older versions of Vagrant couldbe installed through the Ruby Gems facility, modern versions are only provided as distribution specific packages.

NFS Server

The project Vagrantfile uses the NFS shared folders support of VirtualBox to mount the same directory in which theVagrantfile is located on your host machine as a network directory inside the virtual machine. For this to work, yourhost machine must have an NFS server installed and running, and the user under which you run the vagrant commandsmust have sudo permissions to modify the NFS exports configuration and restart the NFS service.

The virtual machine created by Vagrant will still function without a working NFS server on your host machine, butyou will not be able to access the shared folder which includes the entire Traffic Server source tree. You may opt tomodify the Vagrantfile to use a method other than NFS, as per the Vagrant documentation.

Managing Virtual Machines

Listing Available Machines

The included Vagrantfile defines many variations of operating systems, releases, and architectures. To see a completelist of the virtual machine options available to you, run the command vagrant status from within the samedirectory as the Vagrantfile. The command may take a few moments to run as the configurations defined in theVagrantfile are evaluated, and calls are made to the underlying VirtualBox utilities to check for the existence andoperational state of each possibility. You should expect to see output along the lines of:

$ vagrant statusCurrent machine states:

saucy32 not created (virtualbox)raring32 not created (virtualbox)quantal32 not created (virtualbox)precise32 not created (virtualbox)

4.4. Using Vagrant to Test Traffic Server 263

https://www.vagrantup.com/about.html

https://www.chef.io/chef/

https://puppetlabs.com/

http://www.ansible.com/home

https://www.virtualbox.org/

https://www.vagrantup.com/downloads.html

https://docs.vagrantup.com/v2/synced-folders/basic_usage.html


trusty32 not created (virtualbox)saucy64 not created (virtualbox)raring64 not created (virtualbox)quantal64 not created (virtualbox)precise64 not created (virtualbox)trusty64 running (virtualbox)freebsd not created (virtualbox)omnios not created (virtualbox)lucid64 not created (virtualbox)fedora18 not created (virtualbox)centos63 not created (virtualbox)centos59 not created (virtualbox)centos64 not created (virtualbox)debian7 not created (virtualbox)sles11 not created (virtualbox)oel63 not created (virtualbox)

This environment represents multiple VMs. The VMs are all listedabove with their current state. For more information about a specificVM, run `vagrant status NAME`.

Creating and Destroying

Creation and destruction of virtual machines with Vagrant is very simple. To bring a new virtual machine into exis-tence, run the following command from the same directory in which the Vagrantfile is located:

vagrant up <name>

Where <name> should be the specific operating system release you wish to use for the virtual machine. For example,to test Traffic Server in a CentOS 6.4 environment, you would run:

vagrant up centos64

Running the vagrant up command for a virtual machine which does not exist yet (or has previously been deleted)will create a brand new virtual machine, using the appropriate image (called a basebox in Vagrant parlance), as wellas provision the machine according to any configuration management rules specified in the Vagrantfile.

Similarly, you may destroy the virtual machine when you are finished by running the command:

vagrant destroy <name>

Or if you wish to only stop the virtual machine temporarily without deleting it, you may run:

vagrant halt <name>

A halted virtual machine is started back up with the same vagrant up command shown earlier. The difference isthat Vagrant will recognize the box already exists and do nothing more than start the vm process and configure thevirtual networking interfaces on your host. Any configuration management hooks in the Vagrantfile will be skipped.

Logging In

Logging into a virtual machine created with Vagrant may be accomplished in a couple different ways. The easiest isto let Vagrant itself figure out where the machine is and how to properly authenticate you to it:



vagrant ssh <name>

Using that command from within the same directory as the Vagrantfile allows you to skip figuring out what virtualnetwork interface has been attached to the machine, what local port may be assigned to handle SSH forwarding, andso on. As long as the virtual machine was already running, you will be quickly dropped into a local shell in the virtualmachine as the vagrant user.

: Vagrant by default uses a widely-shared private RSA key on newly created virtual machines (that are built on publicbasebox images). The default user on these baseboxes is also configured for password-less sudo permissions. This isvery clearly insecure, but Vagrant is designed for local testing and development, not production (or other public) uses,so the project has made the philosophical trade-off in favor of ease of use.

Alternatively, you may SSH directly to the virtual machine. Because the virtual machines are configured to use only theprivate virtual network layer provided by VirtualBox, you cannot directly. Instead, VirtualBox has created a local portmapping automatically which should be used. There is no fixed, pre-determined port mapping that will be universallyvalid, as Vagrant and VirtualBox may be used together to run an arbitrary number of virtual machines simultaneously,each provisioned in any order, and defined by any number and variety of Vagrantfiles.

The correct way to determine what port Vagrant and VirtualBox have used to map to a given virtual machine is to runthe following command from within the same directory as your Vagrantfile:

vagrant ssh-config <name>

That will output a configuration block, suitable for inclusion in your local ~/.ssh/config file. Note specifically,in addition to the port, the path to the private key you will need to use as your identity when attempting to log into thevirtual machine.

Shared Host Folders

VirtualBox provides a facility for mounting directories from your host machine as filesystems inside the virtual ma-chines. The Traffic Server Vagrantfile makes use of this feature to mount its own source tree in a predictable locationin the virtual environment.

Multiple methods are available for this, including NFS, CIFS, and simulated block devices. The Traffic Server projectopts to use NFS for its simplicity, speed, support for features such as symlinks, and wide interoperability across thevarious guest operating systems included in the Vagrantfile. Within the included Vagrantfile, you can see the followingline:

config.vm.synced_folder ".", "/opt/src/trafficserver.git", :nfs => true

This directs VirtualBox to mount the directory in which the Traffic Server Vagrantfile resides as an NFS mount insidethe virtual machine at the path /opt/src/trafficserver.git. Additional host directories may be mounted inthe same manner should the need arise.

Forwarding Custom Ports

Building Traffic Server Inside Vagrant

Producing Traffic Server builds from within the Vagrant managed virtual machines is effectively no different than inany other environment. The same directory in which the Vagrantfile exists will be mounted via NFS inside the virtualmachine at the path /opt/src/trafficserver.git.

4.4. Using Vagrant to Test Traffic Server 265


: If you have run autoconf or configure from outside the virtual machine environment against the same Gitworking copy as is mounted inside the virtual machine, you will encounter failures should you attempt to run any of theMake targets inside the VM. Any build related commands should be run inside of the virtual machine. Additionally,if you are running more than one virtual machine simultaneously, remember that they are each using the same NFSexport on your host machine.

Debugging and Analysis

Debug Builds

A debugger can set breakpoints in a plugin. Use a Traffic Server debug build and compile the plugin with the -goption. A debugger can also be used to analyze a core dump. To generate core, set the size limit of the core files in therecords.config file to -1 as follows:

CONFIG :ts:cv:`proxy.config.core_limit` INT -1

This is the equivalent of setting ulimit -c unlimited

Debugging Tips:

• Use a Traffic Server debug version.

• Use assertions in your plugin (TSAssert() and TSReleaseAssert()).

Using TSAssert

Debug Tags

Use the API void TSDebug (const char *tag, const char *format_str, ...) to add traces inyour plugin. In this API:

• tag is the Traffic Server parameter that enables Traffic Server to print out ‘‘format_str‘‘

• ... are variables for ‘‘format_str‘‘ in the standard printf style.

Run Traffic Server with the -Ttag option. For example, if the tag is my-plugin, then the debug output goes totraffic.out.See below:

traffic_server -T"my-plugin"

Set the following variables in records.config (in the Traffic Server config directory):

CONFIG proxy.config.diags.debug.enabled INT 1CONFIG proxy.config.diags.debug.tags STRING debug-tag-name

In this case, debug output goes to traffic.out.

Example:

TSDebug ("my-plugin", "Starting my-plugin at %d\n", the_time);



The statement "Starting my-plugin at <time>" appears whenever you run Traffic Server with themy-plugin tag:

traffic_server -T"my-plugin"

Other Useful Internal Debug Tags

Embedded in the base Traffic Server code are many debug tags for internal debugging purposes. These can also beused to follow Traffic Server behavior for testing and analysis.

The debug tag setting (-T and proxy.config.diags.debug.tags) is a anchored regular expression againstwhich the tag for a specific debug message is matched. This means the value “http” matches debug emssages withthe tag “http” but also “http_seq” and “http_trans”. If you want multiple tags then use a pipe symbol to separate thetags. For example “http_tproxy|dns|hostdb” will match any of the message tags “http_tproxy”, “dns”, “hostdb”, or“dns_srv” (but not “http_trans” nor “splitdns”).

Some of the useful HTTP debug tags are:

• http_hdrs - traces all incoming and outgoing HTTP headers.

• http_trans - traces actions in an HTTP transaction.

• http_seq - traces the sequence of the HTTP state machine.

• http_tproxy - transparency related HTTP events

• dns - DNS operations

• hostdb - Host name lookup

• iocore_net - Socket and low level IO (very voluminous)

• socket - socket operations

• ssl - SSL related events

• cache - Cache operations (many subtags, examine the output to narrow the tag set)

• cache_update - Cache updates including writes

• cache_read - Cache read events.

• dir_probe - Cache searches.

• sdk - gives some warning concerning API usage.

Core Dump Analysis

Profiling

Memory Leaks

Memory leaks in a plugin can be detected using e.g. an MRTG graph related to memory - you can use memory dumpinformation. Enable mem dump in records.config as follows:

CONFIG proxy.config.dump_mem_info_frequency INT <value>

This causes Traffic Server to dump memory information to traffic.out at <value> (intervals are in seconds).A zero value means that it is disabled.

4.5. Debugging and Analysis 267


Cache Architecture

The original architectural documents for Traffic Server were lost in the transition to an open source project. Thedocuments in this section are provisional and were written based on the existing code. The purpose is to have a highlevel description of aspects of Traffic Server to better inform ongoing work.

In the final section on “hacking” we try to document our approaches to understanding and modifying the source.

Cache Architecture

Introduction

In addition to being an HTTP proxy, Apache Traffic Server™ is also an HTTP cache. Traffic Server can cache anyoctet stream, although it currently supports only those octet streams delivered by the HTTP protocol. When such astream is cached (along with the HTTP protocol headers) it is termed an object in the cache. Each object is identifiedby a globally unique value called a cache key.

The purpose of this document is to describe the basic structure and implementation details of the Traffic Server cache.Configuration of the cache will be discussed only to the extent needed to understand the internal mechanisms. Thisdocument will be useful primarily to Traffic Server developers working on the Traffic Server codebase or plugins forTraffic Server. It is assumed the reader is already familiar with the Administrator’s Guide and specifically with HTTPProxy Caching and along with the associated configuration files.

Unfortunately, the internal terminology is not particularly consistent, so this document will frequently use terms indifferent ways than they are used in the code in an attempt to create some consistency.

Cache Layout

The following sections describe how persistent cache data is structured. Traffic Server treats its persisent storage as anundifferentiated collection of bytes, assuming no other structure to it. In particular, it does not use the file system ofthe host operating system. If a file is used it is used only to mark out the set of bytes to be used.

Cache storage

The raw storage for the Traffic Server cache is configured in storage.config. Each line in the file defines a cachespan which is treated as a uniform persistent store.

4.1: Two cache spans

This storage organized into a set of cache volumes which are defined in volume.config. These are the units thatare used for all other administator level configuration.

Cache volumes can be defined by a percentage of the total storage or as an absolute amount of storage. By default,each cache volume is spread across all of the cache spans for robustness. The intersection of a cache volume and acache span is a cache stripe. Each cache span is divided into cache stripes and each cache volume is a collection ofthose stripes.



If the cache volumes for the example cache spans were defined as:

Then the actual layout would look like:

Cache stripes are the fundamental unit of cache for the implementation. A cached object is stored entirely in a singlestripe, and therefore in a single cache span. Objects are never split across cache spans or volumes. Objects are assignedto a stripe (and in turn to a cache volume) automatically based on a hash of the URI used to retrieve the object fromthe origin server. It is possible to configure this to a limited extent in hosting.config, which supports contentfrom specific hosts or domain to be stored on specific cache volumes. As of version 4.0.1 it is also possible to controlwhich cache spans (and hence, which cache stripes) are contained in a specific cache volume.

The layout and structure of the cache spans, the cache volumes, and the cache stripes that compose them are derived en-tirely from storage.config and cache.config and is recomputed from scratch when the traffic_serveris started. Therefore, any change to those files can (and almost always will) invalidate the existing cache in its entirety.

Stripe Structure

Traffic Server treats the storage associated with a cache stripe as an undifferentiated span of bytes. Internally eachstripe is treated almost entirely independently. The data structures described in this section are duplicated for eachstripe. Internally the term volume is used for these stripes and implemented primarily in Vol. What a user thinks ofas a volume (and what this document calls a cache volume) is represented by CacheVol.

: Stripe assignment must be done before working with an object because the directory is local to the stripe. Anycached objects for which the stripe assignment is changed are effectively lost as their directory data will not be foundin the new stripe.

Cache Directory

Content in a stripe is tracked via a directory. Each element of the directory is a directory entry and is represented byDir. Each entry refers to a chunk of contiguous storage in the cache. These are referred to variously as fragments,segments, docs, documents, and a few other things. This document will use the term fragment as that is the mostcommon reference in the code. The term Doc (for Doc) will be used to refer to the header data for a fragment.Overall, the directory is treated as a hash with the cache ID as the key. See directory probing for how the cache ID isused to locate a directory entry. The cache ID is in turn computed from a cache key which by default is the URL ofthe content.

The directory is used as a memory resident structure, which means a directory entry is as small as possible (currently10 bytes). This forces some compromises on the data that can be stored there. On the other hand this means that mostcache misses do not require disk I/O, which has a large performance benefit.

4.6. Cache Architecture 269


The directory is always fully sized; once a stripe is initialized the directory size is fixed and never changed. Thissize is related (roughly linearly) to the size of the stripe. It is for this reason the memory footprint of Traffic Serverdepends strongly on the size of the disk cache. Because the directory size does not change, neither does this memoryrequirement, so Traffic Server does not consume more memory as more content is stored in the cache. If there isenough memory to run Traffic Server with an empty cache there is enough to run it with a full cache.

Each entry stores an offset in the stripe and a size. The size stored in the directory entry is an approximate size whichis at least as big as the actual data in the fragment. Exact size data is stored in the fragment header on disk.

: Data in HTTP headers cannot be examined without disk I/O. This includes the original URL for the object. Thecache key is not stored explicitly and therefore cannot be reliably retrieved.

The directory is a hash table that uses chaining for collision resolution. Because each entry is small they are useddirectly as the list header of the hash bucket. Chaining is implemented by imposing grouping structures on theentries in a directory. The first level grouping is a directory bucket. This is a fixed number (currently 4, defined asDIR_DEPTH) of entries. This serves to define the basic hash buckets with the first entry in each cache bucket servingas the root of the hash bucket.

: The term bucket is used in the code to mean both the conceptual bucket for hashing and for a structural groupingmechanism in the directory. These will be qualified as needed to distinguish them. The unqualified term bucket isalmost always used to mean the structural grouping in the directory.

Directory buckets are grouped in to segments. All segments in a stripe have the same number of buckets. The numberof segments in a stripe is chosen so that each segment has as many buckets as possible without exceeding 65,535(216-1) entries in a segment.

Each directory entry has a previous and next index value which is used to link entries in the same segment. Becauseno segment has more than 65,535 entries, 16 bits suffices for storing the index values. The stripe header contains anarray of entry indices which are used as the roots of entry free lists, one for each segment. Active entries are stored viathe bucket structure. When a stripe is initialized the first entry in each bucket is zeroed (marked unused) and all otherentries are put in the corresponding segment free list in the stripe header. This means the first entry of each directorybucket is used as the root of a hash bucket and is therefore marked unused rather than being put a free list. The other


http://en.wikibooks.org/wiki/Data_Structures/Hash_Tables#Collision_resolution


entries in the directory bucket are preferred for adding to the corresponding hash bucket but this is not required. Thesegment free lists are initialized such that the extra bucket entries are added in order; all the seconds, then the thirds,then the fourths. Because the free lists are FIFOs, this means extra entries will be selected from the fourth entriesacross all the buckets first, then the thirds, etc. When allocating a new directory entry in a bucket the entries aresearched from first to last, which maximizes bucket locality (that is, cache IDs that map to the same hash bucket willalso tend to use the same directory bucket).

Entries are removed from the free list when used and returned when no longer in use. When a fragment needs to beput in to the directory the cache ID is used to locate a hash bucket (which also determines the segment and directorybucket). If the first entry in the directory bucket is marked unused, it is used. Otherwise, the other entries in the bucketare searched and if any are on the free list, that entry is used. If none are available then the first entry on the segmentfree list is used. This entry is attached to the hash bucket via the same next and previous indices used for the free listso that it can be found when doing a lookup of a cache ID.

Storage Layout

The storage layout is the stripe metadata followed by cached content. The metadata consists of three parts: the stripeheader, the directory, and the stripe footer. The metadata is stored twice. The header and the footer are instances ofVolHeaderFooter. This is a stub structure which can have a trailing variable sized array. This array is used asthe segment free list roots in the directory. Each contains the segment index of the first element of the free list for thesegment. The footer is a copy of the header without the segment free lists. This makes the size of the header dependenton the directory but not that of the footer.

Each stripe has several values that describe its basic layout:

skip The start of stripe data. This represents either space reserved at the start of a physical device to avoid problemswith the host operating system, or an offset representing use of space in the cache span by other stripes.

start The offset for the start of the content, after the stripe metadata.

length Total number of bytes in the stripe. Vol::len.



data length Total number of blocks in the stripe available for content storage. Vol::data_blocks.

: Great care must be taken with sizes and lengths in the cache code because there are at least three different metrics(bytes, cache blocks, store blocks) used in various places.

The total size of the directory (the number of entries) is computed by taking the size of the cache stripe and dividingby the average object size. The directory always consumes this amount of memory which has the effect that if cachesize is increased so is the memory requirement for Traffic Server. The average object size defaults to 8000 bytes butcan be configured using :ts:cv:‘proxy.config.cache.min_average_object_size‘. Increasing the average object sizewill reduce the memory footprint of the directory at the expense of reducing the number of distinct objects that canbe stored in the cache. The content area stores the actual objects and is used as a circular buffer where new objectsoverwrite the least recently cached objects. The location in a stripe where new cache data is written is called thewrite cursor. This means that objects can be de facto evicted from cache even if they have not expired if the data isoverwritten by the write cursor. If an object is overwritten this is not detected at that time and the directory is notupdated. Instead it will be noted if the object is accessed in the future and the disk read of the fragment fails.

4.2: The write cursor and documents in the cache.

: Cache data on disk is never updated.

This is a key thing to keep in mind. What appear to be updates (such as doing a refresh on stale content and gettingback a 304) are actually new copies of data being written at the write cursor. The originals are left as “dead” spacewhich will be consumed when the write cursor arrives at that disk location. Once the stripe directory is updated (inmemory) the original fragment in the cache is effectively destroyed. This is the general space management techniqueused in other cases as well. If an object needs to removed from cache, only the directory needs to be changed. Noother work (and particularly, no disk I/O) needs to be done.

Object Structure

Objects are stored as two types of data: metadata and content data. Metadata is all the data about the object and thecontent and includes the HTTP headers. The content data is the content of the object, the octet stream delivered to theclient as the object.

Objects are rooted in a Doc structure stored in the cache. Doc serves as the header data for a cache fragment and iscontained at the start of every fragment. The first fragment for an object is termed the first Doc and always containsthe object metadata. Any operation on the object will read this fragment first. The fragment is located by convertingthe cache key for the object to a cache ID and then doing a lookup for a directory entry with that key. The directoryentry has the offset and approximate size of the first fragment which is then read from the disk. This fragment willcontain the request header and response along with overall object properties (such as content length).



Traffic Server supports varying content for objects. These are called alternates. All metadata for all alternates is storedin the first fragment including the set of alternates and the HTTP headers for them. This enables alternate selection tobe done after the first Doc is read from disk. An object that has more than one alternate will have the alternate contentstored separately from the first fragment. For objects with only one alternate the content may or may not be in thesame (first) fragment as the metadata. Each separate alternate content is allocated a directory entry and the key for thatentry is stored in the first fragment metadata.

Prior to version 4.0.1, the header data was stored in the CacheHTTPInfoVector class which was marshaled to avariable length area of the on disk image, followed by information about additional fragments if needed to store theobject.

4.3: Doc layout 3.2.0

This had the problem that with only one fragment table it could not be reliable for objects with more than onealternate[#multiple-alternates]_. Therefore, the fragment data was moved from being a separate variable length sectionof the metadata to being directly incorporated in to the CacheHTTPInfoVector, yielding a layout of the followingform.

4.4: Doc layout 4.0.1

Each element in the vector contains for each alternate, in addition to the HTTP headers and the fragment table (if any),a cache key. This cache key identifies a directory entry that is referred to as the earliest Doc. This is the location wherethe content for the alternate begins.

When the object is first cached, it will have a single alternate and that will be stored (if not too large) in first Doc. Thisis termed a resident alternate in the code. This can only happen on the initial store of the object. If the metadata isupdated (such as a 304 response to an If-Modified-Since request) then unless the object is small, the object datawill be left in the original fragment and a new fragment written as the first fragment, making the alternate non-resident.Small is defined as a length smaller than :ts:cv:‘proxy.config.cache.alt_rewrite_max_size‘.

: The CacheHTTPInfoVector is stored only in the first Doc. Subsequent Doc instances for the object, includingthe earliest Doc, should have an hlen of zero and if not, it is ignored.

Large objects are split in to multiple fragments when written to the cache. This is indicated by a total document lengththat is longer than the content in first Doc or an earliest Doc. In such a case a fragment offset table is stored. Thiscontains the byte offset in the object content of the first byte of content data for each fragment past the first (as theoffset for the first is always zero). This allows range requests to be serviced much more efficiently for large objects,as intermediate fragments that do not contain data in the range can be skipped. The last fragment in the sequence isdetected by the fragment size and offset reaching the end of the total size of the object, there is no explicit end mark.Each fragment is computationally chained from the previous in that the cache key for fragment N is computed by:


http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.44

http://trafficserver.apache.org/docs/trunk/sdk/http-hooks-and-transactions/http-alternate-selection.en.html


key_for_N_plus_one = next_key(key_for_N);

Where next_key is a global function that deterministically computes a new cache key from an existing cache key.

Objects with multiple fragments are laid out such that the data fragments (including the earliest Doc) are written firstand the first Doc is written last. When read from disk, both the first and earliest Doc are validated (tested to ensurethat they haven’t been overwritten by the write cursor) to verify that the entire document is present on disk (as theybookend the other fragments - the write cursor cannot overwrite them without overwriting at least one of the verifiedDoc instances). Note that while the fragments of a single object are ordered they are not necessarily contiguous, asdata from different objects are interleaved as the data arrives in Traffic Server.

4.5: Multi-alternate and multi-fragment object storage

Documents which are pinned into the cache must not be overwritten so they are evacuated from in front of the writecursor. Each fragment is read and rewritten. There is a special lookup mechanism for objects that are being evacuatedso that they can be found in memory rather than the potentially unreliable disk regions. The cache scans ahead of thewrite cursor to discover pinned objects as there is a dead zone immediately before the write cursor from which datacannot be evacuated. Evacuated data is read from disk and placed in the write queue and written as its turn comes up.

Objects can only be pinned via cache.config and while :ts:cv:‘proxy.config.cache.permit.pinning‘ is set to non-zero (it is zero by default). Objects which are in use when the write cursor is near use the same underlying evacuationmechanism but are handled automatically and not via the explicit pinned bit in Dir.

Additional Notes

Some general observations on the data structures.

Cyclone buffer

Because the cache is a cyclone cache, objects are not preserved for an indefinite time. Even if the object is not stale itcan be overwritten as the cache cycles through its volume. Marking an object as pinned preserves the object throughthe passage of the write cursor but this is done by copying the object across the gap, in effect re-storing it in the cache.Pinning large objects or a large number objects can lead to excessive disk activity. The original purpose of pinningwas for small, frequently used objects explicitly marked by the administrator.

This means the purpose of expiration data on objects is simply to prevent them from being served to clients. Theyare not in the standard sense deleted or cleaned up. The space can’t be immediately reclaimed in any event, becausewriting only happens at the write cursor. Deleting an object consists only of removing the directory entries in thevolume directory which suffices to (eventually) free the space and render the document inaccessible.



Historically, the cache was designed this way because web content was relatively small and not particularly consistent.The design also provided high performance and low consistency requirements. There are no fragmentation issues forthe storage, and both cache misses and object deletions require no disk I/O. It does not deal particularly well with longterm storage of large objects. See the Volume Tagging appendix for details on some work in this area.

Disk Failure

The cache is designed to be relatively resistant to disk failures. Because each storage unit in each cache volume ismostly independent, the loss of a disk simply means that the corresponding Vol instances (one per cache volume thatuses the storage unit) becomes unusable. The primary issue is updating the volume assignment table to both preserveassignments for objects on still operational volumes while distributing the assignments from the failed disk to thoseoperational volumes. This mostly done in:

AIO_Callback_handler::handle_disk_failure

Restoring a disk to active duty is a more difficult task. Changing the volume assignment of a cache key renders anycurrently cached data inaccessible. This is not a problem when a disk has failed, but is a bit trickier to decide whichcached objects are to be de facto evicted if a new storage unit is added to a running system. The mechanism for this,if any, is still under investigation.

Implementation Details

Stripe Directory

The in memory volume directory entries are described below.

class DirDefined in P_CacheDir.h.

Name Type Useoffset unsigned int:24 Offset of first byte of metadata (volume relative)big unsigned in:2 Size multipliersize unsigned int:6 Sizetag unsigned int:12 Partial key (fast collision check)phase unsigned int:1 Phase of the Doc (for dir valid check)head unsigned int:1 Flag: first fragment in an objectpinned unsigned int:1 Flag: document is pinnedtoken unsigned int:1 Flag: Unknownnext unsigned int:16 Segment local index of next entry.offset_high inku16 High order offset bits

The stripe directory is an array of Dir instances. Each entry refers to a span in the volume which contains a cachedobject. Because every object in the cache has at least one directory entry this data has been made as small as possible.

The offset value is the starting byte of the object in the volume. It is 40 bits long, split between the offset (lower 24bits) and offset_high (upper 16 bits) members. Note that since there is a directory for every storage unit in a cachevolume, this is the offset in to the slice of a storage unit attached to that volume. The size and big values are used tocalculate the approximate size of the fragment which contains the object. This value is used as the number of bytesto read from storage at the offset value. The exact size is contained in the object metadata in Doc which is consultedonce the read has completed. For this reason, the approximate size needs to be at least as large as the actual size butcan be larger, at the cost of reading the extraneous bytes.

The computation of the approximate size of the fragment is defined as:


https://github.com/apache/trafficserver/blob/master/iocore/cache/P_CacheDir.h


( *size* + 1 ) * 2 ^ ( CACHE_BLOCK_SHIFT + 3 * *big* )

Where CACHE_BLOCK_SHIFT is the bit width of the size of a basic cache block (9, corresponding to a sector sizeof 512). Therefore the value with current defines is:

( *size* + 1 ) * 2 ^ (9 + 3 * *big*)

Because big is 2 bits, the values for the multiplier of size are:

big Multiplier Maximum Size0 512 (2^9) 32768 (2^15)1 4096 (2^12) 262144 (2^18)2 32768 (2^15) 2097152 (2^21)3 262144 (2^18) 16777216 (2^24)

Note also that size is effectively offset by one, so a value of 0 indicates a single unit of the multiplier. The targetfragment size can set with the records.config value :ts:cv:‘proxy.config.cache.target_fragment_size‘.

This value should be chosen so that it is a multiple of a cache entry multiplier. It is not necessary to make it a powerof two[#cache-mult-value]_. Larger fragments increase I/O efficiency but lead to more wasted space. The defaultsize (1M, 2^20) is a reasonable choice in most circumstances, altough in very specific cases there can be benefit fromtuning this parameter. Traffic Server imposes an internal maximum of a 4,194,232 bytes, which is 4M (2^22), less thesize of a struct Doc. In practice, the largest reasonable target fragment size is 4M - 262,144 = 3,932,160.

When a fragment is stored to disk, the size data in the cache index entry is set to the finest granularity permitted by thesize of the fragment. To determine this, consult the cache entry multipler table and find the smallest maximum sizethat is at least as large as the fragment. That will indicate the value of big selected and therefore the granularity of theapproximate size. That represents the largest possible amount of wasted disk I/O when the fragment is read from disk.

The set of index entries for a volume are grouped in to segments. The number of segments for an index is selected sothat there are as few segments as possible such that no segment has more than 2^16 entries. Intra-segment referencescan therefore use a 16 bit value to refer to any other entry in the segment.

Index entries in a segment are grouped buckets, each of DIR_DEPTH (currently 4) entries. These are handled in thestandard hash table manner, giving somewhat less than 2^14 buckets per segment.

Directory Probing

Directory probing is the locating of a specific directory entry in the stripe directory based on a cache ID. This ishandled primarily by the function dir_probe(). This is passed the cache ID (:arg:‘key‘), a stripe (:arg:‘d‘), anda last collision (:arg:‘last_collision‘). The last of these is an in and out parameter, updated as useful during the probe.

Given an ID, the top half (64 bits) is used as a segment index, taken modulo the number of segments in the directory.The bottom half is used as a bucket index, taken modulo the number of buckets per segment. The :arg:‘last_collision‘value is used to mark the last matching entry returned by dir_probe().

After computing the appropriate bucket, the entries in that bucket are searched to find a match. In this case a match isdetected by comparison of the bottom 12 bits of the cache ID (the cache tag). The search starts at the base entry forthe bucket and then proceeds via the linked list of entries from that first entry. If a tag match is found and there is no:arg:‘collision‘ then that entry is returned and :arg:‘last_collision‘ is updated to that entry. If :arg:‘collision‘ is setand if it isn’t the current match, the search continues down the linked list, otherwise :arg:‘collision‘ is cleared and thesearch continues.

The effect of this is that matches are skipped until the last returned match (:arg:‘last_collision‘) is found, after whichthe next match (if any) is returned. If the search falls off the end of the linked list, then a miss result is returned (ifno last collision), otherwise the probe is restarted after clearing the collision on the presumption that the entry for the



collision has been removed from the bucket. This can lead to repeats among the returned values but guarantees that novalid entry will be skipped.

Last collision can therefore be used to restart a probe at a later time. This is important because the match returnedmay not be the actual object. Although the hashing of the cache ID to a bucket and the tag matching is unlikely tocreate false positives, it is possible. When a fragment is read the full cache ID is available and checked and if wrong,that read can be discarded and the next possible match from the directory found because the cache virtual connectiontracks the last collision value.

Cache Operations

Cache activity starts after the HTTP request header has been parsed and remapped. Tunneled transactions do notinteract with the cache because the headers are never parsed.

To understand the logic we must introduce the term cache valid which means something that is directly related to anobject that is valid to be put in the cache (e.g. a DELETE which refers to a URL that is cache valid but cannot becached itself). This is important because Traffic Server computes cache validity several times during a transaction andonly performs cache operations for cache valid results. The criteria used changes during the course of the transactionas well. This is done to avoid the cost of cache activity for objects that cannot be in the cache.

The three basic cache operations are: lookup, read, and write. We will take deleting entries as a special case of writingwhere only the volume directory is updated.

After the client request header is parsed and is determined to be potentially cacheable, a cache lookup is done. Ifsuccessful, a cache read is attempted. If either the lookup or the read fails and the content is considered cacheablethen a cache write is attempted.

Cacheability

The first thing done with a request with respect to cache is to determine whether it is potentially a valid object forthe cache. After initial parsing and remapping, this check is done primarily to detect a negative result, as it allowsfurther cache processing to be skipped. It will not be put in to the cache, nor will a cache lookup be performed. Thereare a number of prerequisites along with configuration options to change them. Additional cacheability checks aredone later in the process, when more is known about the transaction (such as plugin operations and the origin serverresponse). Those checks are described as appropriate in the sections on the relevant operations.

The set of things which can affect cacheability are:

• Built in constraints.

• Settings in records.config.

• Settings in cache.config.

• Plugin operations.

The initial internal checks, along with their records.config overrides[#cacheability-overrides]_, are done inHttpTransact::is_request_cache_lookupable.

The checks that are done are:

Cacheable Method The request must be one of GET, HEAD, POST, DELETE, PUT.

See HttpTransact::is_method_cache_lookupable().

Dynamic URL Traffic Server tries to avoid caching dynamic content because it’s dynamic. A URL isconsidered dynamic if:

• It is not HTTP or HTTPS,



• Has query parameters,

• Ends in asp,

• Has cgi in the path.

This check can be disabled by setting a non-zero value for:ts:cv:‘proxy.config.http.cache.cache_urls_that_look_dynamic‘.

In addition if a TTL is set for rule that matches in cache.config then this check is not done.

Range Request Cache valid only if :ts:cv:‘proxy.config.http.cache.range.lookup‘ in records.config is non-zero. This does not mean the range request can be cached, only that it might besatisfiable from the cache. In addition, :ts:cv:‘proxy.config.http.cache.range.write‘ can be set totry to force a write on a range request. This probably has little value at the moment, but if for ex-ample the origin server ignores the Range: header, this option can allow for the response to becached. It is disabled by default, for best performance.

A plugin can call TSHttpTxnReqCacheableSet() to force the request to be viewed as cache valid.

Cache Lookup

If the initial request is not determined to be cache invalid then a lookup is done. Cache lookup determines if an objectis in the cache and if so, where it is located. In some cases the lookup proceeds to read the first Doc from disk toverify the object is still present in the cache.

The basic steps to a cache lookup are:

1. The cache key is computed.

This is normally computed using the request URL but it can be overridden by a plugin . The cache index stringis not stored, as it is presumed computable from the client request headers.

2. The cache stripe is determined (based on the cache key).

The cache key is used as a hash key in to an array of Vol instances. The construction and arrangement of thisarray is the essence of how volumes are assigned.

3. The cache stripe directory is probed using the index key computed from the cache key.

Various other lookaside directories are checked as well, such as the aggregation buffer.

4. If the directory entry is found the first Doc is read from disk and checked for validity.

This is done in CacheVC::openReadStartHead() or CacheVC::openReadStartEarliest()which are tightly coupled methods.

If the lookup succeeds, then a more detailed directory entry (struct OpenDir) is created. Note that the directory probeincludes a check for an already extant OpenDir which, if found, is returned without additional work.

Cache Read

Cache read starts after a successful cache lookup. At this point the first Doc has been loaded in to memory and can beconsulted for additional information. This will always contain the HTTP headers for all alternates of the object.

Read while write

There is provision in the code to support read while write, that is, serving an object from cache in one transactionwhile it is being written in another. Several settings are needed for it to be used. See Reducing Origin Server



Requests (Avoiding the Thundering Herd). It must specifically enabled in records.config and if not, a cacheread will fail if the object is currently be written or updated.

At this point an alternate for the object is selected. This is done by comparing the client request to the stored responseheaders, but it can be controlled by a plugin using TS_HTTP_ALT_SELECT_HOOK.

The content can now be checked to see if it is stale by calculating the freshness of the object. This is essentiallychecking how old the object is by looking at the headers and possibly other metadata (note that the headers can’t bechecked until we’ve selected an alternate).

Most of this work is done in HttpTransact::what_is_document_freshness.

First, the TTL (time to live) value, which can be set in cache.config, is checked if the request matches theconfiguration file line. This is done based on when the object was placed in the cache, not on any data in the headers.

Next, an internal flag (needs-revalidate-once) is checked if the cache.config valuerevalidate-after is not set, and if set the object is marked stale.

After these checks the object age is calculated by HttpTransactHeaders::calculate_document_age.and then any configured fuzzing is applied. The limits to this age based on available data is calculated byHttpTransact::calculate_document_freshness_limit.

How this age is used is determined by the records.config setting for:ts:cv:‘proxy.config.http.cache.when_to_revalidate‘. If this is 0 then the built caclulations are used whichcompare the freshness limits with document age, modified by any of the client supplied cache control values(max-age, min-fresh, max-stale) unless explicitly overridden in cache.config.

If the object is not stale then it is served to the client. If it is stale, the client request may be changed to an IfModified Since request to revalidate.

The request is served using a standard virtual connection tunnel (HttpTunnel) with the CacheVC acting as theproducer and the client NetVC acting as the sink. If the request is a range request this can be modified with atransform to select the appropriate parts of the object or, if the request contains a single range, it can use the rangeacceleration.

Range acceleration is done by consulting a fragment offset table attached to the earliest Doc which contains offsetsfor all fragments past the first. This allows loading the fragment containing the first requested byte immediately ratherthan performing reads on the intermediate fragments.

Cache Write

Writing to the cache is handled by an instance of the class CacheVC. This is a virtual connection which receives dataand writes it to cache, acting as a sink. For a standard transaction data transfers between virtual connections (VConns)are handled by HttpTunnel. Writing to the cache is done by attaching a CacheVC instance as a tunnel consumer.It therefore operates in parallel with the virtual connection that transfers data to the client. The data does not flow tothe cache and then to the client, it is split and goes both directions in parallel. This avoids any data synchronizationissues between the two.

Writing to disk

The actual write to disk is handled in a separate thread dedicated to I/O operations, the AIO threads. The cachelogic marshals the data and then hands the operation off to the AIO thread which signals back once the operationcompletes.

While each CacheVC handles its transactions independently, they do interact at the volume level as each CacheVCmakes calls to the volume object to write its data to the volume content. The CacheVC accumulates data internally



until either the transaction is complete or the amount of data to write exceeds the target fragment size. In the formercase the entire object is submitted to the volume to be written. In the latter case, a target fragment size amount ofdata is submitted and the CacheVC continues to operate on subsequent data. The volume in turn places these writerequests in an holding area called the aggregation buffer.

For objects under the target fragment size, there is no consideration of order, the object is simply written to the volumecontent. For larger objects, the earliest Doc is written first and the first Doc written last. This provides some detectionability should the object be overwritten. Because of the nature of the write cursor no fragment after the first fragment(in the earliest Doc) can be overwritten without also overwriting that first fragment (since we know at the time theobject was finalized in the cache the write cursor was at the position of the first Doc).

: It is the responsibility of the CacheVC to not submit writes that exceed the target fragment size.

Update

Cache write also covers the case where an existing object in the cache is modified. This occurs when:

• A conditional request is made to the origin server and a 304 - Not Modified response is received.

• An alternate of the object is retrieved from an origin server and added to the object.

• An alternate of the object is removed (e.g., due to a DELETE request).

In every case the metadata for the object must be modified. Because Traffic Server never updates data already in thecache this means the first Doc will be written to the cache again and the volume directory entry updated. Becausea client request has already been processed the first Doc has been read from cache and is in memory. The alternatevector is updated as appropriate (an entry added or removed, or changed to contain the new HTTP headers), and thenwritten to disk. It is possible for multiple alternates to be updated by different CacheVC instances at the same time.The only contention is the first Doc; the rest of the data for each alternate is completely independent.

Aggregation Buffer

Disk writes to cache are handled through an aggregation buffer. There is one for each Vol instance. To minimize thenumber of system calls data is written to disk in units of roughly target fragment size bytes. The algorithm used issimple: data is piled up in the aggregation buffer until no more will fit without going over the target fragment size, atwhich point the buffer is written to disk and the volume directory entries for objects with data in the buffer are updatedwith the actual disk locations for those objects (which are determined by the write to disk action). After the buffer iswritten it is cleared and process repeats. There is a special lookup table for the aggregation buffer so that object lookupcan find cache data in that memory.

Because data in the aggregation buffer is visible to other parts of the cache, particularly cache lookup, there is no needto push a partially filled aggregation buffer to disk. In effect, any such data is memory cached until enough additionalcache content arrives to fill the buffer.

The target fragment size has little effect on small objects because the fragment size is used only to parcel out diskwrite operations. For larger objects the effect very significant as it causes those objects to be broken up in to fragmentsat different locations on in the volume. Each fragment write has its own entry in the volume directory which are com-putationally chained (each cache key is computed from the previous one). If possible, a fragment table is accumulatedin the earliest Doc which has the offsets of the first byte for each fragment.



Evacuation Mechanics

By default, the write cursor will overwrite (de facto evict from cache) objects as it proceeds once it has gone aroundthe cache stripe at least once. In some cases this is not acceptable and the object is evacuated by reading it from thecache and then writing it back to cache which moves the physical storage of the object from in front of the write cursorto behind the write cursor. Objects that are evacuated are handled in this way based on data in stripe data structures(attached to the Vol instance).

Evacuation data structures are defined by dividing up the volume content into a disjoint and contiguous set of regionsof EVACUATION_BUCKET_SIZE bytes. The Vol::evacuate member is an array with an element for eachevacuation region. Each element is a doubly linked list of EvacuationBlock instances. Each instance contains aDir that specifies the fragment to evacuate. It is assumed that an evacuation block is placed in the evacuation bucket(array element) that corresponds to the evacuation region in which the fragment is located although no ordering perbucket is enforced in the linked list (this sorting is handled during evacuation). Objects are evacuated by specifyingthe first or earliest fragment in the evactuation block. The evactuation operation will then continue the evacuation forsubsequent fragments in the object by adding those fragments in evacuation blocks. Note that the actual evacuationof those fragments is delayed until the write cursor reaches the fragments, it is not necessarily done at the time theearliest fragment is evacuated.

There are two types of evacuations: reader based and forced. The EvacuationBlock has a reader count to trackthis. If the reader count is zero, then it is a forced evacuation and the the target, if it exists, will be evacuated whenthe write cursor gets close. If the reader value is non-zero then it is a count of entities that are currently expectingto be able to read the object. Readers increment the count when they require read access to the object, or createthe EvacuationBlock with a count of 1. When a reader is finished with the object it decrements the count andremoves the EvacuationBlock if the count goes to zero. If the EvacuationBlock already exists with a countof zero, the count is not modified and the number of readers is not tracked, so the evacuation is valid as long as theobject exists.

Evacuation is driven by cache writes, essentially in Vol::aggWrite. This method processes the pending cachevirtual connections that are trying to write to the stripe. Some of these may be evacuation virtual connections. If sothen the completion callback for that virtual connection is called as the data is put in to the aggregation buffer.

When no more cache virtual connections can be processed (due to an empty queue or the aggregation buffer filling)then Vol::evac_range is called to clear the range to be overwritten plus an additional EVACUATION_SIZErange. The buckets covering that range are checked. If there are any items in the buckets a new cache virtual connection(a doc evacuator) is created and used to read the evacuation item closest to the write cursor (i.e. with the smallestoffset in the stripe) instead of the aggregation write proceeding. When the read completes it is checked for validityand if valid, the cache virtual connection for it is placed at the front of the write queue for the stripe and the writeaggregation resumed.

Before doing a write, the method Vol::evac_range() is called to start an evacuation. If any fragments are foundin the buckets in the range the earliest such fragment (smallest offset, closest to the write cursor) is selected andread from disk and the aggregation buffer write is suspended. The read is done via a cache virtual connection whichalso effectively serves as the read buffer. Once the read is complete, that cache virtual connection instance (the docevacuator) is placed at the front of the stripe write queue and written out in turn. Because the fragment data is now inmemory it is acceptable to overwrite the disk image.

Note that when normal stripe writing is resumed, this same check is done again, each time evauating (if needed) afragment and queuing them for writing in turn.

Updates to the directory are done when the write for the evacuated fragment completes. Multi-fragment objects aredetected after the read completes for a fragment. If it is not the first fragment then the next fragment is marked forevacuation (which in turn, when it is read, will pull the subsequent fragment). The logic presumes that the end of thealternate is when the next key is not in the directory.

This interacts with the one at a time strategy of the aggregation write logic. If a fragment is close to the fragment beingevacuated, it may end up in the same evacuation bucket. Because the aggregation write checks every time for the nextfragment to evacuate it will find that next fragment and evacuate it before it is overwritten.



Evacuation Operation

The primary source of fragments to be evacuated are active fragments. That is, fragments which are currently open forreading or writing. This is tracked by the reader value in the evacuation blocks noted above.

If object pinning is enabled, then a scan is done on a regular basis as the write cursor moves to detect pinned objectsand mark them for evacuation.

Fragments can also be evacuated through hit evacuation. This is configured by:ts:cv:‘proxy.config.cache.hit_evacuate_percent‘ and :ts:cv:‘proxy.config.cache.hit_evacuate_size_limit‘.When a fragment is read it is checked to see if it is close and in front of the write cursor, close being less than thespecified percent of the size of the stripe. If set at the default value of 10, then if the fragment is withing 10% of thesize of the stripe, it is marked for evacuation. This is cleared if the write cursor passes through the fragment while itremains open (as all open objects are evacuated). If, when the object is closed, the fragment is still marked then it isplaced in the appropriate evacuation bucket.

Initialization

Initialization starts with an instance of Store reading the storage configuration file, by default storage.config.For each valid element in the file an instance of Span is created. These are of basically four types:

• File

• Directory

• Disk

• Raw device

After creating all the Span instances, they are grouped by device ID to internal linked lists attached to theStore::disk array[#store-disk-array]_. Spans that refer to the same directory, disk, or raw device are coalescedin to a single span. Spans that refer to the same file with overlapping offsets are also coalesced5. This is all done inink_cache_init() called during startup.

: The span logic is also used by the HostDB and more than one otherwise inexplicable feature is provided by the spanlogic for that module.

After configuration initialization, the cache processor is started by calling CacheProcessor::start(). Thisdoes a number of things:

For each valid span, an instance of CacheDisk is created. This class is a continuation and so can be used to performpotentially blocking operations on the span. The primary use of these is to be passed to the AIO threads as the callbackwhen an I/O operation completes. These are then dispatched to AIO threads to perform storage unit initialization. Afterall of those have completed, the resulting storage is distributed across the volumes in cplist_reconfigure().The CacheVol instances are created at this time.

Cache stripe assignment setup is done once all stripes have initialized (that is, the stripe header information hasbeen successfully read from disk for all stripes). The assignment information is stored as an array of indices.These are indices in to an array of stripes. Both the assignment and the stripe arrays are stored in an instance ofCacheHostRecord. Assignment initialization consists of populating the assignment array, which is much largerthan the stripe array.

There is an instance of CacheHostRecord for each line in hosting.config and one generic record. For theconfigured instances, the set of stripes is determined from the cache volume specified in the line. If no lines are

5 This linked list is mostly ignored in later processing, causing all but one file or directory storage units on the same device to be ignored. SeeTS-1869.




specified, all stripes are placed in the generic record, otherwise only those stripes marked as default are placed in thegeneric record.

: If hosting records are specified, it is an error to not specify at least one default cache volume.

The assignment table is initialized in build_vol_hash_table() which is called for each CacheHostRecordinstance. For each stripe in the host record, a sequence of pseudo-random numbers is generated. This begins withthe folded hash of the stripe hash identifier, which is the device path followed by the skip and size values for thatstripe, making it unique. This also makes the sequence deterministic for any particular stripe.

Each stripe gets one number in its sequence for every VOL_HASH_ALLOC_SIZE (8 MB currently) of storage. Thesenumbers are paired with the stripe index, combined across all stripes, then sorted by the random values. The resultingarray is sampled for every slot in the stripe assignment table by dividing the maximum random value by the size ofthe assignment table and using the value midway between each multiple of the result of the division. The coalescedpsuedo-random sequence is scanned for each sample in turn and the first number not greater than the sample is found.The stripe associated with that value is used for that assignment table entry.

While this procedure is deterministic, it is sensitive to initial conditions, including the size of each stripe.

Data Structures

class OpenDirAn open directory entry. It contains all the information of a Dir plus additional information from the first Doc.

class CacheVCA virtual connection class which accepts input for writing to cache.

int CacheVC::openReadStartHead(int event, Event *e)Performs the initial read for a cached object.

int CacheVC::openReadStartEarliest(int event, Event *e)Performs the initial read for an alternate of an object.

class HttpTunnelData transfer driver. This contains a set of producers. Each producer is connected to one or more consumers.The tunnel handles events and buffers so that data moves from producers to consumers. The data, as muchas possible, is kept in reference counted buffers so that copies are done only when the data is modified or forsources (which acquire data from outside Traffic Server) and sinks (which move data to outside Traffic Server).

class CacheControlResultHolds the data from a line in cache.config.

class CacheHTTPInfoVectorDefined in P_CacheHttp.h. This is an array of HTTPInfo objects and serves as the respository of informa-tion about alternates of an object. It is marshaled as part of the metadata for an object in the cache.

class HTTPInfoDefined in HTTP.h.

This class is a wrapper for HTTPCacheAlt. It provides the external API for accessing data in the wrappedclass. It contains only a pointer (possibly NULL) to an instance of the wrapped class.

class CacheHTTPInfoA typedef for HTTPInfo.

class HTTPCacheAltDefined in HTTP.h.

This is the metadata for a single alternate for a cached object. It contains, among other data, the following:


https://github.com/apache/trafficserver/blob/master/iocore/cache/P_CacheHttp.h

https://github.com/apache/trafficserver/blob/master/proxy/hdrs/HTTP.h

https://github.com/apache/trafficserver/blob/master/proxy/hdrs/HTTP.h


•The key for the earliest Doc of the alternate.

•The request and response headers.

•The fragment offset table.[#fragment-offset-table]_

•Timestamps for request and response from origin server.

class EvacuationBlockRecord for evacuation.

class VolThis represents a storage unit inside a cache volume.

off_t Vol::segmentsThe number of segments in the volume. This will be roughly the total number of entries divided by thenumber of entries in a segment. It will be rounded up to cover all entries.

off_t Vol::bucketsThe number of buckets in the volume. This will be roughly the number of entries in a segment divided byDIR_DEPTH. For currently defined values this is around 16,384 (2^16 / 4). Buckets are used as the targetsof the index hash.

DLL<EvacuationBlock> Vol::evacuateArray of of EvacuationBlock buckets. This is sized so there is one bucket for every evacuation span.

off_t lenLength of stripe in bytes.

int Vol::evac_range(off_t low, off_t high, int evac_phase)Start an evacuation if there is any EvacuationBlock in the range from :arg:‘low‘ to :arg:‘high‘. Return 0if no evacuation was started, non-zero otherwise.

class CacheVolA cache volume as described in volume.config.

class DocDefined in P_CacheVol.h.

uint32_t Doc::magicValidity check value. Set to DOC_MAGIC for a valid document.

uint32_t Doc::lenThe length of this segment including the header length, fragment table, and this structure.

uint64_t Doc::total_lenTotal length of the entire document not including meta data but including headers.

INK_MD5 Doc::first_keyFirst index key in the document (the index key used to locate this object in the volume index).

INK_MD5 Doc::keyThe index key for this fragment. Fragment keys are computationally chained so that the key for the nextand previous fragments can be computed from this key.

uint32_t Doc::hlenDocument header (metadata) length. This is not the length of the HTTP headers.

uint8_t Doc::ftypeFragment type. Currently only CACHE_FRAG_TYPE_HTTP is used. Other types may be used for cacheextensions if those are ever implemented.

uint24_t Doc::flenFragment table length, if any. Only the first Doc in an object should contain a fragment table.


https://github.com/apache/trafficserver/blob/master/iocore/cache/P_CacheVol.h


The fragment table is a list of offsets relative to the HTTP content (not counting metadata or HTTP head-ers). Each offset is the byte offset of the first byte in the fragment. The first element in the table is thesecond fragment (what would be index 1 for an array). The offset for the first fragment is of course alwayszero and so not stored. The purpose of this is to enable a fast seek for range requests. Given the first Docthe fragment containing the first byte in the range can be computed and loaded directly without furtherdisk access.

Removed as of version 3.3.0.

uint32_t Doc::sync_serialUnknown.

uint32_t Doc::write_serialUnknown.

uint32_t pinnedFlag and timer for pinned objects.

uint32_t checksumUnknown.

class VolHeaderFooter

API functions

void TSHttpTxnReqCacheableSet(TSHttpTxn txnp, int flag)Set a flag that marks a request as cacheable. This is a positive override only, setting :arg:‘flag‘ to 0 restores thedefault behavior, it does not force the request to be uncacheable.

TSReturnCode TSCacheUrlSet(TSHttpTxn txnp, char const* url, int length)Set the cache key for the transaction :arg:‘txnp‘ as the string pointed at by :arg:‘url‘ of :arg:‘length‘ char-acters. It need not be NUL-terminated. This should be called from TS_HTTP_READ_REQUEST_HDR_HOOKwhich is before cache lookup but late enough that the HTTP request header is available.

TSReturnCode TSHttpTxnCacheLookupUrlGet(TSHttpTxn txnp, TSMBuffer bufp, TSMLoc offset)Get the current cache key URL, also referred to as the lookup URL. This must be stored in a properly allocatedURL object, typically created with a TSUrlCreate().

TSReturnCode TSHttpTxnCacheLookupUrlSet(TSHttpTxn txnp, TSMBuffer bufp, TSMLoc offset)Set the current cache key URL, also referred to as the lookup URL. This must be stored in a properly allocatedURL object, typically created with a TSUrlCreate() or TSUrlClone().

The APIs that modify the cache key can be called as early as TS_HTTP_READ_REQUEST_HDR_HOOK but no laterthan TS_HTTP_POST_REMAP_HOOK. The cache key is not only used for a cache lookup before going to origin, butalso to mark the intent to write to cache on an origin response (if possible).

Cache Internals

int DIR_SIZE_WITH_BLOCK(int big)A preprocessor macro which computes the maximum size of a fragment based on the value of :arg:‘big‘. Thisis computed as if the argument where the value of the :arg:‘big‘ field in a struct Dir.

int DIR_BLOCK_SIZE(int big)A preprocessor macro which computes the block size multiplier for a struct Dir where :arg:‘big‘ is the:arg:‘big‘ field value.



Cache Tools

Tools and techniques for cache monitoring and inspection.

• The cache inspector.

Topics to be done

• Resident alternates

• Object refresh

Cache Consistency

The cache is completely consistent, up to and including kicking the power cord out, if the write buffer on consumerdisk drives is disabled. You need to use:

hdparm -W0

The cache validates that all the data for the document is available and will silently mark a partial document as a misson read. There is no gentle shutdown for Traffic Server. You simply kill the process and the recovery code (fsck) isrun every time Traffic Server starts up.

On startup the two versions of the index are checked, and the last valid one is read into memory. Traffic Server thenmoves forward from the last snapped write cursor and reads all the fragments written to disk and updates the directory(as in a log-based file system). It stops reading at the write before the last valid write header it sees (as a write is notnecessarily atomic because of sector reordering). Then the new updated index is written to the invalid version (in caseof a crash during startup) and the system starts.

Volume Tagging

Currently, cache volumes are allocated somewhat arbitrarily from storage elements. This enhancement allowsstorage.config to assign storage units to specific volumes although the volumes must still be listed in volume.config in general and in particular to map domains to specific volumes. A primary use case for this is to be able tomap specific types of content to different storage elements. This can be employed to have different storage devices forvarious types of content (SSD vs. rotational).

Version Upgrade

It is currently the case that any change to the cache format will clear the cache. This is an issue when upgrading theTraffic Server version and should be kept in mind.

Controlling the cache key

The cache key is by default the URL of the request. There are two possible choices, the original (pris-tine) URL and the remapped URL. Which of these is used is determined by the configuration value:ts:cv:‘proxy.config.url_remap.pristine_host_hdr‘.

This is an INT value. If set to 0 (disabled) then the remapped URL is used, and if it is not 0 (enabled) then the originalURL is used. This setting also controls the value of the HOST header that is placed in the request sent to the originserver, using the hostname from the original URL if not 0 and the host name from the remapped URL if 0. It has noother effects.




For caching, this setting is irrelevant if no remapping is done or there is a one-to-one mapping between the originaland remapped URLs.

It becomes significant if multiple original URLs are mapped to the same remapped URL. If pristine headers areenabled, requests to different original URLs will be stored as distinct objects in the cache. If disabled, the remappedURL will be used and there may be collisions. This is bad if the contents different, but quite useful if they are the same(as in situations where the original URLs are just aliases for the same underlying server resource).

This is also an issue if a remapping is changed because it is effectively a time axis version of the previous case. Ifan original URL is remapped to a different server address then the setting determines if existing cached objects willbe served for new requests (enabled) or not (disabled). Similarly, if the original URL mapped to a particular URL ischanged then cached objects from the initial original URL will be served from the updated original URL if pristineheaders is disabled.

These collisions are not by themselves good or bad. An administrator needs to decide which is appropriate for theirsituation and set the value correspondingly.

If a greater degree of control is desired, a plugin must be used to invoke the API callsTSHttpTxnCacheLookupUrlSet() or TSCacheUrlSet() to provide a specific cache key. TheTSCacheUrlSet() API can be called as early as TS_HTTP_READ_REQUEST_HDR_HOOK but no laterthan TS_HTTP_POST_REMAP_HOOK. It can be called only once per transaction; calling it multiple times has noadditional effect.

A plugin that changes the cache key must do so consistently for both cache hit and cache miss requests because twodifferent requests that map to the same cache key will be considered equivalent by the cache. Use of the URL directlyprovides this and so must any substitute. This is entirely the responsibility of the plugin; there is no way for the TrafficServer core to detect such an occurrence.

If TSHttpTxnCacheLookupUrlGet() is called after new cache url set byTSHttpTxnCacheLookupUrlSet() or TSCacheUrlSet(), it should use a URL location created byTSUrlCreate() as its third input parameter instead of getting url_loc from the client request.

It is a requirement that the string be syntactically a URL but otherwise it is completely arbitrary and need not haveany path. For instance, if the company Network Geographics wanted to store certain content under its own cache key,using a document GUID as part of the key, it could use a cache key like

ngeo://W39WaGTPnvg

The scheme ngeo was picked specifically because it is not a valid URL scheme, and so will never collide with anyvalid URL.

This can be useful if the URL encodes both important and unimportant data. Instead of storing potentially identicalcontent under different URLs (because they differ on the unimportant parts) a url containing only the important partscould be created and used.

For example, suppose the URL for Network Geographics content encoded both the document GUID and a referralkey.

http://network-geographics-farm-1.com/doc/W39WaGTPnvg.2511635.UQB_zCc8B8H

We don’t want to serve the same content for every possible referrer. Instead, we could use a plugin to convert this tothe previous example and requests that differed only in the referrer key would all reference the same cache entry. Notethat we would also map the following to the same cache key

http://network-geographics-farm-56.com/doc/W39WaGTPnvg.2511635.UQB_zCc8B8H

This can be handy for sharing content between servers when that content is identical. Plugins can change the cachekey, or not, depending on any data in the request header. For instance, not changing the cache key if the request is not



in the doc directory. If distinguishing servers is important, that can easily be pulled from the request URL and usedin the synthetic cache key. The implementor is free to extract all relevant elements for use in the cache key.

While there is no explicit requirement that the synthetic cache key be based on the HTTP request header, in practiceit is generally necessary due to the consistency requirement. Because cache lookup happens before attempting toconnect to the origin server, no data from the HTTP response header is available, leaving only the request header. Themost common case is the one described above where the goal is to elide elements of the URL that do not affect thecontent to minimize cache footprint and improve cache hit rates.

RAM Cache

New RAM Cache Algorithm (CLFUS)

The new RAM Cache uses ideas from a number of cache replacement policies and algorithms, including LRU, LFU,CLOCK, GDFS and 2Q, called CLFUS (Clocked Least Frequently Used by Size). It avoids any patented algorithmsand includes the following features:

• Balances Recentness, Frequency and Size to maximize hit rate (not byte hit rate).

• Is Scan Resistant and extracts robust hit rates even when the working set does not fit in the RAM Cache.

• Supports compression at 3 levels: fastlz, gzip (libz), and xz (liblzma). Compression can be moved to anotherthread.

• Has very low CPU overhead, only slightly more than a basic LRU. Rather than using an O(lg n) heap, it uses aprobabilistic replacement policy for O(1) cost with low C.

• Has relatively low memory overhead of approximately 200 bytes per object in memory.

The rationale for emphasizing hit rate over byte hit rate is that the overhead of pulling more bytes from secondarystorage is low compared to the cost of a request.

The RAM Cache consists of an object hash fronting 2 LRU/CLOCK lists and a seen hash table. The first cached listcontains objects in memory, while the second contains a history of objects which have either recently been in memoryor are being considered for keeping in memory. The seen hash table is used to make the algorithm scan resistant.

The list entries record the following information:

Value Descriptionkey 16 byte unique object identifierauxkeys 8 bytes worth of version number (in our system, the block in the partition). When the version of an

object changes old entries are purged from the cache.hits Number of hits within this clock period.size size of the object in the cache.len Length of the object, which differs from size because of compression and padding).com-pressed_len

Compressed length of the object.

com-pressed

Compression type, or none if no compression. Possible types are: fastlz, libz, and liblzma.

uncom-pressible

Flag indicating that content cannot be compressed (true), or that it mat be compressed (false).

copy Whether or not this object should be copied in and copied out (e.g. HTTP HDR).LRU linkHASHlinkIOBuffer-Data

Smart point to the data buffer.



The interface to the cache is Get and Put operations. Get operations check if an object is in the cache and are calledon a read attempt. The Put operation decides whether or not to cache the provided object in memory. It is called aftera read from secondary storage.

Seen Hash

The Seen List becomes active after the Cached and History lists become full following a cold start. The purpose isto make the cache scan resistant, which means that the cache state must not be affected at all by a long sequence Getand Put operations on objects which are seen only once. This is essential, and without it not only would the cachebe polluted, but it could lose critical information about the objects that it cares about. It is therefore essential thatthe Cache and History lists are not affected by Get or Put operations on objects seen the first time. The Seen Hashmaintains a set of 16 bit hash tags, and requests which do not hit in the object cache (are in the Cache List or HistoryList) and do not match the hash tag result in the hash tag being updated but are otherwise ignored. The Seen Hash issized to approximately the number of objects in the cache in order to match the number that are passed through it withthe CLOCK rate of the Cached and History Lists.

Cached List

The Cached List contains objects actually in memory. The basic operation is LRU with new entries inserted into aFIFO queue and hits causing objects to be reinserted. The interesting bit comes when an object is being consideredfor insertion. A check is first made against the Object Hash to see if the object is in the Cached List or History. Hitsresult in updating the hit field and reinsertion of the object. History hits result in the hit field being updated and acomparison to see if this object should be kept in memory. The comparison is against the least recently used membersof the Cache List, and is based on a weighted frequency:

CACHE_VALUE = hits / (size + overhead)

A new object must be enough bytes worth of currently cached objects to cover itself. Each time an object is consideredfor replacement the CLOCK moves forward. If the History object has a greater value then it is inserted into the CachedList and the replaced objects are removed from memory and their list entries are inserted into the History List. If theHistory object has a lesser value it is reinserted into the History List. Objects considered for replacement (at least one)but not replaced have their hits field set to 0 and are reinserted into the Cached List. This is the CLOCK operationon the Cached List.

History List

Each CLOCK, the least recently used entry in the History List is dequeued and if the hits field is not greater than 1(it was hit at least once in the History or Cached List) it is deleted. Otherwise, the hits is set to 0 and it is requeuedon the History List.

Compression and Decompression

Compression is performed by a background operation (currently called as part of Put) which maintains a pointer intothe Cached List and runs toward the head compressing entries. Decompression occurs on demand during a Get. In thecase of objects tagged copy, the compressed version is reinserted in the LRU since we need to make a copy anyway.Those not tagged copy are inserted uncompressed in the hope that they can be reused in uncompressed form. This isa compile time option and may be something we want to change.

There are 3 algorithms and levels of compression (speed on an Intel i7 920 series processor using one thread):



Method CompressionRate

DecompressionRate

Notes

fastlz 173 MB/sec 442 MB/sec Basically free since disk or network will limit first; ~53%final size.

libz 55 MB/sec 234 MB/sec Almost free, particularly decompression; ~37% final size.li-blzma

3 MB/sec 50 MB/sec Expensive; ~27% final size.

These are ballpark numbers, and your millage will vary enormously. JPEG, for example, will not compress with anyof these (or at least will only do so at such a marginal level that the cost of compression and decompression is whollyunjustified), and the same is true of many other media and binary file types which embed some form of compression.The RAM Cache does detect compression level and will declare something incompressible if it doesn’t get below 90%of the original size. This value is cached so that the RAM Cache will not attempt to compress it again (at least as longas it is in the history).

Tiered Storage

Tiered storage is an attempt to allow Traffic Server to take advantage of physical storage with different properties.This design concerns only mechanism. Policies to take advantage of these are outside of the scope of this document.Instead we will presume an oracle which implements this policy and describe the queries that must be answered bythe oracle and the effects of the answers.

Beyond avoiding question of tier policy, the design is also intended to be effectively identical to current operations forthe case where there is only one tier.

The most common case for tiers is an ordered list of tiers, where higher tiers are presumed faster but more expensive(or more limited in capacity). This is not required. It might be that different tiers are differentiated by other properties(such as expected persistence). The design here is intended to handle both cases.

The design presumes that if a user has multiple tiers of storage and an ordering for those tiers, they will usually wantcontent stored at one tier level to also be stored at every other lower level as well, so that it does not have to be copiedif evicted from a higher tier.

Configuration

Each storage unit in storage.config can be marked with a quality value which is 32 bit number. Storage unitsthat are not marked are all assigned the same value which is guaranteed to be distinct from all explicit values. Thequality value is arbitrary from the point of view of this design, serving as a tag rather than a numeric value. The user(via the oracle) can impose what ever additional meaning is useful on this value (rating, bit slicing, etc.).

In such cases, all volumes should be explicitly assigned a value, as the default (unmarked) value is not guaranteed tohave any relationship to explicit values. The unmarked value is intended to be useful in situations where the user hasno interest in tiered storage and so wants to let Traffic Server automatically handle all volumes as a single tier.

Operations

After a client request is received and processed, volume assignment is done. For each tier, the oracle would return oneof four values along with a volume pointer:

READ The tier appears to have the object and can serve it.

WRITE The object is not in this tier and should be written to this tier if possible.

RW Treat as READ if possible, but if the object turns out to not in the cache treat as WRITE.

NO_SALE Do not interact with this tier for this object.



The volume returned for the tier must be a volume with the corresponding tier quality value. In effect, the current styleof volume assignment is done for each tier, by assigning one volume out of all of the volumes of the same quality andreturning one of RW or WRITE, depending on whether the initial volume directory lookup succeeds. Note that as withcurrent volume assignment, it is presumed this can be done from in memory structures (no disk I/O required).

If the oracle returns READ or RW for more than one tier, it must also return an ordering for those tiers (it may return anordering for all tiers, ones that are not readable will be ignored). For each tier, in that order, a read of cache storage isattempted for the object. A successful read locks that tier as the provider of cached content. If no tier has a successfulread, or no tier is marked READ or RW then it is a cache miss. Any tier marked RW that fails the read test is demoted toWRITE.

If the object is cached, every tier that returns WRITE receives the object to store in the selected volume (this includesRW returns that are demoted to WRITE). This is a cache to cache copy, not from the origin server. In this case,tiers marked RW that are not tested for read will not receive any data and will not be further involved in the requestprocessing.

For a cache miss, all tiers marked WRITE will receive data from the origin server connection (if successful).

This means, among other things, that if there is a tier with the object all other tiers that are written will get a local copyof the object, and the origin server will not be used. In terms of implementation, currently a cache write to a volumeis done via the construction of an instance of CacheVC which recieves the object stream. For tiered storage, the samething is done for each target volume.

For cache volume overrides (via hosting.config) this same process is used except with only the volumes stripescontained within the specified cache volume.

Copying

It may be necessary to provide a mechanism to copy objects between tiers outside of a client originated transaction. Interms of implementation, this is straight forward using HttpTunnel as if in a transaction, only using a CacheVCinstance for both the producer and consumer. The more difficult question is what event would trigger a possible copy.A signal could be provided whenever a volume directory entry is deleted, although it should be noted that the objectin question may have already been evicted when this event happens.

Additional Notes

As an example use, it would be possible to have only one cache volume that uses tiered storage for a particular setof domains using volume tagging. hosting.config would be used to direct those domains to the selected cachevolume. The oracle would check the URL in parallel and return NO_SALE for the tiers in the target cache volume forother domains. For the other tier (that of the unmarked storage units), the oracle would return RW for the tier in allcases as that tier would not be queried for the target domains.

Plugin Development

Plugin Development Introduction

This chapter provides a foundation for designing and writing plugins. Reading this chapter will help you to understand:

• The asynchronous event mode. This is the design paradigm used throughout Traffic Server; plugins must alsofollow this design. It includes the callback mechanism for Traffic Server to “wake up” your plugin and put it towork.

• Traffic Server’s HTTP processing, with an overview of the HTTP state machine.

• How plugins can hook onto and modify/extend Traffic Server’s HTTP processing.

4.7. Plugin Development 291


• A Roadmap with an overview of the functionality provided by the Traffic Server API.

Roadmap

This chapter has provided an overview of Traffic Server’s HTTP processing, API hooks, and the asynchronous eventmodel. Next, you must understand the capabilities of Traffic Server API functions. These are quite broad:

• HTTP header manipulation functions

Obtain information about and manipulate HTTP headers, URLs, & MIME headers.

• HTTP transaction functions

Get information about and modify HTTP transactions (for example: get the client IP associated to the transac-tion; get the server IP; get parent proxy information)

• IO functions

Manipulate vconnections (virtual connections, used for network and disk I/O)

• Network connection functions

Open connections to remote servers.

• Statistics functions

Define and compute statistics for your plugin’s activity.

• Traffic Server management functions

Obtain values for Traffic Server configuration and statistics variables.

Below are some guidelines for creating a plugin:

1. Decide what you want your plugin to do, based on the capabilities of the API and Traffic Server. The two kindsof example plugins provided with this SDK are HTTP-based (includes header-based and response transformplugins), and non-HTTP-based (a protocol plugin). These examples are discussed in the following chapters.

2. Determine where your plugin needs to hook on to Traffic Server’s HTTP processing (view the HTTP TransactionState Diagram.

3. Read Header-Based Plugin Examples to learn the basics of writing plugins: creating continuations and settingup hooks. If you want to write a plugin that transforms data, then read HTTP Transformations.

4. Figure out what parts of the Traffic Server API you need to use and then read about the details of those APIs inthis manual’s reference chapters.

5. Compile and load your plugin (see Getting Started.

6. Depending on your plugin’s functionality, you might start testing it by issuing requests by hand and checking forthe desired behavior in Traffic Server log files. See the *Traffic Server Administrator’s Guide* for informationabout Traffic Server logs.

Asynchronous Event Model

Traffic Server is a multi-threaded process. There are two main reasons why a server might use multiple threads:

• To take advantage of the concurrency available with multiple CPUs and multiple I/O devices.

• To manage concurrency from having many simultaneous client connections. For example, a server could createone thread for each connection, allowing the operating system (OS) to control switching between threads.



Traffic Server uses multiple threads for the first reason. However, Traffic Server does not use a separate OS thread pertransaction because it would not be efficient when handling thousands of simultaneous connections.

Instead, Traffic Server provides special event-driven mechanisms for efficiently scheduling work: the event system andcontinuations. The event system is used to schedule work to be done on threads. A continuation is a passive, event-driven state machine that can do some work until it reaches a waiting point; it then sleeps until it receives notificationthat conditions are right for doing more work. For example, HTTP state machines (which handle HTTP transactions)are implemented as continuations.

Continuation objects are used throughout Traffic Server. Some might live for the duration of the Traffic Server process,while others are created (perhaps by other continuations) for specific needs and then destroyed. Traffic Server Internals(below) shows how the major components of Traffic Server interact. Traffic Server has several processors, such ascache processor and net processor, that consolidate cache or network I/O tasks. Processors talk to the event system andschedule work on threads. An executing thread calls back a continuation by sending it an event. When a continuationreceives an event, it wakes up, does some work, and either destroys itself or goes back to sleep & waits for the nextevent.

Traffic Server Internals

4.6: Traffic Server Internals

Plugins are typically implemented as continuations. All of the sample code plugins (except hello-world) arecontinuations that are created when Traffic Server starts up; they then wait for events that trigger them into activity.

Traffic Server with Plugins

A plugin may consist of just one static continuation that is called whenever certain events happen. Examples of suchplugins include blacklist-1.c, basic-auth.c, and redirect-1.c. Alternatively, a plugin might dynam-ically create other continuations as needed. Transform plugins are built in this manner: a static parent continuationchecks all transactions to see if any are transformable; when a transaction is transformable, the static continuationcreates a type of continuation called a vconnection. The vconnection lives as long as it takes to complete the trans-form and then destroys itself. This design can be seen in all of the sample transform plugins. Plugins that supportnew protocols also have this architecture: a static continuation listens for incoming client connections and then createstransaction state machines to handle each protocol transaction.



4.7: Traffic Server with Plugins

When you write plugins, there are several ways to send events to continuations. For HTTP plugins, there is a “hook”mechanism that enables the Traffic Server HTTP state machine to send your plugin wakeup calls when needed.Additionally, several Traffic Server API functions trigger Traffic Server sub-processes to send events to plugins:TSContCall, TSVConnRead, TSCacheWrite, and TSMgmtUpdateRegister, to name a few.

Naming Conventions

Traffic Server HTTP State Machine

Traffic Server performs sophisticated HTTP caching and proxying. Important features include checking for alternatesand document freshness, filtering, supporting cache hierarchies, and hosting. Traffic Server handles thousands of clientrequests at a time and each request is handled by an HTTP state machine. These machines follow a complex statediagram that includes all of the states required to support Traffic Server’s features. The Traffic Server API provideshooks to a subset of these states, chosen for their relevance to plugins. You can view the API hooks and correspondingHTTP states in the HTTP Transaction State Diagram.

The example in this section (below) explains how a plugin typically intervenes and extends Traffic Server’s processingof an HTTP transaction. Complete details about hooking on to Traffic Server processes are provided in Hooks andTransactions.

HTTP Transaction

An HTTP transaction consists of a client request for a web document and Traffic Server’s response. The responsecould be the requested web server content or it could be an error message. The content could come from the TrafficServer cache or Traffic Server might fetch it from the origin server. The following diagram shows some states in atypical transaction - specifically, the scenario wherein content is served from cache.

Simplified HTTP Transaction



4.8: Simplified HTTP Transaction

In the diagram above, Traffic Server accepts the client connection, reads the request headers, looks up the originserver’s IP address, and looks for the requested content in the cache. If the content is not in the cache (a “miss”), thenTraffic Server opens a connection to the origin server and issues a request for the content. If the content is in the cache(a “hit”), then Traffic Server checks it for freshness.

If the content is fresh, then Traffic Server sends a reply header to the client. If the content is stale, then Traffic Serveropens a connection to the origin server and requests the content. The figure above, Simplified HTTP Transaction, doesnot show behavior in the event of an error. If there is an error at a any stage, then the HTTP state machine jumps tothe “send reply header” state and sends a reply. If the reply is an error, then the transaction closes. If the reply is notan error, then Traffic Server first sends the response content before it closes the transaction.

API Hooks Corresponding to States

4.9: API Hooks Corresponding to States Listed in

You use hooks as triggers to start your plugin. The name of a hook reflects the Traffic Server state that was justcompleted. For example, the “OS DNS lookup” hook wakes up a plugin right after the origin server DNS lookup.For a plugin that requires the IP address of the requested origin server, this hook is the right one to use. The Blacklist



plugin works in this manner, as shown in the Blacklist Plugin diagram below.

Blacklist Plugin

4.10: Blacklist Plugin

Traffic Server calls the Blacklist plugin right after the origin server DNS lookup. The plugin checks the requested hostagainst a list of blacklisted servers; if the request is allowed, then the transaction proceeds. If the host is forbidden,then the Blacklist plugin sends the transaction into an error state. When the HTTP state machine gets to the “sendreply header” state, it then calls the Blacklist plugin to provide the error message that’s sent to the client.

Types of Hooks

The Blacklist plugin’s hook to the origin server DNS lookup state is a global hook, meaning that the plugin is calledevery time there’s an HTTP transaction with a DNS lookup event. The plugin’s hook to the send reply header state isa transaction hook, meaning that this hook is only invoked for specified transactions (in the Blacklist Plugin example,it’s only used for requests to blacklisted servers). Several examples of setting up hooks are provided in Header-BasedPlugin Examples and HTTP Transformations.

Header manipulation plugins, such as filtering, basic authorization, or redirects, usually have a global hook to theDNS lookup or the read request header states. If specific actions need to be done to the transaction further on, thenthe plugin adds itself to a transaction hook. Transformation plugins require a global hook to check all transactions fortransformability followed by a transform hook, which is a type of transaction hook used specifically for transforms.

Getting Started

A Simple Plugin

This section describes how to write, compile, configure, and run a simple Traffic Server plugin. You’ll follow the stepsbelow:



1. Make sure that your plugin source code contains an TSPluginInit initialization function.

2. Compile your plugin source code, creating a shared library.

3. Add an entry to your plugin’s plugin.config file.

4. Add the path to your plugin shared library into the records.config file.


Compile Your Plugin

The process for compiling a shared library varies with the platform used, so the Traffic Server API provides the tsxstool which you can use to create shared libraries on all the supported Traffic Server platforms.

Example

Assuming the sample program is stored in the file hello-world.c, you could use the following commands to builda shared library

tsxs -o hello-world.so -c hello-world.c

tsxs can be found in trafficserver-dev package.

This shared library will be your plugin. In order to install it, run

sudo tsxs -o hello-world.so -i

or the equivalent to sudo on your platform.

Update the plugin.config File

Your next step is to tell Traffic Server about the plugin by adding the following line to the plugin.config file.Since our simple plugin does not require any arguments, the following plugin.config will work:

# a simple plugin.config for hello-worldhello-world.so

Traffic Server can accommodate multiple plugins. If several plugin functions are triggered by the same event, thenTraffic Server invokes each plugin’s function in the order each was defined in the plugin.config file.

Specify the Plugin’s Location

All plugins must be located in the directory specified by the configuration variable proxy.config.plugin.plugin_dir, which is located in the records.config file. The directory can be specified as an absolute orrelative path.

If a relative path is used, then the starting directory will be the Traffic Server installation directory as specified in/etc/traffic_server. The default value is libexec/trafficserver, but this can vary based on howthe software was configured and built. It is common to use the default directory. Be sure to place the shared libraryhello-world.so inside the directory you’ve configured.



Restart Traffic Server

The last step is to start/restart Traffic Server. Shown below is the output displayed after you’ve created and loadedyour hello-world plugin.

# ls libexec/trafficserverhello-world.so*# bin/traffic_server[Mar 27 19:06:31.669] NOTE: updated diags config[Mar 27 19:06:31.680] NOTE: loading plugin 'libexec/trafficserver/hello-world.so'hello world[Mar 27 19:06:32.046] NOTE: cache disabled (initializing)[Mar 27 19:06:32.053] NOTE: cache enabled[Mar 27 19:06:32.526] NOTE: Traffic Server running

Note: in the example above, Traffic Server notes are directed to the console by specifying E for proxy.config.diags.output.note in records.config. The second note shows Traffic Server attempting to load thehello-world plugin. The third line of Traffic Server output is from your plugin.

Plugin Registration and Version Checking

If you need you make a plugin that will load against multiple versions of Traffic Server, you can check the API versionat both compilation time and run time.

Use the following interfaces:

• TSPluginRegister()

• TSTrafficServerVersionGet()

The plugin registers the plugin and ensures it’s running with a compatible version of Traffic Server.

Naming Conventions

The Traffic Server API adheres to the following naming conventions:

• The TS prefix is used for all function and variable names defined in the Traffic Server API. Examples:TS_EVENT_NONE,TSMutex, and TSContCreate

• Enumerated values are always written in all uppercase letters. Examples: TS_EVENT_NONE andTS_VC_CLOSE_ABORT

• Constant values are all uppercase; enumerated values can be seen as a subset of constants. Examples:TS_URL_SCHEME_FILE and TS_MIME_FIELD_ACCEPT

• The names of defined types are mixed-case. Examples: TSHttpSsn and TSHttpTxn

• Function names are mixed-case. Examples: TSUrlCreate and TSContDestroy

• Function names use the following subject-verb naming style: TS-<subject>-<verb>, where <subject>goes from general to specific. This makes it easier to determine what a function does by reading its name. Forexample: the function to retrieve the password field (the specific subject) from a URL (the general subject) isTSUrlPasswordGet.

• Common verbs like Create, Destroy, Get, Set, Copy, Find, Retrieve, Insert, Remove, andDelete are used only when appropriate.

The Traffic Server API enables you to create plugins, using the C programming language, that customize the behaviorof your Traffic Server installation. This chapter contains the following sections:



• Understanding Traffic Server Plugins – a brief introduction to plugins.

• A Simple Plugin – walks through compiling and loading an example hello world plugin.

• Plugin Registration and Version Checking – shows you how to register your plugin and make sure it’s compatiblewith the version of Traffic Server you’re using.

• Naming Conventions – outlines Traffic Server API naming conventions. For guidelines on creating plugin sourcecode, see Plugin Development Introduction.

Understanding Traffic Server Plugins

Traffic Server enables sophisticated caching and processing of web-related traffic, such as DNS and HTTP requestsand responses.

Traffic Server itself consists of an event-driven loop that can be simplified as follows:

for (;;) {event = get_next_event();handle_event (event);

}

The Role of Plugins

You compile your plugin source code to create a shared library that Traffic Server loads when it is started. Your plugincontains callback functions that are registered for specific Traffic Server events. When Traffic Server needs to processan event, it invokes any and all call-back functions you’ve registered for that event type.

: Since plugins add object code to Traffic Server, programming errors in a plugin can have serious implications.Bugs in your plugin, such as an out-of-range pointer, can cause Traffic Server processes to crash and may ultimatelyresult in unpredictable behavior.

Plugin Process

Possible Uses for Plugins

Possible uses for plugins include the following:

• HTTP processing: plugins can filter, blacklist, authorize users, transform content

• Protocol support: plugins can enable Traffic Server to proxy-cache new protocol content

Some examples of plugins include:

• Blacklisting plugin: denies attempts to access web sites that are off-limits.

• Append transform plugin: adds text to HTTP response content.

• Image conversion plugin: transforms JPEG images to GIF images.

• Compression plugin: sends response content to a compression server that compresses the data (alternatively, acompression library local to the Traffic Server host machine could do the compression).

• Authorization plugin: checks a user’s permissions to access particular web sites. The plugin could consult alocal authorization program or send queries to an authorization server.



4.11: Plugin Process

• A plugin that gathers client information from request headers and enters this information in a database.

• Protocol plugin: listens for specific protocol requests on a designated port and then uses Traffic Server’s proxyserver & cache to serve client requests.

The figure below, Possible Traffic Server Plugins, illustrates several types of plugins.

Possible Traffic Server Plugins

You can find basic examples for many plugins in the SDK sample code:

• append-transform.c adds text from a specified file to HTTP/text responses. This plugin is explained inAppend-Transform Plugin

• The compression plugin in the figure communicates with the server that actually does the compression. Theserver-transform.c plugin shows how to open a connection to a transformation server, have the serverdo the transformation, and send transformed data back to the client. Although the transformation is null inserver-transform.c, a compression or image translation plugin could be implemented in a similar way.

• basic-auth.c performs basic HTTP proxy authorization.

• blacklist-1.c reads blacklisted servers from a configuration file and denies client access to these servers.This plugin is explained in Blacklist Plugin.

Plugin Loading

When Traffic Server is first started, it consults the plugin.config file to determine the names of all shared pluginlibraries that need to be loaded. The plugin.config file also defines arguments that are to be passed to eachplugin’s initialization function, TSPluginInit. The records.config file defines the path to each plugin sharedlibrary, as described in Specify the Plugin’s Location.



4.12: Possible Traffic Server Plugins



: The path for each of these files is <root_dir>/config/, where <root_dir> is where you installed Traffic Server.


The sample plugin.config file below contains a comment line, a blank line, and two plugin configurations:

# This is a comment line.

my-plugin.so junk.example.com trash.example.org garbage.example.edusome-plugin.so arg1 arg2 $proxy.config.http.cache.on

Each plugin configuration in the plugin.config file resembles a UNIX or DOS shell command; each line inplugin.config cannot exceed 1023 characters.

The first plugin configuration is for a plugin named my-plugin.so. It contains three arguments that are to be passedto that plugin’s initialization routine. The second configuration is for a plugin named some-plugin.so; it containsthree arguments. The last argument, ‘‘$proxy.config.http.cache.on‘‘, is actually a configuration variable. Traffic Serverwill look up the specified configuration variable and substitute its value.

Plugins with global variables should not appear more than once in plugin.config. For example, if you enter:

add-header.so header1add-header.so header2

then the second global variable, header2, will be used for both instances. A simple workaround is to give differentnames to different instances of the same plugin. For example:

cp add-header.so add-header1.socp add-header.so add-header2.so

These entries will produce the desired result below:

add-header1.so header1add-header2.so header2

Configuration File Rules

• Comment lines begin with # and continue to the end of the line.

• Blank lines are ignored.

• Plugins are loaded and initialized by Traffic Server in the order they appear in the plugin.config file.

Plugin Initialization

Each plugin must define an initialization function named TSPluginInit that Traffic Server invokes when the pluginis loaded. The TSPluginInit function is commonly used to read configuration information and register hooks forevent notification.

The TSPluginInit function has two arguments:

• The argc argument represents the number of arguments defined in the plugin.config file for that particularplugin



• The argv argument is an array of pointers to the actual arguments defined in the plugin.config file forthat plugin

See TSPluginInit() for details about TSPluginInit.

Building Plugins


The functions discussed in this section do not examine or modify Traffic Server configuration variables. To examineTraffic Server configuration and statistics variables, see Settings and Statistics.

The collection of TSConfig* functions are designed to provide a fast and efficient mechanism for accessing andchanging global configuration information within a plugin. Such a mechanism is simple enough to provide in a single-threaded program, but the translation to a multi-threaded program such as Traffic Server is difficult. A commontechnique is to have a single mutex protect the global configuration information; however, the problem with thissolution is that a single mutex becomes a performance bottleneck very quickly.

These functions define an interface to storing and retrieving an opaque data pointer. Internally, Traffic Server maintainsreference count information about the data pointer so that a call to TSConfigSet() will not disturb another threadusing the current data pointer. The philosophy is that once a user has a hold of the configuration pointer, it is okayfor it to be used even if the configuration changes. All that a user typically wants is a non-changing snapshot of theconfiguration. You should use TSConfigSet() for all global data updates.

Here’s how the interface works:

/* Assume that you have previously defined a plugin configuration

* data structure named ConfigData, along with its constructor

* plugin_config_allocator () and its destructor

* plugin_config_destructor (ConfigData *data)

*/ConfigData *plugin_config;

/* You will need to assign plugin_config a unique identifier of type

* unsigned int. It is important to initialize this identifier to zero

* (see the documentation of the function).

*/static unsigned int my_id = 0;

/* You will need an TSConfig pointer to access a snapshot of the

* current plugin_config.

*/TSConfig config_ptr;

/* Initialize plugin_config. */plugin_config = plugin_config_allocator();

/* Assign plugin_config an identifier using TSConfigSet. */my_id = TSConfigSet (my_id, plugin_config, plugin_config_destructor);

/* Get a snapshot of the current configuration using TSConfigGet. */config_ptr = TSConfigGet (my_id);

/* With an TSConfig pointer to the current configuration, you can

* retrieve the configuration's current data using TSConfigDataGet.

*/plugin_config = (ConfigData*) TSConfigDataGet (config_ptr);



/* Do something with plugin_config here. */

/* When you are done with retrieving or modifying the plugin data, you

* release the pointers to the data with a call to TSConfigRelease.

*/TSConfigRelease (my_id, config_ptr);

/* Any time you want to modify plugin_config, you must repeat these

* steps, starting with

* my_id = TSConfigSet (my_id,plugin_config, plugin_config_destructor);

* and continuing up to TSConfigRelease.

*/

The configuration functions are:

• TSConfigDataGet()

• TSConfigGet()

• TSConfigRelease()

• TSConfigSet()

Plugin Management

This chapter covers the following topics:

Settings and Statistics

Your plugin might need to know information about Traffic Server’s current configuration and performance. The func-tions described in this section read this information from the Traffic Server records.config file. Configurationsettings are stored in CONFIG variables and statistics are stored in PROCESS variables.

: Not all CONFIG and PROCESS variables in records.config are relevant to Traffic Server’s configurationand statistics. Therefore, retrieve only the records.config variables that are documented in the Administra-tor’s Guide.

To retrieve a variable, you need to know its type (int, counter, float, or string). Plugins store the records.config values as an TSMgmtInt, TSMgmtCounter, TSMgmtFloat, or TSMgmtString. You can look uprecords.config variable types in the Administrator’s Guide.

Depending on the result type, you’ll use TSMgmtIntGet, TSMgmtCounterGet, TSMgmtFloatGet, orTSMgmtStringGet to obtain the variable value (see the example for TSMgmtIntGet().

The TSMgmt*Get functions are:

• TSMgmtCounterGet()

• TSMgmtFloatGet()

• TSMgmtIntGet()

• TSMgmtStringGet()



Logging API

The logging API enables your plugin to log entries in a custom text log file that you create with the callTSTextLogObjectCreate(). This log file is part of Traffic Server’s logging system; by default, it is storedin the logging directory. Once you have created the log object, you can set log properties.

The logging API enables you to:

• Establish a custom text log for your plugin: see TSTextLogObjectCreate()

• Set the log header for your custom text log: see TSTextLogObjectHeaderSet()

• Enable or disable rolling your custom text log: see TSTextLogObjectRollingEnabledSet()

• Set the rolling interval (in seconds) for your custom text log: seeTSTextLogObjectRollingIntervalSecSet()

• Set the rolling offset for your custom text log: see TSTextLogObjectRollingOffsetHrSet()

• Set the rolling size for your custom text log: see TSTextLogObjectRollingSizeMbSet()

• Write text entries to the custom text log: see TSTextLogObjectWrite()

• Flush the contents of the custom text log’s write buffer to disk: see TSTextLogObjectFlush()

• Destroy custom text logs when you are done with them: see TSTextLogObjectDestroy()

The steps below show how the logging API is used in the blacklist-1.c sample plugin. For the complete sourcecode, see the Sample Source Code section.

1. A new log file is defined as a global variable.

static TSTextLogObject log;

2. In TSPluginInit, a new log object is allocated:

TSReturnCode error = TSTextLogObjectCreate("blacklist",TS_LOG_MODE_ADD_TIMESTAMP, &log);

The new log is named blacklist.log. Each entry written to the log will have a timestamp. The NULLargument specifies that the new log does not have a log header. The error argument stores the result of the logcreation; if the log is created successfully, then an error will be equal to TS_LOG_ERROR_NO_ERROR.

3. After creating the log, the plugin makes sure that the log was created successfully:

if (error != TS_SUCCESS) {printf("Blacklist plugin: error %d while creating log\n", error);

}

4. The Blacklist Plugin matches the host portion of the URL (in each client request) with a list of blacklisted sites(stored in the array sites[]):

for (i = 0; i < nsites; i++) {if (strncmp (host, sites[i], host_length) == 0) {

/* ... */}

}

If the host matches one of the blacklisted sites (such as sites[i]), then the plugin writes a blacklist entry toblacklist.log:



if (log) { TSTextLogObjectWrite(log, "blacklisting site: %s",sites[i]);

The format of the log entry is as follows:

blacklisting site: sites[i]

The log is not flushed or destroyed in the blacklist-1 plugin - it lives for the life of the plugin.

Actions

Hosts Lookup API

The hosts lookup enables plugins to ask Traffic Server to do a host lookup of a host name, much like a DNS lookup.

The hosts lookup functions are as follows:

• TSHostLookup()

• TSHostLookupResultAddrGet()

An action is a handle to an operation initiated by a plugin that has not yet completed. For example: when a pluginconnects to a remote server, it uses the call TSNetConnect - which takes TSCont as an argument to call back whenthe connection is established. TSNetConnect might not call the continuation back immediately and will return anTSAction structure that the caller can use to cancel the operation. Cancelling the operation does not necessarilymean that the operation will not occur; it simply means that the continuation passed into the operation will not becalled back. In such an example, the connection might still occur if the action is cancelled; however, the continuationthat initiated the connection would not be called back.

In the preceding example, it is also possible that the connection will complete and call back the continuation beforeTSNetConnect returns. If that occurs, then TSNetConnect returns a special action that causes TSActionDoneto return 1. This specifies that the operation has already completed, so it’s pointless to try to cancel the operation. Alsonote that an action will never change from non-completed to completed. When the operation actually succeeds andthe continuation is called back, the continuation must zero out its action pointer to indicate to itself that the operationsucceeded.

The asynchronous nature of all operations in Traffic Server necessitates actions. You should notice from the abovediscussion that once a call to a function like TSNetConnect is made by a continuation and that function returns avalid action (TSActionDone returns 0), it is not safe for the continuation to do anything else except return fromits handler function. It is not safe to modify or examine the continuation’s data because the continuation may havealready been destroyed.

Below is an example of typical usage for an action:

#include <ts/ts.h>static inthandler (TSCont contp, TSEvent event, void *edata){

if (event == TS_EVENT_IMMEDIATE) {TSAction actionp = TSNetConnect (contp, 127.0.0.1, 9999);if (!TSActionDone (actionp)) {

TSContDataSet (contp, actionp);} else {

/* We've already been called back... */return 0;

}} else if (event == TS_EVENT_NET_CONNECT) {



/* Net connection succeeded */TSContDataSet (contp, NULL);return 0;

} else if (event == TS_EVENT_NET_CONNECT_FAILED) {/* Net connection failed */TSContDataSet (contp, NULL);return 0;

}return 0;

}

voidTSPluginInit (int argc, const char *argv[]){

TSCont contp;

contp = TSContCreate (handler, TSMutexCreate ());

/* We don't want to call things out of TSPluginInitdirectly since it's called before the rest of thesystem is initialized. We'll simply schedule an eventon the continuation to occur as soon as the rest ofthe system is started up. */

TSContSchedule (contp, 0);}

The example above shows a simple plugin that creates a continuation and then schedules it to be called immediately.When the plugin’s handler function is called the first time, the event is TS_EVENT_IMMEDIATE. The plugin thentries to open a net connection to port 9999 on localhost (127.0.0.1). The IP description was left in cider notationto further clarify what is going on; also note that the above won’t actually compile until the IP address is modified.The action returned from TSNetConnect is examined by the plugin. If the operation has not completed, then theplugin stores the action in its continuation. Otherwise, the plugin knows it has already been called back and there isno reason to store the action pointer.

A final question might be, “why would a plugin want to cancel an action?” In the above example, a valid reason couldbe to place a limit on the length of time it takes to open a connection. The plugin could schedule itself to get calledback in 30 seconds and then initiate the net connection. If the timeout expires first, then the plugin would cancel theaction. The following sample code implements this:

#include <ts/ts.h>static inthandler (TSCont contp, TSEvent event, void *edata){

switch (event) {case (TS_EVENT_IMMEDIATE):

TSContSchedule (contp, 30000);TSAction actionp = TSNetConnect(contp, 127.0.0.1, 9999);if (!TSActionDone (actionp)) {

TSContDataSet (contp, actionp);} else {

/* We've already been called back ... */}break;

case (TS_EVENT_TIMEOUT):TSAction actionp = TSContDataGet (contp);if (!TSActionDone(actionp)) {



TSActionCancel (actionp);}break;

case (TS_EVENT_NET_CONNECT):/* Net connection succeeded */TSContDataSet (contp, NULL);break;

case (TS_EVENT_NET_CONNECT_FAILED):/* Net connection failed */TSContDataSet (contp, NULL);break;

}return 0;

}


TSCont contp;

contp = TSContCreate (handler, TSMutexCreate ());/* We don't want to call things out of TSPluginInit

directly since it's called before the rest of thesystem is initialized. We'll simply schedule an eventon the continuation to occur as soon as the rest ofthe system is started up. */

TSContSchedule (contp, 0);}

The action functions are:

• TSActionCancel()

• TSActionDone()

Hooks and Transactions

Hooks are points in Traffic Server transaction processing where plugins can step in and do some work. Registeringa plugin function for callback amounts to “adding” the function to a hook. You can register your plugin to be calledback for every single transaction or only for specific transactions.

This chapter contains the following sections:

Adding Hooks

There are several ways to add hooks to your plugin.

• Global HTTP hooks HTTP transaction hooks are set on a global basis using the function TSHttpHookAdd.This means that the continuation specified as the parameter to TSHttpHookAdd is called for every transaction.TSHttpHookAdd must be used in TSPluginInit.

• Transaction hooks Transaction hooks can be used to call plugins back for a specific HTTP transaction. Youcannot add transaction hooks in TSPluginInit; you first need a handle to a transaction. See Accessing theTransaction Being Processed.



• Transformation hooks Transformation hooks are a special case of transaction hooks. SeeTSVConnCacheObjectSizeGet() for more information about transformation hooks. You add a trans-formation hook using TSHttpTxnHookAdd, as described in HTTP Transactions.

• Session hooks An HTTP session starts when a client opens a connection to Traffic Server and ends when theconnection closes. A session can consist of several transactions. Session hooks enable you to hook your pluginto a particular point in every transaction within a specified session (see HTTP Sessions). Session hooks areadded in a manner similar to transaction hooks (ie, you first need a handle to an HTTP session).

• HTTP select alternate hook Alternate selection hooks enable you to hook on to the alternate selection state.These hooks must be added globally, since Traffic Server does not have a handle to a transaction or sessionwhen alternate selection is taking place. See HTTP Alternate Selection for information on the alternate selectionmechanism.

All of the hook addition functions (TSHttpHookAdd(), TSHttpSsnHookAdd(), TSHttpSsnReenable())take TSHttpHookID (identifies the hook to add on to) and TSCont (the basic callback mechanism in Traffic Server).A single TSCont can be added to any number of hooks at a time.

An HTTP hook is identified by the enumerated type TSHttpHookID. The values for TSHttpHookID are:

Values for TSHttpHookID

TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK Called after the HTTP state machine has completed the cachelookup for the document requested in the ongoing transaction. Register this hook via TSHttpTxnHookAddor TSHttpHookAdd. Corresponds to the event TS_EVENT_HTTP_CACHE_LOOKUP_COMPLETE.

TS_HTTP_OS_DNS_HOOK Called immediately after the HTTP state machine has completed a DNS lookup of theorigin server. The HTTP state machine will know the origin server’s IP address at this point, which is useful forperforming both authentication and blacklisting. Corresponds to the event TS_EVENT_HTTP_OS_DNS.

TS_HTTP_POST_REMAP_HOOK Called immediately after remapping occurs, before cache lookup. Corresponds tothe event TS_EVENT_HTTP_POST_REMAP.

TS_HTTP_PRE_REMAP_HOOK Called after the request header is read from the client, before any remapping of theheaders occurs. Corresponds to the event TS_EVENT_HTTP_PRE_REMAP.

TS_HTTP_READ_CACHE_HDR_HOOK Called immediately after the request and response header of a previously-cached object is read from cache. This hook is only called if the document is being served from cache. Corre-sponds to the event TS_EVENT_HTTP_READ_CACHE_HDR.

TS_HTTP_READ_RESPONSE_HDR_HOOK Called immediately after the response header is read from the originserver or parent proxy. Corresponds to the event TS_EVENT_HTTP_READ_RESPONSE_HDR.

TS_HTTP_RESPONSE_TRANSFORM_HOOK See “Transformations” for information about transformation hooks.

TS_HTTP_READ_REQUEST_HDR_HOOK Called immediately after the request header is read from the client. Cor-responds to the event TS_EVENT_HTTP_READ_REQUEST_HDR.

TS_HTTP_REQUEST_TRANSFORM_HOOK See “Transformations” for information about transformation hooks.

TS_HTTP_SELECT_ALT_HOOK See “HTTP Alternate Selection” for information about the alternate selectionmechanism.

TS_HTTP_SEND_RESPONSE_HDR_HOOK Called immediately before the proxy’s response header is writtento the client; this hook is usually used for modifying the response header. Corresponds to the eventTS_EVENT_HTTP_SEND_RESPONSE_HDR.

TS_HTTP_SEND_REQUEST_HDR_HOOK Called immediately before the proxy’s request header is sent to the originserver or the parent proxy. This hook is not called if the document is being served from cache. This hook isusually used for modifying the proxy’s request header before it is sent to the origin server or parent proxy.

TS_HTTP_SSN_CLOSE_HOOK Called when an HTTP session ends. A session ends when the client connection isclosed. You can only add this hook as a global hook



TS_HTTP_SSN_START_HOOK Called when an HTTP session is started. A session starts when a client connects toTraffic Server. You can only add this hook as a global hook.

TS_HTTP_TXN_CLOSE_HOOK Called when an HTTP transaction ends.

TS_HTTP_TXN_START_HOOK Called when an HTTP transaction is started. A transaction starts when either a clientconnects to Traffic Server and data is available on the connection, or a previous client connection that was leftopen for keep alive has new data available.

The function you use to add a global HTTP hook is TSHttpHookAdd().

HTTP Sessions

An HTTP session is an object that is defined for the lifetime of a client’s TCP session. The Traffic Server API enablesyou to add a global hook to the start or end of an HTTP session, as well as add session hooks that call back your pluginfor every transaction within a given session. When a client connects to Traffic Server, it opens up a TCP connectionand sends one or more HTTP requests. An individual request and its response comprise the HTTP transaction. TheHTTP session begins when the client opens the connection and ends when the connection closes.

The HTTP session hooks are:

• TS_HTTP_SSN_START_HOOKCalled when an HTTP session is started (a session starts when a client connectsto Traffic Server). This hook must be added as a global hook.

• TS_HTTP_SSN_CLOSE_HOOK Called when an HTTP session ends (a session ends when the client connectionis closed). This hook must be added as a global hook.

Use the session hooks to get a handle to a session (an TSHttpSsn object). If you want your plugin to be called backfor each transaction within the session, then use TSHttpSsnHookAdd.

Note: you must reenable the session with TSHttpSsnReenable after processing a session hook.

The session hook functions are listed below:

• TSHttpSsnHookAdd()

• TSHttpSsnReenable()

HTTP Transactions

The HTTP transaction functions enable you to set up plugin callbacks to HTTP transactions and obtain/modify infor-mation about particular HTTP transactions.

As described in the section on HTTP sessions, an HTTP transaction is an object defined for the lifetime of a singlerequest from a client and the corresponding response from Traffic Server. The ‘‘TSHttpTxn‘‘ structure is the mainhandle given to a plugin for manipulating a transaction’s internal state. Additionally, an HTTP transaction has areference back to the HTTP session that created it.

The sample code below illustrates how to register locally to a transaction and associate data to the transaction.

/** Simple plugin that illustrates:

* - how to register locally to a transaction

* - how to deal with data that's associated with a tranaction

** Note: for readability, error checking is omitted

*/

#include <ts/ts.h>



#define DBG_TAG "txn"

/* Structure to be associated to txns */typedef struct {

int i;float f;char *s;

} TxnData;

/* Allocate memory and init a TxnData structure */TxnData *txn_data_alloc(){

TxnData *data;data = TSmalloc(sizeof(TxnData));

data->i = 1;data->f = 0.5;data->s = "Constant String";return data;

}

/* Free up a TxnData structure */voidtxn_data_free(TxnData *data){

TSfree(data);}

/* Handler for event READ_REQUEST and TXN_CLOSE */static intlocal_hook_handler (TSCont contp, TSEvent event, void *edata){

TSHttpTxn txnp = (TSHttpTxn) edata;TxnData *txn_data = TSContDataGet(contp);switch (event) {case TS_EVENT_HTTP_READ_REQUEST_HDR:

/* Modify values of txn data */txn_data->i = 2;txn_data->f = 3.5;txn_data->s = "Constant String 2";break;

case TS_EVENT_HTTP_TXN_CLOSE:/* Print txn data values */TSDebug(DBG_TAG, "Txn data i=%d f=%f s=%s", txn_data->i, txn_data->f, txn_data->

→˓s);

/* Then destroy the txn cont and its data */txn_data_free(txn_data);TSContDestroy(contp);break;

default:TSAssert(!"Unexpected event");break;

}



TSHttpTxnReenable(txnp, TS_EVENT_HTTP_CONTINUE);return 1;

}

/* Handler for event TXN_START */static intglobal_hook_handler (TSCont contp, TSEvent event, void *edata){

TSHttpTxn txnp = (TSHttpTxn) edata;TSCont txn_contp;TxnData *txn_data;

switch (event) {case TS_EVENT_HTTP_TXN_START:

/* Create a new continuation for this txn and associate data to it */txn_contp = TSContCreate(local_hook_handler, TSMutexCreate());txn_data = txn_data_alloc();TSContDataSet(txn_contp, txn_data);

/* Registers locally to hook READ_REQUEST and TXN_CLOSE */TSHttpTxnHookAdd(txnp, TS_HTTP_READ_REQUEST_HDR_HOOK, txn_contp);TSHttpTxnHookAdd(txnp, TS_HTTP_TXN_CLOSE_HOOK, txn_contp);break;

default:TSAssert(!"Unexpected event");break;

}

TSHttpTxnReenable(txnp, TS_EVENT_HTTP_CONTINUE);return 1;

}


TSCont contp;

/* Note that we do not need a mutex for this txn since it registers globallyand doesn't have any data associated with it */

contp = TSContCreate(global_hook_handler, NULL);

/* Register gloabally */TSHttpHookAdd(TS_HTTP_TXN_START_HOOK, contp);

}

See Adding Hooks for background on HTTP transactions and HTTP hooks, as well as Hooks and Transactions. Seealso the HTTP Transaction State Diagram for an illustration of the steps involved in a typical HTTP transaction.

The HTTP transaction functions are:

• TSHttpTxnCacheLookupStatusGet()

• TSHttpTxnCachedReqGet() - Note that it is an error to modify cached headers.

• TSHttpTxnCachedRespGet() - Note that it is an error to modify cached headers.

• TSHttpTxnClientReqGet() - Plugins that must read client request headers use this call to retrieve theHTTP header.



• TSHttpTxnClientRespGet()

• TSHttpTxnErrorBodySet()

• TSHttpTxnHookAdd()

• TSHttpTxnNextHopAddrGet()

• TSHttpTxnParentProxySet()

• TSHttpTxnReenable()

• TSHttpTxnServerAddrGet()

• TSHttpTxnServerAddrSet()

• TSHttpTxnServerReqGet()

• TSHttpTxnServerRespGet()

• TSHttpTxnSsnGet()

• TSHttpTxnTransformedRespCache()

• TSHttpTxnTransformRespGet()

• TSHttpTxnUntransformedRespCache()

Intercepting HTTP Transactions

The intercepting HTTP transaction functions enable plugins to intercept transactions either after the request is receivedor upon contact with the origin server. The plugin then acts as the origin server using the TSVConn interface. Theintercepting HTTP transaction function allow for reading POST bodies in plugins as well as using alternative transportsto the origin server.The intercepting HTTP transaction functions are:

• TSHttpTxnIntercept()

Initiate HTTP Connection

This function enables plugins to initiate HTTP transactions. The initiate HTTP connection function is:

• TSHttpConnect()

HTTP Alternate Selection

The HTTP alternate selection functions provide a mechanism for hooking into Traffic Server’s alternate selectionmechanism and augmenting it with additional information. HTTP alternate selection refers to the process of choos-ing between several alternate versions of a document for a specific URL. Alternates arise because the HTTP 1.1specification allows different documents to be sent back for the same URL (depending on the clients request). Forexample, a server might send back a GIF image to a client that only accepts GIF images, and might send back a JPEGimage to a client that only accepts JPEG images.

The alternate selection mechanism is invoked when Traffic Server looks up a URL in its cache. For each URL, TrafficServer stores a vector of alternates. For each alternate in this vector, Traffic Server computes a quality value between0 and 1 that represents how “good” the alternate is. A quality value of 0 means that the alternate is unacceptable; avalue of 1 means that the alternate is a perfect match.

If a plugin hooks onto the TS_HTTP_SELECT_ALT_HOOK, then it will be called back when Traffic Server per-forms alternate selection. You cannot register locally to the hook TS_HTTP_SELECT_ALT_HOOK by using



TSHttpTxnHookAdd - you can only do so by using only TSHttpHookAdd. Since Traffic Server does not ac-tually have an HTTP transaction or an HTTP session on hand when alternate selection is performed, it is only valid tohook onto the global list of TS_HTTP_SELECT_ALT_HOOK. Traffic Server calls each of the select alternate hookswith the TS_EVENT_HTTP_SELECT_ALT event. The void *edata argument that is passed to the continuationis a pointer to an TSHttpAltInfo structure. It can be used later to call the HTTP alternate selection functions listedat the end of this section. Unlike other hooks, this alternate selection callout is non-blocking; the expectation is thatthe quality value for the alternate will be changed by a call to TSHttpAltInfoQualitySet.

: HTTP SM does not have to be reenabled using TSHttpTxnReenable or any other APIs; just return from thefunction.

The sample code below shows how to call the alternate APIs.

static void handle_select_alt(TSHttpAltInfo infop){

TSMBuffer client_req_buf, cache_resp_buf;TSMLoc client_req_hdr, cache_resp_hdr;

TSMLoc accept_transform_field;TSMLoc content_transform_field;

int accept_transform_len = -1, content_transform_len = -1;const char* accept_transform_value = NULL;const char* content_transform_value = NULL;int content_plugin, accept_plugin;

float quality;

/* get client request, cached request and cached response */TSHttpAltInfoClientReqGet (infop, &client_req_buf, &client_req_hdr);TSHttpAltInfoCachedRespGet(infop, &cache_resp_buf, &cache_resp_hdr);

/* get the Accept-Transform field value from the client request */accept_transform_field = TSMimeHdrFieldFind(client_req_buf,

client_req_hdr, "Accept-Transform", -1);if (accept_transform_field) {

TSMimeHdrFieldValueStringGet(client_req_buf, client_req_hdr,accept_transform_field, 0, &accept_transform_value, &accept_transform_len);

TSDebug(DBG_TAG, "Accept-Transform = |%s|",accept_transform_value);

}

/* get the Content-Transform field value from cached server response */content_transform_field = TSMimeHdrFieldFind(cache_resp_buf,

cache_resp_hdr, "Content-Transform", -1);if (content_transform_field) {

TSMimeHdrFieldValueStringGet(cache_resp_buf, cache_resp_hdr,content_transform_field, 0, &content_transform_value, &content_transform_

→˓len);TSDebug(DBG_TAG, "Content-Transform = |%s|",

content_transform_value);}

/* compute quality */accept_plugin = (accept_transform_value && (accept_transform_len > 0) &&

(strncmp(accept_transform_value, "plugin",accept_transform_len) == 0));



content_plugin = (content_transform_value && (content_transform_len >0) &&(strncmp(content_transform_value, "plugin",

content_transform_len) == 0));

if (accept_plugin) {quality = content_plugin ? 1.0 : 0.0;

} else {quality = content_plugin ? 0.0 : 0.5;

}

TSDebug(DBG_TAG, "Setting quality to %3.1f", quality);

/* set quality for this alternate */TSHttpAltInfoQualitySet(infop, quality);

/* cleanup */if (accept_transform_field)

TSHandleMLocRelease(client_req_buf, client_req_hdr,accept_transform_field);

TSHandleMLocRelease(client_req_buf, TS_NULL_MLOC, client_req_hdr);

if (content_transform_field)TSHandleMLocRelease(cache_resp_buf, cache_resp_hdr,

content_transform_field);TSHandleMLocRelease(cache_resp_buf, TS_NULL_MLOC, cache_resp_hdr);

}

static int alt_plugin(TSCont contp, TSEvent event, void *edata){

TSHttpAltInfo infop;

switch (event) {case TS_EVENT_HTTP_SELECT_ALT:

infop = (TSHttpAltInfo)edata;handle_select_alt(infop);break;

default:break;

}

return 0;}

void TSPluginInit (int argc, const char *argv[]){

TSHttpHookAdd(TS_HTTP_SELECT_ALT_HOOK, TSContCreate (alt_plugin,NULL));

}

Traffic Server augments the alternate selection through these callouts using the following algorithm:

1. Traffic Server computes its own quality value for the alternate, taking into account the quality of the acceptmatch, the encoding match, and the language match.

2. Traffic Server then calls out each of the continuations on the global TS_HTTP_SELECT_ALT_HOOK‘s list.

3. It multiplies its quality value with the value returned by each callout. Since all of the values are clamped to be



between 0 and 1, the final value will be between 0 and 1 as well.

4. This algorithm also ensures that a single callout can block the usage of a given alternate by specifying a qualityvalue of 0.

A common usage for the alternate selection mechanism is when a plugin transforms a document for some clients andnot for others, but wants to store both the transformed and unchanged document. The client’s request will specifywhether it accepted the transformed document. The plugin will then determine if the alternate matches this specifica-tion and then set the appropriate quality level for the alternate.

The HTTP alternate selection functions are:

• TSHttpAltInfoCachedReqGet

• TSHttpAltInfoCachedRespGet

• TSHttpAltInfoClientReqGet

• TSHttpAltInfoQualitySet

Hooks

To understand hooks and transactions, you should be familiar with the following terminology:

HTTP Transaction

A transaction consists of a single HTTP request from a client and the response Traffic Server sends to that client.Thus, a transaction begins when Traffic Server receives a request and ends when Traffic Server sends the response.

Traffic Server uses HTTP state machines to process transactions. The state machines follow a complex set of statesinvolved in sophisticated caching and document retrieval (taking into account, for example, alternate selection, fresh-ness criteria, and hierarchical caching). The Traffic Server API provides hooks to a subset of these states, as illustratedin the HTTP Transaction State Diagram below.

Transform Hooks

The two transform hooks, TS_HTTP_REQUEST_TRANSFORM_HOOK andTS_HTTP_RESPONSE_TRANSFORM_HOOK, are called in the course of an HTTP transform. To see where inthe HTTP transaction they are called, look for the “set up transform” ovals in the HTTP Transaction State Diagrambelow.

HTTP Session

A session consists of a single client connection to Traffic Server; it may consist of a single transaction or severaltransactions in succession. The session starts when the client connection opens and ends when the connection closes.

HTTP Transaction State Diagram

HTTP Transacation Timers

For an overview of HTTP transaction timers, refer to the transaction timer diagram below.


http://people.apache.org/~amc/ats/doc/html/ts_8h.html#af4f3a56716e3e97afd582c7fdb14bcb7

http://people.apache.org/~amc/ats/doc/html/ts_8h.html#aff5861ae4a4a7a6ce7b2d669c113b3bb

http://people.apache.org/~amc/ats/doc/html/ts_8h.html#a74d494c6442b6012d8385e92f0e14dee

http://people.apache.org/~amc/ats/doc/html/ts_8h.html#a978b7160a048491d5698e0f4c0c79aad


HTTP Transaction Timers

Transaction Timers at various states

Traffic Server runs a variety of timers at various states of a transaction. Typically, a given transaction may includeupto two connections (one on the UA/client side and the other on the Origin side). Traffic Server supports two kindsof timers “Active” and “Inactive” timers for each side respectively, as applicable at a given state. The below pictureillustrates the specific timers run at various states in the current implementation. The picture only depicts the HTTPtransaction level timers and does not include the TLS handshake states and other more detailed/internal timers in eachindividual state.

4.13: Transaction Timers in various states

Continuations

Activating Continuations

Continuations are activated when they receive an event or by TSContSchedule (which schedules a continuation toreceive an event). Continuations might receive an event because:

• Your plugin calls TSContCall

• The Traffic Server HTTP state machine sends an event corresponding to a particular HTTP hook

• A Traffic Server IO processor (such as a cache processor or net processor) is letting a continuation know thereis data (cache or network) available to read or write. These callbacks are a result of using functions suchTSVConnRead/Write or TSCacheRead/Write

Writing Handler Functions

The handler function is the key component of a continuation. It is supposed to examine the event and event data,and then do something appropriate. The probable action might be to schedule another event for the continuation toreceived, to open up a connection to a server, or simply to destroy itself.

The continuation’s handler function is a function of type TSEventFunc. Its arguments are a continuation, an event,and a pointer to some data (this data is passed to the continuation by the caller - do not confuse this data with thecontinuation’s own data, associated by TSContDataSet()). When the continuation is called back, the continuationand an event are passed to the handler function. The continuation is a handle to the same continuation that is invoked.The handler function typically has a switch statement to handle the events it receives:

static int some_handler (TScont contp, TSEvent event, void *edata){

// .....switch(event) {

case TS_EVENT_SOME_EVENT_1:do_some_thing_1;return;





default: break;}return 0;

}

: You might notice that a continuation cannot determine if more events are “in flight” toward it. Do not useTSContDestroy() to delete a continuation before you make sure that all incoming events, such as those sentbecause of TSHttpTxnHookAdd(), have been handled.

The following table lists events and the corresponding type of void* data passed to handler functions:

Event Event Sender Data TypeTS_EVENT_HTTP_READ_REQUEST_HDR TS_HTTP_READ_REQUEST_HDR_HOOK TSHttpTxnTS_EVENT_HTTP_PRE_REMAP TS_HTTP_PRE_REMAP_HOOK TSHttpTxnTS_EVENT_HTTP_OS_DNS TS_HTTP_OS_DNS_HOOK TSHttpTxnTS_EVENT_HTTP_SEND_REQUEST_HDR TS_HTTP_SEND_REQUEST_HDR_HOOK TSHttpTxnTS_EVENT_HTTP_READ_CACHE_HDR TS_HTTP_READ_CACHE_HDR_HOOK TSHttpTxnTS_EVENT_HTTP_READ_RESPONSE_HDR TS_HTTP_READ_RESPONSE_HDR_HOOK TSHttpTxnTS_EVENT_HTTP_SEND_RESPONSE_HDR TS_HTTP_SEND_RESPONSE_HDR_HOOK TSHttpTxnTS_EVENT_HTTP_SELECT_ALT TS_HTTP_SELECT_ALT_HOOK TSHttpTxnTS_EVENT_HTTP_TXN_START TS_HTTP_TXN_START_HOOK TSHttpTxnTS_EVENT_HTTP_TXN_CLOSE TS_HTTP_TXN_CLOSE_HOOK TSHttpTxnTS_EVENT_HTTP_SSN_START TS_HTTP_SSN_START_HOOK TSHttpSsnTS_EVENT_HTTP_SSN_CLOSE TS_HTTP_SSN_CLOSE_HOOK TSHttpSsnTS_EVENT_NONETS_EVENT_CACHE_LOOKUP_COMPLETE TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK TSHttpTxnTS_EVENT_IMMEDIATE TSVConnClose() TSVIOReenable() TSContSchedule()TS_EVENT_IMMEDIATE TS_HTTP_REQUEST_TRANSFORM_HOOKTS_EVENT_IMMEDIATE TS_HTTP_RESPONSE_TRANSFORM_HOOKTS_EVENT_CACHE_OPEN_READ TSCacheRead() Cache VCTS_EVENT_CACHE_OPEN_READ_FAILED TSCacheRead() TS_CACHE_ERROR codeTS_EVENT_CACHE_OPEN_WRITE TSCacheWrite() Cache VCTS_EVENT_CACHE_OPEN_WRITE_FAILED TSCacheWrite() TS_CACHE_ERROR codeTS_EVENT_CACHE_REMOVE TSCacheRemove()TS_EVENT_CACHE_REMOVE_FAILED TSCacheRemove() TS_CACHE_ERROR codeTS_EVENT_NET_ACCEPT TSNetAccept() TSHttpTxnServerIntercept() TSHttpTxnIntercept() TSNetVConnectionTS_EVENT_NET_ACCEPT_FAILED TSNetAccept() TSHttpTxnServerIntercept() TSHttpTxnIntercept()TS_EVENT_HOST_LOOKUP TSHostLookup() TSHostLookupResultTS_EVENT_TIMEOUT TSContSchedule()TS_EVENT_ERRORTS_EVENT_VCONN_READ_READY TSVConnRead() TSVIOTS_EVENT_VCONN_WRITE_READY TSVConnWrite() TSVIOTS_EVENT_VCONN_READ_COMPLETE TSVConnRead() TSVIOTS_EVENT_VCONN_WRITE_COMPLETE TSVConnWrite() TSVIOTS_EVENT_VCONN_EOS TSVConnRead() TSVIOTS_EVENT_NET_CONNECT TSNetConnect() TSVConnTS_EVENT_NET_CONNECT_FAILED TSNetConnect() TSVConnTS_EVENT_HTTP_CONTINUETS_EVENT_HTTP_ERRORTS_EVENT_MGMT_UPDATE TSMgmtUpdateRegister()



The continuation functions are listed below:

• TSContCall()

• TSContCreate()

• TSContDataGet()

• TSContDataSet()

• TSContDestroy()

• TSContMutexGet()

• TSContSchedule()

The continuation interface is Traffic Server’s basic callback mechanism. Continuations are instances of the opaquedata type TSCont. In its basic form, a continuation represents a handler function and a mutex.

This chapter covers the following topics:

• Mutexes and Data

• Activating Continuations

• Writing Handler Functions

Mutexes and Data

A continuation must be created with a mutex if your continuation does one of the following:

• is registered globally (TSHttpHookAdd or TSHttpSsnHookAdd) to an HTTP hook and usesTSContDataSet/Get

• is registered locally (TSHttpTxnHookAdd), but for multiple transactions uses TSContDataSet/Get

• uses TSCacheXXX, TSNetXXX, TSHostLookup, or TSContSchedule APIs

Before being activated, a caller must grab the continuation’s mutex. This requirement makes it possible for a continua-tion’s handler function to safely access its data and to prevent multiple callers from running it at the same time (see theAbout the Sample Protocol for usage). The data protected by the mutex is any global or continuation data associatedto the continuation by TSContDataSet. This does not include the local data created by the continuation handlerfunction. A typical example of continuations created with associated data structures and mutexes is the transactionstate machine created in the sample Protocol plugin (see One Way to Implement a Transaction State Machine).

A reentrant call occurs when the continuation passed as an argument to the API can be called in the same stack trace asthe function calling the API. For example, if you call TSCacheRead (contp, mykey), it is possible that contp‘shandler will be called directly and then TSCacheRead returns.

Caveats that could cause issues include the following:

• a continuation has data associated with it (TSContDataGet).

• the reentrant call passes itself as a continuation to the reentrant API. In this case, the continuation should not tryto access its data after calling the reentrant API. The reason for this is that data may be modified by the sectionof code in the continuation’s handler that handles the event sent by the API. It is recommended that you alwaysreturn after a reentrant call to avoid accessing something that has been deallocated.

Below is an example, followed by an explanation.

continuation_handler (TSCont contp, TSEvent event, void *edata) {switch (event) {

case event1:TSReentrantCall (contp);



/* Return right away after this call */break;

case event2:TSContDestroy (contp);break;

}}

The above example first assumes that the continuation is called back with event1; it then does the first reentrantcall that schedules the continuation to receive event2. Because the call is reentrant, the processor calls back thecontinuation right away with event2 and the continuation is destroyed. If you try to access the continuation or oneof its members after the reentrant call, then you might access something that has been deallocated. To avoid accessingsomething that has been deallocated, never access the continuation or any of its members after a reentrant call - justexit the handler.

Note: Most HTTP transaction plugin continuations do not need non-null mutexes because they’re called within theprocessing of an HTTP transaction, and therefore have the transaction’s mutex.

It is also possible to specify a continuation’s mutex as NULL. This should be done only when registering a continuationto a global hook, by a call to TSHttpHookAdd. In this case, the continuation can be called simultaneously bydifferent instances of HTTP SM running on different threads. Having a mutex here would slow and/or hinder TrafficServer performance, since all the threads will try to lock the same mutex. The drawback of not having a mutex is thatsuch a continuation cannot have data associated with it (i.e., TSContDataGet/Set cannot be used).

When using a NULL mutex it is dangerous to access the continuation’s data, but usually continuations with NULLmutexes have no data associated with them anyway. An example of such a continuation is one that gets called backevery time an HTTP request is read, and then determines from the request alone if the request should go through or berejected. An HTTP transaction gives its continuation data to the contp.

Mutexes

A mutex is the basic synchronization method used within Traffic Server to protect data from simultaneous access bymultiple threads. A mutex acts as a lock that protects data in one program thread from being accessed by anotherthread.

The Traffic Server API provides two functions that attempt to access and lock the data: TSMutexLockTry() andTSMutexLock(). TSMutexLock is a blocking call - if you use it, you can slow Traffic Server performancebecause transaction processing pauses until the mutex is unlocked. It should be used only on threads created by theplugin TSContThreadCreate. Never use it on a continuation handler called back by the Cache, Net, or EventProcessor. Even if the critical section is very small, do not use it. If you need to update a flag, then set a variableand/or use atomic operations. If TSMutexLock() is used in any case other than the one recommended above, thenthe result will be a serious performance impact.

TSMutexLockTry, on the other hand, attempts to lock the mutex only if it is unlocked (i.e., not being used by anotherthread). It should be used in all cases other than the above mentioned TSMutexLock case. If the TSMutexLockTryattempt fails, then you can schedule a future attempt (which must be at least 10 milliseconds later).

In general, you should use TSMutexLockTry instead of TSMutexLock.

• TSMutexLockTry is required if you are tying to lock Traffic Server internal or system resources (such as thenetwork, cache, event processor, HTTP state machines, and IO buffers).

• TSMutexLockTry is required if you are making any blocking calls (such as network, cache, or file IO calls).

• TSMutexLock might not be necessary if you are not making blocking calls and if you are only accessing localresources.



The Traffic Server API uses the TSMutex type for a mutex. There are two typical uses of mutex. One use is forlocking global data or data shared by various continuations. The other typical usage is for locking data associated witha continuation (i.e., data that might be accessed by other continuations).

Locking Global Data

The blacklist-1.c sample plugin implements a mutex that locks global data. The blacklist plugin reads its blacklistedsites from a configuration file; file read operations are protected by a mutex created in TSPluginInit(). Theblacklist-1.c code uses TSMutexLockTry() instead of TSMutexLock(). For more detailed information, see theblacklist-1.c code; start by looking at the TSPluginInit() function.

General guidelines for locking shared data are as follows:

1. Create a mutex for the shared data with TSMutexCreate().

2. Whenever you need to read or modify this data, first lock it by calling TSMutexLockTry(); then read ormodify the data.

3. When you are done with the data, unlock it with TSMutexUnlock(). If you are unlocking data accessed dur-ing the processing of an HTTP transaction, then you must unlock it before calling TSHttpTxnReenable().

Protecting a Continuation’s Data

You must create a mutex to protect a continuation’s data if it might be accessed by other continuations or processes.Here’s how:

1. Create a mutex for the continuation using TSMutexCreate.For example:

TSMutex mutexp;mutexp = TSMutexCreate ();

2. When you create the continuation, specify this mutex as the continuation’s mutex.For example:

TSCont contp;contp = TSContCreate (handler, mutexp);

If any other functions want to access contp‘s data, then it is up to them to get contp‘s mutex (using, for example,TSContMutexGet) to lock it. For usage, ssee the sample Protocol plugin.

How to Associate a Continuation With Every HTTP Transaction

There could be several reasons why you’d want to create a continuation for each HTTP transaction that calls back yourplugin.

Some potential scenarios are listed below.

• You want to register hooks locally with the new continuation instead of registering them globally to the contin-uation plugin.

• You want to store data specific to each HTTP transaction that you might need to reuse across various hooks.

• You’re using APIs (like TSHostLookup) that will call back the continuation with a certain event.



How to Add the New Continuation

A typical way of adding the new continuation is by registering the plugin continuation to be called back by HTTPtransactions globally when they reach TS_HTTP_TXN_START_HOOK. Refer to the example below, which uses atransaction-specific continuation called txn_contp.

void TSPluginInit(int argc, const char *argv[]){

/* Plugin continuation */TSCont contp;if ((contp = TSContCreate (plugin_cont_handler, NULL)) == TS_ERROR_PTR) {

LOG_ERROR("TSContCreate");} else {

if (TSHttpHookAdd (TS_HTTP_TXN_START_HOOK, contp) == TS_ERROR) {LOG_ERROR("TSHttpHookAdd");

}}

}

In the plugin continuation handler, create the new continuation txn_contp and then register it to be called back atTS_HTTP_TXN_CLOSE_HOOK:

static int plugin_cont_handler(TSCont contp, TSEvent event, void *edata){

TSHttpTxn txnp = (TSHttpTxn)edata;TSCont txn_contp;


/* Create the HTTP txn continuation */txn_contp = TSContCreate(txn_cont_handler, NULL);

/* Register txn_contp to be called back when txnp reaches TS_HTTP_TXN_→˓CLOSE_HOOK */

if (TSHttpTxnHookAdd (txnp, TS_HTTP_TXN_CLOSE_HOOK, txn_contp) == TS_→˓ERROR) {

LOG_ERROR("TSHttpTxnHookAdd");}

break;

default:TSAssert(!"Unexpected Event");break;

}

if (TSHttpTxnReenable(txnp, TS_EVENT_HTTP_CONTINUE) == TS_ERROR) {LOG_ERROR("TSHttpTxnReenable");

}

return 0;}

Remember that the txn_contp handler must destory itself when the HTTP transaction is closed. If you forget to dothis, then your plugin will have a memory leak.

static int txn_cont_handler(TSCont txn_contp, TSEvent event, void *edata){



TSHttpTxn txnp;switch (event) {

case TS_EVENT_HTTP_TXN_CLOSE:txnp = (TSHttpTxn) edata;TSContDestroy(txn_contp);break;


}


}

return 0;}

How to Store Data Specific to Each HTTP Transaction

For the example above, store the data in the txn_contp data structure - this means that you’ll create your own datastructure. Now suppose you want to store the state of the HTTP transaction:

typedef struct {int state;

} ContData;

You need to allocate the memory and initialize this structure for each HTTP txnp. You can do that in the plugincontinuation handler when it is called back with TS_EVENT_HTTP_TXN_START

static int plugin_cont_handler(TSCont contp, TSEvent event, void *edata){

TSHttpTxn txnp = (TSHttpTxn)edata;TSCont txn_contp;ContData *contData;


/* Create the HTTP txn continuation */txn_contp = TSContCreate(txn_cont_handler, NULL);

/* Allocate and initialize the txn_contp data */contData = (ContData*) TSmalloc(sizeof(ContData));contData->state = 0;if (TSContDataSet(txn_contp, contData) == TS_ERROR) {

LOG_ERROR("TSContDataSet");}

/* Register txn_contp to be called back when txnp reaches TS_HTTP_TXN_→˓CLOSE_HOOK */

if (TSHttpTxnHookAdd (txnp, TS_HTTP_TXN_CLOSE_HOOK, txn_contp) == TS_→˓ERROR) {

LOG_ERROR("TSHttpTxnHookAdd");}



break;


}


}

return 0;}

For accessing this data from anywhere, use TSContDataGet:

TSCont txn_contp;ContData *contData;

contData = TSContDataGet(txn_contp);if (contData == TS_ERROR_PTR) {

LOG_ERROR("TSContDataGet");}contData->state = 1;

Remember to free this memory before destroying the continuation:

static int txn_cont_handler(TSCont txn_contp, TSEvent event, void *edata){

TSHttpTxn txnp;ContData *contData;switch (event) {

case TS_EVENT_HTTP_TXN_CLOSE:txnp = (TSHttpTxn) edata;contData = TSContDataGet(txn_contp);if (contData == TS_ERROR_PTR) {

LOG_ERROR("TSContDataGet");} else {

TSfree(contData);}TSContDestroy(txn_contp);break;


}


}

return 0;}



Using Locks

You do not need to use locks when a continuation has registered itself to be called back by HTTP hooks and it onlyuses the HTTP APIs. In the example above, the continuation txn_contp has registered itself to be called back atHTTP hooks and it only uses the HTTP APIs. In this case only, it’s safe to access data shared between txnp andtxn_contp without grabbing a lock. In the example above, txn_contp is created with a NULL mutex. This worksbecause the HTTP transaction txnp is the only one that will call back txn_contp, and you are guaranteed thattxn_contp will be called back only one hook at a time. After processing is finished, txn_contp will reenabletxnp.

In all other cases, you should create a mutex with the continuation. In general, a lock is needed when you’re usingiocore APIs or any other API where txn_contp is scheduled to be called back by a processor (such as the cacheprocessor, the DNS processor, etc.). This ensures that txn_contp is called back sequentially and not simultaneously.In other words, you need to ensure that txn_contp will not be called back by both txnp and the cache processor atthe same time, since this will result in a situation wherein you’re executing two pieces of code in conflict.

Special Case: Continuations Created for HTTP Transactions

If your plugin creates a new continuation for each HTTP transaction, then you probably don’t need to create a newmutex for it because each HTTP transaction (TSHttpTxn object) already has its own mutex.

In the example below, it’s not necessary to specify a mutex for the continuation created in txn_handler:

static voidtxn_handler (TSHttpTxn txnp, TSCont contp) {

TSCont newCont;....

newCont = TSContCreate (newCont_handler, NULL);//It's not necessary to create a new mutex for newCont.

...

TSHttpTxnReenable (txnp, TS_EVENT_HTTP_CONTINUE);}

static inttest_plugin (TSCont contp, TSEvent event, void *edata) {

TSHttpTxn txnp = (TSHttpTxn) edata;

switch (event) {case TS_EVENT_HTTP_READ_REQUEST_HDR:

txn_handler (txnp, contp);return 0;

default:break;

}return 0;

}

The mutex functions are listed below:

• TSMutexCreate()

• TSMutexLock()

• TSMutexLockTry()



IO

This chapter contains the following sections:

Net Vconnections

A network vconnection (or netvconnection) is a wrapper around a TCP socket that enables the socket to work withinthe Traffic Server vconnection framework. See vconnections for more information about the Traffic Server abstractionfor doing asynchronous IO.

The netvconnection functions are listed below:

• [dox ‘TSNetAccept’] in [dox “TSNetAccept” :src_file]

• [dox %TSNetConnect%] in [dox :src_file]

Transformations

Vconnection Implementor’s View

A VConnection implementor writes only transformations. All other VConnections (net VConnections and cacheVConnections) are implemented in iocore. As mentioned earlier, a given vconnection can have a maximum of oneread operation and one write operation being performed on it. The vconnection user gets information about theoperation being performed by examining the VIO returned by a call to TSVConnRead or TSVConnWrite. Theimplementor, in turn, gets a handle on the VIO operation by examining the VIO returned by TSVConnReadVIOGetor TSVConnWriteVIOGet (recall that every vconnection created through the Traffic Server API has an associatedread VIO and write VIO, even if it only supports reading or writing).

For example, the null transform plugin’s transformation examines the input VIO by calling:

input_vio = TSVConnWriteVIOGet (contp);

where contp is the transformation.

A vconnection is a continuation. This means it has a handler function that is run when an event is sent to it, or moreaccurately, when an event that was sent to it is received. It is the handler function’s job to examine the event, thecurrent state of its read VIO and write VIO, and any other internal state the vconnection might have and potentiallymake some progress on the IO operations.

It is common for the handler function for all vconnections to look similar. Their basic form looks something like thecode fragment below:

intvconnection_handler (TSCont contp, TSEvent event, void *edata){if (TSVConnClosedGet (contp)) {

/* Destroy any vconnection specific data here. */TSContDestroy (contp);return 0;

} else {/* Handle the incoming event */

}}

This code fragment basically shows that many vconnections simply want to destroy themselves when they areclosed. However, the situation might also require the vconnection to do some cleanup processing - which is whyTSVConnClose does not simply just destroy the vconnection.



Vconnections are state machines that are animated by the events they receive. An event is sent to the vconnectionwhenever an TSVConnRead, TSVConnWrite, TSVConnClose, TSVConnShutdown, or TSVIOReenablecall is performed. TSVIOReenable indirectly references the vconnection through a back-pointer in the VIO struc-ture to the vconnection. The vconnection itself only knows which call was performed by examining its state andthe state of its VIOs. For example, when TSVConnClose is called, the vconnection is sent an immediate event(TS_EVENT_IMMEDIATE). For every event the vconnection receives, it needs to check its closed flag to see if it hasbeen closed. Similarly, when TSVIOReenable is called, the vconnection is sent an immediate event. For everyevent the vconnection receives, it must check its VIOs to see if the buffers have been modified to a state in which itcan continue processing one of its operations.

Finally, a vconnection is likely the user of other vconnections. It also receives events as the user of these othervconnections. When it receives such an event, like TS_EVENT_VCONN_WRITE_READY, it might just enable anothervconnection that’s writing into the buffer used by the vconnection reading from it. The above description is merelyintended to give the overall idea for what a vconnection needs to do.

Transformation VConnection

A transformation is a specific type of vconnection. It supports a subset of the vconnection functionality that enablesone or more transformations to be chained together. A transformation sits as a bottleneck between an input data sourceand an output data sink, which enables it to view and modify all the data passing through it. Alternatively, sometransformations simply scan the data and pass it on. A common transformation is one that compresses data in somemanner.

A transformation can modify either the data stream being sent to an HTTP client (e.g. the document) or the data streambeing sent from an HTTP client (e.g. post data). To do this, the transformation should hook on to one of the followinghooks:

• TS_HTTP_REQUEST_TRANSFORM_HOOK

• TS_HTTP_RESPONSE_TRANSFORM_HOOK

Note that because a transformation is intimately associated with a given transaction, it is only possible to add the hookto the transaction hooks - not to the global or session hooks. Transformations reside in a chain, so their ordering isquite easily determined: transformations that add themselves to the chain are simply appended to it.

Data is passed in to the transformation by initiating a vconnection write operation on the transformation. As a conse-quence of this design, a transformation must support the vconnection write operation. In other words, your transforma-tion must expect an upstream vconnection to write data to it. The transformation has to read the data, consume it, andtell the upstream vconnection it is finished by sending it an TS_EVENT_WRITE_COMPLETE event. Transformationscannot send the TS_EVENT_VCONN_WRITE_COMPLETE event to the upstream vconnection unless they are finishedconsuming all incoming data. If TS_EVENT_VCONN_WRITE_COMPLETE is sent prematurely, then certain internalTraffic Server data structures will not be deallocated, thereby causing a memory leak.

Here’s how to make sure that all incoming data is consumed:

• After reading or copying data, make sure that you consume the data and increase the value of ndone for the inputVIO, as in the following example taken from null-transform.c:

TSIOBufferCopy (TSVIOBufferGet (data->output_vio),TSVIOReaderGet (input_vio), towrite, 0);/* Tell the read buffer that we have read the data and are no longer interested→˓in it. */TSIOBufferReaderConsume (TSVIOReaderGet (input_vio), towrite);/* Modify the input VIO to reflect how much has been read.*/TSVIONDoneSet (input_vio, TSVIONDoneGet (input_vio) + towrite);

• Before sending TS_EVENT_VCONN_WRITE_COMPLETE, your transformation should check the number ofbytes remaining in the upstream vconnection’s write VIO (input VIO) using the function TSVIONTodoGet



(input_vio). This value should go to zero when all of the upstream data is consumed (TSVIONTodoGet =nbytes - ndone). Do not send TS_EVENT_VCONN_WRITE_COMPLETE events if TSVIONTodoGet isgreater than zero.

• The transformation passes data out of itself by using the output vconnection retrieved byTSTransformOutputVConnGet. Immediately before Traffic Server initiates the write operation(which inputs data into the transformation), it sets the output vconnection either to the next transformation inthe chain of transformations or to a special terminating transformation (if it’s the last transformation in thechain). Since the transformation is handed ownership of the output vconnection, it must close it at some pointin order for it to be deallocated.

• All of the transformations in a transformation chain share the transaction’s mutex. This small restriction (en-forced by TSTransformCreate) removes many of the locking complications of implementing general vcon-nections. For example, a transformation does not have to grab its write VIO mutex before accessing its writeVIO because it knows it already holds the mutex.

The transformation functions are: * TSTransformCreate() * TSTransformOutputVConnGet()

VIOs

A VIO, or virtual IO, is a description of an IO operation that’s currently in progress. The VIO data structure is usedby vconnection users to determine how much progress has been made on a particular IO operation and to re-enablean IO operation when it stalls due to buffer space issues. VIOs are used by vconnection implementors to determinethe buffer for an IO operation, how much work to do on the IO operation, and which continuation to call back whenprogress on the IO operation is made.

The TSVIO data structure itself is opaque, but it could be defined as follows:

typedef struct {TSCont continuation;TSVConn vconnection;TSIOBufferReader reader;TSMutex mutex;int nbytes;int ndone;

} *TSVIO;

The VIO functions below access and modify various parts of the data structure.

• TSVIOBufferGet()

• TSVIOVConnGet()

• TSVIOContGet()

• TSVIOMutexGet()

• TSVIONBytesGet()

• TSVIONBytesSet()

• TSVIONDoneGet()

• TSVIONDoneSet()

• TSVIONTodoGet()

• TSVIOReaderGet()

• TSVIOReenable()



IO Buffers

The IO buffer data structure is the building block of the vconnection abstraction. An IO buffer (TSIOBuffer) iscomposed of a list of buffer blocks that point to buffer data. Both the buffer block (TSIOBufferBlock) and bufferdata (TSIOBufferData) data structures are reference-counted, so they can reside in multiple buffers at the sametime. This makes it extremely efficient to copy data from one IO buffer to another via TSIOBufferCopy, sinceTraffic Server must only copy pointers and adjust reference counts appropriately (and doesn’t actually copy any data).

The IO buffer abstraction provides for a single writer and multiple readers. In order for the readers to have no knowl-edge of each other, they manipulate IO buffers through the TSIOBufferReader data structure. Since only a singlewriter is allowed, there is no corresponding TSIOBufferWriter data structure. The writer simply modifies the IObuffer directly. To see an example that illustrates how to use IOBuffers, refer to the sample code in the description ofTSIOBufferBlockReadStart().

Additional information about IO buffer functions:

• The TSIOBufferReader data structure tracks how much data in TSIOBuffer has been read. It has anoffset number of bytes that is the current start point of a particular buffer reader (for every read operation on anTSIOBuffer, you must allocate an TSIOBufferReader).

• Bytes that have already been read may not necessarily be freed within the TSIOBuffer. To consume bytesthat have been read, you must call TSIOBufferConsume.

Cache API

The cache API enables plugins to read, write, and remove objects in the Traffic Server cache. All cache APIs arekeyed by an object called an TSCacheKey; cache keys are created via TSCacheKeyCreate; keys are destroyedvia TSCacheKeyDestroy. Use TSCacheKeyDigestSet to set the hash of the cache key.

Note that the cache APIs differentiate between HTTP data and plugin data. The cache APIs do not allow you to writeHTTP docs in the cache; you can only write plugin-specific data (a specific type of data that differs from the HTTPtype).

Example:

const unsigned char *key_name = "example key name";

TSCacheKey key;TSCacheKeyCreate (&key);TSCacheKeyDigestSet (key, (unsigned char *) key_name , strlen(key_name));TSCacheKeyDestroy (key);

Cache Reads

TSCacheRead does not really read - it is used for lookups (see the sample Protocol plugin). Possible callback eventsinclude:

• TS_EVENT_CACHE_OPEN_READ - indicates the lookup was successful. The data passed back along with thisevent is a cache vconnection that can be used to initiate a read on this keyed data.

• TS_EVENT_CACHE_OPEN_READ_FAILED - indicates the lookup was unsuccessful. Reasons for this eventcould be that another continuation is writing to that cache location, or the cache key doesn’t refer to a cachedresource. Data payload for this event indicates the possible reason the read failed (TSCacheError).



Cache Writes

Use TSCacheWrite to write to a cache (see the sample Protocol plugin). Possible callback events include:

• TS_EVENT_CACHE_WRITE_READ - indicates the lookup was successful. The data passed back along withthis event is a cache vconnection that can be used to initiate a cache write.

• TS_EVENT_CACHE_OPEN_WRITE_FAILED - event returned when another continuation is currently writingto this location in the cache. Data payload for this event indicates the possible reason for the write failing(TSCacheError).

Cache Removes

Use TSCacheRemove to remove items from the cache. Possible callback events include:

• TS_EVENT_CACHE_REMOVE - the item was removed. There is no data payload for this event.

• TS_EVENT_CACHE_REMOVE_FAILED - indicates the cache was unable to remove the item identified by thecache key. TSCacheError data indicates why the remove failed.

Errors

Errors pertaining to the failure of various cache operations are indicated by TSCacheError (enumeration). Theyare as follows:

• TS_CACHE_ERROR_NO_DOC - the key does not match a cached resource

• TS_CACHE_ERROR_DOC_BUSY - e.g, another continuation could be writing to the cache location

• TS_CACHE_ERROR_NOT_READY - the cache is not ready

Example

In the example below, suppose there is a cache hit and the cache returns a vconnection that enables you to read thedocument from cache. To do this, you need to prepare a buffer (cache_bufp) to hold the document; meanwhile,use TSVConnCachedObjectSizeGet to find out the actual size of the document (content_length). Then,issue TSVConnRead to read the document with the total data length required as content_length. Assume thefollowing data:

TSIOBuffer cache_bufp = TSIOBufferCreate ();TSIOBufferReader cache_readerp = TSIOBufferReaderAlloc (out_bufp);TSVConn cache_vconnp = NULL;TSVIO cache_vio = NULL;int content_length = 0;

In the TS_CACHE_OPEN_READ handler:

cache_vconnp = (TSVConn) data;TSVConnCachedObjectSizeGet (cache_vconnp, &content_length);cache_vio = TSVConnRead (cache_vconn, contp, cache_bufp, content_length);

In the TS_EVENT_VCONN_READ_READY handler:



(usual VCONN_READ_READY handler logic)int nbytes = TSVIONBytesGet (cache_vio);int ntodo = TSVIONTodoGet (cache_vio);int ndone = TSVIONDoneGet (cache_vio);(consume data in cache_bufp)TSVIOReenable (cache_vio);

Do not try to get continuations or VIOs from TSVConn objects for cache vconnections. Also note that the followingAPIs can only be used on transformation vconnections and must not be used on cache vconnections or net vconnec-tions:

• TSVConnWriteVIOGet

• TSVConnReadVIOGet

• TSVConnClosedGet

APIs such as TSVConnRead, TSVConnWrite, TSVConnClose, TSVConnAbort, and TSVConnShutdowncan be used on any kind of vconnections.

When you are finished:

TSCacheKeyDestroy (key);

Vconnections

A User’s Perspective

To use a vconnection, a user must first get a handle to one. This is usually accomplished by having it handed to theuser; the user may also simply issue a call that creates a vconnection (such as TSNetConnect). In the case oftransform plugins, the plugin creates a transformation vconnection viav TSTransformCreate and then accessesthe output vconnection using TSTransformOutputVConnGet.

After getting a handle to a vconnection, the user can then issue a read or write call. It’s important to note that notall vconnections support both reading and writing - as of yet, there has not been a need to query a vconnection aboutwhether it can perform a read or write operation. That ability should be obvious from context.

To issue a read or write operation, a user calls TSVConnRead or TSVConnWrite. These two operations bothreturn VIO (TSVIO). The VIO describes the operation being performed and how much progress has been made.Transform plugins initiate output to the downstream vconnection by calling TSVConnWrite.

A vconnection read or write operation is different from a normal UNIX read(2) or write(2) operation. Specif-ically, the vconnection operation can specify more data to be read or written than exists in the buffer handed to theoperation. For example, it’s typical to issue a read for INT64_MAX (9 quintillion) bytes from a network vconnectionin order to read all the data from the network connection until the end of stream is reached. This contrasts with theusual UNIX fashion of issuing repeated calls to read(2) until one of the calls finally returns 0 to indicate the endof stream was reached (indeed, the underlying implementation of vconnections on UNIX still does issue those calls toread(2), but the interface does not expose that detail).

At most, a given vconnection can have one read operation and one write operation being performed on it. This isrestricted both by design and common sense: if two write operations were performed on a single vconnection, then theuser would not be able to specify which should occur first and the output would occur in an intermingled fashion. Notethat both a read operation and a write operation can happen on a single vconnection at the same time; the restriction isfor more than one operation of the same type.

One obvious issue is that the buffer passed to TSVConnRead and TSVConnWrite won’t be large enough - thereis no reasonable way to make a buffer that can hold INT64_MAX (9 quintillion) bytes! The secret is that vcon-nections engage in a protocol whereby they signal to the user (via the continuation passed to TSVConnRead and



TSVConnWrite) that they have emptied the buffers passed to them and are ready for more data. When this occurs, itis up to the user to add more data to the buffers (or wait for more data to be added) and then wake up the vconnectionby calling TSVIOReenable on the VIO describing the operation. TSVIOReenable specifies that the buffer forthe operation has been modified and that the vconnection should reexamine it to see if it can make further progress.

The null transform plugin provides an example of how this is done. Below is a prototype for TSVConnWrite:

TSVIO TSVConnWrite (TSVConn connp, TSCont contp, TSIOBufferReader readerp, int nbytes)

The connp is the vconnection the user is writing to and contp is the “user” - i.e., the continuation that connp callsback when it has emptied its buffer and is ready for more data.

The call made in the null transform plugin is:

TSVConnWrite (output_conn, contp, data->output_reader, TSVIONBytesGet (input_vio));

In the example above, contp is the transformation vconnection that is writing to the output vconnection. The numberof bytes to be written is obtained from input_vio by TSVIONBytesGet.

When a vconnection calls back its user to indicate that it wants more data (or when some other condition has occurred),it issues a call to TSContCall. It passes the TSVIO describing the operation as the data parameter, and one of thevalues below as the event parameter.

TS_EVENT_ERROR Indicates an error has occurred on the vconnection. This will happen for network IO if theunderlying read(2) or write(2) call returns an error.

TS_EVENT_VCONN_READ_READY The vconnection has placed data in the buffer passed to an TSVConnReadoperation and it would like to do more IO, but the buffer is now full. When the user consumes the data from thebuffer, this should re-enable the VIO so it indicates to the vconnection that the buffer has been modified.

TS_EVENT_VCONN_WRITE_READY The vconnection has removed data from the buffer passed to anTSVConnWrite operation and it would like to do more IO, but the buffer does not have enough data in it.When placing more data in the buffer, the user should re-enable the VIO so it indicates to the vconnection thatthe buffer has been modified.

TS_EVENT_VCONN_READ_COMPLETE The vconnection has read all the bytes specified by an TSVConnReadoperation. The vconnection can now be used to initiate a new IO operation.

TS_EVENT_VCONN_WRITE_COMPLETE The vconnection has written all the bytes specified by anTSVConnWrite operation and can now be used to initiate a new IO operation.

TS_EVENT_VCONN_EOS An attempt was made to read past the end of the stream of bytes during the handlingof an TSVConnRead operation. This event occurs when the number of bytes available for reading from avconnection is less than the number of bytes the user specifies should be read from the vconnection in a call toTSVConnRead. A common case where this occurs is when the user specifies that INT64_MAX bytes are to beread from a network connection.

For example: the null transform plugin’s transformation receives TS_EVENT_VCONN_WRITE_READY andTS_EVENT_VCONN_WRITE_COMPLETE events from the downstream vconnection as a result of the call toTSVConnWrite.

After using a vconnection, the user must call TSVConnClose or TSVConnAbort. While both calls indicate thatthe vconnection can destroy itself, TSVConnAbort should be used when the connection is being closed abnormally.After a call to TSVConnClose or TSVConnAbort, the user will not be called back by the vconnection again.

Sometimes it’s desirable to simply close down the write portion of a connection while keeping the read portion open.This can be accomplished via the TSVConnShutdown function, which shuts down either the read or write portionof a vconnection. Shutdown means that the vconnection will no longer call back the user with events for the portionof the connection that was shut down. For example: if the user shuts down the write portion of a connection, thenthe TS_EVENT_VCONN_WRITE_READY or TS_EVENT_VCONN_WRITE_COMPLETE events will not be produced.



In the null transform plugin, the write operation is shut down with a call to TSVConnShutdown. To learn howvconnections are used in transformation plugins, see Writing Content Transform Plugins.

The vconnection functions are listed below:

• TSVConnAbort()

• TSVConnClose()

• TSVConnClosedGet()

• TSVConnRead()

• TSVConnReadVIOGet()

• TSVConnShutdown()

• TSVConnWrite()

• TSVConnWriteVIOGet()

HTTP Headers

Traffic Server HTTP Header System

No Null-Terminated Strings

It’s not safe to assume that string data contained in marshal buffers (such as URLs and MIME fields) is stored innull-terminated string copies. Therefore, your plugins should always use the length parameter when retrieving ormanipulating these strings. You cannot pass in NULL for string-length return values; string values returned frommarshall buffers are not null-terminated. If you need a null-terminated value, then use TSstrndup to automaticallynull-terminate a string. The strings that come back and are not null-terminated cannot be passed into the commonstr*() routines

: Values returned from a marshall buffer can be NULL, which means the field or object requested does not exist.

For example (from the blacklist-1 sample)

char *host_string;int host_length;host_string = TSUrlHostGet (bufp, url_loc, &host_length);for (i = 0; i < nsites; i++) {if (strncmp (host_string, sites[i], host_length) == 0) {

// ...}

See the sample plugins for additional examples.

Duplicate MIME Fields Are Not Coalesced

MIME headers can contain more than one MIME field with the same name. Earlier versions of Traffic Server joinedmultiple fields with the same name into one field with composite values. This behavior came at a performance costand caused interoperability problems with older clients and servers. Therefore, this version of Traffic Server does notcoalesce duplicate fields.



Properly-behaving plugins should check for the presence of duplicate fields and then iterate over the duplicate fieldsvia TSMimeHdrFieldNextDup().

MIME Fields Always Belong to an Associated MIME Header

When using Traffic Server, you cannot create a new MIME field without an associated MIME header or HTTP header;MIME fields are always seen as part of a MIME header or HTTP header.

To use a MIME field, you must specify the MIME header or HTTP header to which it belongs - this is called the field’sparent header. The TSMimeField* functions in older versions of the SDK have been deprecated, as they do notrequire the parent header as inputs. The current version of Traffic Server uses new functions, the ‘‘TSMimeHdrField‘‘series, which require you to specify the location of the parent header along with the location of the MIME field. Forevery deprecated ‘‘TSMimeField‘‘ function, there is a new, preferred TSMimeHdrField* function. Therefore,you should use the ‘‘TSMimeHdrField‘‘ functions instead of the deprecated ‘‘TSMimeField‘‘ series. Examples areprovided below.

Instead of:

TSMLoc TSMimeFieldCreate (TSMBuffer bufp)

You should use:

TSMLoc TSMimeHdrFieldCreate (TSMBuffer bufp, TSMLoc hdr)

Instead of:

void TSMimeFieldCopyValues (TSMBuffer dest_bufp, TSMLoc dest_offset,TSMBuffer src_bufp, TSMLoc src_offset)

You should use:

void TSMimeHdrFieldCopyValues (TSMBuffer dest_bufp, TSMLoc dest_hdr,TSMLoc dest_field, TSMBuffer src_bufp, TSMLoc src_hdr, TSMLocsrc_field)

In the TSMimeHdrField* function prototypes, the TSMLoc field corresponds to the TSMLoc offset used the dep-recated TSMimeField* functions (see the discussion of parent TSMLoc in the following section).

Release Marshal Buffer Handles

When you fetch a component object or create a new object, you get back a handle to the object location. The handleis either an TSMLoc for an object location or char * for a string location. You can manipulate the object throughthese handles, but when you are finished you need to release the handle to free up system resources.

The general guideline is to release all TSMLoc and string handles you retrieve. The one exception is the string returnedby TSUrlStringGet, which must be freed by a call to TSfree.

The handle release functions expect three arguments: the marshal buffer containing the data, the location of the parentobject, and the location of the object to be released. The parent location is usually clear from the creation of theTSMLoc or string. For example, if your plugin had the following calls:

url_loc = TSHttpHdrUrlGet (bufp, hdr_loc);host_string = TSUrlHostGet (bufp, url_loc, &host_length);

then your plugin would have to call:



TSHandleMLocRelease (bufp, hdr_loc, url_loc);

If an TSMLoc is obtained from a transaction, then it does not have a parent TSMLoc. Use the null TSMLoc constantTS_NULL_MLOC as its parent. For example, if your plugin calls:

TSHttpTxnClientReqGet (txnp, &bufp, &hdr_loc);

then you must release hdr_loc with:

TSHandleMLocRelease (bufp, TS_NULL_MLOC, hdr_loc);

You need to use TS_NULL_MLOC to release any TSMLoc handles retrieved by the TSHttpTxn*Get functions.

Here’s an example using a new TSMimeHdrField function:

TSHttpTxnServerRespGet( txnp, &resp_bufp, &resp_hdr_loc );new_field_loc = TSMimeHdrFieldCreate (resp_bufp, resp_hdr_loc);TSHandleMLocRelease ( resp_bufp, resp_hdr_loc, new_field_loc);TSHandleMLocRelease ( resp_bufp, TS_NULL_MLOC, resp_hdr_loc);

See the sample plugins for many more examples.

: You should release handles before reenabling the HTTP transaction. In other words, callTSHandleMLocRelease before TSHttpTxnReenable.

Header Functions

The Traffic Server API HTTP header functions enable you to work with HTTP header data stored in marshal buffers.

The HTTP header data structure is a parsed version of the HTTP header defined in the HTTP protocol specification.An HTTP header is composed of a request or response line followed by zero or more MIME fields. In fact, an HTTPheader is a subclass of a MIME header; all of the MIME header routines operate on HTTP headers.

An HTTP request line is composed of a method, a URL, and version. A response line is composed of a version,status code, and reason phrase. See About HTTP Headers for additional details and examples.

To facilitate fast comparisons and reduce storage size, Traffic Server defines several pre-allocated method names.These names correspond to the methods defined in the HTTP 1.1 specification

TS_HTTP_METHOD_CONNECT “CONNECT”

TS_HTTP_METHOD_DELETE “DELETE”

TS_HTTP_METHOD_GE “GET”

TS_HTTP_METHOD_HEAD “HEAD”

TS_HTTP_METHOD_ICP_QUERY “ICP_QUERY”

TS_HTTP_METHOD_OPTIONS “OPTIONS”

TS_HTTP_METHOD_POST “POST”

TS_HTTP_METHOD_PURGE “PURGE”

TS_HTTP_METHOD_PUT “PUT”

TS_HTTP_METHOD_TRACE “TRACE”

TS_HTTP_METHOD_PUSH “PUSH”



Traffic Server also defines several common values that appear in HTTP headers.

TS_HTTP_VALUE_BYTES “bytes”

TS_HTTP_VALUE_CHUNKED “chunked”

TS_HTTP_VALUE_CLOSE “close”

TS_HTTP_VALUE_COMPRESS “compress”

TS_HTTP_VALUE_DEFLATE “deflate”

TS_HTTP_VALUE_GZIP “gzip”

TS_HTTP_VALUE_IDENTITY “identity”

TS_HTTP_VALUE_KEEP_ALIVE “keep-alive”

TS_HTTP_VALUE_MAX_AGE “max-age”

TS_HTTP_VALUE_MAX_STALE “max-stale”

TS_HTTP_VALUE_MIN_FRESH “min-fresh”

TS_HTTP_VALUE_MUST_REVALIDATE “must-revalidate”

TS_HTTP_VALUE_NONE “none”

TS_HTTP_VALUE_NO_CACHE “no-cache”

TS_HTTP_VALUE_NO_STORE “no-store”

TS_HTTP_VALUE_NO_TRANSFORM “no-transform”

TS_HTTP_VALUE_ONLY_IF_CACHED “only-if-cached”

TS_HTTP_VALUE_PRIVATE “private”

TS_HTTP_VALUE_PROXY_REVALIDATE “proxy-revalidate”

TS_HTTP_VALUE_PUBLIC “public”

TS_HTTP_VALUE_S_MAX_AGE “s-maxage”

The method names and header values above are defined in ts.h as const char* strings. When Traffic Server setsa method or a header value, it checks to make sure that the new value is one of the known values. If it is, then it storesa pointer into a global table (instead of storing the known value in the marshal buffer). The method names and headervalues listed above are also pointers into this table. This allows simple pointer comparison of the value returned fromTSHttpMethodGet with one of the values listed above. It is also recommended that you use the above values whenreferring to one of the known schemes, since this removes the possibility of a spelling error.

The HTTP Header Functions are listed below:

• TSHttpHdrClone()

• TSHttpHdrCopy()

• TSHttpHdrCreate()

• TSHttpHdrDestroy()

• TSHttpHdrLengthGet()

• TSHttpHdrMethodGet()

• TSHttpHdrMethodSet()

• TSHttpHdrPrint()

• TSHttpHdrReasonGet()



• TSHttpHdrReasonLookup()

• TSHttpHdrReasonSet()

• TSHttpHdrStatusGet()

• TSHttpHdrStatusSet()

• TSHttpHdrTypeGet()

• TSHttpHdrTypeSet()

• TSHttpHdrUrlGet()

• TSHttpHdrUrlSet()

• TSHttpHdrVersionGet()

• TSHttpHdrVersionSet()

• TSHttpParserClear()

• TSHttpParserCreate()

• TSHttpParserDestroy()

• TSHttpHdrParseReq()

• TSHttpHdrParseResp()

MIME Headers

The Traffic Server **MIME header functions ** enable you to retrieve and modify information about HTTP MIMEfields.

An HTTP request or response consists of a header, body, and trailer. The HTTP header contains a request (orresponse) line and a MIME header. A MIME header is composed of zero or more MIME fields. A MIME field iscomposed of a field name, a colon, and zero or more field values (values in a field are separated by commas).

In the example below: Foo is the MIME field name, bar is the first MIME field value, and car is the second MIMEfield value.

Foo: bar, car

The following example is an augmented Backus-Naur Form (BNF) for the form of a MIME header - it specifiesexactly what was described above. A header consists of zero or more fields that contain a name, separating colon,and zero or more values. A name or value is simply a string of tokens that is potentially zero length; a token is anycharacter except certain control characters and separators (such as colons). For the purpose of retrieving a field, fieldnames are not case-sensitive; therefore, the field names Foo, foo and fOO are all equivalent.

MIME-header = *MIME-fieldMIME-field = field-name ":" #field-valuefield-name = *tokenfield-value = *token

The MIME header data structure is a parsed version of a standard Internet MIME header. The MIME header datastructure is similar to the URL data structure (see URLs). The actual data is stored in a marshal buffer; the MIMEheader functions operate on a marshal buffer and a location (TSMLoc) within the buffer.

After a call to TSMimeHdrFieldDestroy, TSMimeHdrFieldRemove, or TSUrlDestroy is made, you mustdeallocate the TSMLoc handle with a call to TSHandleMLocRelease. You do not need to deallocate a NULLhandles. For example: if you call TSMimeHdrFieldValueStringGet to get the value of the content type field



and the field does not exist, then it returns TS_NULL_MLOC. In such a case, you wouldn’t need to deallocate thehandle with a call to TSHandleMLocRelease.

The location (TSMLoc) in the MIME header functions can be either an HTTP header location or a MIME headerlocation. If an HTTP header location is passed to these functions, then the system locates the MIME header associatedwith that HTTP header and executes the corresponding MIME header operations specified by the functions (see theexample in the description of TSMimeHdrCopy()).

Note: MIME headers may contain more than one MIME field with the same name. Previous versions of TrafficServer joined multiple fields with the same name into one field with composite values, but this behavior came at aperformance cost and caused compatability issues with older clients and servers. Hence, the current version of TrafficServer does not coalesce duplicate fields. Correctly-behaving plugins should check for the presence of duplicate fieldsand iterate over the duplicate fields by using TSMimeHdrFieldNextDup.

To facilitate fast comparisons and reduce storage size, Traffic Server defines several pre-allocated field names. Thesefield names correspond to the field names in HTTP and NNTP headers.

TS_MIME_FIELD_ACCEPT “Accept” TS_MIME_LEN_ACCEPT

TS_MIME_FIELD_ACCEPT_CHARSET “Accept-Charset” TS_MIME_LEN_ACCEPT_CHARSET

TS_MIME_FIELD_ACCEPT_ENCODING “Accept-Encoding” TS_MIME_LEN_ACCEPT_ENCODING

TS_MIME_FIELD_ACCEPT_LANGUAGE “Accept-Language” TS_MIME_LEN_ACCEPT_LANGUAGE

TS_MIME_FIELD_ACCEPT_RANGES “Accept-Ranges” TS_MIME_LEN_ACCEPT_RANGES

TS_MIME_FIELD_AGE “Age” TS_MIME_LEN_AGE

TS_MIME_FIELD_ALLOW “Allow” TS_MIME_LEN_ALLOW

TS_MIME_FIELD_APPROVED “Approved” TS_MIME_LEN_APPROVED

TS_MIME_FIELD_AUTHORIZATION “Authorization” TS_MIME_LEN_AUTHORIZATION

TS_MIME_FIELD_BYTES “Bytes” TS_MIME_LEN_BYTES

TS_MIME_FIELD_CACHE_CONTROL “Cache-Control” TS_MIME_LEN_CACHE_CONTROL

TS_MIME_FIELD_CLIENT_IP “Client-ip” TS_MIME_LEN_CLIENT_IP

TS_MIME_FIELD_CONNECTION “Connection” TS_MIME_LEN_CONNECTION

TS_MIME_FIELD_CONTENT_BASE “Content-Base” TS_MIME_LEN_CONTENT_BASE

TS_MIME_FIELD_CONTENT_ENCODING “Content-Encoding” TS_MIME_LEN_CONTENT_ENCODING

TS_MIME_FIELD_CONTENT_LANGUAGE “Content-Language” TS_MIME_LEN_CONTENT_LANGUAGE

TS_MIME_FIELD_CONTENT_LENGTH “Content-Length” TS_MIME_LEN_CONTENT_LENGTH

TS_MIME_FIELD_CONTENT_LOCATION “Content-Location” TS_MIME_LEN_CONTENT_LOCATION

TS_MIME_FIELD_CONTENT_MD5 “Content-MD5” TS_MIME_LEN_CONTENT_MD5

TS_MIME_FIELD_CONTENT_RANGE “Content-Range” TS_MIME_LEN_CONTENT_RANGE

TS_MIME_FIELD_CONTENT_TYPE “Content-Type” TS_MIME_LEN_CONTENT_TYPE

TS_MIME_FIELD_CONTROL “Control” TS_MIME_LEN_CONTROL

TS_MIME_FIELD_COOKIE “Cookie” TS_MIME_LEN_COOKIE

TS_MIME_FIELD_DATE “Date” TS_MIME_LEN_DATE

TS_MIME_FIELD_DISTRIBUTION “Distribution” TS_MIME_LEN_DISTRIBUTION

TS_MIME_FIELD_ETAG “Etag” TS_MIME_LEN_ETAG



TS_MIME_FIELD_EXPECT “Expect” TS_MIME_LEN_EXPECT

TS_MIME_FIELD_EXPIRES “Expires” TS_MIME_LEN_EXPIRES

TS_MIME_FIELD_FOLLOWUP_TO “Followup-To” TS_MIME_LEN_FOLLOWUP_TO

TS_MIME_FIELD_FROM “From” TS_MIME_LEN_FROM

TS_MIME_FIELD_HOST “Host” TS_MIME_LEN_HOST

TS_MIME_FIELD_IF_MATCH “If-Match” TS_MIME_LEN_IF_MATCH

TS_MIME_FIELD_IF_MODIFIED_SINCE “If-Modified-Since” TS_MIME_LEN_IF_MODIFIED_SINCE

TS_MIME_FIELD_IF_NONE_MATCH “If-None-Match” TS_MIME_LEN_IF_NONE_MATCH

TS_MIME_FIELD_IF_RANGE “If-Range” TS_MIME_LEN_IF_RANGE

TS_MIME_FIELD_IF_UNMODIFIED_SINCE “If-Unmodified-Since” TS_MIME_LEN_IF_UNMODIFIED_SINCE

TS_MIME_FIELD_KEEP_ALIVE “Keep-Alive” TS_MIME_LEN_KEEP_ALIVE

TS_MIME_FIELD_KEYWORDS “Keywords” TS_MIME_LEN_KEYWORDS

TS_MIME_FIELD_LAST_MODIFIED “Last-Modified” TS_MIME_LEN_LAST_MODIFIED

TS_MIME_FIELD_LINES “Lines” TS_MIME_LEN_LINES

TS_MIME_FIELD_LOCATION “Location” TS_MIME_LEN_LOCATION

TS_MIME_FIELD_MAX_FORWARDS “Max-Forwards” TS_MIME_LEN_MAX_FORWARDS

TS_MIME_FIELD_MESSAGE_ID “Message-ID” TS_MIME_LEN_MESSAGE_ID

TS_MIME_FIELD_NEWSGROUPS “Newsgroups” TS_MIME_LEN_NEWSGROUPS

TS_MIME_FIELD_ORGANIZATION “Organization” TS_MIME_LEN_ORGANIZATION

TS_MIME_FIELD_PATH “Path” TS_MIME_LEN_PATH

TS_MIME_FIELD_PRAGMA “Pragma” TS_MIME_LEN_PRAGMA

TS_MIME_FIELD_PROXY_AUTHENTICATE “Proxy-Authenticate” TS_MIME_LEN_PROXY_AUTHENTICATE

TS_MIME_FIELD_PROXY_AUTHORIZATION “Proxy-Authorization” TS_MIME_LEN_PROXY_AUTHORIZATION

TS_MIME_FIELD_PROXY_CONNECTION “Proxy-Connection” TS_MIME_LEN_PROXY_CONNECTION

TS_MIME_FIELD_PUBLIC “Public” TS_MIME_LEN_PUBLIC

TS_MIME_FIELD_RANGE “Range” TS_MIME_LEN_RANGE

TS_MIME_FIELD_REFERENCES “References” TS_MIME_LEN_REFERENCES

TS_MIME_FIELD_REFERER “Referer” TS_MIME_LEN_REFERER

TS_MIME_FIELD_REPLY_TO “Reply-To” TS_MIME_LEN_REPLY_TO

TS_MIME_FIELD_RETRY_AFTER “Retry-After” TS_MIME_LEN_RETRY_AFTER

TS_MIME_FIELD_SENDER “Sender” TS_MIME_LEN_SENDER

TS_MIME_FIELD_SERVER “Server” TS_MIME_LEN_SERVER

TS_MIME_FIELD_SET_COOKIE “Set-Cookie” TS_MIME_LEN_SET_COOKIE

TS_MIME_FIELD_SUBJECT “Subject” TS_MIME_LEN_SUBJECTTS_MIME_LEN_SUBJECT

TS_MIME_FIELD_SUMMARY “Summary” TS_MIME_LEN_SUMMARY

TS_MIME_FIELD_TE “TE” TS_MIME_LEN_TE



TS_MIME_FIELD_TRANSFER_ENCODING “Transfer-Encoding” TS_MIME_LEN_TRANSFER_ENCODING

TS_MIME_FIELD_UPGRADE “Upgrade” TS_MIME_LEN_UPGRADE

TS_MIME_FIELD_USER_AGENT “User-Agent” TS_MIME_LEN_USER_AGENT

TS_MIME_FIELD_VARY “Vary” TS_MIME_LEN_VARY

TS_MIME_FIELD_VIA “Via” TS_MIME_LEN_VIA

TS_MIME_FIELD_WARNING “Warning” TS_MIME_LEN_WARNING

TS_MIME_FIELD_WWW_AUTHENTICATE “Www-Authenticate” TS_MIME_LEN_WWW_AUTHENTICATE

TS_MIME_FIELD_XREF “Xref” TS_MIME_LEN_XREF

The header field names above are defined in ts.h as const char* strings. When Traffic Server sets the nameportion of a header field (or any portion for that matter), it quickly checks to see if the new value is one of the knownvalues. If it is, then Traffic Server stores a pointer into a global table instead of storing the known value in the marshalbuffer. The header field names listed above are also pointers into this table, which enables simple pointer comparisonof the value returned from TSMimeHdrFieldNameGet with one of the values listed above. It is recommended thatyou use the above values when referring to one of the known header field names to avoid the possibility of a spellingerror.

Traffic Server adds one important feature to MIME fields that you may not know about: Traffic Server does not printa MIME field if the field name begins with the ‘@‘ symbol. For example: a plugin can add the field “@My-Field”to a header. Even though Traffic Server never sends that field out in a request to an origin server or in a response toa client, it can be printed to Traffic Server logs by defining a custom log configuration file that explicitly logs suchfields. This provides a useful mechanism for plugins to store information about an object in one of the MIME headersassociated with the object. The MIME header functions are listed below:

• TSMimeHdrFieldAppend()

• TSMimeHdrFieldClone()

• TSMimeHdrFieldCopy()

• TSMimeHdrFieldCopyValues()

• TSMimeHdrFieldCreate()

• TSMimeHdrFieldDestroy()

• TSMimeHdrFieldLengthGet()

• TSMimeHdrFieldNameGet()

• TSMimeHdrFieldNameSet()

• TSMimeHdrFieldNext()

• TSMimeHdrFieldNextDup()

• TSMimeHdrFieldValueAppend()

• TSMimeHdrFieldValueAppend()

• TSMimeHdrFieldValueDateGet()

• TSMimeHdrFieldValueDateInsert()

• TSMimeHdrFieldValueDateSet()

• TSMimeHdrFieldValueIntGet()

• TSMimeHdrFieldValueIntSet()

• TSMimeHdrFieldValueStringGet()



• TSMimeHdrFieldValueStringInsert()

• TSMimeHdrFieldValueStringSet()

• TSMimeHdrFieldValueUintGet()

• TSMimeHdrFieldValueUintInsert()

• TSMimeHdrFieldValueUintSet()

• TSMimeHdrFieldValuesClear()

• TSMimeHdrFieldValuesCount()

• TSMimeHdrClone()

• TSMimeHdrCopy()

• TSMimeHdrCreate()

• TSMimeHdrDestroy()

• TSMimeHdrFieldFind()

• TSMimeHdrFieldGet()

• TSMimeHdrFieldRemove()

• TSMimeHdrFieldsClear()

• TSMimeHdrFieldsCount()

• TSMimeHdrLengthGet()

• TSMimeHdrParse()

• TSMimeParserClear()

• TSMimeParserCreate()

• TSMimeParserDestroy()

• TSMimeHdrPrint()

Marshal Buffers

A marshal buffer, or TSMBuffer, is a heap data structure that stores parsed URLs, MIME headers, and HTTPheaders. You can allocate new objects out of marshal buffers and change the values within a marshal buffer. Wheneveryou manipulate an object, you require the handle to the object (TSMLoc) and the marshal buffer containing the object(TSMBuffer).

Routines exist for manipulating the object based on these two pieces of information. For examples, see one of thefollowing:

• HTTP Headers

• URLs

• MIME Headers

The marshal buffer functions enable you to create and destroy Traffic Server’s marshal buffers, which are the datastructures that hold parsed URLs, MIME headers, and HTTP headers.

: Any marshal buffer fetched by TSHttpTxn*Get will be used by other parts of the system. Be careful not todestroy these shared transaction marshal buffers in functions such as those below:



• TSHttpTxnCachedReqGet

• TSHttpTxnCachedRespGet

• TSHttpTxnClientReqGet

• TSHttpTxnClientRespGet

• TSHttpTxnServerReqGet

• TSHttpTxnServerRespGet

• TSHttpTxnTransformRespGet

URLs

API URL functions provide access to URL data stored in marshal buffers. The URL functions can create, copy, retrieveor delete entire URLs; they can also retrieve or modify parts of URLs, such as port or scheme information.

The general form of an Internet URL is:

scheme://user:password@host:port/stuff

The URL data structure includes support for two specific types of internet URLs. HTTP URLs have the form:

http://user:password@host:port/path;params?query#fragment

The URL port is stored as integer. All remaining parts of the URL (scheme, user, etc.) are stored as strings. TrafficServer URL functions are named according to the portion of the URL on which they operate. For instance, the functionthat retrieves the host portion of a URL is named TSUrlHostGet.

To facilitate fast comparisons and reduce storage size, Traffic Server defines several preallocated scheme names.

TS_URL_SCHEME_FILE “file” TS_URL_LEN_FILE

TS_URL_SCHEME_FTP “ftp” TS_URL_LEN_FTP

TS_URL_SCHEME_GOPHER “gopher” TS_URL_LEN_GOPHER

TS_URL_SCHEME_HTTP “http” TS_URL_LEN_HTTP

TS_URL_SCHEME_HTTPS “https” TS_URL_LEN_HTTPS

TS_URL_SCHEME_MAILTO “mailto” TS_URL_LEN_MAILTO

TS_URL_SCHEME_NEWS “news” TS_URL_LEN_NEWS

TS_URL_SCHEME_NNTP “nntp” TS_URL_LEN_NNTP

TS_URL_SCHEME_PROSPERO “prospero” TS_URL_LEN_PROSPERO

TS_URL_SCHEME_TELNET “telnet” TS_URL_LEN_TELNET

TS_URL_SCHEME_WAIS “wais” TS_URL_LEN_WAIS

The scheme names above are defined in ts.h as const char* strings. When Traffic Server sets the scheme portionof the URL (or any portion for that matter), it quickly checks to see if the new value is one of the known values. If itis, then it stores a pointer into a global table (instead of storing the known value in the marshal buffer). The schemevalues listed above are also pointers into this table. This allows simple pointer comparison of the value returnedfrom TSUrlSchemeGet with one of the values listed above. You should use the Traffic Server-defined values whenreferring to one of the known schemes, since doing so can prevent the possibility of spelling errors.

Traffic Server URL functions are listed below:


http://people.apache.org/~amc/ats/doc/html/InkAPI_8cc.html#a889b626142157077f4f3cfe479e8b8e2

http://people.apache.org/~amc/ats/doc/html/InkAPI_8cc.html#ae8f24b8dabb5008ad11620a11682ffd6

http://people.apache.org/~amc/ats/doc/html/InkAPI_8cc.html#acca66f22d0f87bf8f08478ed926006a5

http://people.apache.org/~amc/ats/doc/html/InkAPI_8cc.html#a92349c8363f72b1f6dfed3ae80901fff

http://people.apache.org/~amc/ats/doc/html/ts_8h.html#aac2343a8b47bf9150f3ff7cd4e692d57

http://people.apache.org/~amc/ats/doc/html/ts_8h.html#a39e8bfb199eadabb54c067ff25a9a400

http://people.apache.org/~amc/ats/doc/html/InkAPI_8cc.html#a20367f5469e8b7e73621c1316091d578


TSUrlClone() TSUrlCopy() TSUrlCreate() TSUrlDestroy() TSUrlPrint()TSUrlFtpTypeGet() TSUrlFtpTypeSet() TSUrlHostGet() TSUrlHostSet()TSUrlHttpFragmentGet() TSUrlHttpFragmentSet() TSUrlHttpParamsGet()TSUrlHttpParamsSet() TSUrlHttpQueryGet() TSUrlHttpQuerySet() TSUrlLengthGet()TSUrlParse() TSUrlPasswordGet() TSUrlPasswordSet() TSUrlPathGet() TSUrlPathSet()TSUrlPortGet() TSUrlPortSet() TSUrlSchemeGet() TSUrlSchemeSet() TSUrlStringGet()TSUrlUserGet() TSUrlUserSet()

An HTTP message consists of the following:

• HTTP header

• body

• trailer

The HTTP header consists of:

• A request or response line

– An HTTP request line contains a method, URL, and version

– A response line contains a version, status code, and reason phrase

• A MIME header

A MIME header is comprised of zero or more MIME fields. A MIME field is composed of a field name, a colon,and (zero or more) field values. The values in a field are separated by commas. An HTTP header containing a requestline is usually referred to as a request. The following example shows a typical request header:

GET http://www.tiggerwigger.com/ HTTP/1.0Proxy-Connection: Keep-AliveUser-Agent: Mozilla/5.0 [en] (X11; I; Linux 2.2.3 i686)Host: www.tiggerwigger.comAccept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */*Accept-Encoding: gzipAccept-Language: enAccept-Charset: iso-8859-1, *, utf-8

The response header for the above request might look like the following:

HTTP/1.0 200 OKDate: Fri, 13 Nov 2009 06:57:43 GMTContent-Location: http://locutus.tiggerwigger.com/index.htmlEtag: "07db14afa76be1:1074"Last-Modified: Thu, 05 Nov 2009 20:01:38 GMTContent-Length: 7931Content-Type: text/htmlServer: Microsoft-IIS/4.0Age: 922Proxy-Connection: close

The following figure illustrates an HTTP message with an expanded HTTP header.

Figure 10.1. HTTP Request/Response and Header Structure

The figure below shows example HTTP request and response headers.

Figure 10.2. Examples of HTTP Request and Response Headers

The marshal buffer or TSMBuffer is a heap data structure that stores parsed URLs, MIME headers, and HTTP head-ers. You can allocate new objects out of marshal buffers and change the values within the marshal buffer. Whenever



4.14: HTTP Request/Response and Header Structure

4.15: Examples of HTTP Request and Response Headers



you manipulate an object, you must require the handle to the object (TSMLoc) and the marshal buffer containing theobject (TSMBuffer).

Figure 10.3. Marshal Buffers and Header Locations

4.16: Marshal Buffers and Header Locations

The figure above shows the following:

• The marshal buffer containing the HTTP request, reqest_bufp

• TSMLoc location pointer for the HTTP header (http_hdr_loc)

• TSMLoc location pointer for the request URL (url_loc)

• TSMLoc location pointers for the MIME header (mime_hdr_loc)

• TSMLoc location pointers for MIME fields (fieldi_loc)

• TSMLoc location pointer for the next duplicate MIME field (next_dup_loc)

The diagram also shows that an HTTP header contains pointers to the URL location and the MIME header location.You can obtain the URL location from an HTTP header using the function TSHttpHdrUrlGet. To work withMIME headers, you can pass either a MIME header location or an HTTP header location to MIME header functions



. If you pass an HTTP header to a MIME header function, then the system locates the associated MIME header andexecutes the MIME header function on the MIME header location.

HTTP Transformations

Transform plugins examine or transform HTTP message body content. For example, transform plugins can:

• Append text to HTML documents

• Compress images

• Do virus checking (on client POST data or server response data)

• Do content-based filtering (filter out HTML documents that contain certain terms or expressions)

This chapter explains how to write transform plugins. The following examples are discussed in detail:

Sample Null Transform Plugin

This section provides a step-by-step description of what the null transform plugin does, along with sections of codethat apply. For context, you can find each code snippet in the complete source code. Some of the error checking detailsare left out - to give the description a step-by-step flow, only the highlights of the transform are included.

Below is an overview of the null transform plugin:

1. Gets a handle to HTTP transactions.

voidTSPluginInit (int argc, const char *argv[]) {

TSHttpHookAdd (TS_HTTP_READ_RESPONSE_HDR_HOOK,TSContCreate (transform_plugin, NULL));

With this TSPluginInit routine, the plugin is called back every time Traffic Server reads a response header.

2. Checks to see if the transaction response is transformable.

static int transform_plugin (TSCont contp, TSEvent event, void *edata) {TSHttpTxn txnp = (TSHttpTxn) edata;switch (event) {

case TS_EVENT_HTTP_READ_RESPONSE_HDR:if (transformable (txnp)) {

transform_add (txnp);}

The default behavior for transformations is to cache the transformed content (you can also tell Traffic Server tocache untransformed content, if you want). Therefore, only responses received directly from an origin serverneed to be transformed. Objects served from cache are already transformed. To determine whether the responseis from the origin server, the routine transformable checks the response header for the “200 OK” serverresponse.

static int transformable (TSHttpTxn txnp){

TSMBuffer bufp;TSMLoc hdr_loc;TSHttpStatus resp_status;TSHttpTxnServerRespGet (txnp, &bufp, &hdr_loc);

if (TS_HTTP_STATUS_OK == (resp_status =



TSHttpHdrStatusGet (bufp, hdr_loc)) ) {return 1;

} else {return 0;

}}

3. If the response is transformable, then the plugin creates a transformation vconnection that gets called back whenthe response data is ready to be transformed (as it is streaming from the origin server).

static void transform_add (TSHttpTxn txnp){

TSVConn connp;connp = TSTransformCreate (null_transform, txnp);TSHttpTxnHookAdd (txnp, TS_HTTP_RESPONSE_TRANSFORM_HOOK, connp);

}

The previous code fragment shows that the handler function for the transformation vconnection isnull_transform.

4. Get a handle to the output vconnection (that receives data from the tranformation).

output_conn = TSTransformOutputVConnGet (contp);

5. Get a handle to the input VIO. (See the handle_transform function.)

input_vio = TSVConnWriteVIOGet (contp);

This is so that the transformation can get information about the upstream vconnection’s write operation to theinput buffer.

6. Initiate a write to the output vconnection of the specified number of bytes. When the write is initiated, the trans-formation expects to receive WRITE_READY, WRITE_COMPLETE, or ERROR events from the output vconnec-tion. See the handle_transform function for the following code fragment:

data->output_vio = TSVConnWrite (output_conn, contp,data->output_reader, TSVIONBytesGet (input_vio));

7. Copy data from the input buffer to the output buffer. See the handle_transform function for the followingcode fragment:

TSIOBufferCopy (TSVIOBufferGet (data->output_vio),TSVIOReaderGet (input_vio), towrite, 0);

8. Tell the input buffer that the transformation has read the data. See the handle_transform function for thefollowing code fragment:

TSIOBufferReaderConsume (TSVIOReaderGet (input_vio), towrite);

9. Modify the input VIO to tell it how much data has been read (increase the value of ndone). See thehandle_transform function for the following code fragment:

TSVIONDoneSet (input_vio, TSVIONDoneGet (input_vio) + towrite);

10. If there is more data left to read ( if ndone < nbytes), then the handle_transform function wakes up thedownstream vconnection with a reenable and wakes up the upstream vconnection by sending it WRITE_READY:



if (TSVIONTodoGet (input_vio) > 0) {if (towrite > 0) {

TSVIOReenable (data->output_vio);

TSContCall (TSVIOContGet (input_vio),TS_EVENT_VCONN_WRITE_READY, input_vio);

}} else {

The process of passing data through the transformation is illustrated in the following diagram. The downstreamvconnections send WRITE_READY events when they need more data; when data is available, the upstreamvconnections reenable the downstream vconnections. In this instance, the TSVIOReenable function sendsTS_EVENT_IMMEDIATE.

Passing Data Through a Transformation {#PassingDataThroughaTransformation}

4.17: Passing Data Through a Transformation

11. If the handle_transform function finds there is no more data to read, then it sets nbytes to ndone onthe output (downstream) VIO and wakes up the output vconnection with a reenable. It then triggers the end ofthe write operation from the upstream vconnection by sending the upstream vconnection a WRITE_COMPLETEevent.

TSVIONBytesSet (data->output_vio, TSVIONDoneGet (input_vio));TSVIOReenable (data->output_vio);TSContCall (TSVIOContGet (input_vio),

TS_EVENT_VCONN_WRITE_COMPLETE, input_vio);}

When the upstream vconnection receives the WRITE_COMPLETE event, it will probably shut down the writeoperation.

12. Similarly, when the downstream vconnection has consumed all of the data, it sends the transformation aWRITE_COMPLETE event. The transformation handles this event with a shut down (the transformation shutsdown the write operation to the downstream vconnection). See the null_plugin function for the followingcode fragment:

case TS_EVENT_VCONN_WRITE_COMPLETE:TSVConnShutdown (TSTransformOutputVConnGet (contp), 0, 1break;



The following diagram illustrates the flow of events:

Ending the Transformation {#EndingTransformation}

4.18: Ending the Transformation

Append-Transform Plugin

The append-transform plugin appends text to the body of an HTTP response. It obtains this text from a file; the nameof the file containing the append text is a parameter you specify in plugin.config, as follows:

append-transform.so path/to/file

The append-transform plugin is based on null-transform.c. The only difference is that after the plugin feedsthe document through the transformation, it adds text to the response.

Below is a list of the functions in append-transform.c, in the order they appear in the source code. Below eachentry is a description of what the function does:

• ‘‘my_data_alloc‘‘

Allocates and initializes a MyData structure. The plugin defines a struct, MyData, as follows:

typedef struct {TSVIO output_vio;TSIOBuffer output_buffer;TSIOBufferReader output_reader;int append_needed;

} MyData;

The MyData structure is used to represent data that the transformation (vconnection) needs. The transforma-tion’s data pointer is set to a MyData pointer using TSContDataSet in the handle_transform routine.

• ‘‘my_data_destroy‘‘

Destroys objects of type MyData. To deallocate the transform’s data, the append_transform routine (seebelow) calls my_data_destroy when the transformation is complete.

• ‘‘handle_transform‘‘



This function does the actual data transformation. The transformation is created in transform_add (seebelow). handle_transform is called by append_transform.

• ‘‘append_transform‘‘

This is the handler function for the transformation vconnection created in transform_add. It is the imple-mentation of the vconnection.

– If the transformation vconnection has been closed, then append_transform callsmy_data_destroy to destroy the vonnection.

– If append_transform receives an error event, then it calls back the continuation to let it know it hascompleted the write operation.

– If it receives a WRITE_COMPLETE event, then it shuts down the write portion of its vconnection.

– If it receives a WRITE_READY or any other event (such as TS_HTTP_RESPONSE_TRANSFORM_HOOK),then it calls handle_transform to attempt to transform more data.

• ‘‘transformable‘‘

The plugin transforms only documents that have a content type of text/html. This function examines theContent-Type MIME header field in the response header. If the value of the MIME field is text/html,then the function returns 1; otherwise, it returns zero.

• ‘‘transform_add‘‘

Creates the transformation for the current transaction and sets up a transformation hook. The handler functionfor the transformation is append_transform.

• ‘‘transform_plugin‘‘

This is the handler function for the main continuation for the plugin. Traffic Server calls this function wheneverit reads an HTTP response header. transform_plugin does the following:

– Gets a handle to the HTTP transaction being processed

– Calls transformable to determine whether the response document content is of type text/html

– If the content is transformable, then it calls transform_add to create the transformation.

– Calls TSHttpTxnReenable to continue the transaction

• ‘‘load‘‘

Opens the file containing the text to be appended and loads the contents of the file into an TSIOBuffer calledappend_buffer.

• ‘‘TSPluginInit‘‘

Does the following:

– Checks to make sure that the required configuration information (the append text filename) is entered inplugin.config correctly.

– If there is a filename, then TSPluginInit calls load to load the text.

– Creates a continuation for the plugin. The handler for this continuation is transform_plugin.

– Adds the plugin’s continuation to TS_HTTP_READ_RESPONSE_HDR_HOOK. In other words, it sets upa callback of the plugin’s continuation when Traffic Server reads HTTP response headers.



Sample Buffered Null Transform Plugin

The buffered null transform, bnull-transform.c, reads the response content into a buffer and then writes thefull buffer out to the client. Many examples of transformations, such as compression, require you to gather the fullresponse content in order to perform the transformation.

The buffered null transform uses a state variable to keep track of when it is (a) reading data into the buffer and (b)writing the data from the buffer to the downstream vconnection.

The following is a step-by-step walk through the buffered null transform:

1. Gets a handle to HTTP transactions.

voidTSPluginInit (int argc, const char *argv[]) {

TSHttpHookAdd (TS_HTTP_READ_RESPONSE_HDR_HOOK,TSContCreate (transform_plugin, NULL)); }

With this TSPluginInit routine, the plugin is called back every time Traffic Server reads a response header.

2. Checks to see if the transaction response is transformable.

static int transform_plugin (TSCont contp, TSEvent event, void *edata) {TSHttpTxn txnp = (TSHttpTxn) edata;switch (event) {

case TS_EVENT_HTTP_READ_RESPONSE_HDR:if (transformable (txnp)) {

transform_add (txnp);}

The default behavior for transformations is to cache the transformed content (if desired, you also can tell TrafficServer to cache untransformed content). Therefore, only responses received directly from an origin server needto be transformed. Objects served from the cache are already transformed. To determine whether the response isfrom the origin server, the routine transformable checks the response header for the “200 OK” server response.

{TSMBuffer bufp;TSMLoc hdr_loc;TSHttpStatus resp_status;

TSHttpTxnServerRespGet (txnp, &bufp, &hdr_loc);

if(TS_HTTP_STATUS_OK==(resp_status=TSHttpHdrStatusGet(bufp,hdr_loc)))

{return 1;

}else {

return 0;}

}

3. If the response is transformable, then the plugin creates a transformation vconnection that gets called back whenthe response data is ready to be transformed (as it is streaming from the origin server).

static void transform_add (TSHttpTxn txnp){

TSVConn connp;connp = TSTransformCreate (bnull_transform, txnp);



TSHttpTxnHookAdd (txnp, TS_HTTP_RESPONSE_TRANSFORM_HOOK, connp);}

The previous code fragment shows that the handler function for the transformation vconnection isbnull_transform.

4. The bnull_transform function has to handle ERROR, WRITE_COMPLETE, WRITE_READY, andIMMEDIATE events. If the transform is just beginning, the event received is probably IMMEDIATE. Thebnull_transform function calls handle_transform to handle WRITE_READY and IMMEDIATE.

5. The handle_transform function examines the data parameter for the continuation passed to it (the contin-uation passed to handle_transform is the transformation vconnection). The data structure keeps track oftwo states: copying the data into the buffer (STATE_BUFFER_DATA) and writing the contents of the buffer tothe output vconnection (STATE_OUTPUT_DATA).

6. Get a handle to the input VIO (see the handle_buffering function). input_vio =TSVConnWriteVIOGet (contp); This is so that the transformation can get information about the up-stream vconnection’s write operation to the input buffer.

7. Copy data from the input buffer to the output buffer. See the handle_buffering function for the followingcode fragment:

TSIOBufferCopy (data->output_buffer,TSVIOReaderGet (write_vio), towrite, 0);

8. Tell the input buffer that the transformation has read the data. See the handle_buffering function for thefollowing code fragment:

TSIOBufferReaderConsume (TSVIOReaderGet (write_vio), towrite);

9. Modify the input VIO to tell it how much data has been read (increase the value of ndone). See thehandle_buffering function for the following code fragment:

TSVIONDoneSet (write_vio, TSVIONDoneGet (write_vio) + towrite); }

10. If there is more data left to read ( if ndone < nbytes), then the handle_buffering function wakes up theupstream vconnection by sending it WRITE_READY:

if (TSVIONTodoGet (write_vio) > 0) {if (towrite > 0) {

TSContCall (TSVIOContGet (write_vio),TS_EVENT_VCONN_WRITE_READY, write_vio);

}} else {

The process of passing data through the transformation is illustrated in the following diagram. The transforma-tion sends WRITE_READY events when it needs more data; when data is available, the upstream vconnectionreenables the transformation with an IMMEDIATE event.

The following diagram illustrates the read from an input vconnection:

Reading Data Into the Buffer (the ‘‘STATE_BUFFER_DATA‘‘ State) {#ReadingDataIntoBuffer}

11. When the data is read into the output buffer, the handle_buffering function sets the state of the trans-formation’s data structure to STATE_OUTPUT_DATA and calls the upstream vconnection back with theWRITE_COMPLETE event.



4.19: Reading Data Into the Buffer the STATE_BUFFER_DATA State

data->state = STATE_OUTPUT_DATA;TSContCall (TSVIOContGet (write_vio),

TS_EVENT_VCONN_WRITE_COMPLETE, write_vio);

12. The upstream vconnection will probably shut down the write operation when it receives the WRITE_COMPLETEevent. The handler function of the transformation, bnull_transform, receives an IMMEDIATEevent and calls the handle_transform function. This time, the state is STATE_OUTPUT_DATA, sohandle_transform calls handle_output.

13. The handle_output function gets a handle to the output vconnection: output_conn =TSTransformOutputVConnGet (contp);

14. The handle_output function writes the buffer to the output vconnection:

data->output_vio =TSVConnWrite (output_conn, contp, data->output_reader,TSIOBufferReaderAvail (data->output_reader) );

The following diagram illustrates the write to the output vconnection:

Writing the Buffered Data to the Output Vconnection {#WritingBufferedtDataIntoVConnection}

Writing Content Transform Plugins

Content transformation plugins transform HTTP response content (such as images or HTML documents) and HTTPrequest content (such as client POST data). Because the data stream to be transformed is of variable length, theseplugins must use a mechanism that passes data from buffer to buffer and checks to see if the end of the data stream isreached. This mechanism is provided by virtual connections (VConnections) and virtual IO descriptors (VIOs).

A VConnection is an abstraction for a data pipe that allows its users to perform asynchronous reads and writeswithout knowing the underlying implementation. A transformation is a specific type of VConnection. A transfor-mation connects an input data source and an output data sink; this feature enables it to view and modify all the datapassing through it.



4.20: Writing the Buffered Data to the Output Vconnection

Transformations can be chained together, one after the other, so that multiple transformations can be performedon the same content. The VConnection type, TSVConn, is actually a subclass of TSCont, which means thatVConnections (and transformations) are continuations. VConnections and transformations can thus exchangeevents, informing one another that data is available for reading or writing, or that the end of a data stream is reached.

A VIO is a description of an IO operation that is in progress. Every VConnection has an associated input VIO andan associated output VIO. When VConnections are transferring data to one another, one VConnection‘s inputVIO is another VConnection‘s output VIO. A VConnection‘s input VIO is also called its write ‘‘VIO‘‘ becausethe input VIO refers to a write operation performed on the VConnection itself. Similarly, the output VIO is alsocalled the read ‘‘VIO‘‘. For transformations, which are designed to pass data in one direction, you can picture therelationship between the transformation VConnection and its VIOs as follows:

4.21: A Transformation and its VIOs

Because the Traffic Server API places transformations directly in the response or request data stream, the transforma-tion VConnection is responsible only for reading the data from the input buffer, transforming it, and then writing itto the output buffer. The upstream VConnectionwrites the incoming data to the transformation’s input buffer. In thefigure above, A Transformation and its VIOs, the input VIO describes the progress of the upstream VConnection‘swrite operation on the transformation, while the output VIO describes the progress of the transformation’s write op-eration on the output (downstream) VConnection. The nbytes value in the VIO is the total number of bytes to bewritten. The ndone value is the current progress, or the number of bytes that have been written at a specific point in



time.

When writing a transformation plugin, you must understand implementation as well as the use of VConnections.The implementor’s side refers to how to implement a VConnection that others can use. At minimum, a transformplugin creates a transformation that sits in the data stream and must be able to handle the events that the upstream anddownstream VConnections send to it. The user’s side refers to how to use a VConnection to read or write data.At the very least, transformations output (write) data.

Transformations

VIOs

A VIO``*or virtual IO is a description of an in progress IO operation. The``VIO data structure is used by VConnection users to determine how much progress has been made on a particu-lar IO operation, and to reenable an IO operation when it stalls due to buffer space. VConnection implementors useVIOs to determine the buffer for an IO operation, how much work to do on the IO operation, and which continuationto call back when progress on the IO operation is made.

The TSVIO data structure itself is opaque, but it might have been defined as follows:

typedef struct {TSCont continuation;TSVConn vconnection;TSIOBufferReader reader;TSMutex mutex;int nbytes;int ndone;

} *TSVIO;

IO Buffers

The IO buffer data structure is the building block of the VConnection abstraction. An IO buffer is composed of alist of buffer blocks which, in turn, point to buffer data. Both the buffer block (TSIOBufferBlock) and buffer data(TSIOBufferData) data structures are reference counted so they can reside in multiple buffers at the same time.This makes it extremely efficient to copy data from one IO buffer to another using TSIOBufferCopy, since TrafficServer only needs to copy pointers and adjust reference counts appropriately (instead of actually copying any data).

The IO buffer abstraction provides for a single writer and multiple readers. In order for the readers to have no knowl-edge of each other, they manipulate IO buffers through theTSIOBufferReader data structure. Since only a singlewriter is allowed, there is no corresponding TSIOBufferWriter data structure. The writer simply modifies the IObuffer directly.

Transaction Data Sink

The hook TS_HTTP_RESPONSE_CLIENT_HOOK is a hook that supports a special type of transformation, one withonly input and no output. Although the transformation doesn’t provide data back to Traffic Server it can do anythingelse with the data, such as writing it to another output device or process. It must, however, consume all the data for thetransaction. There are two primary use cases.

1. Tap in to the transaction to provide the data for external processing.

2. Maintain the transaction.



For the latter it is important to note that if all consumers of a transaction (primarily the user agent) shut down thetransaction is also terminated, including the connection to the origin server. A data sink transform, unlike a standardtransform, is considered to be a consumer and will keep the transaction and the origin server connection up. This isuseful when the transaction is in some way expensive and should run to completion even if the user agent disconnects.Examples would be a standard transform that is expensive to initiate, or expensive origin server connections that shouldbe :ts:cv:‘shared <proxy.config.http.server_session_sharing.match>‘.

There is an example plugin that demonstrates this used as a pure data sink to keep the transaction up regardless ofwhether the user agent disconnects.

New Protocol Plugins

The new protocol APIs enable you to extend Traffic Server to be a web proxy for any protocol. This chapter describesnew protocol APIs and the plugins that support new protocols. It also provides a detailed review of code for a sampleProtocol plugin that supports a very simple artificial HTTP-like protocol.

About the Sample Protocol

The sample protocol enables a client to ask a server for a file. Clients send requests to a specific Traffic Server port(specified in plugin.config); each request has the following structure:

server_name file_name

Using the Protocol plugin, Traffic Server can accept these requests, parse them, and act as a proxy cache (i.e., requestthe file from the origin server on the client’s behalf and store copies of response messages in cache). The Protocolplugin is a state machine that flows through the states illustrated in the Sample Protocol State Diagram. This figureillustrates the steps that Traffic Server and the Protocol plugin go through in order to support the sample protocol.

In more specific terms, Traffic Server and the Protocol plugin must:

• Listen for and accept client connections (on the accept port specified in plugin.config)

• Read incoming client requests

• Look up the requested content in the Traffic Server cache

• Serve content from cache if the request is a cache hit (this simple example does not do freshness checking)

• Open a connection to the origin server if the request is a cache miss (on the server port specified in plugin.config)

• Forward the request to the origin server

• Receive the origin server response

• Cache the response and send it on to the client

Sample Protocol State Diagram

Protocol Plugin Structure

To see how the Protocol plugin works, you need to understand some broader concepts. This section assumes you’refamiliar with the concepts of continuation, Traffic Server’s asynchronous event model, and basic Traffic Serverplugin structure. If you are not familiar with these concepts, then you may want to begin with the Getting Startedsection.


https://github.com/apache/trafficserver/blob/master/example/txn-data-sink/txn-data-sink.c


4.22: Sample Protocol State Diagram



Continuations in the Protocol Plugin

The Protocol plugin creates a static continuation that is an “accept” state machine - that is, a state machine whosejob is to accept client connections on the appropriate port. When Traffic Server accepts a net connection from a clienton that port, the accept state machine is activated. It then creates a new continuation: a transaction state machine.The accept state machine creates one transaction state machine for each transaction (where a transaction consists ofa client request and Traffic Server’s response). Each transaction state machine lives until the transaction completes;then it is destroyed. If the client’s request for content is a cache miss, then a transaction state machine might need toopen a connection to the origin server. This is illustrated in the Protocol Plugin Overview diagram below.

Protocol Plugin Overview

4.23: Protocol Plugin Overview

The first steps for writing the Protocol plugin are now clear: in TSPluginInit, you must create a continuationthat listens for net connections on the client port specified in plugin.config (this continuation is the accept statemachine).

Below is a summary of the continuations implemented for the Protocol plugin:

• An accept state machine that listens for client connections, and then creates transaction state machines when-ever Traffic Server accepts a new client connection. The accept state machine lives as long as Traffic Server isrunning.

• Transaction state machines that read client requests, process them, and are then destroyed when the transactionis finished.

Event Flow

Implementing the rest of the Protocol plugin requires that you understand the flow of events during the course of atransaction. Unlike HTTP transaction plugins, this plugin must read data from network connections and then read/writedata to the Traffic Server cache. This means that its continuations do not receive HTTP state machine events; theyreceive events from Traffic Server’s processor subsystems. For example: the accept state machine is activated byan TS_EVENT_NET_ACCEPT event from Traffic Server’s Net Processor; the handler function for the accept statemachine must therefore be able to handle that event.



The transaction state machines are activated when the client connection receives incoming request data. The NetProcessor notifies the transaction state machine of incoming data. The transaction state machine reads the data; whenfinished, it initiates a cache lookup of the requested file. When the cache lookup completes, the transaction statemachine is activated by the Traffic Server Cache Processor.

If the transaction state machine needs to open a connection to the origin server to fetch content (in the case of a cachemiss), then the transaction state machine initiates a DNS lookup of the server name. The transaction state machine isactivated by a DNS lookup event from the Traffic Server Host Database Processor. If the transaction must connectto the origin server, then the transaction state machine initiates a net connection and waits for an event from the NetProcessor.

Protocol Plugin Flow of Events

4.24: Protocol Plugin Flow of Events

The flow of events is illustrated in the Protocol Plugin Flow of Events diagram above. The thin straight lines show NetProcessor event flow, the thin dashed lines represent Host Database event flow, and the thick dashed lines show Cacheevent flow.

Notice that this flow of events is independent of the Protocol plugin’s design (i.e., whether you build accept ortransaction state machines). Any plugin that supports network connections uses the net vconnection interfaces(TSNetAccept, TSNetConnect) and thus receives events from the Net Processor. Any plugin that performscache lookups or cache writes uses TSCacheRead, TSCacheWrite, TSVConnRead, and TSVConnWrite andthus receives events from the Cache Processor and Traffic Server event system. Similarly, any plugin that does DNSlookups receives events from the Host Database Processor.

One Way to Implement a Transaction State Machine

Transaction state machines (TSMs) in the Protocol plugin must do the following:

• Keep track of the state of the transaction

• Handle events received (based on the state of the transaction and the event received)

• Update the state of the transaction as it changes

Below is one way you can implement TSMs. Details about how the Protocol plugin does this are provided in the nextsection.

• Create a data structure for transactions that contains all of the state data you need to keep track of. In the Protocolplugin this is a struct, Txn_SM.



• When you create the TSM’s continuation, initialize data of type Txn_SM. Initialize the data to the initial state ofa transaction (in this case, a net connection has just been accepted). Associate this data to the TSM continuationusing TSContDataSet.

• Write state handler functions that handle the expected events for each state.

• Write the handler for the TSM. Its job is to receive events, examine the current state, and execute the appropriatestate handler function. In the Protocol plugin, the handler is main_handler. main_handler calls the statehandler functions to handle each state.

The steps below describe the flow of execution illustrated in “How Transaction State Machines are Implemented inthe Protocol Plugin”.

1. The handler for the TSM, (called main_handler in the Protocol plugin) receives events from the TSM.

2. main_handler examines the state of the transaction-in particular, it examines the current handler.

3. main_handler calls the current_handler (which is one of the state handler functions), and thenpasses the current event to current_handler. In the image below below, the current handler is calledstate2_handler.

4. The current_handler handles the event and updates the data. In the image below below, the stateis changed from state2 to state3 (and the current handler is changed from state2_handlerto state3_handler). The next time main_handler receives an event, it will be processed bystate3_handler.

5. state2_handler arranges the next callback of the TSM. Typically, it gives Traffic Server additional workto do (such as writing a file to cache) so that it can progress to the next state. The TSM (main_handler) thenwaits for the next event to arrive from Traffic Server.

How Transaction State Machines are Implemented in the Protocol Plugin

Processing a Typical Transaction

The code is contained in the following files:

• Protocol.c and Protocol.h

• Accept.c and Accept.h

• TxnSM.c and TxnSM.h

Below is a step-by-step walk-through of the code that processes a typical transaction.

1. The TSPluginInit function is in the Protocol.c file. It checks the validity of the plugin.configentries (there must be two: a client accept port and a server port) and runs an initialization routine, init.

2. The init function (in Protocol.c) creates the plugin’s log file using TSTextLogObjectCreate.

3. The init function creates the accept state machine using AcceptCreate. The code for AcceptCreate isin the Accept.c file.

4. The accept state machine, like the transaction state machine, keeps track of its state with a data structure. Thisdata structure, Accept, is defined in the Accept.h file. State data in AcceptCreate is associated with thenew accept state machine via TSContDataSet.

5. The init function arranges the callback of the accept state machine when there is a network connection byusing TSNetAccept.

6. The handler for the accept state machine is accept_event in the Accept.c file. When Traffic Server’s NetProcessor sends TS_EVENT_NET_ACCEPT to the accept state machine, accept_event creates a transactionstate machine (txn_sm) by calling TxnSMCreate. Notice that accept_event creates a mutex for thetransaction state machine, since each transaction state machine has its own mutex.



4.25: How Transaction State Machines are Implemented in the Protocol Plugin



7. The TxnSMCreate function is in the TxnSM.c file. The first thing it does is initialize the transaction’s data,which is of type TxnSM (as defined in TxnSM.h). Notice that the current handler (q_current_handler)is set to state_start.

8. TxnSMCreate then creates a transaction state machine using TSContCreate. The handler for the transac-tion state machine is main_handler, which is in the TxnSM.c file.

9. When accept_event receives TS_EVENT_NET_ACCEPT, it calls the transaction state machine (TSContCall (txn_sm, 0, NULL); ). The event passed to main_handler is 0 (TS_EVENT_NONE).

10. The first thing main_handler does is examine the current txn_sm state by calling TSContDataGet. Thestate is state_start.

11. main_handler then invokes the handler for state_start by using the function pointer TxnSMHandler(as defined in TxnSM.h).

12. The state_start handler function (in the TxnSM.c file) is handed an event (at this stage, the event isTS_EVENT_NET_ACCEPT) and a client vconnection. state_start checks to see if this client vconnec-tion is closed; if it is not, then state_start attempts to read data from the client vconnection into anTSIOBuffer (state_start is handling the event it receives).

13. state_start changes the current handler to state_interface_with_client (that is, it updates thestate of the transaction to the next state).

14. state_start initiates a read of the client vconnection (arranges for Traffic Server to sendTS_EVENT_VCONN_READ_READY events to the TSM) by calling TSVConnRead.

15. state_interface_with_client is activated by the next event from Traffic Server. It checks for errorsand examines the read VIO for the read operation initiated by TSVConnRead.

16. If the read VIO is the client_read_VIO (which we are expecting at this stage in the transaction), thenstate_interface_with_client updates the state to state_read_request_from_client .

17. state_read_request_from_client handles actual TS_EVENT_READ_READY events and reads theclient request.

18. state_read_request_from_client parses the client request.

19. state_read_request_from_client updates the current state to the next state,state_handle_cache_lookup .

20. state_read_request_from_client arranges for Traffic Server to call back the TSM with the next setof events (initiating the cache lookup) by calling TSCacheRead.

21. When the TSCacheRead sends the TSM either TS_EVENT_OPEN_READ (a cachehit) or TS_EVENT_OPEN_READ_FAILED (a cache miss), main_handler callsstate_handle_cache_lookup.

Plugin Interfaces

Most of the functions in the Traffic Server API provide an interface to specific code modules within Traffic Server.The miscellaneous functions described in this chapter provide some useful general capabilities. They are categorizedas follows:

The C library already provides functions such as printf, malloc, and fopen to perform these tasks. The TrafficServer API versions, however, overcome various C library limitations (such as portability to all Traffic Server-supportplatforms).



TSfopen Family

The fopen family of functions in C is normally used for reading configuration files, since fgets is an easy way toparse files on a line-by-line basis. The TSfopen family of functions aims at solving the same problem of bufferedIO and line at a time IO in a platform-independent manner. The fopen family of C library functions can only opena file if a file descriptor less than 256 is available. Since Traffic Server often has more than 2000 file descriptors openat once, however, the likelihood of an available file descriptor less than 256 very small. To solve this problem, theTSfopen family can open files with descriptors greater than 256.

The TSfopen family of routines is not intended for high speed IO or flexibility - they are blocking APIs (not asyn-chronous). For performance reasons, you should not directly use these APIs on a Traffic Server thread (when beingcalled back on an HTTP hook); it is better to use a separate thread for doing the blocking IO. The TSfopen family isintended for reading and writing configuration information when corresponding usage of the fopen family of func-tions is inappropriate due to file descriptor and portability limitations. The TSfopen family of functions consists ofthe following:

• TSfclose()

• TSfflush()

• TSfgets()

• TSfopen()

• TSfread()

• TSfwrite()

Memory Allocation

Traffic Server provides five routines for allocating and freeing memory. These routines correspond to similar routinesin the C library. For example, TSrealloc behaves like the C library routine realloc.

There are two main reasons for using the routines provided by Traffic Server. The first is portability: the Traffic ServerAPI routines behave the same on all of Traffic Server’s supported platforms. For example, realloc does not acceptan argument of NULL on some platforms. The second reason is that the Traffic Server routines actually track thememory allocations by file and line number. This tracking is very efficient, always turned on, and quite useful whentracking down memory leaks.

The memory allocation functions are:

• TSfree()

• TSmalloc()

• TSrealloc()

• TSstrdup()

• TSstrndup()

Thread Functions

The Traffic Server API thread functions enable you to create, destroy, and identify threads within Traffic Server.Multithreading enables a single program to have more than one stream of execution and to process more than onetransaction at a time. Threads serialize their access to shared resources and data using the TSMutex type, as describedin Mutexes.

The thread functions are listed below:



• TSThreadCreate()

• TSThreadDestroy()

• TSThreadInit()

• TSThreadSelf()

Debugging Functions

• TSDebug() prints out a formatted statement if you are running Traffic Server in debug mode.

• TSIsDebugTagSet() checks to see if a debug tag is set. If the debug tag is set, then Traffic Server prints outall debug statements associated with the tag.

• TSError() prints error messages to Traffic Server’s error log

• TSAssert() enables the use of assertion in a plugin.

• TSReleaseAssert() enables the use of assertion in a plugin.

Adding Statistics

This chapter describes how to add statistics to your plugins. Statistics can be coupled or uncoupled. Coupled statisticsare quantities that are related and must therefore be updated together. The Traffic Server API statistics functions addyour plugin’s statistics to the Traffic Server statistics system. You can view your plugin statistics as you would anyother Traffic Server statistic, using Traffic Line (Traffic Server’s command line interface). This chapter contains thefollowing topics:

Uncoupled Statistics

A statistic is an object of type TSStat. The value of the statistic is of type TSStatType. The possibleTSStatTypes are:

• TSSTAT_TYPE_INT64

• TSSTAT_TYPE_FLOAT

There is no TSSTAT_TYPE_INT32.

To add uncoupled statistics, follow the steps below:

1. Declare your statistic as a global variable in your plugin. For example:

static TSStat my_statistic;

2. In TSPluginInit, create new statistics using TSStatCreate. When you create a new statistic, you needto give it an “external” name that the Traffic Server command line interface (Traffic Line) uses to access thestatistic. For example:

my_statistic = TSStatCreate ("my.statistic", TSSTAT_TYPE_INT64);

3. Modify (increment, decrement, or other modification) your statistic in plugin functions.



Coupled Statistics

Use coupled statistics for quantities that are related and therefore must be updated jointly.

As a very simple example, suppose you have three statistics: sum, part_1, and part_2. They must always preservethe relationship that sum = part_1 + part_2. If you update part_1 without updating sum at the same time,then the equation becomes untrue. Therefore, the statistics are said to be coupled.

The mechanism for updating coupled statistics jointly is to create local copies of global coupled statistics inthe routines that modifiy them. When each local copy is updated appropriately, do a global update usingTSStatsCoupledUpdate. To specify which statistics are related to one another, establish a coupled statisticcategory and make sure that each coupled statistic belongs to the appropriate category. When it is time to do the globalupdate, specify the category to be updated.

: The local statistic copy must have a duplicate set of statistics as that of the master copy. Local statistics must alsobe added to the local statistic category in the same order as their master copy counterparts were originally added.

Below are the steps you need to follow, along with a code example taken from the redirect-1.c sample plugin.

To add coupled statistics

1. Declare the global category for your coupled statistics as a global TSCoupledStat variable in your plugin.

2. Declare your coupled statistics as global TSStat variables in your plugin.

3. In TSPluginInit, create a new global coupled category using TSStatCoupledGlobalCategoryCreate.

4. In TSPluginInit, create new global coupled statistics using TSStatCoupledGlobalAdd. When youcreate a new statistic, you need to give it an “external” name that the Traffic Server command line interface(Traffic Line) uses to access the statistic.

5. In any routine wherein you want to modify (increment, decrement, or other modification) your coupled statistics,declare local copies of the coupled category and coupled statistics.

6. Create local copies using TSStatCoupledLocalCopyCreate and TSStatCoupledLocalAdd.

7. Modify the local copies of your statistics. Then call TSStatsCoupledUpdate to update the global copiesjointly.

8. When you are finished, you must destroy all of the local copies in the category viaTSStatCoupledLocalCopyDestroy.

Example Using the redirect-1.c Sample Plugin

static TSCoupledStat request_outcomes;

static TSStat requests_all;static TSStat requests_redirects;static TSStat requests_unchanged;

request_outcomes = TSStatCoupledGlobalCategoryCreate ("request_outcomes");

requests_all = TSStatCoupledGlobalAdd (request_outcomes, "requests.all", TSSTAT_TYPE_→˓FLOAT);requests_redirects = TSStatCoupledGlobalAdd (request_outcomes, "requests.redirects",

TSSTAT_TYPE_INT64);



requests_unchanged = TSStatCoupledGlobalAdd (request_outcomes, "requests.unchanged",TSSTAT_TYPE_INT64);

TSCoupledStat local_request_outcomes;TSStat local_requests_all;TSStat local_requests_redirects;TSStat local_requests_unchanged;

local_request_outcomes = TSStatCoupledLocalCopyCreate("local_request_outcomes",request_outcomes);

local_requests_all = TSStatCoupledLocalAdd(local_request_outcomes, "requests.all.local→˓",

TSSTAT_TYPE_FLOAT);local_requests_redirects = TSStatCoupledLocalAdd(local_request_outcomes,

"requests.redirects.local", TSSTAT_TYPE_INT64);local_requests_unchanged = TSStatCoupledLocalAdd(local_request_outcomes,

"requests.unchanged.local", TSSTAT_TYPE_INT64);

TSStatFloatAddTo( local_requests_all, 1.0 ) ;...TSStatIncrement (local_requests_unchanged);TSStatsCoupledUpdate(local_request_outcomes);

TSStatCoupledLocalCopyDestroy(local_request_outcomes);

Viewing Statistics Using Traffic Line

To view statistics for your plugin, follow the steps below:

1. Make sure you know the name of your statistic (i.e., the name used in the TSStatCoupledGlobalAdd,TSStatCreate, or TSStatCoupledGlobalCategoryCreate call).

2. In your <Traffic Server>/bin directory, enter the following:

./traffic\_line -r the\_name

Example Plugins

Basic Authorization Plugin

The sample basic authorization plugin, basic-auth.c, checks for basic HTTP proxy authorization. In HTTP basicproxy authorization, client user names and passwords are contained in the Proxy-Authorization header. Thepassword is encoded using base64 encoding. The plugin checks all incoming requests for the authorization header,user name, and password. If the plugin does not find all of the these, then it reenables with an error (effectivelystopping the transaction) and adds a transaction hook to the send response header event.

Creating the Plugin’s Parent Continuation and Global Hook

The parent continuation and global hook are created as follows:

TSHttpHookAdd (TS_HTTP_OS_DNS_HOOK, TSContCreate (auth_plugin, NULL));



: Authorization plugins which attach to TS_HTTP_OS_DNS_HOOK (as shown in the example above) will not operateas expected unless :ts:cv:‘proxy.config.http.doc_in_cache_skip_dns‘ is set to 0. Disabling this feature ensures thatDNS hooks will still be executed even when a matching document has been located in the cache. The downside is thatthe performance gain by skipping otherwise unnecessary DNS lookups is lost.

Implementing the Handler and Getting a Handle to the Transaction

The handler function for the plugin’s parent continuation is implemented as follows:

static intauth_plugin (TSCont contp, TSEvent event, void *edata){

TSHttpTxn txnp = (TSHttpTxn) edata;switch (event) {case TS_EVENT_HTTP_OS_DNS:

handle_dns (txnp, contp);return 0;

case TS_EVENT_HTTP_SEND_RESPONSE_HDR:handle_response (txnp);return 0;

default:break;

}

return 0;}

Working With HTTP Headers

The plugin checks all client request headers for the Proxy-Authorization MIME field, which should containthe user name and password. The plugin’s continuation handler, auth-plugin, calls handle_dns tocheck the Proxy-Authorization field. The handle_dns routine uses TSHttpTxnClientReqGet andTSMimeHdrFieldFind to obtain the Proxy-Authorization field:

{TSMBuffer bufp;TSMLoc hdr_loc;TSMLoc field_loc;const char *val;char *user, *password;

if (!TSHttpTxnClientReqGet (txnp, &bufp, &hdr_loc)) {TSError ("[basic_authorization] Couldn't retrieve client request header");goto done;

}

field_loc = TSMimeHdrFieldFind (bufp, hdr_loc,TS_MIME_FIELD_PROXY_AUTHORIZATION);

If the Proxy-Authorization field is present, then the plugin checks that the authentication type is “Basic”, andthe user name and password are present and valid:



val = TSMimeHdrFieldValueStringGet (bufp, hdr_loc, field_loc, -1, &authval_length);if (!val) {

TSError ("[basic_authorization] No value in Proxy-Authorization field");TSHandleMLocRelease (bufp, hdr_loc, field_loc);TSHandleMLocRelease (bufp, TS_NULL_MLOC, hdr_loc);goto done;

}

if (strncmp (val, "Basic", 5) != 0) {TSError ("[basic_authorization] No Basic auth type in Proxy-Authorization");TSHandleMLocRelease (bufp, hdr_loc, field_loc);TSHandleMLocRelease (bufp, TS_NULL_MLOC, hdr_loc);goto done;

}

val += 5;while ((*val == ' ') || (*val == '\t')) {

val += 1;}

user = base64_decode (val);password = strchr (user, ':');if (!password) {

TSError ("[basic_authorization] No password in authorization information");TSfree (user);TSHandleMLocRelease (bufp, hdr_loc, field_loc);TSHandleMLocRelease (bufp, TS_NULL_MLOC, hdr_loc);goto done;

}

*password = '\0';password += 1;

if (!authorized (user, password)) {TSError ("[basic_authorization] %s:%s not authorized", user, password);TSfree (user);TSHandleMLocRelease (bufp, hdr_loc, field_loc);TSHandleMLocRelease (bufp, TS_NULL_MLOC, hdr_loc);goto done;

}

TSfree (user);TSHandleMLocRelease (bufp, hdr_loc, field_loc);TSHandleMLocRelease (bufp, TS_NULL_MLOC, hdr_loc);TSHttpTxnReenable (txnp, TS_EVENT_HTTP_CONTINUE);return;

Setting a Transaction Hook

If the request does not have the Proxy-Authorization field set to Basic authorization or a valid user-name/password, then the plugin sends the 407 Proxy authorization required status code back to the client. Theclient will then prompt the user for a username and password, and then resend the request.

In the handle_dns routine, the following lines handle the authorization error case:

done:TSHttpTxnHookAdd (txnp, TS_HTTP_SEND_RESPONSE_HDR_HOOK, contp);



TSHttpTxnReenable (txnp, TS_EVENT_HTTP_ERROR);

If handle_dns does not find the Proxy-Authorization field set to Basic authorization or a valid user-name/password, then it adds a SEND_RESPONSE_HDR_HOOK to the transaction being processed. This means thatTraffic Server will call the plugin back when sending the client response. handle_dns reenables the transactionwith TS_EVENT_HTTP_ERROR, which means that the plugin wants Traffic Server to terminate the transaction.

When Traffic Server terminates the transaction, it sends the client an error message. Because of theSEND_RESPONSE_HDR_HOOK, Traffic Server calls the plugin back. The auth-plugin routine callshandle_response to send the client a 407 status code. When the client resends the request with theProxy-Authorization field, a new transaction begins.

handle_dns calls base64_decode to decode the username and password; handle_dns also callsauthorized to validate the username and password. In this plugin, sample NT code is provided for passwordvalidation. UNIX programmers can supply their own validation mechanism.

Blacklist Plugin

The sample blacklisting plugin included in the Traffic Server SDK is blacklist-1.c. This plugin checks everyincoming HTTP client request against a list of blacklisted web sites. If the client requests a blacklisted site, then theplugin returns an Access forbidden message to the client.

The flow of HTTP processing with the blacklist plugin is illustrated in the figure titled Blacklist Plugin. This ex-ample also contains a simple configuration management interface. It can read a list of blacklisted sites from a file(blacklist.txt) that can be updated by a Traffic Server administrator. When the configuration file is updated,Traffic Server sends an event to the plugin that wakes it up to do some work.

Creating the Parent Continuation

You create the static parent continuation in the mandatory TSPluginInit function. This parent continuation ef-fectively is the plugin: the plugin executes only when this continuation receives an event from Traffic Server. TrafficServer passes the event as an argument to the continuation’s handler function. When you create continuations, youmust create and specify their handler functions.

You can specify an optional mutex lock when you create continuations. The mutex lock protects data shared byasynchronous processes. Because Traffic Server has a multi-threaded design, race conditions can occur if severalthreads try to access the same continuation’s data.

Here is how the static parent continuation is created in blacklist-1.c:


// ...TSCont contp;

contp = TSContCreate (blacklist_plugin, NULL);// ...

}

The handler function for the plugin is blacklist_plugin, and the mutex is null. The continuation handler func-tion’s job is to handle the events that are sent to it; accordingly, the blacklist_plugin routine consists of a switchstatement that covers each of the events that might be sent to it:



static intblacklist_plugin (TSCont contp, TSEvent event, void *edata){

TSHttpTxn txnp = (TSHttpTxn) edata;switch (event) {

case TS_EVENT_HTTP_OS_DNS:handle_dns (txnp, contp);return 0;


default:TSDebug ("blacklist_plugin", "This event was unexpected: %d\n", );break;

}return 0;

}

When you write handler functions, you have to anticipate any events that might be sent to the handler by hooks orby other functions. In the Blacklist plugin, TS_EVENT_OS_DNS is sent because of the global hook establishedin TSPluginInit, TS_EVENT_HTTP_SEND_RESPONSE_HDR is sent because the plugin contains a transactionhook (see Setting Up a Transaction Hook). It is good practice to have a default case in your switch statements.

Setting a Global Hook

Global hooks are always added in TSPluginInit using TSHttpHookAdd. The two arguments ofTSHttpHookAdd are the hook ID and the continuation to call when processing the event corresponding to thehook. In blacklist-1.c, the global hook is added as follows:

TSHttpHookAdd (TS_HTTP_OS_DNS_HOOK, contp);

Above, TS_HTTP_OS_DNS_HOOK is the ID for the origin server DNS lookup hook and contp is the parent contin-uation created earlier.

This means that the Blacklist plugin is called at every origin server DNS lookup. When it is called, the handlerfunctio blacklist_plugin receives TS_EVENT_HTTP_OS_DNS and calls handle_dns to see if the requestis forbidden.

Accessing the Transaction Being Processed

A continuation’s handler function is of type TSEventFunc; the prototype is as follows:

static int function_name (TSCont contp, TSEvent event, void *edata)

In general, the return value of the handler function is not used. The continuation argument is the continuation beingcalled back, the event is the event being sent to the continuation, and the data pointed to by void *edata dependson the type of event. The data types for each event type are listed in Writing Handler Functions

The key here is that if the event is an HTTP transaction event, then the data passed to the continuation’s handler is oftype TSHttpTxn (a data type that represents HTTP transactions). Your plugin can then do things with the transaction.Here’s how it looks in the code for the Blacklist plugin’s handler:

static intblacklist_plugin (TSCont contp, TSEvent event, void *edata){



TSHttpTxn txnp = (TSHttpTxn) edata;switch (event) {

case TS_EVENT_HTTP_OS_DNS:handle_dns (txnp, contp);return 0;


default:break;

}return 0;

}

For example: when the origin server DNS lookup event is sent, blacklist_plugin can call handle_dnsandpass txnp as an argument.

Setting Up a Transaction Hook

The Blacklist plugin sends “access forbidden” messages to clients if their requests are directed to blacklisted hosts.Therefore, the plugin needs a transaction hook so it will be called back when Traffic Server’s HTTP state machinereaches the “send response header” event. In the Blacklist plugin’s handle_dns routine, the transaction hook isadded as follows:

TSMutexLock (sites_mutex);for (i = 0; i < nsites; i++) {

if (strncmp (host, sites[i], host_length) == 0) {printf ("blacklisting site: %s\n", sites[i]);TSHttpTxnHookAdd (txnp,

TS_HTTP_SEND_RESPONSE_HDR_HOOK,contp);

TSHandleMLocRelease (bufp, hdr_loc, url_loc);TSHandleMLocRelease (bufp, TS_NULL_MLOC, hdr_loc);TSHttpTxnReenable (txnp, TS_EVENT_HTTP_ERROR);TSMutexUnlock (sites_mutex);return;

}}TSMutexUnlock (sites_mutex);done:TSHttpTxnReenable (txnp, TS_EVENT_HTTP_CONTINUE);

This code fragment shows some interesting features. The plugin is comparing the requested site to the list of blacklistedsites. While the plugin is using the blacklist, it must acquire the mutex lock for the blacklist to prevent configurationchanges in the middle of a blacklisting operation. If the requested site is blacklisted, then the following things happen:

1. A transaction hook is added with TSHttpTxnHookAdd; the plugin is called back at the “send response header”event (i.e., the plugin sends an Access forbidden message to the client). You can see that in order to add atransaction hook, you need a handle to the transaction being processed.

2. The transaction is reenabled using TSHttpTxnReenable with TS_EVENT_HTTP_ERROR as its event argu-ment. Reenabling with an error event tells the HTTP state machine to stop the transaction and jump to the “sendresponse header” state. Notice that if the requested site is not blacklisted, then the transaction is reenabled withthe TS_EVENT_HTTP_CONTINUE event.

3. The string and TSMLoc data stored in the marshal buffer bufp is released by TSHandleMLocRelease (seeMarshal Buffers). Release these handles before re-enabling the transaction.



In general, whenever the plugin is doing something to a transaction, it must reenable the transaction when it is finished.In other words: every time your handler function handles a transaction event, it must call TSHttpTxnReenablewhen it is finished. Similarly, after your plugin handles session events (TS_EVENT_HTTP_SSN_START andTS_EVENT_HTTP_SSN_CLOSE), it must reenable the session with TSHttpSsnReenable. Reenabling the trans-action twice in the same plugin routine is a bad error.

Working with HTTP Header Functions

The Blacklist plugin examines the host header in every client transaction. This is done in the handle_dns routine,using TSHttpTxnClientReqGet, TSHttpHdrUrlGet, and TSUrlHostGet.

static voidhandle_dns (TSHttpTxn txnp, TSCont contp){TSMBuffer bufp;TSMLoc hdr_loc;TSMLoc url_loc;const char *host;int i;int host_length;

if (TSHttpTxnClientReqGet(txnp, &bufp, &hdr_loc) != TS_SUCCESS) {TSError("[blacklist] Couldn't retrieve client request header");goto done;

}

if (TSHttpHdrUrlGet(bufp, hdr_loc, &url_loc) != TS_SUCCESS) {TSError("[blacklist] Couldn't retrieve request url");TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);goto done;

}

host = TSUrlHostGet(bufp, url_loc, &host_length);if (!host) {

TSError("[blacklist] couldn't retrieve request hostname");TSHandleMLocRelease(bufp, hdr_loc, url_loc);TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);goto done;

}

To access the host header, the plugin must first get the client request, retrieve the URL portion, and then obtain thehost header. See HTTP Headers for more information about these calls. See Marshal Buffers for guidelines on usingTSHandleMLocRelease.

Sample Source Code

blacklist-1.c

The sample blacklisting plugin included in the Traffic Server SDK is blacklist-1.c. This plugin checks everyincoming HTTP client request against a list of blacklisted web sites. If the client requests a blacklisted site, then theplugin returns an Access forbidden message to the client.

This plugin illustrates:

• An HTTP transaction extension



• How to examine HTTP request headers

• How to use the logging interface

• How to use the plugin configuration management interface

Query Remap Plugin

Example: Query Remap Plugin

The sample remap plugin, query_remap.c, maps client requests to a number of servers based on a hash of therequest’s URL query parameter. This can be useful for spreading load for a given type of request among backendservers, while still maintaining “stickiness” to a single server for similar requests. For example, a search engine maywant to send repeated queries for the same keywords to a server that has likely cached the result from a prior query.

Configuration of query_remap

The query remap plugin will allow the query parameter name to be specified, along with the hostnames of the serversto hash across. Sample remap.config rules using query_remap will look like:

map http://www.example.com/search http://srch1.example.com/search @plugin=query_remap.→˓so @pparam=q @pparam=srch1.example.com @pparam=srch2.example.com @pparam=srch3.→˓example.commap http://www.example.com/profiles http://prof1.example.com/profiles @plugin=query_→˓remap.so @pparam=user_id @pparam=prof1.example.com @pparam=prof2.example.com

The first @pparam specifies the query param key for which the value will be hashed. The remaining parameters list thehostnames of the servers. A request for http://www.example.com/search?q=apache will match the firstrule. The plugin will look for the ‘‘q‘‘ parameter and hash the value ‘apache‘ to pick from among srch_[1-3]_.example.com to send the request.

If the request does not include a ‘‘q‘‘ query parameter and the plugin decides not to modify the request, the defaulttoURL ‘http://srch1.example.com/search‘ will be used by TS.

The parameters are passed to the plugin’s tsremap_new_instance function. In query_remap,tsremap_new_instance creates a plugin-defined query_remap_info struct to store its configuration pa-rameters. The ihandle, an opaque pointer that can be used to pass per-instance data, is set to this struct pointer and willbe passed to the tsremap_remap function when it is triggered for a request.

typedef struct _query_remap_info {char *param_name;size_t param_len;char **hosts;int num_hosts;

} query_remap_info;

int tsremap_new_instance(int argc,char *argv[],ihandle *ih,char *errbuf,int errbuf_→˓size){

int i;

if (argc param_name = strdup(argv[2]);qri->param_len = strlen(qri->param_name);qri->num_hosts = argc - 3;qri->hosts = (char**) TSmalloc(qri->num_hosts*sizeof(char*));



for (i=0; i num_hosts; ++i) {qri->hosts[i] = strdup(argv[i+3]);

}

*ih = (ihandle)qri;return 0;

}

Another way remap plugins may want handle more complex configuration is to specify a configuration filename as apparam and parse the specified file during instance initialization.

Performing the Remap

The plugin implements the tsremap_remap function, which is called when TS has read the client HTTP requestheaders and matched the request to a remap rule configured for the plugin. The TSRemapRequestInfo structcontains input and output members for the remap operation.

tsremap_remap uses the configuration information passed via the ihandle and checks the request_queryfor the configured query parameter. If the parameter is found, the plugin sets a new_host to modify the request host:

int tsremap_remap(ihandle ih, rhandle rh, TSRemapRequestInfo *rri){

int hostidx = -1;query_remap_info *qri = (query_remap_info*)ih;

if (!qri) {TSError("[remap] NULL ihandle");return 0;

}

if (rri && rri->request_query && rri->request_query_size > 0) {char *q, *s, *key;

//make a copy of the query, as it is read onlyq = (char*) TSmalloc(rri->request_query_size+1);strncpy(q, rri->request_query, rri->request_query_size);q[rri->request_query_size] = '\0';

s = q;//parse query parametersfor (key = strsep(&s, "&"); key != NULL; key = strsep(&s, "&")) {

char *val = strchr(key, '=');if (val && (size_t)(val-key) == qri->param_len &&

!strncmp(key, qri->param_name, qri->param_len)) {++val;//the param key matched the configured param_name//hash the param value to pick a hosthostidx = hash_fnv32(val, strlen(val)) % (uint32_t)qri->num_hosts;break;

}}

TSfree(q);

if (hostidx >= 0) {rri->new_host_size = strlen(qri->hosts[hostidx]);



if (rri->new_host_size new_host, qri->hosts[hostidx], rri->new_host_size);return 1; //host has been modified

}}

}

//the request was not modified, TS will use the toURL from the remap rulereturn 0;

}

The Remap plugin provides a more flexible, dynamic way of specifying remap rules. It is not built on top of the TrafficServer APIs and exists solely for the purpose of URL remapping. The remap plugin is not global –it is configuredon a per-remap rule basis, which enables you to customize how URLs are redirected based on individual rules in theremap.config file.

The Traffic Server Remap API enables a plugin to dynamically map a client request to a target URL. Each pluginis associated with one or more remap rules in remap.config (an “instance”). If a request URL matches a remaprule’s “fromURL”, then Traffic Server calls the plugin-defined remap function for that request.

((Editor’s note: additional text TBD; text in this chapter is still under development))

Remap Header File

The remap.h header file contains the Traffic Server remap API. By default, the header file location is: /usr/local/include/ts/remap.h

Required Functions

A remap plugin is required to implement the following functions:

• TSRemapInit: the remap initialization function, called once when the plugin is loaded

• TSRemapNewInstance: a new instance is created for each rule associated with the plugin. Called each time theplugin used in a remap rule (this function is what processes the pparam values)

• TSRemapDoRemap: the entry point used by Traffic Server to find the new URL to which it remaps; called everytime a request comes in

Configuration

To associate a remap plugin with a remap rule, use the @plugin parameter. See the Admin Guide section (?TBD?)for details on configuring remap plugins

Header-Based Plugin Examples

Header-based plugins read or modify the headers of HTTP messages that Traffic Server sends and receives. Readingthis chapter will help you to understand the following topics:

• Creating continuations for your plugins

• Adding global hooks

• Adding transaction hooks

• Working with HTTP header functions


http://people.apache.org/~amc/ats/doc/html/remap_8h.html#af7e9b1eee1c38c6f8dcc67a65ba02c24

http://people.apache.org/~amc/ats/doc/html/remap_8h.html#a963de3eeed2ed7a2da483acf77dc42ca

http://people.apache.org/~amc/ats/doc/html/remap_8h.html#acf73f0355c591e145398211b3c0596fe


The two sample plugins discussed in this chapter are blacklist-1.c and basic-auth.c.

Overview

Header-based plugins take actions based on the contents of HTTP request or response headers. Examples includefiltering (on the basis of requested URL, source IP address, or other request header), user authentication, or userredirection. Header-based plugins have the following common elements:

• The plugin has a static parent continuation that scans all Traffic Server headers (either request headers, responseheaders, or both).

• The plugin has a global hook. This enables the plugin to check all transactions to determine if the plugin needsto do something.

• The plugin gets a handle to the transaction being processed through the global hook.

• If the plugin needs to do something to transactions in specific cases, then it sets up a transaction hook for aparticular event.

• The plugin obtains client header information and does something based on that information.

This chapter demonstrates how these components are implemented in SDK sample code.

Configuration Variable Implementation

Adding a new configuration variable in records.config requires a number of steps which are mostly documentedhere.

Before adding a new configuration variable, please discuss it on the mailing list. It will commonly be the case that abetter name, or a more general approach to the problem which solves several different issues, may be suggested.

Defining the Variable

To begin, the new configuration variables must be added to RecordsConfig.cc. This contains a long array ofconfiguration variable records. The fields for each record are:

type:RecT Type of record. The valid values are:

RECT_NULL Undefined record.

RECT_CONFIG General configuration variable.

RECT_PROCESS Process related statistic.

RECT_NODE Local statistic.

RECT_CLUSTER Cluster statistic.

RECT_LOCAL Configuration variable that is explicitly not shared across a cluster.

RECT_PLUGIN Plugin created statistic.

In general, RECT_CONFIG should be used unless it is required that the value not be shared among members ofa cluster, in which case RECT_LOCAL should be used. If you use RECT_LOCAL, you must also start the linewith LOCAL instead of CONFIG and the name should use .local. instead of .config..

name:char const* The fully qualified name of the configuration variable. Although there appears to be a hierar-chial naming scheme, that’s just a convention, and it is not actually used by the code. Nonetheless, new variablesshould adhere to the hierarchial scheme.


https://github.com/apache/trafficserver/blob/master/mgmt/RecordsConfig.cc


value_type:RecDataT The data type of the value. It should be one of RECD_INT, RECD_STRING, RECD_FLOATas appropriate.

default:char const* The default value for the variable. This is always a string regardless of the value_type.

update:RecUpdateT Information about how the variable is updated. The valid values are:

RECU_NULL Behavior is unknown or unspecified.

RECU_DYNAMIC This can be updated via command line tools.

RECD_RESTART_TS The traffic_server process must be restarted for a new value to take effect.

RECD_RESTART_TM The traffic_manager process must be restarted for a new value to take ef-fect.

RECD_RESTART_TC The traffic_cop process must be restarted for a new value to take effect.

required:RecordRequiredType Effectively a boolean that specifies if the record is required to be present,with RR_NULL meaning not required and RR_REQUIRED indicating that it is required. Given that usingRR_REQUIRED would be a major incompatibility, RR_NULL is generally the better choice.

check:RecCheckT Additional type checking. It is unclear if this is actually implemented. The valid values are:

RECC_NULL No additional checking.

RECC_STR Verify the value is a string.

RECC_INT Verify the value is an integer.

RECC_IP Verify the value is an IP address. Unknown if this checks for IPv6.

pattern:char const* This provides a regular expressions (PCRE format) for validating the value, beyond thebasic type validation performed by RecCheckT. This can be NULL if there is no regular expression to use.

access:RecAccessT Access control. The valid values are:

RECA_NULL The value is read / write.

RECA_READ_ONLY The value is read only.

RECA_NO_ACCESS No access to the value; only privileged level parts of ATS can access the value.

Variable Infrastructure

The primary effort in defining a configuration variable is handling updates, generally via traffic_ctl configreload. This is handled in a generic way, as described in the next section, or in a more specialized way (built on topof the generic mechanism) for HTTP related configuration variables. This is only needed if the variable is marked asdynamically updateable (RECU_DYNAMIC) although HTTP configuration variables should be dynamic if possible.

Documentation and Defaults

A configuration variable should be documented in records.config. There are many examples in the file alreadythat can be used for guidance. The general format is to use the tag

.. ts:cv:`variable.name.here`

The arguments to this are the same as for the configuration file. The documentation generator will pick out key bitsand use them to decorate the entry. In particular if a value is present it will be removed and used as the default value.You can attach some additional options to the variable. These are:

reloadable The variable can be reloaded via command line on a running Traffic Server.

4.8. Configuration Variable Implementation 377


metric Specify the units for the value. This is critical for variables that use unexpected or non-obvious metrics, suchas minutes instead of seconds, or disk sectors instead of bytes.

deprecated Mark a variable as deprecated.

Example

:ts:cv:‘custom.variable‘

reloadable

metric minutes

deprecated

If you need to refer to another configuration variable in the documentation, you can use the form

:ts:cv:`the.full.name.of.the.variable`

This will display the name as a link to the full definition.

In general, a new configuration variable should not be present in the default records.config. If it is added, suchdefaults should be added to the file proxy/config/records.config.default.in. This is used to generatethe default records.config. Just add the variable to the file in an appropriate place with a proper default as thiswill now override whatever default you put in the code for new installs.

Handling Updates

The simplest mechanism for handling updates is the REC_EstablishStaticConfigXXX family of functions.This mechanism will cause the value in the indicated instance to be updated in place when an update to records.config occurs. This is done asynchronously using atomic operations. Use of these variables must keep that inmind.

If a variable requires additional handling when updated a callback can be registered which is called when the variableis updated. This is what the REC_EstablishStaticConfigXXX calls do internally with a callback that simplyreads the new value and writes it to storage indicated by the call parameters. The functions used are the link_XXXstatic functions in RecCore.cc.

To register a configuration variable callback, call RecRegisterConfigUpdateCb with the arguments:

char const* name The variable name.

callback A function with the signature <int (char const* name, RecDataT type, RecData data,void* cookie)>. The :arg:‘name‘ value passed is the same as the :arg:‘name‘ passed to the registrationfunction as is the :arg:‘cookie‘ argument. The :arg:‘type‘ and :arg:‘data‘ are the new value for the variable.The return value is currently ignored. For future compatibility return REC_ERR_OKAY.

void* cookie A value passed to the callback. This is only for the callback, the internals simply store it and pass iton.

callback is called under lock so it should be quick and not block. If that is necessary a continuation should be scheduledto handle the required action.

: The callback occurs asynchronously. For HTTP variables as described in the next section, this is handled by themore specialized HTTP update mechanisms. Otherwise it is the implementor’s responsibility to avoid race conditions.


https://github.com/apache/trafficserver/blob/master/lib/records/RecCore.cc


HTTP Configuation Values

Variables used for HTTP processing should be declared as members of the HTTPConfigParams structure (but seeOverridable Variables for further details) and use the specialized HTTP update mechanisms which handle synchro-nization and initialization issues.

The configuration logic maintains two copies of the HTTPConfigParams structure, the master copy and the currentcopy. The master copy is kept in the m_master member of the HttpConfig singleton. The current copy is kept inthe ConfigProcessor. The goal is to provide a (somewhat) atomic update for configuration variables which are loadedindividually in to the master copy as updates are received and then bulk copied to a new instance which is then swappedin as the current copy. The HTTP state machine interacts with this mechanism to avoid race conditions.

For each variable, a mapping between the variable name and the appropriate member in the master copy shouldbe established between in the HTTPConfig::startup method. The HttpEstablishStaticConfigXXXfunctions should be used unless there is a strong, explicit reason to not do so.

The HTTPConfig::reconfigure method handles the current copy of the HTTP configuration variables. Logicshould be added here to copy the value from the master copy to the current copy. Generally this will be a simpleassignment. If there are dependencies between variables, those should be checked and enforced in this method.

Overridable Variables

HTTP related variables that are changeable per transaction are stored in the OverridableHttpConfigParamsstructure, an instance of which is the oride member of HTTPConfigParams and therefore the points in theprevious section still apply. The only difference for that is the further .oride member specifier in the structurereferences.

The variable is required to be accessible from the transaction API. In addition to any custom API functions used toaccess the value, the following items are required for generic access:

1. Add a value to the TSOverridableConfigKey enumeration in apidefs.h.in.

2. Augment the TSHttpTxnConfigFind function to return this enumeration value when given the name of theconfiguration variable. Be sure to count the charaters very carefully.

3. Augment the _conf_to_memberp function in InkAPI.cc to return a pointer to the appropriate member ofOverridableHttpConfigParams and set the type if not a byte value.

4. Update the testing logic in InkAPITest.cc by adding the string name of the configuration variable to theSDK_Overridable_Configs array.

5. Update the Lua plugin enumeration TSLuaOverridableConfigKey in ts_lua_http_config.c.

6. Update the documentation of TSHttpOverridableConfig in TSHttpOverridableConfig.en.rst.

API Reference

For an introduction to the Traffic Server API, how to link it to your plugin, and the methods of passing parameters toyour plugin at runetime, please refer to TSAPI().

API Types

This section documents all of the types defined by the Traffic Server C API. While each type’s documentation refer-ences the C header file in which they are defined, ts/apidefs.h, please note that your plugins should never needto include this file directly. Including the ts/ts.h header is sufficient.

4.9. API Reference 379

https://github.com/apache/trafficserver/blob/master/lib/ts/apidefs.h.in

https://github.com/apache/trafficserver/blob/master/proxy/api/InkAPI.cc

https://github.com/apache/trafficserver/blob/master/proxy/api/InkAPITest.cc

https://github.com/apache/trafficserver/blob/master/plugins/experimental/ts_lua/ts_lua_http_config.c

https://github.com/apache/trafficserver/blob/master/doc/reference/api/TSHttpOverridableConfig.en.rst


TSCacheDataType

Synopsis

#include <ts/apidefs.h>

TSCacheDataType

Enum typedef.

Enumeration Members

TSCacheDataType TS_CACHE_DATA_TYPE_NONE

TSCacheDataType TS_CACHE_DATA_TYPE_HTTP

TSCacheDataType TS_CACHE_DATA_TYPE_OTHER

Description

TSCacheError

Synopsis


TSCacheError

Enum typedef.

Enumeration Members

TSCacheError TS_CACHE_ERROR_NO_DOC

TSCacheError TS_CACHE_ERROR_DOC_BUSY

TSCacheError TS_CACHE_ERROR_NOT_READY

Description

TSCacheLookupResult

Synopsis


TSCacheLookupResult

Enum typedef.



Enumeration Members

TSCacheLookupResult TS_CACHE_LOOKUP_MISS

TSCacheLookupResult TS_CACHE_LOOKUP_HIT_STALE

TSCacheLookupResult TS_CACHE_LOOKUP_HIT_FRESH

TSCacheLookupResult TS_CACHE_LOOKUP_SKIPPED

Description

TSCacheScanResult

Synopsis


TSCacheScanResult

Enum typedef.

Enumeration Members

TSCacheScanResult TS_CACHE_SCAN_RESULT_DONE

TSCacheScanResult TS_CACHE_SCAN_RESULT_CONTINUE

TSCacheScanResult TS_CACHE_SCAN_RESULT_DELETE

TSCacheScanResult TS_CACHE_SCAN_RESULT_DELETE_ALL_ALTERNATES

TSCacheScanResult TS_CACHE_SCAN_RESULT_UPDATE

TSCacheScanResult TS_CACHE_SCAN_RESULT_RETRY

Description

TSEvent

Synopsis


TSEvent

Enum typedef defining the possible events which may be passed to a continuation callback.

Enumeration Members

TSEvent TS_EVENT_NONE

TSEvent TS_EVENT_IMMEDIATE

TSEvent TS_EVENT_TIMEOUT



TSEvent TS_EVENT_ERROR

TSEvent TS_EVENT_CONTINUE

TSEvent TS_EVENT_VCONN_READ_READY

TSEvent TS_EVENT_VCONN_WRITE_READY

TSEvent TS_EVENT_VCONN_READ_COMPLETE

TSEvent TS_EVENT_VCONN_WRITE_COMPLETE

TSEvent TS_EVENT_VCONN_EOS

TSEvent TS_EVENT_VCONN_INACTIVITY_TIMEOUT

TSEvent TS_EVENT_VCONN_ACTIVE_TIMEOUT

TSEvent TS_EVENT_NET_CONNECT

TSEvent TS_EVENT_NET_CONNECT_FAILED

TSEvent TS_EVENT_NET_ACCEPT

TSEvent TS_EVENT_NET_ACCEPT_FAILED

TSEvent TS_EVENT_INTERNAL_206







TSEvent TS_EVENT_HOST_LOOKUP

TSEvent TS_EVENT_CACHE_OPEN_READ

TSEvent TS_EVENT_CACHE_OPEN_READ_FAILED

TSEvent TS_EVENT_CACHE_OPEN_WRITE

TSEvent TS_EVENT_CACHE_OPEN_WRITE_FAILED

TSEvent TS_EVENT_CACHE_REMOVE

TSEvent TS_EVENT_CACHE_REMOVE_FAILED

TSEvent TS_EVENT_CACHE_SCAN

TSEvent TS_EVENT_CACHE_SCAN_FAILED

TSEvent TS_EVENT_CACHE_SCAN_OBJECT

TSEvent TS_EVENT_CACHE_SCAN_OPERATION_BLOCKED

TSEvent TS_EVENT_CACHE_SCAN_OPERATION_FAILED

TSEvent TS_EVENT_CACHE_SCAN_DONE

TSEvent TS_EVENT_CACHE_LOOKUP

TSEvent TS_EVENT_CACHE_READ

TSEvent TS_EVENT_CACHE_DELETE



TSEvent TS_EVENT_CACHE_WRITE

TSEvent TS_EVENT_CACHE_WRITE_HEADER

TSEvent TS_EVENT_CACHE_CLOSE

TSEvent TS_EVENT_CACHE_LOOKUP_READY

TSEvent TS_EVENT_CACHE_LOOKUP_COMPLETE

TSEvent TS_EVENT_CACHE_READ_READY

TSEvent TS_EVENT_CACHE_READ_COMPLETE


TSEvent TS_AIO_EVENT_DONE

TSEvent TS_EVENT_HTTP_CONTINUE

TSEvent TS_EVENT_HTTP_ERROR

TSEvent TS_EVENT_HTTP_READ_REQUEST_HDR

TSEvent TS_EVENT_HTTP_OS_DNS

TSEvent TS_EVENT_HTTP_SEND_REQUEST_HDR

TSEvent TS_EVENT_HTTP_READ_CACHE_HDR

TSEvent TS_EVENT_HTTP_READ_RESPONSE_HDR

TSEvent TS_EVENT_HTTP_SEND_RESPONSE_HDR

TSEvent TS_EVENT_HTTP_REQUEST_TRANSFORM

TSEvent TS_EVENT_HTTP_RESPONSE_TRANSFORM

TSEvent TS_EVENT_HTTP_SELECT_ALT

TSEvent TS_EVENT_HTTP_TXN_START

TSEvent TS_EVENT_HTTP_TXN_CLOSE

TSEvent TS_EVENT_HTTP_SSN_START

TSEvent TS_EVENT_HTTP_SSN_CLOSE

TSEvent TS_EVENT_HTTP_CACHE_LOOKUP_COMPLETE

TSEvent TS_EVENT_HTTP_PRE_REMAP

TSEvent TS_EVENT_HTTP_POST_REMAP

TSEvent TS_EVENT_LIFECYCLE_PORTS_INITIALIZED

TSEvent TS_EVENT_LIFECYCLE_PORTS_READY

TSEvent TS_EVENT_LIFECYCLE_CACHE_READY

TSEvent TS_EVENT_LIFECYCLE_SERVER_SSL_CTX_INITIALIZED

TSEvent TS_EVENT_LIFECYCLE_CLIENT_SSL_CTX_INITIALIZED

TSEvent TS_EVENT_VCONN_PRE_ACCEPT

TSEvent TS_EVENT_MGMT_UPDATE






Description

TSFetchWakeUpOptions

Synopsis


TSFetchWakeUpOptions

Enum typedef.

Enumeration Members

TSFetchWakeUpOptions NO_CALLBACK

TSFetchWakeUpOptions AFTER_HEADER

TSFetchWakeUpOptions AFTER_BODY

Description

TSHttpHookID

Synopsis


TSHttpHookID

Enum typedef defining the possible Hooks for setting up Continuations callbacks.

Enumeration Members

TSHttpHookID TS_HTTP_READ_REQUEST_HDR_HOOK

TSHttpHookID TS_HTTP_OS_DNS_HOOK

TSHttpHookID TS_HTTP_SEND_REQUEST_HDR_HOOK

TSHttpHookID TS_HTTP_READ_CACHE_HDR_HOOK

TSHttpHookID TS_HTTP_READ_RESPONSE_HDR_HOOK

TSHttpHookID TS_HTTP_SEND_RESPONSE_HDR_HOOK

TSHttpHookID TS_HTTP_REQUEST_TRANSFORM_HOOK

TSHttpHookID TS_HTTP_RESPONSE_TRANSFORM_HOOK

TSHttpHookID TS_HTTP_SELECT_ALT_HOOK

TSHttpHookID TS_HTTP_TXN_START_HOOK

TSHttpHookID TS_HTTP_TXN_CLOSE_HOOK



TSHttpHookID TS_HTTP_SSN_START_HOOK

TSHttpHookID TS_HTTP_SSN_CLOSE_HOOK

TSHttpHookID TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK

TSHttpHookID TS_HTTP_PRE_REMAP_HOOK

TSHttpHookID TS_HTTP_POST_REMAP_HOOK

TSHttpHookID TS_HTTP_RESPONSE_CLIENT_HOOK

TSHttpHookID TS_SSL_FIRST_HOOK

TSHttpHookID TS_VCONN_PRE_ACCEPT_HOOK

TSHttpHookID TS_SSL_SNI_HOOK

TSHttpHookID TS_SSL_CERT_HOOK

TSHttpHookID TS_SSL_LAST_HOOK

TSHttpHookID TS_HTTP_LAST_HOOK

Description

TSHttpStatus

Synopsis


TSHttpStatus

Enum typedef which defines the possible return values from TSHttpStatusGet().

Enumeration Members

TSHttpStatus TS_HTTP_STATUS_NONE

TSHttpStatus TS_HTTP_STATUS_CONTINUE

TSHttpStatus TS_HTTP_STATUS_SWITCHING_PROTOCOL

TSHttpStatus TS_HTTP_STATUS_OK

TSHttpStatus TS_HTTP_STATUS_CREATED

TSHttpStatus TS_HTTP_STATUS_ACCEPTED

TSHttpStatus TS_HTTP_STATUS_NON_AUTHORITATIVE_INFORMATION

TSHttpStatus TS_HTTP_STATUS_NO_CONTENT

TSHttpStatus TS_HTTP_STATUS_RESET_CONTENT

TSHttpStatus TS_HTTP_STATUS_PARTIAL_CONTENT

TSHttpStatus TS_HTTP_STATUS_MULTI_STATUS

TSHttpStatus TS_HTTP_STATUS_ALREADY_REPORTED

TSHttpStatus TS_HTTP_STATUS_IM_USED

TSHttpStatus TS_HTTP_STATUS_MULTIPLE_CHOICES



TSHttpStatus TS_HTTP_STATUS_MOVED_PERMANENTLY

TSHttpStatus TS_HTTP_STATUS_MOVED_TEMPORARILY

TSHttpStatus TS_HTTP_STATUS_SEE_OTHER

TSHttpStatus TS_HTTP_STATUS_NOT_MODIFIED

TSHttpStatus TS_HTTP_STATUS_USE_PROXY

TSHttpStatus TS_HTTP_STATUS_TEMPORARY_REDIRECT

TSHttpStatus TS_HTTP_STATUS_PERMANENT_REDIRECT

TSHttpStatus TS_HTTP_STATUS_BAD_REQUEST

TSHttpStatus TS_HTTP_STATUS_UNAUTHORIZED

TSHttpStatus TS_HTTP_STATUS_PAYMENT_REQUIRED

TSHttpStatus TS_HTTP_STATUS_FORBIDDEN

TSHttpStatus TS_HTTP_STATUS_NOT_FOUND

TSHttpStatus TS_HTTP_STATUS_METHOD_NOT_ALLOWED

TSHttpStatus TS_HTTP_STATUS_NOT_ACCEPTABLE

TSHttpStatus TS_HTTP_STATUS_PROXY_AUTHENTICATION_REQUIRED

TSHttpStatus TS_HTTP_STATUS_REQUEST_TIMEOUT

TSHttpStatus TS_HTTP_STATUS_CONFLICT

TSHttpStatus TS_HTTP_STATUS_GONE

TSHttpStatus TS_HTTP_STATUS_LENGTH_REQUIRED

TSHttpStatus TS_HTTP_STATUS_PRECONDITION_FAILED

TSHttpStatus TS_HTTP_STATUS_REQUEST_ENTITY_TOO_LARGE

TSHttpStatus TS_HTTP_STATUS_REQUEST_URI_TOO_LONG

TSHttpStatus TS_HTTP_STATUS_UNSUPPORTED_MEDIA_TYPE

TSHttpStatus TS_HTTP_STATUS_REQUESTED_RANGE_NOT_SATISFIABLE

TSHttpStatus TS_HTTP_STATUS_EXPECTATION_FAILED

TSHttpStatus TS_HTTP_STATUS_UNPROCESSABLE_ENTITY

TSHttpStatus TS_HTTP_STATUS_LOCKED

TSHttpStatus TS_HTTP_STATUS_FAILED_DEPENDENCY

TSHttpStatus TS_HTTP_STATUS_UPGRADE_REQUIRED

TSHttpStatus TS_HTTP_STATUS_PRECONDITION_REQUIRED

TSHttpStatus TS_HTTP_STATUS_TOO_MANY_REQUESTS

TSHttpStatus TS_HTTP_STATUS_REQUEST_HEADER_FIELDS_TOO_LARGE

TSHttpStatus TS_HTTP_STATUS_INTERNAL_SERVER_ERROR

TSHttpStatus TS_HTTP_STATUS_NOT_IMPLEMENTED

TSHttpStatus TS_HTTP_STATUS_BAD_GATEWAY

TSHttpStatus TS_HTTP_STATUS_SERVICE_UNAVAILABLE



TSHttpStatus TS_HTTP_STATUS_GATEWAY_TIMEOUT

TSHttpStatus TS_HTTP_STATUS_HTTPVER_NOT_SUPPORTED

TSHttpStatus TS_HTTP_STATUS_VARIANT_ALSO_NEGOTIATES

TSHttpStatus TS_HTTP_STATUS_INSUFFICIENT_STORAGE

TSHttpStatus TS_HTTP_STATUS_LOOP_DETECTED

TSHttpStatus TS_HTTP_STATUS_NOT_EXTENDED

TSHttpStatus TS_HTTP_STATUS_NETWORK_AUTHENTICATION_REQUIRED

Description

TSHttpType

Synopsis


TSHttpType

Enum typedef which defines the possible HTTP types assigned to an HTTP header, as returned byTSHttpHdrTypeGet(). Headers created by TSHttpHdrCreate() receive TS_HTTP_TYPE_UNKNOWN bydefault and may be modified once by using TSHttpHdrTypeSet().

Enumeration Members

TSHttpType TS_HTTP_TYPE_UNKNOWN

Default for new headers created by TSHttpHdrCreate().

TSHttpType TS_HTTP_TYPE_REQUEST

HTTP request headers.

TSHttpType TS_HTTP_TYPE_RESPONSE

HTTP response headers.

Description

TSIOBuffersSizeIndex

Synopsis


TSIOBuffersSizeIndex

Enum typedef.



Enumeration Members

TSIOBuffersSizeIndex TS_IOBUFFER_SIZE_INDEX_128



TSIOBuffersSizeIndex TS_IOBUFFER_SIZE_INDEX_1K










TSIOBuffersSizeIndex TS_IOBUFFER_SIZE_INDEX_1M

TSIOBuffersSizeIndex TS_IOBUFFER_SIZE_INDEX_2M

Description

TSLifecycleHookID

Synopsis


TSLifeCycleHookID

Enum typedef used to indicate the event hook being called during a continuation invocation.

Enumeration Members

TSLifecycleHookID TS_LIFECYCLE_PORTS_INITIALIZED_HOOK

TSLifecycleHookID TS_LIFECYCLE_PORTS_READY_HOOK

TSLifecycleHookID TS_LIFECYCLE_CACHE_READY_HOOK

TSLifecycleHookID TS_LIFECYCLE_SERVER_SSL_CTX_INITIALIZED_HOOK

TSLifecycleHookID TS_LIFECYCLE_CLIENT_SSL_CTX_INITIALIZED_HOOK

TSLifecycleHookID TS_LIFECYCLE_LAST_HOOK



Description

TSLookingUpType

Synopsis


TSLookingUpType

Enum typedef.

Enumeration Members

TSLookingUpType TS_LOOKUP_UNDEFINED_LOOKUP

TSLookingUpType TS_LOOKUP_ICP_SUGGESTED_HOST

TSLookingUpType TS_LOOKUP_PARENT_PROXY

TSLookingUpType TS_LOOKUP_ORIGIN_SERVER

TSLookingUpType TS_LOOKUP_INCOMING_ROUTER

TSLookingUpType TS_LOOKUP_HOST_NONE

Description

TSMiletonesType

Synopsis


TSMiletonesType

Enum typedef.

Enumeration Members

TSMiletonesType TS_MILESTONE_NULL

TSMiletonesType TS_MILESTONE_UA_BEGIN

TSMiletonesType TS_MILESTONE_UA_READ_HEADER_DONE

TSMiletonesType TS_MILESTONE_UA_BEGIN_WRITE

TSMiletonesType TS_MILESTONE_UA_CLOSE

TSMiletonesType TS_MILESTONE_SERVER_FIRST_CONNECT

TSMiletonesType TS_MILESTONE_SERVER_CONNECT

TSMiletonesType TS_MILESTONE_SERVER_CONNECT_END

TSMiletonesType TS_MILESTONE_SERVER_BEGIN_WRITE

TSMiletonesType TS_MILESTONE_SERVER_FIRST_READ



TSMiletonesType TS_MILESTONE_SERVER_READ_HEADER_DONE

TSMiletonesType TS_MILESTONE_SERVER_CLOSE

TSMiletonesType TS_MILESTONE_CACHE_OPEN_READ_BEGIN

TSMiletonesType TS_MILESTONE_CACHE_OPEN_READ_END

TSMiletonesType TS_MILESTONE_CACHE_OPEN_WRITE_BEGIN

TSMiletonesType TS_MILESTONE_CACHE_OPEN_WRITE_END

TSMiletonesType TS_MILESTONE_DNS_LOOKUP_BEGIN

TSMiletonesType TS_MILESTONE_DNS_LOOKUP_END

TSMiletonesType TS_MILESTONE_SM_START

TSMiletonesType TS_MILESTONE_SM_FINISH

TSMiletonesType TS_MILESTONE_LAST_ENTRY

Description

TSOverridableConfigKey

Synopsis



Enum typedef.

Enumeration Members

TSOverridableConfigKey TS_CONFIG_NULL

TSOverridableConfigKey TS_CONFIG_URL_REMAP_PRISTINE_HOST_HDR

TSOverridableConfigKey TS_CONFIG_HTTP_CHUNKING_ENABLED

TSOverridableConfigKey TS_CONFIG_HTTP_NEGATIVE_CACHING_ENABLED

TSOverridableConfigKey TS_CONFIG_HTTP_NEGATIVE_CACHING_LIFETIME

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_WHEN_TO_REVALIDATE

TSOverridableConfigKey TS_CONFIG_HTTP_KEEP_ALIVE_ENABLED_IN

TSOverridableConfigKey TS_CONFIG_HTTP_KEEP_ALIVE_ENABLED_OUT

TSOverridableConfigKey TS_CONFIG_HTTP_KEEP_ALIVE_POST_OUT

TSOverridableConfigKey TS_CONFIG_HTTP_SHARE_SERVER_SESSIONS, // DEPRECATED

TSOverridableConfigKey TS_CONFIG_HTTP_SERVER_SESSION_SHARING_POOL

TSOverridableConfigKey TS_CONFIG_HTTP_SERVER_SESSION_SHARING_MATCH

TSOverridableConfigKey TS_CONFIG_NET_SOCK_RECV_BUFFER_SIZE_OUT

TSOverridableConfigKey TS_CONFIG_NET_SOCK_SEND_BUFFER_SIZE_OUT

TSOverridableConfigKey TS_CONFIG_NET_SOCK_OPTION_FLAG_OUT



TSOverridableConfigKey TS_CONFIG_HTTP_FORWARD_PROXY_AUTH_TO_PARENT

TSOverridableConfigKey TS_CONFIG_HTTP_ANONYMIZE_REMOVE_FROM

TSOverridableConfigKey TS_CONFIG_HTTP_ANONYMIZE_REMOVE_REFERER

TSOverridableConfigKey TS_CONFIG_HTTP_ANONYMIZE_REMOVE_USER_AGENT

TSOverridableConfigKey TS_CONFIG_HTTP_ANONYMIZE_REMOVE_COOKIE

TSOverridableConfigKey TS_CONFIG_HTTP_ANONYMIZE_REMOVE_CLIENT_IP

TSOverridableConfigKey TS_CONFIG_HTTP_ANONYMIZE_INSERT_CLIENT_IP

TSOverridableConfigKey TS_CONFIG_HTTP_RESPONSE_SERVER_ENABLED

TSOverridableConfigKey TS_CONFIG_HTTP_INSERT_SQUID_X_FORWARDED_FOR

TSOverridableConfigKey TS_CONFIG_HTTP_SERVER_TCP_INIT_CWND

TSOverridableConfigKey TS_CONFIG_HTTP_SEND_HTTP11_REQUESTS

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_HTTP

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_CLUSTER_CACHE_LOCAL

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_IGNORE_CLIENT_NO_CACHE

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_IGNORE_CLIENT_CC_MAX_AGE

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_IMS_ON_CLIENT_NO_CACHE

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_IGNORE_SERVER_NO_CACHE

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_CACHE_RESPONSES_TO_COOKIES

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_IGNORE_AUTHENTICATION

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_CACHE_URLS_THAT_LOOK_DYNAMIC

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_REQUIRED_HEADERS

TSOverridableConfigKey TS_CONFIG_HTTP_INSERT_REQUEST_VIA_STR

TSOverridableConfigKey TS_CONFIG_HTTP_INSERT_RESPONSE_VIA_STR

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_HEURISTIC_MIN_LIFETIME

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_HEURISTIC_MAX_LIFETIME

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_GUARANTEED_MIN_LIFETIME

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_GUARANTEED_MAX_LIFETIME

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_MAX_STALE_AGE

TSOverridableConfigKey TS_CONFIG_HTTP_KEEP_ALIVE_NO_ACTIVITY_TIMEOUT_IN

TSOverridableConfigKey TS_CONFIG_HTTP_KEEP_ALIVE_NO_ACTIVITY_TIMEOUT_OUT

TSOverridableConfigKey TS_CONFIG_HTTP_TRANSACTION_NO_ACTIVITY_TIMEOUT_IN

TSOverridableConfigKey TS_CONFIG_HTTP_TRANSACTION_NO_ACTIVITY_TIMEOUT_OUT

TSOverridableConfigKey TS_CONFIG_HTTP_TRANSACTION_ACTIVE_TIMEOUT_OUT

TSOverridableConfigKey TS_CONFIG_HTTP_ORIGIN_MAX_CONNECTIONS

TSOverridableConfigKey TS_CONFIG_HTTP_CONNECT_ATTEMPTS_MAX_RETRIES

TSOverridableConfigKey TS_CONFIG_HTTP_CONNECT_ATTEMPTS_MAX_RETRIES_DEAD_SERVER



TSOverridableConfigKey TS_CONFIG_HTTP_CONNECT_ATTEMPTS_RR_RETRIES

TSOverridableConfigKey TS_CONFIG_HTTP_CONNECT_ATTEMPTS_TIMEOUT

TSOverridableConfigKey TS_CONFIG_HTTP_POST_CONNECT_ATTEMPTS_TIMEOUT

TSOverridableConfigKey TS_CONFIG_HTTP_DOWN_SERVER_CACHE_TIME

TSOverridableConfigKey TS_CONFIG_HTTP_DOWN_SERVER_ABORT_THRESHOLD

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_FUZZ_TIME

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_FUZZ_MIN_TIME

TSOverridableConfigKey TS_CONFIG_HTTP_DOC_IN_CACHE_SKIP_DNS

TSOverridableConfigKey TS_CONFIG_HTTP_BACKGROUND_FILL_ACTIVE_TIMEOUT

TSOverridableConfigKey TS_CONFIG_HTTP_RESPONSE_SERVER_STR

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_HEURISTIC_LM_FACTOR

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_FUZZ_PROBABILITY

TSOverridableConfigKey TS_CONFIG_HTTP_BACKGROUND_FILL_COMPLETED_THRESHOLD

TSOverridableConfigKey TS_CONFIG_NET_SOCK_PACKET_MARK_OUT

TSOverridableConfigKey TS_CONFIG_NET_SOCK_PACKET_TOS_OUT

TSOverridableConfigKey TS_CONFIG_HTTP_INSERT_AGE_IN_RESPONSE

TSOverridableConfigKey TS_CONFIG_HTTP_CHUNKING_SIZE

TSOverridableConfigKey TS_CONFIG_HTTP_FLOW_CONTROL_ENABLED

TSOverridableConfigKey TS_CONFIG_HTTP_FLOW_CONTROL_LOW_WATER_MARK

TSOverridableConfigKey TS_CONFIG_HTTP_FLOW_CONTROL_HIGH_WATER_MARK

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_RANGE_LOOKUP

TSOverridableConfigKey TS_CONFIG_HTTP_NORMALIZE_AE_GZIP

TSOverridableConfigKey TS_CONFIG_HTTP_DEFAULT_BUFFER_SIZE

TSOverridableConfigKey TS_CONFIG_HTTP_DEFAULT_BUFFER_WATER_MARK

TSOverridableConfigKey TS_CONFIG_HTTP_REQUEST_HEADER_MAX_SIZE

TSOverridableConfigKey TS_CONFIG_HTTP_RESPONSE_HEADER_MAX_SIZE

TSOverridableConfigKey TS_CONFIG_HTTP_NEGATIVE_REVALIDATING_ENABLED

TSOverridableConfigKey TS_CONFIG_HTTP_NEGATIVE_REVALIDATING_LIFETIME

TSOverridableConfigKey TS_CONFIG_HTTP_ACCEPT_ENCODING_FILTER_ENABLED

TSOverridableConfigKey TS_CONFIG_SSL_HSTS_MAX_AGE

TSOverridableConfigKey TS_CONFIG_SSL_HSTS_INCLUDE_SUBDOMAINS

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_OPEN_READ_RETRY_TIME

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_MAX_OPEN_READ_RETRIES

TSOverridableConfigKey TS_CONFIG_HTTP_CACHE_RANGE_WRITE

TSOverridableConfigKey TS_CONFIG_HTTP_POST_CHECK_CONTENT_LENGTH_ENABLED

TSOverridableConfigKey TS_CONFIG_HTTP_GLOBAL_USER_AGENT_HEADER



TSOverridableConfigKey TS_CONFIG_LAST_ENTRY

Description

TSParseResult

Synopsis


TSParseResult

Enum typedef defining the possible return values from TSHttpHdrParseResp().

Enumeration Members

TSParseResult TS_PARSE_ERROR

TSParseResult TS_PARSE_DONE

TSParseResult TS_PARSE_OK

TSParseResult TS_PARSE_CONT

Description

TSRecordAccessType

Synopsis


TSRecordAccessType

Enum typedef.

Enumeration Members

TSRecordAccessType TS_RECORDACCESS_NULL

TSRecordAccessType TS_RECORDACCESS_NO_ACCESS

TSRecordAccessType TS_RECORDACCESS_READ_ONLY

Description

TSRecordCheckType

Synopsis


TSRecordCheckType



Enum typedef.

Enumeration Members

TSRecordCheckType TS_RECORDCHECK_NULL

TSRecordCheckType TS_RECORDCHECK_STR

TSRecordCheckType TS_RECORDCHECK_INT

TSRecordCheckType TS_RECORDCHECK_IP

Description

TSRecordDataType

Synopsis


TSRecordDataType

Enum typedef.

Enumeration Members

TSRecordDataType TS_RECORDDATATYPE_NULL

TSRecordDataType TS_RECORDDATATYPE_INT

TSRecordDataType TS_RECORDDATATYPE_FLOAT

TSRecordDataType TS_RECORDDATATYPE_STRING

TSRecordDataType TS_RECORDDATATYPE_COUNTER

TSRecordDataType TS_RECORDDATATYPE_STAT_CONST

TSRecordDataType TS_RECORDDATATYPE_STAT_FX

TSRecordDataType TS_RECORDDATATYPE_MAX

Description

TSRecordModeType

Synopsis


TSRecordModeType

Enum typedef.



Enumeration Members

TSRecordModeType TS_RECORDMODE_NULL

TSRecordModeType TS_RECORDMODE_CLIENT

TSRecordModeType TS_RECORDMODE_SERVER

TSRecordModeType TS_RECORDMODE_STAND_ALONE

Description

TSRecordPersistType

Synopsis


TSRecordPersistType

Enum typedef.

Enumeration Members

TSRecordPersistType TS_RECORDP_NULL

TSRecordPersistType TS_RECORDP_PERSISTENT

TSRecordPersistType TS_RECORDP_NON_PERSISTENT

Description

TSRecordType

Synopsis


TSRecordType

Enum typedef.

Enumeration Members

TSRecordType TS_RECORDTYPE_NULL

TSRecordType TS_RECORDTYPE_CONFIG

TSRecordType TS_RECORDTYPE_PROCESS

TSRecordType TS_RECORDTYPE_NODE

TSRecordType TS_RECORDTYPE_CLUSTER

TSRecordType TS_RECORDTYPE_LOCAL



TSRecordType TS_RECORDTYPE_PLUGIN

TSRecordType TS_RECORDTYPE_ALL

Description

TSRecordUpdateType

Synopsis


TSRecordUpdateType

Enum typedef.

Enumeration Members

TSRecordUpdateType TS_RECORDUPDATE_NULL

TSRecordUpdateType TS_RECORDUPDATE_DYNAMIC

TSRecordUpdateType TS_RECORDUPDATE_RESTART_TS

TSRecordUpdateType TS_RECORDUPDATE_RESTART_TM

TSRecordUpdateType TS_RECORDUPDATE_RESTART_TC

Description

TSReturnCode

Synopsis


TSReturnCode

Enum typedef.

Enumeration Members

TSReturnCode TS_ERROR

TSReturnCode TS_SUCCESS

Description

TSSDKVersion

Synopsis




TSSDKVersion

Enum typedef.

Enumeration Members

TSSDKVersion TS_SDK_VERSION_2_0

TSSDKVersion TS_SDK_VERSION_3_0

Description

TSServerSessionSharingMatchType

Synopsis


TSServerSessionSharingMatchType

Enum typedef.

Enumeration Members

TSServerSessionSharingMatchType TS_SERVER_SESSION_SHARING_MATCH_NONE

TSServerSessionSharingMatchType TS_SERVER_SESSION_SHARING_MATCH_BOTH

TSServerSessionSharingMatchType TS_SERVER_SESSION_SHARING_MATCH_IP

TSServerSessionSharingMatchType TS_SERVER_SESSION_SHARING_MATCH_HOST

Description

TSServerSessionSharingPoolType

Synopsis


TSServerSessionSharingPoolType

Enum typedef.

Enumeration Members

TSServerSessionSharingPoolType TS_SERVER_SESSION_SHARING_POOL_GLOBAL

TSServerSessionSharingPoolType TS_SERVER_SESSION_SHARING_POOL_THREAD



Description

TSServerState

Synopsis


TSServerState

Enum typedef.

Enumeration Members

TSServerState TS_SRVSTATE_STATE_UNDEFINED

TSServerState TS_SRVSTATE_ACTIVE_TIMEOUT

TSServerState TS_SRVSTATE_BAD_INCOMING_RESPONSE

TSServerState TS_SRVSTATE_CONNECTION_ALIVE

TSServerState TS_SRVSTATE_CONNECTION_CLOSED

TSServerState TS_SRVSTATE_CONNECTION_ERROR

TSServerState TS_SRVSTATE_INACTIVE_TIMEOUT

TSServerState TS_SRVSTATE_OPEN_RAW_ERROR

TSServerState TS_SRVSTATE_PARSE_ERROR

TSServerState TS_SRVSTATE_TRANSACTION_COMPLETE

TSServerState TS_SRVSTATE_CONGEST_CONTROL_CONGESTED_ON_F

TSServerState TS_SRVSTATE_CONGEST_CONTROL_CONGESTED_ON_M

Description

TSSslVConnOp

Synopsis


TSSslVConnOp

Enum typedef.

Enumeration Members

TSSslVConnOp TS_SSL_HOOK_OP_DEFAULT

TSSslVConnOp TS_SSL_HOOK_OP_TUNNEL

TSSslVConnOp TS_SSL_HOOK_OP_TERMINATE

TSSslVConnOp TS_SSL_HOOK_OP_LAST



Description

TSThreadPool

Synopsis


TSThreadPool

Enum typedef.

Enumeration Members

TSThreadPool TS_THREAD_POOL_DEFAULT

TSThreadPool TS_THREAD_POOL_NET

TSThreadPool TS_THREAD_POOL_TASK

TSThreadPool TS_THREAD_POOL_SSL

TSThreadPool TS_THREAD_POOL_DNS

TSThreadPool TS_THREAD_POOL_REMAP

TSThreadPool TS_THREAD_POOL_CLUSTER

TSThreadPool TS_THREAD_POOL_UDP

Description

TSVConnCloseFlags

Synopsis


TSVConnCloseFlags

Enum typedef.

Enumeration Members

TSVConnCloseFlags TS_VC_CLOSE_ABORT

TSVConnCloseFlags TS_VC_CLOSE_NORMAL

Description

API Functions

TSAPI

Introduction to the Apache Traffic Server API.



Synopsis

#include <ts/ts.h> #include <ts/remap.h>

Description

The Apache Traffic Server API enables you to create plugins, using the C programming language, that customize thebehavior of your Traffic Server installation.

Traffic Server enables sophisticated caching and processing of web-related traffic, such as DNS and HTTP requestsand responses. Traffic Server itself consists of an event-driven loop that can be simplified as follows:

for (;;) {event = get_next_event();handle_event (event);

}

You compile your plugin source code to create a shared library that Traffic Server loads when it is started. Your plugincontains callback functions that are registered for specific Traffic Server events. When Traffic Server needs to processan event, it invokes any and all call-back functions you’ve registered for that event type.

Possible uses for plugins include the following:

• HTTP processing plugins can filter, blacklist, authorize users or transform content.

• Protocol plugins can enable Traffic Server to proxy-cache new protocol content.

• A blacklisting plugin denies attempts to access web sites that are off-limits.

• Append transform plugins add data to HTTP response content.

• An image conversion plugin transforms JPEG images to GIF images.

• Compression plugins send response content to a compression server that compresses the data (alternatively, acompression library local to the Traffic Server host machine could do the compression).

• Authorization plugins check a user’s permissions to access particular web sites. The plugin could consult a localauthorization program or send queries to an authorization server.

• A plugin that gathers client information from request headers and enters this information in a database.

• A protocol plugin listen for specific protocol requests on a designated port and then uses Traffic Server’s proxyserver and cache to serve client requests.

Naming Conventions

The Traffic Server API adheres to the following naming conventions:

• The TS prefix is used for all function and variable names defined in the Traffic Server API. For example,TS_EVENT_NONE, TSMutex, and TSContCreate().

• Enumerated values are always written in all uppercase letters. For example, TS_EVENT_NONE andTS_VC_CLOSE_ABORT.

• Constant values are all uppercase; enumerated values can be seen as a subset of constants. For example,TS_URL_SCHEME_FILE and TS_MIME_FIELD_ACCEPT.

• The names of defined types are mixed-case. For example, TSHttpSsn and TSHttpTxn(). TSDebug()

• Function names are mixed-case. For example, TSUrlCreate() and TSContDestroy().



• Function names use the following subject-verb naming style: TS-<subject>-<verb>, where <subject> goes fromgeneral to specific. This makes it easier to determine what a function does by reading its name. For ex-ample, the function to retrieve the password field (the specific subject) from a URL (the general subject) isTSUrlPasswordGet().

• Common verbs like Create, Destroy, Get, Set, Copy, Find, Retrieve, Insert, Remove, and Delete are used onlywhen appropriate.

Plugin Loading and Configuration

When Traffic Server is first started, it consults the plugin.config file to determine the names of all shared plugin librariesthat need to be loaded. The plugin.config file also defines arguments that are to be passed to each plugin’s initializationfunction, TSPluginInit(). The records.config file defines the path to each plugin shared library.

The sample plugin.config file below contains a comment line, a blank line, and two plugin configurations:

# This is a comment line.my-plugin.so www.junk.com www.trash.com www.garbage.comsome-plugin.so arg1 arg2 $proxy.config.http.cache.on

Each plugin configuration in the plugin.config file resembles a UNIX or DOS shell command; each line inplugin.config cannot exceed 1023 characters.

The first plugin configuration is for a plugin named my-plugin.so. It contains three arguments that are to be passed tothat plugin’s initialization routine. The second configuration is for a plugin named some-plugin.so; it contains threearguments. The last argument, $proxy.config.http.cache.on, is actually a configuration variable. Traffic Server willlook up the specified configuration variable and substitute its value.

Plugins are loaded and initialized by Traffic Server in the order they appear in the plugin.config file.

Plugin Initialization

Each plugin must define an initialization function named TSPluginInit() that Traffic Server invokes when theplugin is loaded. TSPluginInit() is commonly used to read configuration information and register hooks forevent notification.

Files

plugin.config, records.config

See Also

TSPluginInit(3ts)

TSActionCancel

Synopsis

#include <ts/ts.h>

void TSActionCancel(TSAction actionp)



Description

TSActionDone

Synopsis

#include <ts/ts.h>

int TSActionDone(TSAction actionp)

Description

TSCacheRead

Synopsis

#include <ts/ts.h>

TSAction TSCacheRead(TSCont contp, TSCacheKey key)

Description

Asks the Traffic Server cache if the object corresponding to :arg:‘key‘ exists in the cache and can be read.

If the object can be read, the Traffic Server cache calls the continuation :arg:‘contp‘ back with the eventTS_EVENT_CACHE_OPEN_READ. In this case, the cache also passes :arg:‘contp‘ a cache vconnection and:arg:‘contp‘ can then initiate a read operation on that vconnection using TSVConnRead.

If the object cannot be read, the cache calls :arg:‘contp‘ back with the eventTS_EVENT_CACHE_OPEN_READ_FAILED. The user (:arg:‘contp‘) has the option to cancel the action re-turned by TSCacheRead. Note that reentrant calls are possible, i.e. the cache can call back the user (:arg:‘contp‘)in the same call.

TSCacheRemove

Synopsis

#include <ts/ts.h>

TSAction TSCacheRemove(TSCont contp, TSCacheKey key)

Description

Removes the object corresponding to :arg:‘key‘ from the cache.

If the object was removed successfully, the cache calls :arg:‘contp‘ back with the eventTS_EVENT_CACHE_REMOVE. If the object was not found in the cache, the cache calls :arg:‘contp‘ backwith the event TS_EVENT_CACHE_REMOVE_FAILED.

In both of these callbacks, the user (:arg:‘contp‘) does not have to do anything. The user does not get anyvconnection from the cache, since no data needs to be transferred. When the cache calls :arg:‘contp‘ back withTS_EVENT_CACHE_REMOVE, the remove has already been commited.



TSCacheWrite

Synopsis

#include <ts/ts.h>

TSAction TSCacheWrite(TSCont contp, TSCacheKey key)

Description

Asks the Traffic Server cache if :arg:‘contp‘ can start writing the object corresponding to :arg:‘key‘ to the cache.

If the object can be written, the cache calls :arg:‘contp‘ back with the event TS_EVENT_CACHE_OPEN_WRITE. Inthis case, the cache also passes :arg:‘contp‘ a cache vconnection and :arg:‘contp‘ can then initiate a write operationon that vconnection using TSVConnWrite. The object is not committed to the cache until the vconnection is closed.When all data has been transferred, the user (:arg:‘contp‘) must do an TSVConnClose. In case of any errors, theuser must do an TSVConnAbort(contp, 0).

If the object cannot be written, the cache calls :arg:‘contp‘ back with the eventTS_EVENT_CACHE_OPEN_WRITE_FAILED. This can happen, for example, if there is another object withthe same :arg:‘key‘ being written to the cache. The user (:arg:‘contp‘) has the option to cancel the action returnedby TSCacheWrite.

Note that reentrant calls are possible, i.e. the cache can call back the user (:arg:‘contp‘) in the same call.

TSConfigDataGet

Synopsis

#include <ts/ts.h>

void* TSConfigDataGet(TSConfig configp)

Description

TSConfigGet

Synopsis

#include <ts/ts.h>

TSConfig TSConfigGet(unsigned int id)

Description

TSConfigRelease

Synopsis

#include <ts/ts.h>

void TSConfigRelease(unsigned int id, TSConfig configp)



Description

TSConfigSet

Synopsis

#include <ts/ts.h>

unsigned int TSConfigSet(unsigned int id, void * data, TSConfigDestroyFunc funcp)

Description

TSContCall

Synopsis

#include <ts/ts.h>

int TSContCall(TSCont contp, TSEvent event, void * edata)

Description

TSContCreate

Synopsis

#include <ts/ts.h>

TSCont TSContCreate(TSEventFunc funcp, TSMutex mutexp)

Description

TSContDataGet

Synopsis

#include <ts/ts.h>

void* TSContDataGet(TSCont contp)

Description

TSContDataSet

Synopsis

#include <ts/ts.h>

void TSContDataSet(TSCont contp, void * data)



Description

TSContDestroy

Synopsis

#include <ts/ts.h>

void TSContDestroy(TSCont contp)

Description

TSContMutexGet

Synopsis

#include <ts/ts.h>

TSMutex TSContMutexGet(TSCont contp)

Description

TSContSchedule

Synopsis

#include <ts/ts.h>

TSAction TSContSchedule(TSCont contp, ink_hrtime timeout, TSThreadPool tp)

Description

TSDebug

Traffic Server Debugging APIs.

Synopsis

#include <ts/ts.h>

void TSDebug(const char * tag, const char * format, ...)

void TSError(const char * tag, const char * format, ...)

int TSIsDebugTagSet(const char * tag)

void TSDebugSpecific(int debug_flag, const char * tag, const char * format, ...)

void TSHttpTxnDebugSet(TSHttpTxn txnp, int on)

void TSHttpSsnDebugSet(TSHttpSsn ssn, int on)

int TSHttpTxnDebugGet(TSHttpTxn txnp)



int TSHttpSsnDebugGet(TSHttpSsn ssn)

const char* TSHttpServerStateNameLookup(TSServerState state)

const char* TSHttpHookNameLookup(TSHttpHookID hook)

const char* TSHttpEventNameLookup(TSEvent event)

void TSAssert(...)

void TSReleaseAssert(...)

Description

TSError() is similar to printf except that instead of writing the output to the C standard output, it writes outputto the Traffic Server error log.

TSDebug() is the same as TSError() except that it only logs the debug message if the given debug :arg:‘tag‘ isenabled. It writes output to the Traffic Server debug log.

TSIsDebugTagSet() returns non-zero if the given debug :arg:‘tag‘ is enabled.

In debug mode, TSAssert Traffic Server to prints the file name, line number and expression, and then aborts.In release mode, the expression is not removed but the effects of printing an error message and aborting are.TSReleaseAssert prints an error message and aborts in both release and debug mode.

TSDebugSpecific() emits a debug line even if the debug :arg:‘tag‘ is turned off, as long as debug flagis enabled. This can be used in conjunction with TSHttpTxnDebugSet(), TSHttpSsnDebugSet(),TSHttpTxnDebugGet() and TSHttpSsnDebugGet() to enable debugging on specific session and transac-tion objects.

TSHttpServerStateNameLookup(), TSHttpHookNameLookup() and TSHttpEventNameLookup()converts the respective internal state to a string representation. This can be useful in debugging (TSDebug()), loggingand other types notifications.

Examples

This example uses TSDebugSpecific() to log a message when a specific debugging flag is enabled:

#include <ts/ts.h>

// Produce information about a hook receiving an eventTSDebug(PLUGIN_NAME, "Entering hook=%s, event=%s",

TSHttpHookNameLookup(hook), TSHttpEventNameLookup(event));

// Emit debug message if "tag" is enabled or the txn debug// flag is set.TSDebugSpecifc(TSHttpTxnDebugGet(txn), "tag" ,

"Hello World from transaction %p", txn);

See Also

TSAPI(3ts), printf(3)



TSHostLookup

Synopsis

#include <ts/ts.h>

TSAction TSHostLookup(TSCont contp, const char * hostname, size_t namelen)

Description

TSHostLookupResultAddrGet

Synopsis

#include <ts/ts.h>

sockaddr const* TSHostLookupResultAddrGet(TSHostLookupResult lookup_result)

Description

TSHttpConnect

Synopsis

#include <ts/ts.h>

TSVConn TSHttpConnect(sockaddr const * addr)

Description

Allows the plugin to initiate an HTTP connection.

The TSVConn the plugin receives as the result of successful operates identically to one created throughTSNetConnect. Aside from allowing the plugin to set the client ip and port for logging, the functional-ity of TSHttpConnect() is identical to connecting to localhost on the proxy port with TSNetConnect().TSHttpConnect() is more efficient than TSNetConnect() to localhost since it avoids the overhead of pass-ing the data through the operating system.

TSHttpConnectWithPluginId

Allows the plugin to initiate an http connection. This will tag the HTTP state machine with extra data that can beaccessed by the logging interface. The connection is treated as an HTTP transaction as if it came from a client.

Synopsis

#include <ts/ts.h>

TSVConn TSHttpConnectWithPluginId(sockaddr const * addr, char const * tag, int64_t id)



Description

This call attempts to create an HTTP state machine and a virtual connection to that state machine. This is more efficientthan using TSNetConnect() because it avoids using the operating system stack via the loopback interface.

:arg:‘addr‘ This is the network address of the target of the connection. This includes the port which should be storedin the sockaddr structure pointed at by :arg:‘addr‘.

:arg:‘tag‘ This is a tag that is passed through to the HTTP state machine. It must be a persistent string that has alifetime longer than the connection. It is accessible via the log field pitag. This is intended as a class or typeidentifier that is consistent across all connections for this plugin. In effect, the name of the plugin. This can beNULL.

:arg:‘id‘ This is a numeric identifier that is passed through to the HTTP state machine. It is accessible viathe log field piid. This is intended as a connection identifier and should be distinct for every call toTSHttpConnectWithPluginId(). The easiest mechanism is to define a plugin global value and incre-ment it for each connection. The value 0 is reserved to mean “not set” and can be used as a default if thisfunctionality is not needed.

The virtual connection returned as the TSCVonn is API equivalent to a network virtual connection both to the pluginand to internal mechanisms. Data is read and written to the connection (and thence to the target system) by readingand writing on this virtual connection.

: This function only opens the connection. To drive the transaction an actual HTTP request must be sent and the HTTPresponse handled. The transaction is handled as a standard HTTP transaction and all of the standard configurationoptions and plugins will operate on it.

The combination of :arg:‘tag‘ and :arg:‘id‘ is intended to enable correlation in log post processing. The :arg:‘tag‘identifies the connection as related to the plugin and the :arg:‘id‘ can be used in conjuction with plugin generated logsto correlate the log records.

Notes

The SPDY implementation uses this to correlate client sessions with SPDY streams. Each client connection is assigneda distinct numeric identifier. This is passed as the :arg:‘id‘ to TSHttpConnectWithPluginId(). The :arg:‘tag‘is selected to be the NPN string for the client session protocol, e.g. “spdy/3” or “spdy/3.1”. Log post processing canthen count the number of connections for the various supported protocols and the number of SPDY virtual streams foreach real client connection to Traffic Server.

See Also

TSHttpConnect(3ts), TSNetConnect(3ts), TSAPI(3ts)

TSHttpHdrClone

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpHdrClone(TSMBuffer dest_bufp, TSMBuffer src_bufp, TSMLoc src_hdr, TSMLoc* locp)



Description

TSHttpHdrCopy

Copies the contents of the HTTP header located at :arg:‘src_loc‘ within :arg:‘src_bufp‘ to the HTTP header locatedat :arg:‘dest_loc‘ within :arg:‘dest_bufp‘.

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpHdrCopy(TSMBuffer dest_bufp, TSMLoc dest_offset, TSMBuffer src_bufp, TSM-Loc src_offset)

Description

TSHttpHdrCopy() works correctly even if :arg:‘src_bufp‘ and :arg:‘dest_bufp‘ point to different MarshalBuffers. Make sure that you create the destination HTTP header before copying into it.

: TSHttpHdrCopy() appends the port number to the domain of the URL portion of the header. For example, acopy of http://www.example.com appears as http://www.example.com:80 in the destination buffer.

TSHttpHdrCreate

Synopsis

#include <ts/ts.h>

TSMLoc TSHttpHdrCreate(TSMBuffer bufp)

Description

TSHttpHdrDestroy

Synopsis

#include <ts/ts.h>

void TSHttpHdrDestroy(TSMBuffer bufp, TSMLoc offset)

Description

Destroys the HTTP header located at :arg:‘hdr_loc‘ within the marshal buffer :arg:‘bufp‘.

: Do not forget to release the handle :arg:‘hdr_loc‘ with a call to TSHandleMLocRelease().


http://www.example.com

http://www.example.com:80


TSHttpHdrHostGet

Synopsis

#include <ts/ts.h>

const char* TSHttpHdrHostGet(TSMBuffer bufp, TSMLoc offset, int * length)

Description

Get the host for the request. :arg:‘bufp‘ and :arg:‘offset‘ must reference an HTTP request header. A pointer to thehost is returned and the length is stored in the int pointed at by :arg:‘length‘. Note the returned text may not be nullterminated. The URL in the request is checked first then the Host header field.

: This is much faster than calling TSHttpEffectiveUrlStringGet() and extracting the host from the result.

TSHttpHdrLengthGet

Synopsis

#include <ts/ts.h>

int TSHttpHdrLengthGet(TSMBuffer bufp, TSMLoc offset)

Description

TSHttpHdrMethodGet

Synopsis

#include <ts/ts.h>

const char* TSHttpHdrMethodGet(TSMBuffer bufp, TSMLoc offset, int * length)

Description

TSHttpHdrMethodSet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpHdrMethodSet(TSMBuffer bufp, TSMLoc offset, const char * value, int length)



Description

TSHttpHdrPrint

Synopsis

#include <ts/ts.h>

void TSHttpHdrPrint(TSMBuffer bufp, TSMLoc offset, TSIOBuffer iobufp)

Description

TSHttpHdrReasonGet

Synopsis

#include <ts/ts.h>

const char* TSHttpHdrReasonGet(TSMBuffer bufp, TSMLoc offset, int * length)

Description

TSHttpHdrReasonLookup

Synopsis

#include <ts/ts.h>

const char* TSHttpHdrReasonLookup(TSHttpStatus status)

Description

TSHttpHdrReasonSet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpHdrReasonSet(TSMBuffer bufp, TSMLoc offset, const char * value, int length)

Description

TSHttpHdrStatusGet

Synopsis

#include <ts/ts.h>

TSHttpStatus TSHttpHdrStatusGet(TSMBuffer bufp, TSMLoc offset)



Description

TSHttpHdrStatusSet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpHdrStatusSet(TSMBuffer bufp, TSMLoc offset, TSHttpStatus status)

Description

TSHttpHdrTypeGet

Synopsis

#include <ts/ts.h>

TSHttpType TSHttpHdrTypeGet(TSMBuffer bufp, TSMLoc offset)

Description

TSHttpHdrTypeSet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpHdrTypeSet(TSMBuffer bufp, TSMLoc offset, TSHttpType type)

Description

TSHttpHdrUrlGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpHdrUrlGet(TSMBuffer bufp, TSMLoc offset, TSMLoc * locp)

Description

TSHttpHdrUrlSet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpHdrUrlSet(TSMBuffer bufp, TSMLoc offset, TSMLoc url)



Description

TSHttpHdrVersionGet

Synopsis

#include <ts/ts.h>

int TSHttpHdrVersionGet(TSMBuffer bufp, TSMLoc offset)

Description

TSHttpHdrVersionSet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpHdrVersionSet(TSMBuffer bufp, TSMLoc offset, int ver)

Description

TSHttpHookAdd

Intercept Traffic Server events.

Synopsis

#include <ts/ts.h>

void TSHttpHookAdd(TSHttpHookID id, TSCont contp)

void TSHttpSsnHookAdd(TSHttpSsn ssnp, TSHttpHookID id, TSCont contp)

void TSHttpTxnHookAdd(TSHttpTxn txnp, TSHttpHookID id, TSCont contp)

Description

Hooks are points in Apache Traffic Server transaction HTTP processing where plugins can step in and do some work.Registering a plugin function for callback amounts to adding the function to a hook. You can register your plugin tobe called back for every single transaction, or for specific transactions only.

HTTP transaction hooks are set on a global basis using the function TSHttpHookAdd(). This means that thecontinuation specified as the parameter to TSHttpHookAdd() is called for every transaction. TSHttpHookAdd()is typically called from TSPluginInit() or TSRemapInit().

TSHttpSsnHookAdd() adds :arg:‘contp‘ to the end of the list of HTTP session hooks specified by :arg:‘id‘.This means that :arg:‘contp‘ is called back for every transaction within the session, at the point specified by thehook ID. Since :arg:‘contp‘ is added to a session, it is not possible to call TSHttpSsnHookAdd() from the plugininitialization routine; the plugin needs a handle to an HTTP session.



TSHttpTxnHookAdd() adds :arg:‘contp‘ to the end of the list of HTTP transaction hooks specified by :arg:‘id‘.Since :arg:‘contp‘ is added to a transaction, it is not possible to call TSHttpTxnHookAdd() from the plugininitialization routine but only when the plugin has a handle to an HTTP transaction.

Return Values

None. Adding hooks is always successful.

Examples

The following example demonstrates how to add global, session and transaction hooks:

#include <ts/ts.h>

static inthandler(TSCont contp, TSEvent event, void *edata){

TSHttpSsn ssnp;TSHttpTxn txnp;

switch (event){case TS_EVENT_HTTP_SSN_START:

ssnp = (TSHttpSsn) edata;// Add a session hook ...TSHttpSsnHookAdd(ssnp, TS_HTTP_TXN_START_HOOK, contp);TSHttpSsnReenable(ssnp, TS_EVENT_HTTP_CONTINUE);return 0;

case TS_EVENT_HTTP_TXN_START:txnp = (TSHttpTxn) edata;// Add a transaction hook ...TSHttpTxnHookAdd(txnp, TS_HTTP_READ_REQUEST_HDR_HOOK, contp);TSHttpTxnReenable(txnp, TS_EVENT_HTTP_CONTINUE);return 0;

default:break;

}

return 0;}


TSCont contp;contp = TSContCreate(handler, NULL);TSHttpHookAdd(TS_HTTP_SSN_START_HOOK, contp);

}

See Also

TSAPI(3ts), TSContCreate(3ts), TSLifecycleHookAdd(3ts)



TSHttpIsInternalRequest

Test whether a request is internally-generated.

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpIsInternalRequest(TSHttpTxn txnp)

TSReturnCode TSHttpIsInternalSession(TSHttpSsn ssnp)

Description

TSHttpIsInternalRequest() tests whether a HTTP transaction was originated within Traffic Server.

Deprecated

Deprecated in favor of TSHttpTxnIsInternal().

TSHttpIsInternalSession() tests whether a HTTP session was originated within Traffic Server.

Deprecated

Deprecated in favor of TSHttpSsnIsInternal().

Return Values

Both these APIs returns a TSReturnCode, indicating whether the object was internal (TS_SUCCESS) or not(TS_ERROR).

See Also

TSAPI(3ts), TSHttpSsnIsInternal(3ts), TSHttpTxnIsInternal(3ts)

TSHttpOverridableConfig

Synopsis

#include <ts/ts.h>


TSReturnCode TSHttpTxnConfigIntSet(TSHttpTxn txnp, TSOverridableConfigKey key,TSMgmtInt value)

TSReturnCode TSHttpTxnConfigIntGet(TSHttpTxn txnp, TSOverridableConfigKey key,TSMgmtInt* value)

TSReturnCode TSHttpTxnConfigFloatSet(TSHttpTxn txnp, TSOverridableConfigKey key, TSMgmt-Float value)



TSReturnCode TSHttpTxnConfigFloatGet(TSHttpTxn txnp, TSOverridableConfigKey key, TSMgmt-Float* value)

TSReturnCode TSHttpTxnConfigStringSet(TSHttpTxn txnp, TSOverridableConfigKey key, constchar* value, int length)

TSReturnCode TSHttpTxnConfigStringGet(TSHttpTxn txnp, TSOverridableConfigKey key, constchar** value, int* length)

TSReturnCode TSHttpTxnConfigFind(const char* name, int length, TSOverridableConfigKey* key,TSRecordDataType* type)

Description

Some of the values that are set in records.config can be changed for a specific transaction. It is important to notethat these functions change the configuration values stored for the transation, which is not quite the same as changingthe actual operating values of the transaction. The critical effect is the value must be changed before it is used by thetransaction - after that, changes will not have any effect.

All of the ...Get functions store the internal value in the storage indicated by the :arg:‘value‘ argument. For strings:arg:‘length*‘ will receive the length of the string.

The values are identified by the enumeration TSOverridableConfigKey . String values can be used indirectlyby first passing them to TSHttpTxnConfigFind() which, if the string matches an overridable value, return thekey and data type.

Configurations

The following configurations (from records.config) are overridable.

:ts:cv:‘proxy.config.url_remap.pristine_host_hdr‘:ts:cv:‘proxy.config.http.chunking_enabled‘:ts:cv:‘proxy.config.http.negative_caching_enabled‘:ts:cv:‘proxy.config.http.negative_caching_lifetime‘:ts:cv:‘proxy.config.http.cache.when_to_revalidate‘:ts:cv:‘proxy.config.http.keep_alive_enabled_in‘:ts:cv:‘proxy.config.http.keep_alive_enabled_out‘:ts:cv:‘proxy.config.http.keep_alive_post_out‘:ts:cv:‘proxy.config.net.sock_recv_buffer_size_out‘:ts:cv:‘proxy.config.net.sock_send_buffer_size_out‘:ts:cv:‘proxy.config.net.sock_option_flag_out‘:ts:cv:‘proxy.config.http.forward.proxy_auth_to_parent‘:ts:cv:‘proxy.config.http.anonymize_remove_from‘:ts:cv:‘proxy.config.http.anonymize_remove_referer‘:ts:cv:‘proxy.config.http.anonymize_remove_user_agent‘:ts:cv:‘proxy.config.http.anonymize_remove_cookie‘:ts:cv:‘proxy.config.http.anonymize_remove_client_ip‘:ts:cv:‘proxy.config.http.anonymize_insert_client_ip‘:ts:cv:‘proxy.config.http.response_server_enabled‘:ts:cv:‘proxy.config.http.insert_squid_x_forwarded_for‘:ts:cv:‘proxy.config.http.server_tcp_init_cwnd‘



:ts:cv:‘proxy.config.http.send_http11_requests‘:ts:cv:‘proxy.config.http.cache.http‘:ts:cv:‘proxy.config.http.cache.cluster_cache_local‘:ts:cv:‘proxy.config.http.cache.ignore_client_no_cache‘:ts:cv:‘proxy.config.http.cache.ignore_client_cc_max_age‘:ts:cv:‘proxy.config.http.cache.ims_on_client_no_cache‘:ts:cv:‘proxy.config.http.cache.ignore_server_no_cache‘:ts:cv:‘proxy.config.http.cache.cache_responses_to_cookies‘:ts:cv:‘proxy.config.http.cache.ignore_authentication‘:ts:cv:‘proxy.config.http.cache.cache_urls_that_look_dynamic‘:ts:cv:‘proxy.config.http.cache.required_headers‘:ts:cv:‘proxy.config.http.insert_request_via_str‘:ts:cv:‘proxy.config.http.insert_response_via_str‘:ts:cv:‘proxy.config.http.cache.heuristic_min_lifetime‘:ts:cv:‘proxy.config.http.cache.heuristic_max_lifetime‘:ts:cv:‘proxy.config.http.cache.guaranteed_min_lifetime‘:ts:cv:‘proxy.config.http.cache.guaranteed_max_lifetime‘:ts:cv:‘proxy.config.http.cache.max_stale_age‘:ts:cv:‘proxy.config.http.keep_alive_no_activity_timeout_in‘:ts:cv:‘proxy.config.http.keep_alive_no_activity_timeout_out‘:ts:cv:‘proxy.config.http.transaction_no_activity_timeout_in‘:ts:cv:‘proxy.config.http.transaction_no_activity_timeout_out‘:ts:cv:‘proxy.config.http.transaction_active_timeout_out‘:ts:cv:‘proxy.config.websocket.no_activity_timeout‘:ts:cv:‘proxy.config.websocket.active_timeout‘:ts:cv:‘proxy.config.http.origin_max_connections‘:ts:cv:‘proxy.config.http.connect_attempts_max_retries‘:ts:cv:‘proxy.config.http.connect_attempts_max_retries_dead_server‘:ts:cv:‘proxy.config.http.connect_attempts_rr_retries‘:ts:cv:‘proxy.config.http.connect_attempts_timeout‘:ts:cv:‘proxy.config.http.post_connect_attempts_timeout‘:ts:cv:‘proxy.config.http.down_server.cache_time‘:ts:cv:‘proxy.config.http.down_server.abort_threshold‘:ts:cv:‘proxy.config.http.cache.fuzz.time‘:ts:cv:‘proxy.config.http.cache.fuzz.min_time‘:ts:cv:‘proxy.config.http.doc_in_cache_skip_dns‘:ts:cv:‘proxy.config.http.background_fill_active_timeout‘:ts:cv:‘proxy.config.http.response_server_str‘:ts:cv:‘proxy.config.http.cache.heuristic_lm_factor‘:ts:cv:‘proxy.config.http.cache.fuzz.probability‘:ts:cv:‘proxy.config.http.background_fill_completed_threshold‘:ts:cv:‘proxy.config.net.sock_packet_mark_out‘:ts:cv:‘proxy.config.net.sock_packet_tos_out‘:ts:cv:‘proxy.config.http.insert_age_in_response‘:ts:cv:‘proxy.config.http.chunking.size‘:ts:cv:‘proxy.config.http.flow_control.enabled‘:ts:cv:‘proxy.config.http.flow_control.low_water‘



:ts:cv:‘proxy.config.http.flow_control.high_water‘:ts:cv:‘proxy.config.http.cache.range.lookup‘:ts:cv:‘proxy.config.http.normalize_ae_gzip‘:ts:cv:‘proxy.config.http.default_buffer_size‘:ts:cv:‘proxy.config.http.default_buffer_water_mark‘:ts:cv:‘proxy.config.http.request_header_max_size‘:ts:cv:‘proxy.config.http.response_header_max_size‘:ts:cv:‘proxy.config.http.negative_revalidating_enabled‘:ts:cv:‘proxy.config.http.negative_revalidating_lifetime‘:ts:cv:‘proxy.config.http.accept_encoding_filter_enabled‘:ts:cv:‘proxy.config.http.cache.range.write‘:ts:cv:‘proxy.config.http.global_user_agent_header‘:ts:cv:‘proxy.config.http.slow.log.threshold‘:ts:cv:‘proxy.config.http.cache.generation‘:ts:cv:‘proxy.config.body_factory.template_base‘:ts:cv:‘proxy.config.http.cache.open_write_fail_action‘:ts:cv:‘proxy.config.http.redirection_enabled‘:ts:cv:‘proxy.config.http.number_of_redirections‘:ts:cv:‘proxy.config.http.cache.max_open_write_retries‘:ts:cv:‘proxy.config.http.redirect_use_orig_cache_key‘

Examples

Enable transaction buffer control with a high water mark of 262144 and a low water mark of 65536.

int callback(TSCont contp, TSEvent event, void* data){

TSHttpTxn txnp = static_cast<TSHttpTxn>(data);TSHttpTxnConfigIntSet(txnp, TS_CONFIG_HTTP_FLOW_CONTROL_ENABLED, 1);TSHttpTxnConfigIntSet(txnp, TS_CONFIG_HTTP_FLOW_CONTROL_HIGH_WATER_MARK, 262144);TSHttpTxnConfigIntSet(txnp, TS_CONFIG_HTTP_FLOW_CONTROL_LOWER_WATER_MARK, 65536);return 0;

}

See Also

TSAPI(3ts)

TSHttpParserCreate

Parse HTTP headers from memory buffers.

Synopsis

#include <ts/ts.h>

TSHttpParser TSHttpParserCreate(void)



void TSHttpParserClear(TSHttpParser parser)

void TSHttpParserDestroy(TSHttpParser parser)

TSParseResult TSHttpHdrParseReq(TSHttpParser parser, TSMBuffer bufp, TSMLoc offset, const char** start, const char * end)

TSParseResult TSHttpHdrParseResp(TSHttpParser parser, TSMBuffer bufp, TSMLoc offset, const char** start, const char * end)

Description

TSHttpParserCreate() creates an HTTP parser object. The parser’s data structure contains information aboutthe header being parsed. A single HTTP parser can be used multiple times, though not simultaneously. Before beingused again, the parser must be cleared by calling TSHttpParserClear().

TSHttpHdrParseReq() parses an HTTP request header. The HTTP header :arg:‘offset‘ must already be created,and must reside inside the marshal buffer :arg:‘bufp‘. The :arg:‘start‘ argument points to the current position of thestring buffer being parsed and the :arg:‘end‘ argument points to one byte after the end of the buffer to be parsed. Onreturn, :arg:‘start‘ is modified to point past the last character parsed.

It is possible to parse an HTTP request header a single byte at a time using repeated calls toTSHttpHdrParseReq(). As long as an error does not occur, the TSHttpHdrParseReq() func-tion will consume that single byte and ask for more. TSHttpHdrParseReq() should be called afterTS_HTTP_READ_REQUEST_HDR_HOOK.

TSHttpHdrParseResp() operates in the same manner as TSHttpHdrParseReq() except it parses an HTTPresponse header. It should be called after TS_HTTP_READ_RESPONSE_HDR_HOOK.

TSHttpParserClear() clears the specified HTTP parser so it may be used again.

TSHttpParserDestroy() destroys the TSHttpParser object pointed to by :arg:‘parser‘. The :arg:‘parser‘pointer must not be NULL.

Return Salues

TSHttpHdrParseReq() and TSHttpHdrParseResp() both return a TSParseResult value.TS_PARSE_ERROR is returned on error, TS_PARSE_CONT is returned if parsing of the header stopped be-cause the end of the buffer was reached, and TS_PARSE_DONE or TS_PARSE_OK when a \r\n\r\n pattern isencountered, indicating the end of the header.

Known Bugs

The distinction between the TS_PARSE_DONE and TS_PARSE_OK results is not well-defined. Plugins should expectboth status codes and treat them equivalently.

See Also

TSAPI(3ts)



TSHttpSsnClientFdGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpSsnClientFdGet(TSHttpTxn txnp, int *fdp)

Description

Get the socket descriptor for the client socket in :arg:‘fdp‘ for the transaction :arg:‘ssn‘.

TSHttpSsnReenable

Synopsis

#include <ts/ts.h>

void TSHttpSsnReenable(TSHttpSsn ssnp, TSEvent event)

Description

TSHttpTxnCacheLookupStatusGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnCacheLookupStatusGet(TSHttpTxn txnp, int * lookup_status)

Description

TSHttpTxnCacheLookupUrlGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnCacheLookupUrlGet(TSHttpTxn txnp, TSMBuffer bufp, TSMLoc offset)

Description

Get the current cache key URL, also referred to as the lookup URL. This must be stored in a properly allocated URLobject, typically created with a TSUrlCreate().



TSHttpTxnCacheLookupUrlSet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnCacheLookupUrlSet(TSHttpTxn txnp, TSMBuffer bufp, TSMLoc offset)

Description

Set the current cache key URL, also referred to as the lookup URL. This must be stored in a properly allocated URLobject, typically created with a TSUrlCreate() or TSUrlClone().

This API can be called as early as TS_HTTP_READ_REQUEST_HDR_HOOK but no later thanTS_HTTP_POST_REMAP_HOOK. This is the preferred and most efficient way to modify the cache key, butan alternative is to use the old TSCacheUrlSet(), which takes a simple string as argument.

TSHttpTxnCachedReqGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnCachedReqGet(TSHttpTxn txnp, TSMBuffer * bufp, TSMLoc * offset)

Description

TSHttpTxnCachedRespGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnCachedRespGet(TSHttpTxn txnp, TSMBuffer * bufp, TSMLoc * offset)

Description

TSHttpTxnClientFdGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnClientFdGet(TSHttpTxn txnp, int *fdp)

Description

Get the socket descriptor for the client socket in :arg:‘fdp‘ for the transaction :arg:‘txnp‘.



TSHttpTxnClientPacketDscpSet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnClientPacketDscpSet(TSHttpTxn txnp, int dscp)

Description

Change packet :arg:‘dscp‘ for the client side connection.

: Changes take effect immediately.

See Also

Traffic Shaping

TSHttpTxnClientPacketMarkSet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnClientPacketMarkSet(TSHttpTxn txnp, int mark)

Description

Change packet firewall :arg:‘mark‘ for the client side connection.

: Changes take effect immediately.

See Also

TSHttpTxnClientPacketTosSet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnClientPacketTosSet(TSHttpTxn txnp, int tos)


https://cwiki.apache.org/confluence/display/TS/Traffic+Shaping


Description

Change packet :arg:‘tos‘ for the client side connection.

: Changes take effect immediately

: TOS is deprecated and replaced by DSCP. This is still used to set DSCP, however the first 2 bits of this value willbe ignored as they now belong to the ECN field.

See Also

TSHttpTxnClientReqGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnClientReqGet(TSHttpTxn txnp, TSMBuffer * bufp, TSMLoc * offset)

Description

TSHttpTxnClientRespGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnClientRespGet(TSHttpTxn txnp, TSMBuffer * bufp, TSMLoc * offset)

Description

TSHttpTxnErrorBodySet

Sets an error type body to a transaction.

Synopsis

#include <ts/ts.h>

void TSHttpTxnErrorBodySet(TSHttpTxn txnp, char * buf, size_t buflength, char * mimetype)

Description

Note that both string arguments must be allocated with TSmalloc() or TSstrdup(). The :arg:‘mimetype‘ isoptional, and if not provided it defaults to text/html. Sending an empty string would prevent setting a content typeheader (but that is not adviced).



TSHttpTxnIncomingAddrGet

Get the incoming proxy address on which a client’s connection is accepted.

Synopsis

#include <ts/ts.h>

sockaddr const* TSHttpTxnIncomingAddrGet(TSHttpTxn txnp)

Description

: The pointer is valid only for the current callback. Clients that need to keep the value across callbacks must maintaintheir own storage.

TSHttpTxnInfoIntGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnInfoIntGet(TSHttpTxn txnp, TSHttpTxnInfoKey key, TSMgmtInt *value)

Description

TSHttpTxnInfoIntGet() returns arbitrary integer-typed info about a transaction as defined inTSHttpTxnInfoKey. The API will be part of a generic API umbrella that can support returning arbitraryinfo about a transaction using custom log tags. It works on multiple hooks depending on the requested info. Forexample, cache related info may be available only at or after TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOKhook.

The TSHttpTxnInfoKey currently supports the below integer-based info about a transaction

TS_TXN_INFO_CACHE_HIT_RAM

This info is available at or after TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK hook. Avalue of 1 indicates that the response is returned from RAM cache. A value of 0 indicatesotherwise.

TS_TXN_INFO_CACHE_COMPRESSED_IN_RAM

This info is available at or after TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK hook. Avalue of 1 indicates that the response is returned from RAM cache and is compressed. A valueof 0 indicates otherwise.

TS_TXN_INFO_CACHE_HIT_RWW

This info is available at or after TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK hook. Avalue of 1 indicates that the response is returned via Read-While-Writer functionality. A valueof 0 indicates otherwise.

TS_TXN_INFO_CACHE_OPEN_READ_TRIES



This info is available at or after TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK hook. Thevalue indicates the number of cache open read reattempts made by the transaction on cacheopen read failure.

TS_TXN_INFO_CACHE_OPEN_WRITE_TRIES

This info is available at or after TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK hook. Thevalue indicates the number of cache open write reattempts made by the transaction on cacheopen write failure.

TS_TXN_INFO_CACHE_VOLUME

This info is available at or after TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK hook. Thevalue indicates the cache volume ID used for the cache object associated with the transaction.

Return values

The API returns TS_SUCCESS, if the requested info is supported, TS_ERROR otherwise.

TSHttpTxnIntercept

Allows a plugin take over the servicing of the request as though it was the origin server.

Synopsis

#include <ts/ts.h>

void TSHttpTxnIntercept(TSCont contp, TSHttpTxn txnp)

Description

:arg:‘contp‘ will be sent TS_EVENT_NET_ACCEPT. The edata passed with TS_NET_EVENT_ACCEPT is anTSVConn just as it would be for a normal accept. The plugin must act as if it is an HTTP server and read theHTTP request and body off the TSVConn and send an HTTP response header and body.

TSHttpTxnIntercept() must be called be called from only TS_HTTP_READ_REQUEST_HOOK. UsingTSHttpTxnIntercept will bypass the Traffic Server cache. If response sent by the plugin should be cached, useTSHttpTxnServerIntercept() instead. TSHttpTxnIntercept() primary use is allow plugins to servedata about their functioning directly.

TSHttpTxnIntercept() must only be called once per transaction.

TSHttpTxnIsInternal

Test whether a request is internally-generated.

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnIsInternal(TSHttpTxn txnp)

TSReturnCode TSHttpSsnIsInternal(TSHttpSsn ssnp)



Description

TSHttpTxnIsInternal() tests whether a HTTP transaction was originated within Traffic Server.

TSHttpSsnIsInternal() tests whether a HTTP session was originated within Traffic Server.

Return Values

Both these APIs returns a TSReturnCode, indicating whether the object was internal (TS_SUCCESS) or not(TS_ERROR).

Examples

The ESI plugin uses TSHttpTxnIsInternal() to ignore requests that is had generated while fetching portionsof an ESI document:

See also

TSAPI(3ts)

TSHttpTxnMilestoneGet

Get a specified :arg:‘milestone‘ timer value for the current transaction.

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnMilestoneGet(TSHttpTxn txnp, TSMilestonesType milestone, TSHRTime* time)

Description

TSHttpTxnMilestoneGet() will fetch a specific :arg:‘milestone‘ timer value for the transaction :arg:‘txnp‘.These timers are calculated during the lifetime of a transaction and are measured in nanoseconds from the beginningof the transaction. :arg:‘time‘ is used a pointer to storage to update if the call is successful.

TSMilestonesType



Value Milestone

:const:‘TS_MILESTONE_UA_BEGIN‘The client connection is accepted.

:const:‘TS_MILESTONE_UA_READ_HEADER_DONE‘The request header from the client has been read andparsed.

:const:‘TS_MILESTONE_UA_BEGIN_WRITE‘The response header write to the client starts.

:const:‘TS_MILESTONE_UA_CLOSE‘Last I/O activity on the client socket, or connection abort.

:const:‘TS_MILESTONE_SERVER_FIRST_CONNECT‘First time origin server connect attempted or shared sharedsession attached.

:const:‘TS_MILESTONE_SERVER_CONNECT‘Most recent time origin server connect attempted orshared session attached.

:const:‘TS_MILESTONE_SERVER_CONNECT_END‘More recent time a connection attempt was resolved.

:const:‘TS_MILESTONE_SERVER_BEGIN_WRITE‘First byte is written to the origin server connection.

:const:‘TS_MILESTONE_SERVER_FIRST_READ‘First byte is read from connection to origin server.

:const:‘TS_MILESTONE_SERVER_READ_HEADER_DONE‘Origin server response has been read and parsed.

:const:‘TS_MILESTONE_SERVER_CLOSE‘Last I/O activity on origin server connection.

:const:‘TS_MILESTONE_CACHE_OPEN_READ_BEGIN‘Initiate read of the cache.

:const:‘TS_MILESTONE_CACHE_OPEN_READ_END‘Initial cache read has resolved.

:const:‘TS_MILESTONE_CACHE_OPEN_WRITE_BEGIN‘Start open for cache write.

:const:‘TS_MILESTONE_CACHE_OPEN_WRITE_END‘Cache has been opened for write.

:const:‘TS_MILESTONE_DNS_LOOKUP_BEGIN‘Initiate host resolution in HostDB

:const:‘TS_MILESTONE_DNS_LOOKUP_END‘Host resolution resolves.

:const:‘TS_MILESTONE_SM_START‘Transaction state machine is initialized.

:const:‘TS_MILESTONE_SM_FINISH‘Transaction has finished, state machine final logging hasstarted.

:const:‘TS_MILESTONE_PLUGIN_ACTIVE‘Amount of time plugins were active plus start time.

:const:‘TS_MILESTONE_PLUGIN_TOTAL‘Wall time while plugins were active plus start time.

• The server connect times predate the transmission of the :literal:‘SYN packet. That is, before a connection tothe origin server is completed.

• A connection attempt is resolved when no more connection related activity remains to be done, and the connec-tion is either established or has failed.

• :const:‘TS_MILESTONE_UA_CLOSE‘ and :const:‘TS_MILESTONE_SERVER_CLOSE‘ are updatedcontinuously during the life of the transaction, every time there is I/O activity. The updating stops when thecorresponding connection is closed, leaving the last I/O time as the final value.

• The cache OPEN milestones time only the initial setup, the open, not the full read or write.

• :const:‘TS_MILESTONE_PLUGIN_ACTIVE‘ and :const:‘TS_MILESTONE_PLUGIN_TOTAL‘ are dif-ferent from the other milestones as they measure elapsed time, not event time. The value is the elapsedtime plus :const:‘TS_MILESTONE_SM_START‘. This was decided to be more convenient because thenthese milestones can be handled / displayed in the same way as the other milestones, as offsets from:const:‘TS_MILESTONE_SM_START‘.

:const:‘TS_MILESTONE_PLUGIN_ACTIVE‘ value is the amount of time the plugin was active, thatis performing computation. :const:‘TS_MILESTONE_PLUGIN_TOTAL‘ is the wall time which in-



cludes any time the transaction was blocked while a plugin was active. For instance if a plugin waits onan external event, that waiting time will be in :const:‘TS_MILESTONE_PLUGIN_TOTAL‘ but not in:const:‘TS_MILESTONE_PLUGIN_ACTIVE‘.

Return Values

:const:‘TS_SUCCESS‘ if successful and :arg:‘time‘ was updated, otherwise :const:‘TS_ERROR‘.

See Also

TSAPI(3ts)

TSHttpTxnNextHopAddrGet

Synopsis

#include <ts/ts.h>

sockaddr const * TSHttpTxnNextHopAddrGet(TSHttpTxn txnp)

Description

Get the next hop address for transaction :arg:‘txnp‘.


TSHttpTxnOutgoingAddrGet

Get the outgoing address used in origin connection.

Synopsis

#include <ts/ts.h>

sockaddr const* TSHttpTxnOutgoingAddrGet(TSHttpTxn txnp)

Description




TSHttpTxnParentProxySet

Sets the parent proxy :arg:‘hostname‘ and :arg:‘port‘.

Synopsis

#include <ts/ts.h>

void TSHttpTxnParentProxySet(TSHttpTxn txnp, const char * hostname, int port)

Description

The string :arg:‘hostname‘ is copied into the TSHttpTxn. You can modify or delete the string after callingTSHttpTxnParentProxySet().

TSHttpTxnReenable

Notifies the HTTP transaction txnp that the plugin is finished processing the current hook.

Synopsis

#include <ts/ts.h>

void TSHttpTxnReenable(TSHttpTxn txnp, TSEvent event)

Description

The plugin tells the transaction :arg:‘txnp‘ to either continue (TS_EVENT_HTTP_CONTINUE) or stop(TS_EVENT_HTTP_ERROR).

: You must always reenable the HTTP transaction after the processing of each transaction event. However, neverreenable twice. Reenabling twice is a serious error.

TSHttpTxnServerAddrGet

Synopsis

#include <ts/ts.h>

sockaddr const* TSHttpTxnServerAddrGet(TSHttpTxn txnp)

Description

Get the origin server address for transaction :arg:‘txnp‘.




TSHttpTxnServerAddrSet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnServerAddrGet(TSHttpTxn txnp, struct sockaddr const* addr)

Description

Set the origin server address for transaction :arg:‘txnp‘. This includes the port in :arg:‘addr‘. The address family isalso set by the contents of :arg:‘addr‘. The address data is copied out of :arg:‘addr‘ so there is no dependency onthe lifetime of that object.

This hook must be called no later than TS_HTTP_OS_DNS_HOOK. If this is called then DNS resolution will not bedone as the address of the server is already know.

An error value is returned if :arg:‘addr‘ does not contain a valid IPv4 or IPv6 address with a valid (non-zero) port.

TSHttpTxnServerFdGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnServerFdGet(TSHttpTxn txnp, int *fdp)

Description

Get the socket descriptor for the server socket in :arg:‘fdp‘ for the transaction :arg:‘txnp‘.

TSHttpTxnServerIntercept

Intercept origin server requests.

Synopsis

#include <ts/ts.h>

void TSHttpTxnServerIntercept(TSCont contp, TSHttpTxn txnp)



Description

TSHttpTxnServerIntercept() allows a plugin take over the servicing of the request as though it was the originserver. In the event a request needs to be made to the server for transaction :arg:‘txnp‘, :arg:‘contp‘ will be sent aTS_EVENT_NET_ACCEPT event. The :arg:‘edata‘ passed with TS_NET_EVENT_ACCEPT is an TSVConn just asit would be for a normal accept. The plugin must act as if it is an HTTP server and read the HTTP request and bodyfrom the TSVConn and send an HTTP response header and body.

TSHttpTxnServerIntercept() must be not be called after the connection to the server has taken place. Thismeans that the last hook that it can be called from is TS_HTTP_READ_CACHE_HDR_HOOK. If a connection to theserver is not necessary, the continuation :arg:‘contp‘ will be sent a TS_EVENT_NET_ACCEPT_FAILED event whenthe transaction completes.

The response from the plugin is cached subject to standard and configured HTTP caching rules. Should the pluginwish the response not be cached, the plugin must use appropriate HTTP response headers to prevent caching. Theprimary purpose of TSHttpTxnServerIntercept() is allow plugins to provide gateways to other protocols orto allow to plugin to its own transport for the next hop to the server. TSHttpTxnServerIntercept() overridesparent cache configuration.

TSHttpTxnServerIntercept() must only be called once per transaction. The continuation :arg:‘contp‘ musthave been created with a valid TSMutex.

See Also

TSAPI(3ts)

TSHttpTxnServerPacketDscpSet

Change packet DSCP for the server side connection.

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnServerPacketDscpSet(TSHttpTxn txnp, int dscp)

Description

: The change takes effect immediately. If no OS connection has been made, then this sets the mark that will be used.If an OS connection is established

See Also

Traffic Shaping

TSHttpTxnServerPacketMarkSet

Change packet firewall mark for the server side connection.


https://cwiki.apache.org/confluence/display/TS/Traffic+Shaping


Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnServerPacketMarkSet(TSHttpTxn txnp, int mark)

Description


See Also

TSHttpTxnServerPacketTosSet

Change packet TOS for the server side connection.

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnServerPacketTosSet(TSHttpTxn txnp, int tos)

Description


TOS is deprecated and replaced by DSCP. This is still used to set DSCP, however the first 2 bits of this value will beignored as they now belong to the ECN field.

See Also

TSHttpTxnServerReqGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnServerReqGet(TSHttpTxn txnp, TSMBuffer * bufp, TSMLoc * offset)



Description

TSHttpTxnServerRespGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnServerRespGet(TSHttpTxn txnp, TSMBuffer * bufp, TSMLoc * offset)

Description

TSHttpTxnSsnGet

Synopsis

#include <ts/ts.h>

TSHttpSsn TSHttpTxnSsnGet(TSHttpTxn txnp)

Description

TSHttpTxnTransformRespGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSHttpTxnTransformRespGet(TSHttpTxn txnp, TSMBuffer * bufp, TSMLoc * offset)

Description

TSHttpTxnTransformedRespCache

Synopsis

#include <ts/ts.h>

void TSHttpTxnTransformedRespCache(TSHttpTxn txnp, int on)

Description

TSHttpTxnUntransformedRespCache

Synopsis

#include <ts/ts.h>

void TSHttpTxnUntransformedRespCache(TSHttpTxn txnp, int on)



Description

TSIOBufferBlockReadStart

Synopsis

#include <ts/ts.h>

const char * TSIOBufferBlockReadStart(TSIOBufferBlock blockp, TSIOBufferReader readerp, int64_t* avail)

Description

TSIOBufferCopy

Synopsis

#include <ts/ts.h>

int64_t TSIOBufferCopy(TSIOBuffer bufp, TSIOBufferReader readerp, int64_t length, int64_t offset)

Description

TSIOBufferCreate

Traffic Server IO buffer API.

Synopsis

#include <ts/ts.h>

TSIOBuffer TSIOBufferCreate(void)

TSIOBuffer TSIOBufferSizedCreate(TSIOBufferSizeIndex index)

void TSIOBufferDestroy(TSIOBuffer bufp)

int64_t TSIOBufferWrite(TSIOBuffer bufp, const void * buf, int64_t length)

void TSIOBufferProduce(TSIOBuffer bufp, int64_t nbytes)

int64_t TSIOBufferWaterMarkGet(TSIOBuffer bufp)

void TSIOBufferWaterMarkSet(TSIOBuffer bufp, int64_t water_mark)

Description

The TSIOBuffer data structure is the building block of the TSVConn abstraction. An IO buffer is composed ofa list of buffer blocks which are reference counted so that they can reside in multiple buffers at the same time. Thismakes it extremely efficient to copy data from one IO buffer to another using TSIOBufferCopy() since TrafficServer only needs to copy pointers and adjust reference counts appropriately and not actually copy any data. However,applications should still strive to ensure data blocks are a reasonable size.



The IO buffer abstraction provides for a single writer and multiple readers. In order for the readers to have no knowl-edge of each other, they manipulate IO buffers through the TSIOBufferReader data structure. Since only a singlewriter is allowed, there is no corresponding TSIOBufferWriter data structure. The writer simply modifies the IObuffer directly.

TSIOBufferCreate() creates an empty TSIOBuffer.

TSIOBufferSizedCreate() creates an empty TSIOBuffer with an initial capacity of :arg:‘index‘ bytes.

TSIOBufferDestroy() destroys the IO buffer :arg:‘bufp‘. Since multiple IO buffers can share data, this does notnecessarily free all of the data associated with the IO buffer but simply decrements the appropriate reference counts.

TSIOBufferWrite() appends :arg:‘length‘ bytes from the buffer :arg:‘buf‘ to the IO buffer :arg:‘bufp‘ andreturns the number of bytes successfully written into the IO buffer.

TSIOBufferProduce() makes :arg:‘nbytes‘ of data available for reading in the IO buffer :arg:‘bufp‘. A com-mon pattern for writing to an IO buffer is to copy data into a buffer block and then call INKIOBufferProduce to makethe new data visible to any readers.

The watermark of an TSIOBuffer is the minimum number of bytes of data that have to be in the buffer before callingback any continuation that has initiated a read operation on this buffer. As a writer feeds data into the TSIOBuffer,no readers are called back until the amount of data reaches the watermark. Setting a watermark can improve perfor-mance because it avoids frequent callbacks to read small amounts of data. TSIOBufferWaterMarkGet() getsthe current watermark for the IO buffer :arg:‘bufp‘.

TSIOBufferWaterMarkSet() gets the current watermark for the IO buffer :arg:‘bufp‘ to :arg:‘water_mark‘bytes.

See Also

TSAPI(3ts), TSIOBufferReaderAlloc(3ts)

TSInstallDirGet

Return Traffic Server installation directories.

Synopsis

#include <ts/ts.h>

const char * TSInstallDirGet(void)

const char * TSConfigDirGet(void)

const char * TSPluginDirGet(void)

Description

TSInstallDirGet() returns the path to the root of the Traffic Server installation. TSConfigDirGet() andTSPluginDirGet() return the complete, absolute path to the configuration directory and the plugin installationdirectory, respectively.

Return Values

These functions all return a NUL-terminated string that must not be modified or freed.



Examples

To load a file that is located in the Traffic Server configuration directory:

#include <ts/ts.h>#include <stdio.h>

char * path;asprintf(&path, "%s/example.conf", TSConfigDirGet());

See Slso

TSAPI(3ts)

TSLifecycleHookAdd

Synopsis

#include <ts/ts.h>

void TSLifecycleHookAdd(TSLifecycleHookID id, TSCont contp)

Description

TSLifecycleHookAdd() adds :arg:‘contp‘ to the list of lifecycle hooks specified by :arg:‘id‘. Lifecycle hooksare based on the Traffic Server process, not on any specific transaction or session. These will typically be called onlyonce during the execution of the Traffic Server process and therefore should be added in TSPluginInit() (whichcould itself be considered a lifecyle hook). Unlike other hooks, lifecycle hooks may not have a well defined orderingand use of them should not assume that one of the hooks is always called before another unless specifically mentioned.

TS_LIFECYCLE_PORTS_INITIALIZED_HOOKCalled after the :ts:cv:‘proxy server port <proxy.config.http.server_ports>‘ data structures have been initial-ized but before connections are accepted on those ports. The sockets corresponding to the ports may or may notbe open depending on how the traffic_server process was invoked. Other API functions that depend onserver ports should be called from this hook and not TSPluginInit().

Invoked with the event TS_EVENT_LIFECYCLE_PORTS_INITIALIZED and NULL data.

TS_LIFECYCLE_PORTS_READY_HOOKCalled after enabling connections on the proxy server ports. Because Traffic Server is threaded this may or maynot be called before any connections are accepted. The hook code may assume that any connection to TrafficServer started after this hook is called will be accepted by Traffic Server, making this a convenient place tosignal external processes of that.

Invoked with the event TS_EVENT_LIFECYCLE_PORTS_READY and NULL data.

TS_LIFECYCLE_CACHE_READY_HOOKCalled after Traffic Server cache initialization has finished.

Invoked with the event TS_EVENT_LIFECYCLE_CACHE_READY and NULL data.



Ordering

TS_LIFECYCLE_PORTS_INITIALIZED_HOOK will always be called beforeTS_LIFECYCLE_PORTS_READY_HOOK.

Examples

The following example demonstrates how to correctly use TSNetAcceptNamedProtocol(), which requires theproxy ports to be initialized and therefore does not work if called from TSPluginInit() directly.

#include <ts/ts.h>

#define SSL_PROTOCOL_NAME "whatever"

static intssl_proto_handler(TSCont contp, TSEvent event, void* data){

/// Do named protocol handling.}

static intlocal_ssl_init(TSCont contp, TSEvent event, void * edata){

if (TS_EVENT_LIFECYCLE_PORTS_INITIALIZED == event) { // just to be safe.TSNetAcceptNamedProtocol(

TSContCreate(ssl_proto_handler, TSMutexCreate()),SSL_PROTOCOL_NAME

);}return 0;

}

voidTSPluginInit (int argc, const char * argv[]){

TSLifecycleHookAdd(TS_LIFECYCLE_PORTS_INITIALIZED_HOOK, TSContCreate(local_ssl_→˓init, NULL));}

History

Lifecycle hooks were introduced to solve process initialization ordering issues (TS-1487). Different API calls requireddifferent modules of Traffic Server to be initialized for the call to work, but others did not work that late in initialization,which was problematic because all of them could effectively only be called from TSPluginInit() . The solutionwas to move TSPluginInit() as early as possible in the process initialization and provide hooks for API calls thatneeded to be invoked later which served essentially as additional pluging initialization points.

See Also

TSAPI(3ts), TSContCreate(3ts)



TSMBufferCreate

Traffic Server marshall buffer API.

Synopsis

#include <ts/ts.h>

TSMBuffer TSMBufferCreate(void)

TSReturnCode TSMBufferDestroy(TSMBuffer bufp)

TSReturnCode TSHandleMLocRelease(TSMBuffer bufp, TSMLoc parent, TSMLoc mloc)

Description

The marshal buffer or TSMBuffer is a heap data structure that stores parsed URLs, MIME headers and HTTP head-ers. You can allocate new objects out of marshal buffers, and change the values within the marshal buffer. Wheneveryou manipulate an object, you require the handle to the object (TSMLoc) and the marshal buffer containing the object(TSMBuffer).

Any marshal buffer fetched by TSHttpTxn*Get() will be used by other parts of the system. Be careful not todestroy these shared, transaction marshal buffers.

TSMBufferCreate() creates a new marshal buffer and initializes the reference count. TSMBufferDestroy()Ignores the reference count and destroys the marshal buffer bufp. The internal data buffer associated with the marshalbuffer is also destroyed if the marshal buffer allocated it.

TSHandleMLocRelease()Releases the TSMLocmloc created from the TSMLoc parent. If a TSMLoc is obtainedfrom a transaction, it does not have a parent TSMLoc. Use the the constant TS_NULL_MLOC as its parent.

Return values

TSMBufferDestroy() and TSHandleMLocRelease() return TS_SUCCESS on success, or TS_ERROR onfailure. TSMBufferCreate() returns the new TSMBuffer.

Examples

#include <ts/ts.h>

static voidcopyResponseMimeHdr (TSCont pCont, TSHttpTxn pTxn){

TSMBuffer respHdrBuf, tmpBuf;TSMLoc respHttpHdrLoc, tmpMimeHdrLoc;

if (!TSHttpTxnClientRespGet(pTxn, &respHdrBuf, &respHttpHdrLoc)) {TSError("couldn't retrieve client response header0);TSHandleMLocRelease(respHdrBuf, TS_NULL_MLOC, respHttpHdrLoc);goto done;

}

tmpBuf = TSMBufferCreate();tmpMimeHdrLoc = TSMimeHdrCreate(tmpBuf);



TSMimeHdrCopy(tmpBuf, tmpMimeHdrLoc, respHdrBuf, respHttpHdrLoc);TSHandleMLocRelease(tmpBuf, TS_NULL_MLOC, tmpMimeHdrLoc);TSHandleMLocRelease(respHdrBuf, TS_NULL_MLOC, respHttpHdrLoc);TSMBufferDestroy(tmpBuf);

done:TSHttpTxnReenable(pTxn, TS_EVENT_HTTP_CONTINUE);

}

See also

TSAPI(3ts)

TSMgmtCounterGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSMgmtCounterGet(const char * var_name, TSMgmtCounter * result)

Description

TSMgmtFloatGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSMgmtFloatGet(const char * var_name, TSMgmtFloat * result)

Description

TSMgmtIntGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSMgmtIntGet(const char * var_name, TSMgmtInt * result)

Description

TSMgmtStringGet

Synopsis

#include <ts/ts.h>

TSReturnCode TSMgmtStringGet(const char * var_name, TSMgmtString * result)



Description

TSMgmtUpdateRegister

Synopsis

#include <ts/ts.h>

void TSMgmtUpdateRegister(TSCont contp, const char * plugin_name)

Description

TSMimeHdrClone

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrClone(TSMBuffer dest_bufp, TSMBuffer src_bufp, TSMLoc src_hdr, TSMLoc* locp)

Description

Copies a specified MIME header to a specified marshal buffer, and returns the location of the copied MIME headerwithin the destination marshal buffer.

Unlike TSMimeHdrCopy(), you do not have to create the destination MIME header before cloning. Release thereturned TSMLoc handle with a call to TSHandleMLocRelease().

TSMimeHdrCopy

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrCopy(TSMBuffer dest_bufp, TSMLoc dest_offset, TSMBuffer src_bufp, TSM-Loc src_offset)

Description

Copies the contents of the MIME header located at :arg:‘src_loc‘ within :arg:‘src_bufp‘ to the MIME header locatedat :arg:‘dest_loc‘ within :arg:‘dest_bufp‘.

TSMimeHdrCopy() works correctly even if :arg:‘src_bufp‘ and :arg:‘dest_bufp‘ point to different marshalbuffers.

: You must create the destination MIME header before copying into it, using TSMimeHdrCreate().



TSMimeHdrCreate

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrCreate(TSMBuffer bufp, TSMLoc * locp)

Description

Creates a new MIME header within :arg:‘bufp‘.

Release with a call to TSHandleMLocRelease().

TSMimeHdrDestroy

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrDestroy(TSMBuffer bufp, TSMLoc offset)

Description

Destroys the MIME header located at :arg:‘hdr_loc‘ within :arg:‘bufp‘.

TSMimeHdrFieldAppend

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldAppend(TSMBuffer bufp, TSMLoc hdr, TSMLoc field)

Description

Returns the TSMLoc location of a specified MIME field from within the MIME header located at :arg:‘hdr‘.

The retrieved_str parameter specifies which field to retrieve. For each MIME field in the MIME header, a pointercomparison is done between the field name and retrieved_str. This is a much quicker retrieval function thanTSMimeHdrFieldFind() since it obviates the need for a string comparison. However, retrieved_str must beone of the predefined field names of the form TS_MIME_FIELD_XXX for the call to succeed. Release the returnedTSMLoc handle with a call to TSHandleMLocRelease().

TSMimeHdrFieldClone

Synopsis

#include <ts/ts.h>



TSReturnCode TSMimeHdrFieldClone(TSMBuffer dest_bufp, TSMLoc dest_hdr, TSMBuffer src_bufp,TSMLoc src_hdr, TSMLoc src_field, TSMLoc * locp)

Description

TSMimeHdrFieldCopy

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldCopy(TSMBuffer dest_bufp, TSMLoc dest_hdr, TSMLoc dest_field, TSM-Buffer src_bufp, TSMLoc src_hdr, TSMLoc src_field)

Description

TSMimeHdrFieldCopyValues

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldCopyValues(TSMBuffer dest_bufp, TSMLoc dest_hdr, TSM-Loc dest_field, TSMBuffer src_bufp, TSMLoc src_hdr,TSMLoc src_field)

Description

TSMimeHdrFieldCreate

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldCreate(TSMBuffer bufp, TSMLoc hdr, TSMLoc * locp)

Description

TSMimeHdrFieldDestroy

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldDestroy(TSMBuffer bufp, TSMLoc hdr, TSMLoc field)



Description

Destroys the MIME :arg:‘field‘ located at :arg:‘field‘ within :arg:‘bufp‘.

You must release the TSMLoc field with a call to TSHandleMLocRelease().

TSMimeHdrFieldFind

Synopsis

#include <ts/ts.h>

TSMLoc TSMimeHdrFieldFind(TSMBuffer bufp, TSMLoc hdr, const char * name, int length)

Description

Retrieves the TSMLoc location of a specified MIME field from within the MIME header located at :arg:‘hdr‘.

The :arg:‘name‘ and :arg:‘length‘ parameters specify which field to retrieve. For each MIME field in theMIME header, a case insensitive string comparison is done between the field name and :arg:‘name‘. IfTSMimeHdrFieldFind() cannot find the requested field, it returns TS_NULL_MLOC. Release the returnedTSMLoc handle with a call to TSHandleMLocRelease().

TSMimeHdrFieldGet

Synopsis

#include <ts/ts.h>

TSMLoc TSMimeHdrFieldGet(TSMBuffer bufp, TSMLoc hdr, int idx)

Description

Retrieves the location of a specified MIME field within the MIME header located at :arg:‘hdr_loc‘ within :arg:‘bufp‘.

The :arg:‘idx‘ parameter specifies which field to retrieve. The fields are numbered from 0 toTSMimeHdrFieldsCount(bufp, hdr_loc) - 1. If :arg:‘idx‘ does not lie within that range thenTSMimeHdrFieldGet returns 0. Release the returned handle with a call to TSHandleMLocRelease.

TSMimeHdrFieldLengthGet

Synopsis

#include <ts/ts.h>

int TSMimeHdrFieldLengthGet(TSMBuffer bufp, TSMLoc hdr, TSMLoc field)



Description

TSMimeHdrFieldNameGet

Synopsis

#include <ts/ts.h>

const char * TSMimeHdrFieldNameGet(TSMBuffer bufp, TSMLoc hdr, TSMLoc field, int * length)

Description

TSMimeHdrFieldNameSet

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldNameSet(TSMBuffer bufp, TSMLoc hdr, TSMLoc field, const char* name, int length)

Description

TSMimeHdrFieldNext

Synopsis

#include <ts/ts.h>

TSMLoc TSMimeHdrFieldNext(TSMBuffer bufp, TSMLoc hdr, TSMLoc field)

Description

TSMimeHdrFieldNextDup

Synopsis

#include <ts/ts.h>

TSMLoc TSMimeHdrFieldNextDup(TSMBuffer bufp, TSMLoc hdr, TSMLoc field)

Description

TSMimeHdrFieldRemove

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldRemove(TSMBuffer bufp, TSMLoc hdr, TSMLoc field)



Description

Removes the MIME field located at :arg:‘field‘ within :arg:‘bufp‘ from the header located at :arg:‘hdr‘ within:arg:‘bufp‘.

If the specified field cannot be found in the list of fields associated with the header then nothing is done.

: Removing the field does not destroy the field. It only detaches the field, hiding it from the printed output.The field can be reattached with a call to TSMimeHdrFieldAppend(). If you do not use the detached fieldyou should destroy it with a call to TSMimeHdrFieldDestroy() and release the handle field with a call toTSHandleMLocRelease().

TSMimeHdrFieldValueAppend

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldValueAppend(TSMBuffer bufp, TSMLoc hdr, TSMLoc field, int idx,const char * value, int length)

Description

TSMimeHdrFieldValueDateInsert

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldValueDateInsert(TSMBuffer bufp, TSMLoc hdr, TSMLoc field,time_t value)

Description

TSMimeHdrFieldValueDateSet

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldValueDateSet(TSMBuffer bufp, TSMLoc hdr, TSMLoc field,time_t value)

Description

TSMimeHdrFieldValueIntSet



Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldValueIntSet(TSMBuffer bufp, TSMLoc hdr, TSMLoc field, int idx,int value)

Description

TSMimeHdrFieldValueStringGet

Get HTTP MIME header values.

Synopsis

#include <ts/ts.h>

const char * TSMimeHdrFieldValueStringGet(TSMBuffer bufp, TSMLoc hdr, TSMLoc field, int idx,int * value_len_ptr)

int TSMimeHdrFieldValueIntGet(TSMBuffer bufp, TSMLoc hdr, TSMLoc field, int idx)

int64_t TSMimeHdrFieldValueInt64Get(TSMBuffer bufp, TSMLoc hdr, TSMLoc field, int idx)

unsigned int TSMimeHdrFieldValueUintGet(TSMBuffer bufp, TSMLoc hdr, TSMLoc field, int idx)

time_t TSMimeHdrFieldValueDateGet(TSMBuffer bufp, TSMLoc hdr, TSMLoc field)

Description

MIME headers and fields can be components of request headers, response headers, or standalone headers cre-ated within a Traffic Server plugin. The functions here are all used to access header values of specific types,but it is up to the caller to know if a header has appropriate semantics for the API used. For all butTSMimeHdrFieldValueStringGet(), an appropriate data conversion algorithm is applied to the header fieldstring.

All the APIs take a TSMBuffer marshal buffer argument :arg:‘bufp‘, and a TSMLoc argument :arg:‘hdr‘ indicatingthe location of the HTTP headers. The required :arg:‘field‘ argument is the locator of a specific header value, asreturned by an accessor function such as TSMimeHdrFieldFind().

Within the header field, comma-separated values can be retrieved with an index :arg:‘idx‘ ranging from 0 to themaximum number of fields for this value; this maximum is retrieved using TSMimeHdrFieldValuesCount().An :arg:‘idx‘ value of -1 has the semantics of retrieving the entire header value, regardless of how many comma-separated values there are. If a header is not comma-separated, an :arg:‘idx‘ of 0 or -1 are the same, but the latter ispreferred.

TSMimeHdrFieldValueStringGet() returns a pointer to the header value, and populated:arg:‘value_len_ptr‘ with the length of the value in bytes. The returned header value is not NUL-terminated.

Return Values

All functions returns the header value with a type matching the respective function name. UsingTSMimeHdrFieldValueDateGet() on a header which does not have date-time semantics always returns 0.



Examples

This examples show how to retrieve and copy a specific header.

#include <string.h>#include <ts/ts.h>

intget_content_type(TSHttpTxn txnp, char* buf, size_t buf_size){

TSMBuffer bufp;TSMLoc hdrs;TSMLoc ctype_field;int len = -1;

if (TS_SUCCESS == TSHttpTxnServerRespGet(txnp, &bufp, &hdrs)) {ctype_field = TSMimeHdrFieldFind(bufp, hdrs, TS_MIME_FIELD_CONTENT_TYPE, TS_MIME_

→˓LEN_CONTENT_TYPE);

if (TS_NULL_MLOC != ctype_field) {const char* str = TSMimeHdrFieldValueStringGet(bufp, hdrs, ctype_field, -1, &

→˓len);

if (len > buf_size)len = buf_size;

memcpy(buf, str, len);TSHandleMLocRelease(bufp, hdrs, ctype_field);

}TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdrs);

}

return len;}

See Slso

TSAPI(3ts), TSMBufferCreate(3ts), TSMimeHdrFieldValuesCount(3ts)

TSMimeHdrFieldValueStringInsert

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldValueStringInsert(TSMBuffer bufp, TSMLoc hdr, TSMLoc field,int idx, const char * value, int length)

Description

TSMimeHdrFieldValueStringSet



Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldValueStringSet(TSMBuffer bufp, TSMLoc hdr, TSMLoc field, int idx,const char * value, int length)

Description

TSMimeHdrFieldValueUintInsert

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldValueUintInsert(TSMBuffer bufp, TSMLoc hdr, TSMLoc field,int idx, unsigned int value)

Description

TSMimeHdrFieldValueUintSet

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldValueUintSet(TSMBuffer bufp, TSMLoc hdr, TSMLoc field, int idx,unsigned int value)

Description

TSMimeHdrFieldValuesClear

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldValuesClear(TSMBuffer bufp, TSMLoc hdr, TSMLoc field)

Description

TSMimeHdrFieldValuesCount

Synopsis

#include <ts/ts.h>

int TSMimeHdrFieldValuesCount(TSMBuffer bufp, TSMLoc hdr, TSMLoc field)



Description

TSMimeHdrFieldsClear

Synopsis

#include <ts/ts.h>

TSReturnCode TSMimeHdrFieldsClear(TSMBuffer bufp, TSMLoc offset)

Description

Removes and destroys all the MIME fields within the MIME header located at :arg:‘hdr_loc‘ within the marshalbuffer :arg:‘bufp‘.

TSMimeHdrFieldsCount

Synopsis

#include <ts/ts.h>

int TSMimeHdrFieldsCount(TSMBuffer bufp, TSMLoc offset)

Description

Returns a count of the number of MIME fields within the MIME header located at :arg:‘hdr_loc‘ within the marshalbuffer :arg:‘bufp‘.

TSMimeHdrLengthGet

Synopsis

#include <ts/ts.h>

int TSMimeHdrLengthGet(TSMBuffer bufp, TSMLoc offset)

Description

Calculates the length of the MIME header located at :arg:‘hdr_loc‘ in buffer :arg:‘bufp‘ if it were returned as astring, in its unparsed form.

TSMimeHdrParse

Synopsis

#include <ts/ts.h>

TSParseResult TSMimeHdrParse(TSMimeParser parser, TSMBuffer bufp, TSMLoc offset, const char** start, const char * end)



Description

Parses a MIME header.

The MIME header must have already been allocated and both :arg:‘bufp‘ and :arg:‘hdr_loc‘ must point within thatheader. It is possible to parse a MIME header a single byte at a time using repeated calls to TSMimeHdrParse().As long as an error does not occur, TSMimeHdrParse() consumes each single byte and asks for more.

TSMimeHdrPrint

Synopsis

#include <ts/ts.h>

void TSMimeHdrPrint(TSMBuffer bufp, TSMLoc offset, TSIOBuffer iobufp)

Description

Formats the MIME header located at :arg:‘hdr_loc‘ within :arg:‘bufp‘ into the TSIOBuffer :arg:‘iobufp‘.

TSMimeParserClear

Synopsis

#include <ts/ts.h>

void TSMimeParserClear(TSMimeParser parser)

Description

Clears the specified MIME :arg:‘parser‘ so that it can be used again.

TSMimeParserCreate

Synopsis

#include <ts/ts.h>

TSMimeParser TSMimeParserCreate(void)

Description

Creates a MIME parser.

The parser’s data structure contains information about the header being parsed. A single MIME parser can beused multiple times, though not simultaneously. Before being used again, the parser must be cleared by callingTSMimeParserClear().



TSMimeParserDestroy

Synopsis

#include <ts/ts.h>

void TSMimeParserDestroy(TSMimeParser parser)

Description

Destroys the specified MIME :arg:‘parser‘ and frees the associated memory.

TSMutexCreate

Synopsis

#include <ts/ts.h>

TSMutex TSMutexCreate(void)

Description

TSMutexDestroy

Synopsis

#include <ts/ts.h>

void TSMutexDestroy(TSMutex mutexp)

Description

Destroys the indicated :arg:‘mutex‘ previously created via TSMutexCreate().

TSMutexLock

Synopsis

#include <ts/ts.h>

void TSMutexLock(TSMutex mutexp)

Description

TSMutexLockTry

Synopsis

#include <ts/ts.h>



TSReturnCode TSMutexLockTry(TSMutex mutexp)

Description

TSMutexUnlock

Synopsis

#include <ts/ts.h>

void TSMutexUnlock(TSMutex mutexp)

Description

TSNetAccept

Synopsis

#include <ts/ts.h>

TSAction TSNetAccept(TSCont contp, int port, int domain, int accept_threads)

Description

TSNetAcceptNamedProtocol

Listen on all SSL ports for connections for the specified protocol name.

Synopsis

#include <ts/ts.h>

TSReturnCode TSNetAcceptNamedProtocol(TSCont contp, const char * protocol)

Description

TSNetAcceptNamedProtocol registers the specified :arg:‘protocol‘ for all statically configured TLSports. When a client using the TLS Next Protocol Negotiation extension negotiates the requested protocol,TrafficServer will route the request to the given handler :arg:‘contp‘.

: Be aware that the protocol is not registered on ports opened by other plugins.

The event and data provided to the handler are the same as for TSNetAccept(). If a connection is successfullyaccepted, the event code will be TS_EVENT_NET_ACCEPT and the event data will be a valid TSVConn bound tothe accepted connection.

: Neither :arg:‘contp‘ nor :arg:‘protocol‘ are copied. They must remain valid for the lifetime of the plugin.



TSNetAcceptNamedProtocol fails if the requested protocol cannot be registered on all of the configured TLSports. If it fails, the protocol will not be registered on any ports. There is no partial failure.

TSNetConnect

Synopsis

#include <ts/ts.h>

TSAction TSNetConnect(TSCont contp, sockaddr const * addr)

Description

TSPluginInit

Traffic Server plugin loading and registration.

Synopsis

#include <ts/ts.h>

void TSPluginInit(int argc, const char* argv[])

TSReturnCode TSPluginRegister(TSPluginRegistrationInfo* plugin_info)

Description

TSPluginInit() must be defined by all plugins. Traffic Server calls this initialization routine when it loads theplugin and sets :arg:‘argc‘ and :arg:‘argv‘ appropriately based on the values in plugin.config.

:arg:‘argc‘ is a count of the number of arguments in the argument vector, :arg:‘argv‘. The count is at least onebecause the first argument in the argument vector is the plugins name, which must exist in order for the plugin to beloaded.

TSPluginRegister() registers the appropriate SDK version specific in :arg:‘sdk_version‘ for your plugin. Usethis function to make sure that the version of Traffic Server on which your plugin is running supports the plugin.

Return Values

TSPluginRegister() returns :const:‘TS_ERROR‘ if the plugin registration failed.

Examples

#include <ts/ts.h>


TSPluginRegistrationInfo info;info.plugin_name = "hello-world";



info.vendor_name = "MyCompany";info.support_email = "[email protected]";

if (TSPluginRegister(&info) != TS_SUCCESS) {TSError("Plugin registration failed. 0);

}}

See Also

TSAPI(3ts), TSInstallDirGet(3ts)

TSRemapInit

Traffic Server remap plugin entry points.

Synopsis


TSReturnCode TSRemapInit(TSRemapInterface * api_info, char * errbuf, int errbuf_size)

void TSRemapDone(void)

TSRemapStatus TSRemapDoRemap(void * ih, TSHttpTxn rh, TSRemapRequestInfo * rri)

TSReturnCode TSRemapNewInstance(int argc, char * argv[], void ** ih, char * errbuf, int errbuf_size)

void TSRemapDeleteInstance(void *)

void TSRemapOSResponse(void * ih, TSHttpTxn rh, int os_response_type)

Description

The Traffic Server remap interface provides a simplified mechanism for plugins to manipulate HTTP transactions. Aremap plugin is not global; it is configured on a per-remap rule basis, which enables you to customize how URLsare redirected based on individual rules in remap.config. Writing a remap plugin consists of implementing oneor more of the remap entry points and configuring remap.config to route the transaction through your plugin.Multiple remap plugins can be specified for a single remap rule, resulting in a remap plugin chain where each pluginis given an opportunity to examine the HTTP transaction.

TSRemapInit() is a required entry point. This function will be called once when Traffic Server loads the plugin. Ifthe optional TSRemapDone() entry point is available, Traffic Server will call then when unloading the remap plugin.

A remap plugin may be invoked for different remap rules. Traffic Server will call the entry point each time aplugin is specified in a remap rule. When a remap plugin instance is no longer required, Traffic Server will callTSRemapDeleteInstance().

TSRemapDoRemap() is called for each HTTP transaction. This is a mandatory entry point. In this function, theremap plugin may examine and modify the HTTP transaction.



Return Values

TSRemapInit() and TSRemapNewInstance() should return TS_SUCCESS on success, and TS_ERROR oth-erwise. A return value of TS_ERROR is unrecoverable.

TSRemapDoRemap() returns a status code that indicates whether the HTTP transaction has been modified andwhether Traffic Server should continue to evaluate the chain of remap plugins. If the transaction was modified,the plugin should return TSREMAP_DID_REMAP or TSREMAP_DID_REMAP_STOP; otherwise it should returnTSREMAP_NO_REMAP or TSREMAP_NO_REMAP_STOP. If Traffic Server should not send the transaction to subse-quent plugins in the remap chain, return TSREMAP_NO_REMAP_STOP or TSREMAP_DID_REMAP_STOP. Return-ing TSREMAP_ERROR causes Traffic Server to stop evaluating the remap chain and respond with an error.

See Also

TSAPI(3ts)

TSSslContextFindByName

Synopsis

#include <ts/ts.h>

TSSslContext TSSslContextFindByName(const char * name)

Description

Look for a SSL context created from ssl_multicert.config. Use the server :arg:‘name‘ to search.

TSSslContextFindByAddr

Synopsis

#include <ts/ts.h>

TSSslContext TSSslContextFindByAddr(struct sockaddr const*)

Description

Look for a SSL context created from ssl_multicert.config. Use the server address to search.

Type

TSSslContext

Corresponds to the SSL_CTX * value in openssl.

TSTextLogObjectCreate

Traffic Server text logging API.



Synopsis

#include <ts/ts.h>

TSReturnCode TSTextLogObjectCreate(const char * filename, int mode, TSTextLogObject* new_log_obj)

TSReturnCode TSTextLogObjectWrite(TSTextLogObject the_object, const char * format, ...)

void TSTextLogObjectFlush(TSTextLogObject the_object)

TSReturnCode TSTextLogObjectDestroy(TSTextLogObject the_object)

void TSTextLogObjectHeaderSet(TSTextLogObject the_object, const char * header)

TSReturnCode TSTextLogObjectRollingEnabledSet(TSTextLogObject the_object,int rolling_enabled)

void TSTextLogObjectRollingIntervalSecSet(TSTextLogObject the_object,int rolling_interval_sec)

void TSTextLogObjectRollingOffsetHrSet(TSTextLogObject the_object, int rolling_offset_hr)

void TSTextLogObjectRollingSizeMbSet(TSTextLogObject the_object, int rolling_size_mb)

Description

TSTextLogObjectRollingEnabledSet() sets the log rolling mode for the given object. This API mustbe used once the object is created and before writing into logs. The :arg:‘rolling_enabled‘ argument must be avalid :ts:cv:‘proxy.config.log.rolling_enabled‘ values. If TSTextLogObjectRollingEnabledSet() is nevercalled, the log object takes it’s log rolling mode from the global :ts:cv:‘proxy.config.log.rolling_enabled‘ setting.

It’s also important to call TSTextLogObjectRollingEnabledSet() before any of the other APIs that modifiesthe rolling object. This is due to the fact that this API dictates which rolling mode should be used, and therefore affectsfurther modifications of the logging object.

See Also

TSAPI(3ts)

TSThreadCreate

Synopsis

#include <ts/ts.h>

TSThread TSThreadCreate(TSThreadFunc func, void * data)

Description

TSThreadDestroy

Synopsis

#include <ts/ts.h>



void TSThreadDestroy(TSThread thread)

Description

TSThreadInit

Synopsis

#include <ts/ts.h>

TSThread TSThreadInit(void)

Description

TSThreadSelf

Synopsis

#include <ts/ts.h>

TSThread TSThreadSelf(void)

Description

TSTrafficServerVersionGet

Return Traffic Server version information.

Synopsis

#include <ts/ts.h>

const char * TSTrafficServerVersionGet(void)

int TSTrafficServerVersionGetMajor(void)

int TSTrafficServerVersionGetMinor(void)

int TSTrafficServerVersionGetPatch(void)

Description

TSTrafficServerVersionGet() returns a pointer to a string of characters that indicates the Traffic Serverrelease version. This string must not be modified.

The other APIs return an integer from the relevant component of the version number string.



Example

#include <stdio.h>#include <ts/ts.h>

intcheck_ts_version(){

const char *ts_version = TSTrafficServerVersionGet();int result = 0;

if (ts_version) {int major_ts_version = 0;int minor_ts_version = 0;int patch_ts_version = 0;

if (sscanf(ts_version, "%d.%d.%d", &major_ts_version,&minor_ts_version, &patch_ts_version) != 3) {

return 0;}

/* We need at least Traffic Server 3.0 */if (major_ts_version >= 3) {

result = 1;}

}

return result;}


TSPluginRegistrationInfo info;info.plugin_name = "hello-world";info.vendor_name = "MyCompany";info.support_email = "[email protected]";

if (TSPluginRegister(&info) != TS_SUCCESS) {TSError("Plugin registration failed. 0);

}

if (!check_ts_version()) {TSError("Plugin requires Traffic Server 3.0 or later0);return;

}

TSDebug("debug-hello", "Hello World!0);}

See Also

TSAPI(3ts)



TSTransformCreate

Synopsis

#include <ts/ts.h>

TSVConn TSTransformCreate(TSEventFunc event_funcp, TSHttpTxn txnp)

Description

TSTransformOutputVConnGet

Synopsis

#include <ts/ts.h>

TSVConn TSTransformOutputVConnGet(TSVConn connp)

Description

TSAPI Types

Synopsis


Description

The Apache Traffic Server API provides large number of types. Many of them are specific to a particular API functionor function group, but others are used more widely. Those are described on this page.

ink_hrtime

TSAction

TSCacheKey

TSConfig

TSConfigDestroyFunc

TSContAn opaque type that represents a Traffic Server continuation.

TSEventTSEvents are sent to continuations when they are called back.

The TSEvent provides the continuation’s handler function with information about the callback. Based on theevent it receives, the handler function can decide what to do.

TSEventFunc

TSFile

TSHostLookupResult



TSHRTime“High Resolution Time”

A 64 bit time value, measured in nanoseconds.

TSHttpHookIDAn enumeration that identifies a specific type of hook for HTTP transactions.

TSHttpParser

TSHttpSsnAn opaque type that represents a Traffic Server session.

TSHttpStatusThis set of enums represents possible return values from TSHttpHdrStatusGet(), which retrieves thestatus code from an HTTP response header (TSHttpHdrStatusGet() retrieves status codes only fromheaders of type TS_HTTP_TYPE_RESPONSE).

You can also set the TSHttpStatus of a response header using TSHttpHdrStatusSet().

TSHttpTxnAn opaque type that represents a Traffic Server HTTP transaction.

TSHttpTypeThis set of enums represents the possible HTTP types that can be assigned to an HTTP header.

When a header is created with TSHttpHdrCreate(), it is automatically assigned a type ofTS_HTTP_TYPE_UNKNOWN . You can modify the HTTP type ONCE after it the header is created, us-ing TSHttpHdrTypeSet(). After setting the HTTP type once, you cannot set it again. UseTSHttpHdrTypeGet() to obtain the TSHttpType of an HTTP header.

TSIOBuffer

TSIOBufferBlock

TSIOBufferReader

TSIOBufferSizeIndex

TSLifecycleHookIDAn enumeration that identifies a life cycle hook.

TSMBuffer

TSMgmtCounter

TSMgmtFloatThe type used internally for a floating point value. This corresponds to the value:const:‘TS_RECORDDATATYPE_FLOAT‘ for TSRecordDataType.

TSMgmtIntThe type used internally for an integer. This corresponds to the value :const:‘TS_RECORDDATATYPE_INT‘for TSRecordDataType.

TSMgmtString

TSMimeParser

TSMLoc

TSMutex

TSParseResultThis set of enums are possible values returned by TSHttpHdrParseReq() andTSHttpHdrParseResp().



TSPluginRegistrationInfoThe following struct is used by TSPluginRegister().

It stores registration information about the plugin.

TSRecordDataTypeAn enumeration that specifies the type of a value in an internal data structure that is accessible via the API.

TSRemapInterface

TSRemapRequestInfo

TSRemapStatus

TSReturnCodeAn indicator of the results of an API call. A value of :const:‘TS_SUCCESS‘ means the call was successful.Any other value indicates a failure and is specific to the API call.

TSSDKVersionStarting 2.0, SDK now follows same versioning as Traffic Server.

TSServerState

TSTextLogObjectThis type represents a custom log file that you create with TSTextLogObjectCreate().

Your plugin writes entries into this log file using TSTextLogObjectWrite().

TSThread

TSThreadFunc

TSThreadPool

TSVConn

TSVIO

TSUrlCreate

Traffic Server URL object construction API.

Synopsis

#include <ts/ts.h>

TSReturnCode TSUrlCreate(TSMBuffer bufp, TSMLoc * locp)

TSReturnCode TSUrlClone(TSMBuffer dest_bufp, TSMBuffer src_bufp, TSMLoc src_url, TSMLoc * locp)

TSReturnCode TSUrlCopy(TSMBuffer dest_bufp, TSMLoc dest_url, TSMBuffer src_bufp, TSMLoc src_url)

TSParseResult TSUrlParse(TSMBuffer bufp, TSMLoc offset, const char ** start, const char * end)

Description

The URL data structure is a parsed version of a standard internet URL. The Traffic Server URL API provides access toURL data stored in marshal buffers. The URL functions can create, copy, retrieve or delete entire URLs, and retrieveor modify parts of URLs, such as their host, port or scheme information.



TSUrlCreate() creates a new URL within the marshal buffer :arg:‘bufp‘. Release the resulting handle with a callto TSHandleMLocRelease().

TSUrlClone() copies the contents of the URL at location :arg:‘src_url‘ within the marshal buffer :arg:‘src_bufp‘to a location within the marshal buffer :arg:‘dest_bufp‘. Release the returned handle with a call toTSHandleMLocRelease().

TSUrlCopy() copies the contents of the URL at location :arg:‘src_url‘ within the marshal buffer :arg:‘src_bufp‘to the location :arg:‘dest_url‘ within the marshal buffer dest_bufp. TSUrlCopy() works correctly even if:arg:‘src_bufp‘ and :arg:‘dest_bufp‘ point to different marshal buffers. It is important for the destination URL(its marshal buffer and TSMLoc) to have been created before copying into it.

TSUrlParse() parses a URL. The :arg:‘start‘ pointer is both an input and an output parameter and marks thestart of the URL to be parsed. After a successful parse, the :arg:‘start‘ pointer equals the :arg:‘end‘ pointer. The:arg:‘end‘ pointer must be one byte after the last character you want to parse. The URL parsing routine assumes thateverything between :arg:‘start‘ and :arg:‘end‘ is part of the URL. It is up to higher level parsing routines, such asTSHttpHdrParseReq(), to determine the actual end of the URL.

Return Values

The TSUrlParse() function returns a TSParseResult, where TS_PARSE_ERROR indicates an error. Suc-cess is indicated by one of TS_PARSE_DONE, TS_PARSE_OK or TS_PARSE_CONT. The other APIs all return aTSReturnCode, indicating success (TS_SUCCESS) or failure (TS_ERROR) of the operation.

See Also

TSAPI(3ts), TSMBufferCreate(3ts), TSUrlHostGet(3ts), TSUrlHostSet(3ts),TSUrlStringGet(3ts), TSUrlPercentEncode(3ts)

TSUrlDestroy

Synopsis

#include <ts/ts.h>

TSReturnCode TSUrlDestroy(TSMBuffer bufp, TSMLoc offset)

Description

Destroys the URL located at :arg:‘url_loc‘ within the marshal buffer :arg:‘bufp‘.

Do not forget to release the TSMLoc url_loc with a call to TSHandleMLocRelease().

Deprecated

There is no reason to destroy the URL, just release the marshal buffers.

Should be removed for v5.0.0



TSUrlFtpTypeGet

Synopsis

#include <ts/ts.h>

int TSUrlFtpTypeGet(TSMBuffer bufp, TSMLoc offset)

Description

Retrieves the FTP type of the URL located at :arg:‘url_loc‘ within :arg:‘bufp‘.

TSUrlFtpTypeSet

Synopsis

#include <ts/ts.h>

TSReturnCode TSUrlFtpTypeSet(TSMBuffer bufp, TSMLoc offset, int type)

Description

Sets the FTP type portion of the URL located at :arg:‘url_loc‘ within :arg:‘bufp‘ to the value :arg:‘type‘.

TSUrlHostGet

Traffic Server URL component retrieval API.

Synopsis

#include <ts/ts.h>

const char * TSUrlHostGet(TSMBuffer bufp, TSMLoc offset, int * length)

const char * TSUrlSchemeGet(TSMBuffer bufp, TSMLoc offset, int * length)

const char * TSUrlUserGet(TSMBuffer bufp, TSMLoc offset, int * length)

const char * TSUrlPasswordGet(TSMBuffer bufp, TSMLoc offset, int* length)

int TSUrlPortGet(TSMBuffer bufp, TSMLoc offset)

const char * TSUrlPathGet(TSMBuffer bufp, TSMLoc offset, int * length)

const char * TSUrlHttpQueryGet(TSMBuffer bufp, TSMLoc offset, int * length)

const char * TSUrlHttpParamsGet(TSMBuffer bufp, TSMLoc offset, int * length)

const char * TSUrlHttpFragmentGet(TSMBuffer bufp, TSMLoc offset, int * length)



Description


TSUrlSchemeGet(), TSUrlUserGet(), TSUrlPasswordGet(), TSUrlHostGet(),TSUrlHttpParamsGet(), TSUrlHttpQueryGet() and TSUrlHttpFragmentGet() each retrievean internal pointer to the specified portion of the URL from the marshall buffer :arg:‘bufp‘. The length of thereturned string is placed in :arg:‘length‘ and a pointer to the URL portion is returned.

TSUrlPortGet() retrieves the port number portion of the URL located at :arg:‘offset‘ within the marshal buffer:arg:‘bufp‘. If there is no explicit port number in the URL, a canonicalized valued is returned based on the URLscheme.

Return Values

All APIs except TSUrlPortGet() returns a string, which is not guaranteed to be NULL terminated. You musttherefore always use the :arg:‘length‘ value to determine the actual length of the returned string.

TSUrlPortGet() simply returns the port number as an integer, possibly canonicalized with 80 for HTTP and 443for HTTPS schemes. If there is neither port nor scheme information available in the URL, 0 is returned.

See Also

TSAPI(3ts), TSUrlCreate(3ts), TSUrlHostSet(3ts), TSUrlStringGet(3ts),TSUrlPercentEncode(3ts)

TSUrlHostSet

Traffic Server URL component manipulation API.

Synopsis

#include <ts/ts.h>

TSReturnCode TSUrlHostSet(TSMBuffer bufp, TSMLoc offset, const char * value, int length)

TSReturnCode TSUrlSchemeSet(TSMBuffer bufp, TSMLoc offset, const char * value, int length)

TSReturnCode TSUrlUserSet(TSMBuffer bufp, TSMLoc offset, const char * value, int length)

TSReturnCode TSUrlPasswordSet(TSMBuffer bufp, TSMLoc offset, const char * value, int length)

TSReturnCode TSUrlPortSet(TSMBuffer bufp, TSMLoc offset, int port)

TSReturnCode TSUrlPathSet(TSMBuffer bufp, TSMLoc offset, const char * value, int length)

TSReturnCode TSUrlHttpQuerySet(TSMBuffer bufp, TSMLoc offset, const char * value, int length)

TSReturnCode TSUrlHttpParamsSet(TSMBuffer bufp, TSMLoc offset, const char * value, int length)

TSReturnCode TSUrlHttpFragmentSet(TSMBuffer bufp, TSMLoc offset, const char * value, int length)



Description


TSUrlSchemeSet(), TSUrlUserSet(), TSUrlPasswordSet(), TSUrlHostSet(),TSUrlHttpParamsSet(), TSUrlHttpQuerySet() and TSUrlHttpFragmentSet() each set thespecified portion of the URL located at offset within the marshal buffer :arg:‘bufp‘ to the string value. If:arg:‘length‘ is -1 then these functions assume that value is NULL-terminated. Otherwise, the length of the:arg:‘string‘ value is taken to be the value of :arg:‘length‘. These functions copy the string to within :arg:‘bufp‘,so it can be subsequently modified or deleted.

TSUrlPortSet() sets the port number portion of the URL located at :arg:‘offset‘ within the marshal buffer:arg:‘bufp‘ to the value port. Normal canonicalization based on the URL scheme still applies.

Return Values

All these APIs returns a TSReturnCode, indicating success (TS_SUCCESS) or failure (TS_ERROR) of the opera-tion.

See Also

TSAPI(3ts), TSUrlCreate(3ts), TSUrlHostGet(3ts), TSUrlStringGet(3ts),TSUrlPercentEncode(3ts)

TSUrlPercentEncode

Traffic Server URL percent encoding API.

Synopsis

#include <ts/ts.h>

TSReturnCode TSUrlPercentEncode(TSMBuffer bufp, TSMLoc offset, char * dst, size_t dst_size, size_t* length, const unsigned char * map)

TSReturnCode TSStringPercentEncode(const char * str, int str_len, char * dst, size_t dst_size, size_t* length, const unsigned char * map)

TSReturnCode TSStringPercentDecode(const char * str, size_t str_len, char * dst, size_t dst_size, size_t* length)

Description


TSUrlPercentEncode() performs percent-encoding of the URL object, storing the new string in the :arg:‘dst‘buffer. The :arg:‘length‘ parameter will be set to the new (encoded) string length, or 0 if the encoding failed.



TSStringPercentEncode() is similar but operates on a string. If the optional :arg:‘map‘ parameter is pro-vided (not NULL) , it should be a map of characters to encode.

TSStringPercentDecode() perform percent-decoding of the string in the :arg:‘str‘ buffer, writing to the:arg:‘dst‘ buffer. The source and destination can be the same, in which case they overwrite. The decoded stringis always guaranteed to be no longer than the source string.

Return Values

All these APIs returns a TSReturnCode, indicating success (TS_SUCCESS) or failure (TS_ERROR) of the opera-tion.

See Also

TSAPI(3ts), TSUrlCreate(3ts), TSUrlHostGet(3ts), TSUrlHostSet(3ts),TSUrlStringGet(3ts)

TSUrlStringGet

Traffic Server URL string representations API.

Synopsis

#include <ts/ts.h>

char * TSUrlStringGet(TSMBuffer bufp, TSMLoc offset, int * length)

int TSUrlLengthGet(TSMBuffer bufp, TSMLoc offset)

void TSUrlPrint(TSMBuffer bufp, TSMLoc offset, TSIOBuffer iobufp)

Description


TSUrlStringGet() constructs a string representation of the URL located at :arg:‘offset‘ within the marshal buffer:arg:‘bufp‘. TSUrlStringGet() stores the length of the allocated string in the parameter :arg:‘length‘. This isthe same length that TSUrlLengthGet() returns. The returned string is allocated by a call to TSmalloc() andmust be freed by a call to TSfree(). If length is NULL then no attempt is made to de-reference it.

TSUrlLengthGet() calculates the length of the URL located at :arg:‘offset‘ within the marshal buffer bufp as ifit were returned as a string. This length will be the same as the length returned by TSUrlStringGet().

TSUrlPrint() formats a URL stored in an TSMBuffer to an TSIOBuffer.

See Also

TSAPI(3ts), TSmalloc(3ts), TSUrlCreate(3ts), TSUrlHostGet(3ts), TSUrlHostSet(3ts),TSUrlPercentEncode(3ts)



TSVConnAbort

Synopsis

#include <ts/ts.h>

void TSVConnAbort(TSVConn connp, int error)

Description

TSVConnCacheObjectSizeGet

Synopsis

#include <ts/ts.h>

int64_t TSVConnCacheObjectSizeGet(TSVConn connp)

Description

TSVConnClose

Synopsis

#include <ts/ts.h>

void TSVConnClose(TSVConn connp)

Description

TSVConnClosedGet

Synopsis

#include <ts/ts.h>

int TSVConnClosedGet(TSVConn connp)

Description

TSVConnFdCreate

Create a TSVConn from a socket.

Synopsis

#include <ts/ts.h>

TSVConn TSVConnFdCreate(int fd)



Description

TSVConnFdCreate() accepts a network socket :arg:‘fd‘ and returns a new TSVConn constructed from the socket.The socket descriptor must be an already connected socket. It will be placed into non-blocking mode.

Return Values

On success, the returned TSVConn object owns the socket and the caller must not close it. If TSVConnFdCreate()fails, NULL is returned, the socket is unchanged and the caller must close it.

Examples

The example below is excerpted from example/intercept/intercept.cc in the Traffic Server source distribution. Itdemonstrates how to use TSVConnFdCreate() to construct a TSVConn from a connected socket.

See Also

TSAPI(3ts)

TSVConnIsSsl

Synopsis

#include <ts/ts.h>

int TSVConnIsSsl(TSVConn svc)

Description

Determines whether the connection associated with :arg:‘svc‘ is being processed as an SSL connection. Returns 1 ifit is being processed as SSL and 0 otherwise.

TSVConnRead

Synopsis

#include <ts/ts.h>

TSVIO TSVConnRead(TSVConn connp, TSCont contp, TSIOBuffer bufp, int64_t nbytes)

Description

TSVConnReadVIOGet

Synopsis

#include <ts/ts.h>



TSVIO TSVConnReadVIOGet(TSVConn connp)

Description

TSVConnReenable

Synopsis

#include <ts/ts.h>

void TSSslVConnReenable(TSVConn svc)

Description

Reenable the SSL connection :arg:‘svc‘. If a plugin hook is called, ATS processing on that connnection will notresume until this is invoked for that connection.

If the server is running OpenSSL 1.0.1 with the appropraite patch installed or it is running OpenSSL 1.0.2, the pluginwriter can pause SSL handshake processing by not reenabling the connection. Without the OpenSSL patch or runningan OpenSSL versions older than 1.0.2, the handshake processing in SSL_accept will not be stopped even if the SNIcallback does not reenable the connection.

Additional processing could reenable the virtual connection causing the SSL_accept to be called again to completethe handshake exchange. In the case of a blind tunnel conversion, the SSL handshake will never be completed byTraffic Server.

This call does appropriate locking and scheduling, so it is safe to call from another thread.

TSVConnShutdown

Synopsis

#include <ts/ts.h>

void TSVConnShutdown(TSVConn connp, int read, int write)

Description

TSVConnSslConnectionGet

Synopsis

#include <ts/ts.h>

TSSslVConnection TSVConnSslConnectionGet(TSVConn svc)

Description

Get the SSL (per connection) object from the SSl connection :arg:‘svc‘.



Types

TSSslVConnection

The SSL (per connection) object. This is an opaque type that can be cast to the appropriate type (SSL * for theOpenSSL library).

TSVConnTunnel

Synopsis

#include <ts/ts.h>

TSReturnCode TSVConnTunnel(TSVConn svc)

Description

Set the SSL connection :arg:‘svc‘ to convert to a blind tunnel. Can be called from TS_VCONN_PRE_ACCEPT_HOOKor TS_SSL_SNI_HOOK/TS_SSL_CERT_HOOK.

For this to work from the TS_SSL_SNI_HOOK or TS_SSL_CERT_HOOK, either the server must be runningOpenSSL 1.0.2 or a version of OpenSSL 1.0.1 with the appropriate patch installed.

TSVConnWrite

Synopsis

#include <ts/ts.h>

TSVIO TSVConnWrite(TSVConn connp, TSCont contp, TSIOBufferReader readerp, int64_t nbytes)

Description

TSVConnWriteVIOGet

Synopsis

#include <ts/ts.h>

TSVIO TSVConnWriteVIOGet(TSVConn connp)

Description

TSVIOBufferGet

Synopsis

#include <ts/ts.h>

TSIOBuffer TSVIOBufferGet(TSVIO viop)



Description

TSVIOContGet

Synopsis

#include <ts/ts.h>

TSCont TSVIOContGet(TSVIO viop)

Description

TSVIOMutexGet

Synopsis

#include <ts/ts.h>

TSMutex TSVIOMutexGet(TSVIO viop)

Description

TSVIONBytesGet

Synopsis

#include <ts/ts.h>

int64_t TSVIONBytesGet(TSVIO viop)

Description

TSVIONBytesSet

Synopsis

#include <ts/ts.h>

void TSVIONBytesSet(TSVIO viop, int64_t nbytes)

Description

TSVIONDoneGet

Synopsis

#include <ts/ts.h>

int64_t TSVIONDoneGet(TSVIO viop)



Description

TSVIONDoneSet

Synopsis

#include <ts/ts.h>

void TSVIONDoneSet(TSVIO viop, int64_t ndone)

Description

TSVIONTodoGet

Synopsis

#include <ts/ts.h>

int64_t TSVIONTodoGet(TSVIO viop)

Description

TSVIOReaderGet

Synopsis

#include <ts/ts.h>

TSIOBufferReader TSVIOReaderGet(TSVIO viop)

Description

TSVIOReenable

Synopsis

#include <ts/ts.h>

void TSVIOReenable(TSVIO viop)

Description

TSVIOVConnGet

Synopsis

#include <ts/ts.h>

TSVConn TSVIOVConnGet(TSVIO viop)



Description

TSfclose

Synopsis

#include <ts/ts.h>

void TSfclose(TSFile filep)

Description

Closes the file to which :arg:‘filep‘ points and frees the data structures and buffers associated with it.

If the file was opened for writing, any pending data is flushed.

TSfflush

Synopsis

#include <ts/ts.h>

void TSfflush(TSFile filep)

Description

Flushes pending data that has been buffered up in memory from previous calls to TSfwrite().

TSfgets

Synopsis

#include <ts/ts.h>

char* TSfgets(TSFile filep, char * buf, size_t length)

Description

Reads a line from the file pointed to by :arg:‘filep‘ into the buffer :arg:‘buf‘.

Lines are terminated by a line feed character, ‘ ‘. The line placed in the buffer includes the line feed character and isterminated with a NULL. If the line is longer than length bytes then only the first length-minus-1 bytes are placed in:arg:‘buf‘.

TSfopen

Synopsis

#include <ts/ts.h>



TSFile TSfopen(const char * filename, const char * mode)

Description

Opens a file for reading or writing and returns a descriptor for accessing the file.

The current implementation cannot open a file for both reading and writing. See TSfopen Family for examples.

TSfread

Synopsis

#include <ts/ts.h>

size_t TSfread(TSFile filep, void * buf, size_t length)

Description

Attempts to read :arg:‘length‘ bytes of data from the file pointed to by :arg:‘filep‘ into the buffer :arg:‘buf‘.

TSfwrite

Synopsis

#include <ts/ts.h>

size_t TSfwrite(TSFile filep, const void * buf, size_t length)

Description

Attempts to write :arg:‘length‘ bytes of data from the buffer :arg:‘buf‘ to the file :arg:‘filep‘.

Make sure that :arg:‘filep‘ is open for writing. You might want to check the number of bytes written (TSfwrite()returns this value) against the value of :arg:‘length‘. If it is less, there might be insufficient space on disk, for example.

TSmalloc

Traffic Server memory allocation API.

Synopsis

#include <ts/ts.h>

void * TSmalloc(size_t size, const char * path)

void * TSrealloc(void * ptr, size_t size, const char * path)

char * TSstrdup(const char * str)

char * TSstrndup(const char * str, size_t size)

size_t TSstrlcpy(char * dst, const char * src, size_t size)



size_t TSstrlcat(char * dst, const char * src, size_t size)

void TSfree(void * ptr)

Description

Traffic Server provides a number of routines for allocating and freeing memory. These routines correspond to similarroutines in the C library. For example, TSrealloc() behaves like the C library routine realloc(). There are tworeasons to use the routines provided by Traffic Server. The first is portability. The Traffic Server API routines behavethe same on all of Traffic Servers supported platforms. For example, realloc() does not accept an argument ofNULL on some platforms. The second reason is that the Traffic Server routines actually track the memory allocationsby file and line number. This tracking is very efficient, is always turned on, and is useful for tracking down memoryleaks.

TSmalloc() returns a pointer to size bytes of memory allocated from the heap. Traffic Server uses TSmalloc()internally for memory allocations. Always use TSfree() to release memory allocated by TSmalloc(); do not usefree().

TSstrdup() returns a pointer to a new string that is a duplicate of the string pointed to by str. The memory for thenew string is allocated using TSmalloc() and should be freed by a call to TSfree(). TSstrndup() returns apointer to a new string that is a duplicate of the string pointed to by str and size bytes long. The new string will beNUL-terminated. This API is very useful for transforming non NUL-terminated string values returned by APIs suchas TSMimeHdrFieldValueStringGet() into NUL-terminated string values. The memory for the new string isallocated using TSmalloc() and should be freed by a call to TSfree().

TSstrlcpy() copies up to size - 1 characters from the NUL-terminated string src to dst, NUL-terminating theresult.

TSstrlcat() appends the NUL-terminated string src to the end of dst. It will append at most size - strlen(dst) - 1bytes, NUL-terminating the result.

TSfree() releases the memory allocated by TSmalloc() or TSrealloc(). If ptr is NULL, TSfree() doesno operation.

See also

TSAPI(3ts)

Continuous Integration

Documentation

Conventions

The conventions detailed in this chapter should be followed when modifying the project documentation to maintainreadability and consistency.

Typographic

Italic Used to introduce new terms. Italics used solely for emphasis should be avoided. When introducing new terms,only the first use should be set in italics, and its introduction should be followed with a definition or descriptionof the term.

4.10. Continuous Integration 475


Example:

The |ATS| object storage is based on a *cyclone buffer* architecture.Cyclone buffers are most simply described as a fixed size, butcontinually looping block of storage updated by a single writerprocess, wherein the writer continually reclaims the oldest allocationsfor the most recent object updates.

Bold Bold typesetting is to be reserved for section, table, and glossary headings and should be avoided in paragraphcopy.

Monospace Used to indicate filesystem paths, variable names, language keywords and functions, and command out-put. Note that in the case of variables and functions, whenever possible references to their documentation shouldbe used in place of simple monospace markup.

Example:

Documentation source files for |TS| are located in the ``doc/``directory of the project source tree.

Bracketed Monospace Used to indicate, within command or source code examples, variables for which the readershould substitute a value.

Example:

To examine a performance statistic of a running |TS| instance, you mayrun the comand ``traffic_line -r <name>``, replacing ``<name>`` withthe statistic you wish to examine.

Ellipsis Used to indicate the omission of irrelevant or unimportant information. May also be used to separate matterto be treated in detail elsewhere.

Layout Styles

Block Content

Notes Use of .. note:: blocks should be sparing. In most rendered forms, the note block will appear quiteprominently and draw the readers’ eyes away from the surrounding copy. It is therefore advisable that the noteitself provide enough context and detail to be read on its own, without a full reading of the surrounding copy.

Important Notes The use of .. important:: callout blocks should be limited only to those situations in whichcritical information needs to be prominently displayed. Suitable use would include noting that resizing a cachevoiume will result in Traffic Server resetting the cache contents to empty when the service is started. Thisis information that may not be obvious, or safe to assume, for the reader but which can significantly (andnegatively) impact the use and performance of Traffic Server in their environment. Important note blocks shouldnot be used for behavior or actions which generally do not have potential negative side effects.

Sidebars The use of .. sidebar:: blocks in Traffic Server documentation should be avoided, and note blocksfavored in their place.

Code Samples Content should be set within .. code:: blocks whenever a full line or multiple lines of source codeare being included in the documentation, or when example shell or network commands are being demonstrated.Blank lines may be used to separate lines of code within the block for readability, and all normal indentationpractices should be followed. No additional markup should be used within the content block.

Definition Lists Definition lists may be used in multiple ways, not just for the term listings in the glossary section.They may be used to provide individual detailed treatment for a list of function or command arguments or whereany series of terms need to be explained outside of the formal glossary.



Ordered Lists Explicityly numbered ordered lists should be avoided. reStructuredText provides two methods ofmarking up ordered, numbered lists, and the automatic numbering form should be used in all cases wheresurrounding paragraphs do not need to reference individual list entries.

Tables

Tabular content may be used in any situation where a selection of items, terms, options, formats, etc. are beingcompared or where a grouping of the same are being presented alongside a small number of attributes which do notrequire detailed expositions.

The Traffic Server project documentation supplies a custom styling override which causes cell contents to wrap withinwide tables whenever possible in cases where it is necessary to prevent the content from overflowing into the margin.If, however, the cell content cannot be wrapped because there are no breaking spaces present (for example, a longvariable name containing only letters, periods, underscores, dashes, and so no, but no whitespace), the table may stillrequire overflowing into the page margin. Whenever possible, please try to avoid the use of tables when presentinginformation that will lead to this, as it greatly hampers readibility on smaller screens, especially tablets and mobiledevices. Alternatives such as a definition list may be better suited to the content.

Tables may be marked up using any of the reStructuredText styles, though it is generally easiest to maintain thoseusing the simple table markup style.

Structural

Common Definitions

The Traffic Server project documentation maintains a common definitions, abbreviations, and shortcut listing in thefile doc/common.defs. This file should be included by all reStructuredText source files after the standard projectcopyright notice.

The file should always be included using a relative path from the current file’s location. Any commonly or repeatedlyused abbreviations, especialy those of product, company, or person names, should be added to the definitions file asuseful to avoid repetitive typing and ensure accurate spellings or legal usage.

Tables of Content

Any chapters of non-trivial scope should begin with a table of contents on the chapter index. The :depth: optionfor the table of contents may be set to a value appropriate to the amount of content in the chapter sections. A depthof 2 will generally provide a balance between usefully describing the contents of the chapter without overwhelming areader scanning for topics relevant to their needs or interests.

Sections and Headings

Each chapter section should be located in a separate reStructuredText file. Each file should contain the standard projectcopyright notice first, followed by a unique reference marker for the section.

While reStructuredText itself does not define a fixed ordering of section markers, Traffic Server documentation sourcefiles should use the same set of single line section markings, proceeding through the section levels without skipping.For consistency, the following section line markers should be used in order:

Top Level

*********

4.11. Documentation 477


Second Level============

Third Level-----------

Fourth Level~~~~~~~~~~~~

Any section file which reaches or has need to exceed the fourth level style of section line markings is an excellentcandidate for breaking into several smaller, and ideally more focused, reStructuredText source files and referenced byan index with its own table of contents.

Footnotes and Endnotes

Both footnotes and endnotes should be avoided. The Traffic Server documentation is intended primarily for onlineviewing and the positioning of footnotes in the rendered output is not always intuitive. In cases where a footnote mighthave been appropriate in print-oriented material for referencing an external resource, that reference is more ideallyintegrated as a standard reStructuredText reference.

For more descriptive content that might have been included as a footnote, it is less disruptive and more useful to choosebetween reformullating the text to simply include the additional wording, or consider the use of an inline note block.

Grammatical

Structure

The Apache Traffic Server™ documentation is presented as a single manual, sub-divided into the following sections:

Preface The introductory section to the manual as a whole, with a brief exposition on Traffic Server and its generalrole in a network and application infrastructure. The typographic conventions used in the manual should becovered here to ease readers’ comprehension of the following sections, as well as pointers to additional resourcesavailable outside of the manual.

Getting Started Aimed at administrators and developers new to Traffic Server who wish to install and configure aTraffic Server instance in the least amount of time necessary, without delving into the full featureset or theinternals of the Traffic Server architecture.

Administrator’s Guide Provides in-depth coverage of all Traffic Server features and configurations for use by ad-ministrators responsible for installing, configuring, and maintaining Traffic Server instances. Troubleshooting,performance tuning, and plugin usage should all be covered in this section of the manual.

Developer’s Guide Documentation of internal Traffic Server architecture as well as guides on plugin developmentand debugging are presented in this section.

Appendices Glossary, common references to both administrator and developer guides (such as status code tables andcommand line utility usage), frequently asked questions, and errata should all be located in this final section ofthe manual.

Within each top-level section, a short introduction should be provided to the content of that section, as well as a briefnote on the intended audience. A very high level treatment of each chapter should be presented to allow readers tomore easily locate the most relevant chapters for their current needs.



Building the Documentation

All documentation and related files are located in the source tree under the doc/ directory. Makefiles are generatedautomatiically by the main configure script. For simplicity’s sake, it is recommended that contributors new to thedocumentation make use of the included Vagrant configurations, as these will take care of providing all dependencies.Please refer to the Using Vagrant to Test Traffic Server chapter for complete details on using Vagrant to build and testthe Traffic Server source tree.

With a configured source tree, building the documentation requires only the invocation make html from withindoc/. For repeated builds while working on the documentation, it is advisable to clean out the built and intermediatefiles each time by running the following instead (again, from within the doc/ directory of the Traffic Server sourcetree):

make clean && make && make html

This will ensure that make doesn’t inadvertantly skip the regeneration of any targets.

To view the built documentation, you may point any browser to the directory doc/docbuild/html/. If you arebuilding the documentation on your local machine, you may access the HTML documentation files directly withoutthe need for a full-fledged web server, as all necessary resources (CSS, Javascript, and images) are referenced usingrelative paths and there are no server-side scripts necessary to render the documentation.

reStructuredText and Sphinx

The Traffic Server documentation source files are written using reStructuredText (RST) markup. This is a plain-textformat intended to be easily readable both in its source and final rendered forms. While not entirely dissimilar to otherplain text markup formats like Markdown, RST provides additional functionality for defining internal links betweendocuments, producing tables of contents, and indices. Additionally, RST works in hand with Sphinx for providingadvanced features such as markup domains, of which this documenentation makes extensive use.

Both reStructuredText and Sphinx have official documentation resources which detail their full capabilities and allmarkup options, as well as methods of extending them with custom markup. This section will explain the mostcommon markup features used, but the following resources may be consulted for further details.

• reStructuredText Markup Specification

• Sphinx Documentation Generator

• RST Primary by Sphinx

Traffic Server Specific Markup

This section covers markup used specific to the Traffic Server documentation and custom domains provided by localextensions.

Types

Data types are documented using the standard :c:type: markup provided by Sphinx. Types provided by the C APIshould be documented in doc/developer-guide/api/types/

Constants

Functions


http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html

http://sphinx-doc.org/

http://sphinx-doc.org/rest.html


Custom Domains

The Traffic Server documentation provides a Sphinx extension, located at doc/ext/traffic-server.py whichdefines custom domains for various purposes. Those domains and their usage are documented here.

Configuration Variables

Configuration variables are documented with the :ts:cv: domain, which takes three required arguments (the scope,which is the literal string CONFIG, the variable name, and the data type) and a fourth optional argument (the defaultvalue).

:ts:cv: <scope> <variable name> <data type> <value>

Definition

Scope The scope of the variable. For configuration variables, this will always be the literal string CONFIG.

Variable Name The full and exact configuration variable name.

Data Type Indicates the data type of the variable and must be one of the following literal strings:

INT Any integer value. Values may optionally be expressed with a binary order of magnitude suffix; e.g. K forvalue * 1024, M for value * 1024^2, G for value * 1024^3, or T for value * 1024^4.

FLOAT Any floating point value.

STRING Any alphanumeric string.

Value The default value of the configuration variable. It is preferable to not use any order of magnitude suffix, asTraffic Server will rewrite its configuration files under varioua circumstances, and when doing so it does notmaintain those suffixes.

Options

The domain for configuration variables takes only a single option at this time.

deprecated A simple flag option which, when attached to a configuration variable, is used to indicate that the variablehas been deprecated and should no longer be used or relied upon, as it may be removed at any time by futurereleases.

References

References to configuration variables from elsewhere in the documentation should be made using the standard domainreference markup:

:ts:cv:`full.variable.name`

Statistics

Traffic Server statistics are documented using the domain :ts:stat:. The domain takes three required arguments(collection, name, data type) and an option fourth (an example value).



:ts:stat: <collection> <statistic name> <data type> <example-value>

Definition

Collection The key name of the collection in the returned JSON data from the Stats Over HTTP Plugin plugin. Formost statistics, this is the literal sting global. Required

Statistic Name The exact and full name of the statistic. Required

Data Type One of the following literal string values: integer, float, boolean, or string. Required

Example Value A valid example of the value which may be exported by the statistic. In cases where the descriptionof the statistic makes note of particular features of the values, such as the case of string statistics which may beformatted in specific ways, providing the optional example is strongly recommended. Optional

The statistics domain also supports several options which can provide even more metadata about the statistic. Theseare currently:

Options

type Defines the type of metric exposed by the statistic. Valid values are:

counter Numeric values which only increment based on the accumulation or occurrence of underlying events.Examples include the total number of incoming connections, or the total number of bytes transferred.

gauge Used for moment-in-time metrics, such as the current number of Traffic Server processes running, orthe current number of connections open to origin servers.

flag Indicates that values for this statistic will be an integer with each value indicating a particular state. Moststatistics of this type are booleans, where 1 indicates a truth or an on state, and 0 the opposite.

derivative Statistics of this type presents values that are calculated, or derived, from other staistics. Theydo not expose a number or state gathered directly. Typical statistics of this type are representations of astatistic over a given period (e.g. average origin connections per second), ratio or percentage of a statisticas part of a set (e.g. the percentage of total dns lookups which have failed), or any other statistic whosecomputation depends on the value(s) of one or more other statistics.

unit Indicates the units of measurement that should be assumed for the given statistic’s value.

introduced May be used to indicate the version of Traffic Server in which the statistic was first available. The valueof this option should be a valid, human-readable version number string, e.g. 5.3.0.

deprecated Used to indicate that the statistic is no longer supported and may be removed in later versions. This optionmay be used as a simple flag without any given value, or may have a value associated in which case it shouldbe a valid, human-readable version number string for the Traffic Server release which was first to deprecate thestatistic.

ungathered A simple flag option, without any associated values, indicating that while the statistic is included in theoutput of plugins like Stats Over HTTP Plugin there is no underlying data gathered for the statistic. If a statisticis thus marked, it should be assumed to be invalid or simply unimplemented.

References

To reference a statistic from elsewhere in the documentation, the standard domain reference markup should be used:



:ts:stat:`full.statistic.name`

References should not include the collection name, data type, or any other components aside from the statistic name.

Creating New Domains

In the event a new type of object or reference needs to be documented, and none of the existing markup options ordomains are appropriate, it is possible to extend reStructuredText and Sphinx by adding custom domains.

Each domain may be designed to accept any number of required and optional arguments, as well as any collectionof domain options, and each option may be designed to support arbitrary values, restricted (enumerated) values, or tosimply act as flags.

All custom domain definitions should be located in doc/ext/traffic-server.py and consist of, at a bareminimum, a domain class definition and a domain reference class definition. Sphinx domains are implemented inPython.

For this section, we will use the contrived example of creating a domain which permits us to define and reference a setof variables which are constrained by the following characteristics:

1. Each variable in the domain must be one of known list of data types, which we will limit here to the possibilitiesof integeer, float, string.

2. Where the data type is not specified, we can assume it is string.

3. Variables which are numeric in their type may have a range of permissible values.

4. Variables in the domain may still be present and supported in the system, but are planned to be removed in somefuture release.

5. Every variable is associated with a single URI protocol, though there is no validation performed on the valueused to represent the protocol name.

As stated, this example is fairly contrived and would not match any particularly likely real-world needs, but it willallow us to demonstrate the full extent of custom domain definition without needless complexity, reader’s suspensionof disbelief permitting.

For this chapter’s purpose, we will call this domain simply Variables, and we will construct classes which allow us todocument variables thusly:

.. ts:variable:: http_enabled http integer:deprecated:

Enables (any postive, non-zero value) or disables (any zero or negativevalue) processing of HTTP requests.

And referencing of those variables defined with this domain via:

:ts:variable:`http_enabled`

Defining the Domain

Each domain is defined by a class which inherits from std.Target. Several class attributes are expected, whichdetermine how domain object definitions are processed. Traffic Server convention is to name each domain’s class incamel case, beginning with TS to prevent any class name collisions with builtin Sphinx classes.



class TSVariable(std.Target):

We have named the domain’s defining class as TSVariable and inherited from the std.Target class. Given theearlier stated requirements, we need a domain which supports at least two required attributes (a name, of course, and aURI protocol with which it is associated) and a commonly defined, though optional, third attribute (a data type). We’lldeal with the value ranges and deprecation status later.

class TSVariable(std.Target):required_arguments = 2optional_arguments = 1final_argument_whitespace = Falsehas_content = True

We’ve now specified the appropriate number of required and optional arguments, though not what each one happensto be or in what order the required arguments need be written. Additionally, we’ve declared that definitions using thisdomain do not permit whitespace in the final argument, but definitions can have a block of text content which followsthem and should be associated with the item being defined.

: Permitting whitespace in the final argument causes the final value of a valid definition to slurp the remaining contentof the definition. Normally, each argument is separated by whitespace, thus foo bar baz would only be a validdefinition if the domain’s required and optional argument counts added up to exactly three. If the domain defined onlytwo arguments as expected, but sets final_argument_whitespace to True, then the definition would be validand the second argument in this case would be bar baz.

Our requirements also state support for optional value ranges, and a flag to indicate whether the variable is beingdeprecated. These can easily be supported through the option_spec, which allows for options to be tagged on to adomain item, on the lines immediately following its definition.

class TSVariable(std.Target):...option_spec = {

'deprecated' : rst.directives.flag,'range' : rst.directives.unchanged

}

For our example, deprecated is simply a boolean flag, and range will be an arbitrary string on which we willperform no particular transformation or validation (good behavior will be left up to those documenting their variableswith this domain). The rst.directives module may be consulted for a wider range of predefined option types,including the ability to define your own types which can perform any complexity of validation you may desire toimplement.

It would be good form to also include a docstring for the class explaining the expected arguments in brief. With thatincluded, our class now looks like:

class TSVariable(std.Target):"""Description of a Traffic Server protocol variable.

Required arguments, in order, are:

URI ProtocolVariable name

Optional argument is the data type of the variable, with "string" thethe default. Possible values are: "string", "integer", and "float".



Options supported are:

:deprecated: - A simple flag option indicating whether the variableis slated for removal in future releases.

:range: - A string describing the permissible range of values thevariable may contain.

"""

option_spec = {'deprecated' : rst.directives.flag,'range' : rst.directives.unchanged

}

required_arguments = 2optional_arguments = 1final_argument_whitespace = Falsehas_content = True

Every domain class must also provide a run method, which is called every time an item definition using the domainis encountered. This method is where all argument and option validations are performed, and where transformation ofthe definition into the documentation’s rendered output occurs.

The core responsibilities of the run method in a domain class are to populate the domain’s data dictionary, for use byreferences, as well as to transform the item’s definition into a document structure suitable for rendering. The defaulttitle to be used for references will be constructed in this method, and all arguments and options will be processed.

Our variables domain might have the following run method:

def run(self):var_name, var_proto = self.arguments[0:2]var_type = 'string'

if (len(self.arguments) > 2):var_type = self.arguments[2]

# Create a documentation node to use as the parent.node = sphinx.addnodes.desc()node.document = self.state.documentnode['objtype'] = 'variable'

# Add the signature child node for permalinks.title = sphinx.addnodes.desc_signature(var_name, '')title['ids'].append(nodes.make_id('variable-'+var_name))title['names'].append(var_name)title['first'] = Falsetitle['objtype'] = node['objtype']self.add_name(title)title.set_class('ts-variable-title')

title += sphinx.addnodes.desc_name(var_name, var_name)node.append(title)

env.domaindata['ts']['variable'][var_name] = env.docname

# Create table detailing all provided domain optionsfl = nodes.field_list()



if ('deprecated' in self.options):fl.append(self.make_field('Deprecated', 'Yes'))

if ('range' in self.options):fl.append(self.make_field('Value range:', self.options['range']))

# Parse any associated block content for the item's descriptionnn = nodes.compound()self.state.nested_parse(self.content, self.content_offset, nn)

# Create an index node so Sphinx will list this variable and its# references in the index section.indexnode = sphinx.addnodes.index(entries=[])indexnode['entries'].append(

('single', _('%s') % var_name, nodes.make_id(var_name), ''))

return [ indexnode, node, fl, nn ]

Defining the Domain Reference

Domain reference definitions are quite simple in comparison to the full domain definition. As with the domain itself,they are defined by a single class, but inherit from XRefRole instead. There are no attributes necessary, and only asingle method, process_link need be defined.

For our variables domain references, the class definition is a very short one. Traffic Server convention is to namethe reference class the same as the domain class, but with Ref appended to the name. Thus, the domain classTSVariable is accompanied by a TSVariableRef reference class.

class TSVariableRef(XRefRole):def process_link(self, env, ref_node, explicit_title_p, title, target):

return title, target

The process_link method will receive several arguments, as described below, and should return two values: astring containing the title of the reference, and a hyperlink target to be used for the rendered documentation.

The process_link method receives the following arguments:

self The reference instance object, as per Python method conventions.

env A dictionary object containing the environment of the documentation processor in its state at the time of thereference encounter.

ref_node The node object of the reference as encountered in the documentation source.

explicit_title_p Contains the text content of the reference’s explicit title overriding, if present in the referencemarkup.

title The processed form of the reference title, which may be the result of domain class transformations or anoverriding of the reference title within the reference itself.

target The computed target of the reference, suitable for use by Sphinx to construct hyperlinks to the location ofthe item’s definition, wherever it may reside in the final rendered form of the documentation.

In our reference class, we have simply returned the processed title (allowing the documentation to override the vari-able’s name if desired, or defaulting to the domain class’s representation of the variable name in all other cases) andthe parser’s computed target.



It is recommended to leave the target untouched, however you may choose to perform any transformations youwish on the value of the title, bearing in mind that whatever string is returned will appear verbatim in the rendereddocumentation everywhere references for this domain are used.

Exporting the Domain

With both the domain itself and references to it now defined, the final step is to register those classes as domain andreference handlers in a namespace. This is done for Traffic Server (in its :ts: namespace) quite easily by modifyingthe TrafficServerDomain class, also located in doc/ext/traffic-server.py.

The following dictionaries defined by that class should be updated to include the new domain and reference. In eachcase, the key used when adding to the dictionary should be the string you wish to use in documentation markup foryour new domain. In our example’s case, we will choose variable since it aligns with the Python classes we’vecreated above, and their contrived purpose.

object_types Used to define the actual markup string

directives Defines which class is used to implement a given domain.

roles Defines the class used to implement references to a domain.

initial_data Used to initialized the dictionary which tracks all encountered instances of each domain. This shouldalways be set to an empty dictionary for each domain.

dangling_warnings May be used to provide a default warning if a reference is attempted to a non-existent item for adomain.

Documenting Plugins

Plugins can provide tremendously useful functionality for users of Traffic Server, but it does them little good if thereare no instructions on how to use the plugins properly. In the interest of encouraging useful documentation on all thefeatures available to those deploying Traffic Server in their infrastructure, this section lays out a series of guidelinesand offers a basic template for adding new entries to the Plugins chapter of the Administrator’s Guide.

Structure

Ideally, every plugin will provide documentation covering the following general topics (preferably in top-level sectionsnamed to match):

Introduction

A brief section in which the plugin’s core functionality is explained in a quickly digestable paragraph or two. For thesake of brevity, this section should omit discussion of configuration details in favor of providing a more 10,000 footview of the plugin features.

Purpose

Why would someone want to use this particular plugin? What problems does it solve, or what features not attainableotherwise (through core functionality or other plugins) does it provide to an administrator or user? While not exactlya product pitch, this is the section in which the case for using the plugin at all should be made.



Dependencies

If the plugin has external dependencies, they should be listed in full, with the exception of items already required by abare-bones Traffic Server build. If the plugin has no unusual or non-standard dependencies, this section may be safelyomitted.

For example, there is no need to list a C++ compiler as a requirement, but if the plugin requires OpenCL and a specificbrand of GPU expansion card to perform some wacky image transformations on cached objects - that really should bedisclosed.

If any of the dependencies are optional, that should be noted as well as the functionality that may be missing shouldthe dependency not be present.

Installation

Detailed instructions on how to install the plugin if it is not built by default with Traffic Server or requires any additionallibraries, services, or applications outside of Traffic Server to be present.

If the plugin is marked stable and included by default in Traffic Server builds, then it may not be necessary to includeany installation steps beyond noting that it is part of the default Traffic Server.

Configuration

Detailed instructions on how and where to provide configuration data for the plugin’s features. It is important touse reStructuredText references for all mentions of Traffic Server configuration variables, as well as their containingconfiguration files.

If configurations are generated by scripts included with the plugin source code, those scripts should be mentioned andtheir usage should be shown.

Caveats (or Limitations)

Notes about any peculiarities with the plugin, incompatibilities or strange behaviors when used in conjunction withother features, or special considerations for its deployment that might affect an administrator’s use of the plugin ortheir infrastructure’s architecture.

If the plugin contains known limitations (perhaps it only works with some protocols, can understand only some mediatypes, or is especially non-performant under specific configurations) they should be noted. If possible, workaroundsfor these limitations should be suggested, or if none are available that should be noted.

Examples

Every plugin should aim to have at least one, complete example of its usage beginning with its basic installation,showing a complete (and ideally working) configuration, as well as sample output from Traffic Server in which theplugin’s effects may be observed.

Further Reading

Consolidated list of references to outside materials and documents which may be relevant to the plugin, such as RFCsand white papers. If the plugin is related to external projects, or has been the subject of benchmarks or articles, thosemay be useful to list as well.



Additional Sections

Not all plugins will fit perfectly within that list of sections. Some may require additional details, or such extendeddiscussion that it is best to break the plugin’s documentation down a bit further. In general, it is recommended to placethese additional documentation sections in between Configuration and Caveats or Limitations.

This ordering permits a quick scanning of the plugin’s purpose, basic method of installation, and initial overview ofits configuration before becoming mired down in dissertation length details of every possible configuration, warning,and trade-off that may affect deployments.

In cases where the array of configuration possibilities is extensive, it is recommended that the initial Configurationsection describe the most common configuration scenario(s), while noting that there are many other possibilities thatwill be covered in the sections that follow. This, it is hoped, will get the usual case out of the way quickly for mostTraffic Server administrators while limiting the paralysis of choice they may face otherwise.

Documentation Template

For the convenience of plugin developers, this manual includes a plugin documentation template which you may useas a base. The template includes placeholders, with markup, for common features of the documentation which may berelevant for your plugin. Developers are very strongly encouraged to use this template whenever possible, as a meansof producing and maintaining a consistent format for plugin documentation.

Host Resolution Proposal

Introduction

The current mechanism for resolving host names to IP addresses for Traffic Server is contained the HostDB and DNSlibraries. These take hostnames and provide IP addresses for them.

The current implementation is generally considered inadequate, both from a functionality point of view and difficultyin working with it in other parts of Traffic Server. As Traffic Server is used in more complex situtations this inadequacypresents increasing problems.

Goals

Updating the host name resolution (currently referred to as “HostDB”) has several functions goals

• Enable additional processing layers to be easily added.

• Enable plugins to directly access the name resolution logic

• Enable plugins to provide name resolution

• Asynchronous (immediate resolve or callback on block)

• Minimize allocations – in particular no allocations for cached resolutions

• Simplify interactions with the resolution, particularly with regard to nameservers, origin server failover, andaddress family handling.

It is also necessary to support a number of specific features that are either currently available or strongly desired.

• SplitDNS or its equivalent

• Use of a hosts file (e.g. /etc/hosts)

• Simultaneous IPv4 and IPv6 queries



• IP family control

• Negative caching * Server connection failures * Query failures * Nameserver failures.

• Address validity time out control

• Address round robin support

• SRV record support (weighted records)

• Nameserver round robin

• Plugin access to nameserver data (add, remove, enumerate)

• Plugin provision of resolvers.

• Hooks for plugin detection / recovery from resolution events.

One issue is persistence of the cached resolutions. This creates problems for the current implementation (because ofsize limits it imposes on the cached data) but also allows for quicker restarts in a busy environment.

Basics

The basic design is to separate the functionality into chainable layers so that a resolver with the desired attributes canbe assembled from those layers. The core interface is that of a lazy iterator. This object returns one of four resultswhen asked for an address

• An IP address

• Done(no more addresses are available)

• Wait(an address may be available in the future)

• Fail (no address is available and none will be so in the future)

Each layer (except the bottom) uses this API and also provides it. This enables higher level logic such as the statemachine to simply use the resolver as a list without having to backtrack states in the case of failures, or have specialcases for different resolution sources.

To perform a resolution, a client creates a query object (potentially on the stack), initializes it with the required data(at least the hostname) and then starts the resolution. Methods on the query object allow its state and IP address datato be accessed.

Required Resolvers

Nameserver A bottom level resolver that directly queries a nameserver for DNS data. This contains much of thefunctionality currently in the iocore/dns directory.

SplitDNS A resolver that directs requests to one of several resolvers. To emulate current behavior these would beNameserver instances.

NameserverGroup A grouping mechanism for Nameserver instances that provides failover, round robin, and order-ing capabilities. It may be reasonable to merge this with the SplitDNS resolver.

HostFile A resolver that uses a local file to resolve names.

AddressCache A resolver that also has a cache for resolution results. It requires another resolver instance to performthe actual resolution.

Preloaded A resolver that can contain one or more explicitly set IP addresses which are returned. When those areexhausted it falls back to another resolver.

4.12. Host Resolution Proposal 489


Configuration

To configuration the resolution, each resolver would be assigned a tag. It is not, however, sufficient to simply providethe list of resolver tags because some resolvers require additional configuration. Unfortunately this will likely requirea separate configuration file outside of records.config, although we would be able to remove splitdns.config. In this case we would need chain start / end markers around a list of resolver tags. Each tag would the beable to take additional resolver configuration data. For instance, for a SplitDNS resolver the nameservers.

Examples

Transparent operations would benefit from the Preloaded resolver. This would be loaded with the origin host addressprovided by the client connection. This could be done early in processing and then no more logic would be required toskip DNS processing as it would happen without additional action by the state machine. It would handle the problemof de facto denial of service if an origin server becomes unavailable in that configuration, as Preloaded would switchto alternate addresses automatically.

Adding host file access would be easier as well, as it could be done in a much more modular fashion and then added tothe stack at configuration time. Whether such addresses were cached would be controlled by chain arrangement ratheryet more configuration knobs.

The default configuration would be Preloaded : AddressCache : Nameserver.

In all cases the state machine makes requests against the request object to get IP addresses as needed.

Issues

Request object allocation

The biggest hurdle is being able to unwind a resolver chain when a block is encountered. There are some ways to dealwith this.

1) Set a maximum resolver chain length and declare the request instance so that there is storage for state for that manyresolvers. If needed and additional value of maximum storage per chain could be set as well. The expected number ofelements in a chain is expected to be limited, 10 would likely be a reaosnable limit. If settable at source configurationtime this should be sufficient.

2) Embed class allocators in resolver chains and mark the top / outermost / first resolver. The maximum state size fora resolution can be calculated when the chain is created and then the top level resolver can use an allocation pool toefficiently allocate request objects. This has an advantage that with a wrapper class the request object can be passedalong cheaply. Whether that’s an advantage in practice is unclear.

Plugin resolvers

If plugins can provide resolvers, how can these can integrated in to existing resolver chains for use by the HTTP SMfor instance?

Feedback

It should be possible for a client to provide feedback about addresses (e.g., the origin server at this address is notavailable). Not all resolvers will handle feedback but some will and that must be possible.

Related to this is that caching resolvers (such as AddressCache) must be able to iterator over all resolved addresseseven if their client does not ask for them. In effect they must background fill the address data.


CHAPTER 5

Appendices

Command Line Utilities

traffic_cop

Description

traffic_cop is a watchdog program that is responsible for starting traffic_manager and traffic_server and moni-toring them for responsiveness. If either of these processes are determined to be unresponsive, traffic_cop willkill and restart them.

On Linux, traffic_cop will also monitor available memory and swap space, restarting the watched pro-cesses if the available memory falls below a minimum threshold. The memory thresholds can be configured withthe :ts:cv:‘proxy.config.cop.linux_min_swapfree_kb‘ and :ts:cv:‘proxy.config.cop.linux_min_memfree_kb‘ vari-ables.

Options

-d, --debugEmit debugging messages.

-o, --stdouttraffic_cop ordinarily logs to syslog, however for debugging purposes, this option causes it to print mes-sages to standard output instead.

-s, --stopKill children using SIGSTOP instead of SIGKILL. This option is primarily for debugging.

-V, --versionPrint version information and exit.

491


See also

syslog(1), traffic_manager(8), traffic_server(8)

traffic_crashlog

Synopsis

traffic_crashlog [options]

Description

traffic_crashlog is a helper process that catches Traffic Server crashes and writes a crash report log to thelogging directory. Other than for debugging or development purposes, traffic_crashlog is not intended forusers to run directly.

When traffic_server starts, it will launch a traffic_crashlog process and keep it stopped, activating it only if acrash occurs.

Options

--host TRIPLEThis option specifies the host triple for the process that traffic_crashlog should examine. If a supportedhost triple is specified, traffic_crashlog expects to receive a siginfo_t structure on it’s standard input,followed by a ucontext_t.

--target PIDSpecifies the process ID of the crashing traffic_server process. If this option is not specified,traffic_crashlog assumes it should examine it’s parent process.

--syslogThis option causes traffic_crashlog to log the name of the crash log it creates to the system log.

--debugThis option enables debugging mode. In this mode, traffic_crashlog emits the log to it’s standard output.

--waitThis option causes traffic_crashlog to stop itself immediately after it is launched. traffic_serverwill allow it to continue only when a crash occurs.

Caveats

traffic_crashlog makes use of various Traffic Server management APIs. If traffic_manager is not available,the crash log will be incomplete.

traffic_crashlog may generate a crash log containing information you would rather not share outside yourorganization. Please examine the crash log carefully before posting it in a public forum.

See also

records.config(5), traffic_manager(8), traffic_server(8)

492 Chapter 5. Appendices


traffic_ctl

Synopsis

traffic_ctl [OPTIONS] SUBCOMMAND [OPTIONS]

Description

traffic_ctl is used to display and manipulate configure a running Traffic Server. traffic_ctl includes anumber of subcommands that control different aspects of Traffic Server:

traffic_ctl alarm Display and manipulate Traffic Server alarms

traffic_ctl cluster Stop, restart and examine the cluster

traffic_ctl config Manipulate and display configuration records

traffic_ctl metric Manipulate performance and status metrics

traffic_ctl server Stop, restart and examine the server

traffic_ctl storage Manipulate cache storage

Options

--debugEnable debugging output.


Subcommands

traffic_ctl alarm

listList all alarm events that have not been acknowledged (cleared).

clearClear (acknowledge) all current alarms.

resolve ALARM [ALARM...]Clear (acknowledge) an alarm event. The arguments are a specific alarm number (e.g. “1’‘), or an alarm stringidentifier (e.g. ‘’MGMT_ALARM_PROXY_CONFIG_ERROR’‘).

traffic_ctl cluster

restart [--drain] [--manager]Shut down and immediately restart Traffic Server, node by node across the cluster. The –drain and –manageroptions have the same behavior as for the traffic_ctl server restart subcommand.

statusShow the current cluster status.

5.1. Command Line Utilities 493


traffic_ctl config

defaults [--records]Display the default values for all configuration records. The –records flag has the same behavior astraffic_ctl config get --records.

describe RECORD [RECORD...]Display all the known information about a configuration record. This includes the current and default values,the data type, the record class and syntax checking expression.

diff [--records]Display configuration records that have non-default values. The –records flag has the same behavior astraffic_ctl config get --records.

get [--records] RECORD [RECORD...]Display the current value of a configuration record.

--recordsIf this flag is provided, traffic_ctl config get will emit results in records.config format.

match [--records] REGEX [REGEX...]Display the current values of all configuration variables whose names match the given regular expression. The–records flag has the same behavior as traffic_ctl config get --records.

reloadInitiate a Traffic Server configuration reload. Use this command to update the running configuration after anyconfiguration file modification.

The timestamp of the last reconfiguration event (in seconds since epoch) is published in theproxy.node.config.reconfigure_time metric.

set RECORD VALUESet the named configuration record to the specified value. Refer to the records.config documentation fora list of the configuration variables you can specify. Note that this is not a synchronous operation.

statusDisplay detailed status about the Traffic Server configuration system. This includes version information, whetherthe internal configuration store is current and whether any daemon processes should be restarted.

traffic_ctl metric

clear [--cluster]Reset all statistics to zero. The –cluster option applies this across all cluster nodes.

get METRIC [METRIC...]Display the current value of the specifies statistics.

match REGEX [REGEX...]Display the current values of all statistics whose names match the given regular expression.

zero [--cluster] METRIC [METRIC...]Reset the named statistics to zero. The –cluster option applies this across all cluster nodes.

traffic_ctl server

restartShut down and immediately restart Traffic Server



--drainThis option modifies the behavior of traffic_ctl server restart such that traffic_serveris not shut down until the number of active client connections drops to the number given by the:ts:cv:‘proxy.config.restart.active_client_threshold‘ configuration variable.

--managerThe default behavior of traffic_ctl server restart is to restart traffic_server. If this optionis specified, traffic_manager is also restarted.

statusShow the current proxy server status, indicating if we’re running or not.

backtraceShow a full stack trace of all the traffic_server threads.

traffic_ctl storage

offline DEVICE [DEVICE ...]Mark a cache storage device as offline. The storage is identified by a path which must match exactly a pathspecified in storage.config. This removes the storage from the cache and redirects requests that wouldhave used this storage to other storage. This has exactly the same effect as a disk failure for that storage. Thisdoes not persist across restarts of the traffic_server process.

Examples

Configure Traffic Server to log in Squid format:

$ traffic_ctl config set proxy.config.log.squid_log_enabled 1$ traffic_ctl config set proxy.config.log.squid_log_is_ascii 1$ traffic_ctl config reload

See also

records.config(5), storage.config(5)

traffic_line

: This utility is deprecated as of v6.0.0, and replaced with traffic_ctl. You should change your tools / scripts touse this new application instead.

Synopsis

traffic_line [options]

Description

traffic_line is used to execute individual Traffic Server commands and to script multiple commands in a shell.



Options

-B, --bounce_clusterBounce all Traffic Server nodes in the cluster. Bouncing Traffic Server shuts down and immediately restartsTraffic Server, node-by-node.

-b, --bounce_localBounce Traffic Server on the local node. Bouncing Traffic Server shuts down and immediately restarts theTraffic Server node.

-C, --clear_clusterClears accumulated statistics on all nodes in the cluster.

-c, --clear_nodeClears accumulated statistics on the local node.

--drainThis option modifies the behavior of traffic_line -b and traffic_line -L such thattraffic_server is not shut down until the number of active client connections drops to the number givenby the :ts:cv:‘proxy.config.restart.active_client_threshold‘ configuration variable.

-h, --helpPrint usage information and exit.

-L, --restart_localRestart the traffic_manager and traffic_server processes on the local node.

-M, --restart_clusterRestart the traffic_manager process and the traffic_server process on all the nodes in a cluster.

-m REGEX, --match_var REGEXDisplay the current values of all performance statistics or configuration variables whose names match the givenregular expression.

-r VAR, --read_var VARDisplay specific performance statistics or a current configuration setting.

-s VAR, --set_var VARSet the configuration variable named VAR. The value of the configuration variable is given by thetraffic_line -v option. Refer to the records.config documentation for a list of the configurationvariables you can specify.

-S, --shutdownShut down Traffic Server on the local node.

-U, --startupStart Traffic Server on the local node.

-v VALUE, --value VALUESpecify the value to set when setting a configuration variable.


-x, --reread_configInitiate a Traffic Server configuration file reread. Use this command to update the running configuration afterany configuration file modification.

The timestamp of the last reconfiguration event (in seconds since epoch) is published in theproxy.node.config.reconfigure_time metric.



-Z, --zero_clusterReset performance statistics to zero across the cluster.

-z, --zero_nodeReset performance statistics to zero on the local node.

--offline PATHMark a cache storage device as offline. The storage is identified by a path which must match exactly a pathspecified in storage.config. This removes the storage from the cache and redirects requests that wouldhave used this storage to other storage. This has exactly the same effect as a disk failure for that storage. Thisdoes not persist across restarts of the traffic_server process.

--alarmsList all alarm events that have not been acknowledged (cleared).

--clear_alarms [all | #event | name]Clear (acknowledge) an alarm event. The arguments are “all” for all current alarms, a specific alarm number(e.g. “1’‘), or an alarm string identifier (e.g. ‘’MGMT_ALARM_PROXY_CONFIG_ERROR’‘).

--statusShow the current proxy server status, indicating if we’re running or not.

Performance Statistics

proxy.process.ssl.user_agent_other_errors Total number of other ssl client connection errors (counts ssl errors thatare not captured in other user agent stats below)

proxy.process.ssl.user_agent_expired_cert Total number of ssl client connection failures where the cert was ex-pired.

proxy.process.ssl.user_agent_revoked_cert Total number of ssl client connection failures where the cert was re-voked.

proxy.process.ssl.user_agent_unknown_cert Total number of ssl client connection failures related to the cert, butspecific error was unknown.

proxy.process.ssl.user_agent_cert_verify_failed Total number of ssl client connection failures where cert verifica-tion failed.

proxy.process.ssl.user_agent_bad_cert Total number of ssl client connection failures where the cert is bad.

proxy.process.ssl.user_agent_decryption_failed Total number of ssl client connection decryption failures (duringnegotiation).

proxy.process.ssl.user_agent_wrong_version Total number of ssl client connections that provided an invalid proto-col version.

proxy.process.ssl.user_agent_unknown_ca Total number of ssl client connection that failed due to unknown ca.

proxy.process.ssl.origin_server_other_errors Total number of other ssl origin server connection errors (counts sslerrors that are not captured in other origin server stats below).

proxy.process.ssl.origin_server_expired_cert Total number of ssl origin server connection failures where the certwas expired.

proxy.process.ssl.origin_server_revoked_cert Total number of ssl origin server connection failures where the certwas revoked.

proxy.process.ssl.origin_server_unknown_cert Total number of ssl origin server connection failures related to thecert where specific error was unknown.



proxy.process.ssl.origin_server_cert_verify_failed Total number of ssl origin server connection failures where certverification failed.

proxy.process.ssl.origin_server_bad_cert Total number of ssl origin server connection failures where the cert is bad.

proxy.process.ssl.origin_server_decryption_failed Total number of ssl origin server connection decryption failures(during negotiation).

proxy.process.ssl.origin_server_wrong_version Total number of ssl origin server connections that provided an in-valid protocol version.

proxy.process.ssl.origin_server_unknown_ca Total number of ssl origin server connection that failed due to un-known ca.

proxy.process.ssl.user_agent_sessions Total number of ssl/tls sessions created.

proxy.process.ssl.user_agent_session_hit Total number of session hits. A previous session was reused which re-sulted in an abbreviated ssl client negotiation.

proxy.process.ssl.user_agent_session_miss Total number of session misses. The ssl client provided a session id thatwas not found in cache and, therefore, could not be used.

proxy.process.ssl.user_agent_session_timeout Total number of session timeouts. The ssl client provided a session,but it could not be used because it was past the session timeout.

proxy.process.ssl.cipher.user_agent.{CIPHERNAME} Total number of ssl client connections that used ci-pherName. The list of cipher statistics is dynamic and depends upon the installed ciphers and the:ts:cv:‘proxy.config.ssl.server.cipher_suite‘ configuration. The set of cipher statistics can be discovered withtraffic_line -m. For example:

$ traffic_line -m proxy.process.ssl.cipher.user_agent.proxy.process.ssl.cipher.user_agent.ECDHE-RSA-AES256-GCM-SHA384 0proxy.process.ssl.cipher.user_agent.ECDHE-ECDSA-AES256-GCM-SHA384 0proxy.process.ssl.cipher.user_agent.ECDHE-RSA-AES256-SHA384 0proxy.process.ssl.cipher.user_agent.ECDHE-ECDSA-AES256-SHA384 0...

Cache Statistics

Cache statistics come in two varieties, global and per cache volume. These will be listed here in the global form.To get a cache volume statistic add .volume_# to the name after cache where # is 1-based index of the volume instorage.config. For example the statistic proxy.process.cache.sync.bytes is a global statistic. The value for thethird cache volume is proxy.process.cache.volume_3.sync.bytes.

proxy.process.cache.sync.bytes The total number of bytes written to disk to synchronize the cache directory.

proxy.process.cache.sync.time The total time, in nanoseconds, during which the cache directory was being writtento disk.

proxy.process.cache.sync.count The number of times a cache directory sync has been done.

proxy.process.cache.wrap_count The number of times a cache stripe has cycled. Each stripe is a circular buffer andthis is incremented each time the write cursor is reset to the start of the stripe.

Examples

Configure Traffic Server to log in Squid format:



$ traffic_line -s proxy.config.log.squid_log_enabled -v 1$ traffic_line -s proxy.config.log.squid_log_is_ascii -v 1$ traffic_ctl config reload

See also

records.config(5), storage.config(5)

traffic_logcat

Synopsis

traffic_logcat [-o output-file | -a] [-CEhSVw2] [input-file ...]

Description

To analyse a binary log file using standard tools, you must first convert it to ASCII. traffic_logcat does exactlythat.

Options

-o PATH, --output_file PATH

Specifies where the command output is directed.

-a, --auto_filename

Automatically generates the output filename based on the input filename. If the input is from stdin, then this option isignored. For example:

traffic_logcat -a squid-1.blog squid-2.blog squid-3.blog

generates:

squid-1.log squid-2.log squid-3.log

-f, --follow

Follows the file, like tail(1) -f

-C, --clf

Attempts to transform the input to Netscape Common format, if possible.

-E, --elf

Attempts to transform the input to Netscape Extended format, if possible.

-S, --squid

Attempts to transform the input to Squid format, if possible.

-2, --elf2

Attempt to transform the input to Netscape Extended-2 format, if possible.

-T, --debug_tags



-w, --overwrite_output



: Use only one of the following options at any given time: -S, -C, -E, or -2.

If no input files are specified, then traffic_logcat reads from the standard input (stdin). If you do not specifyan output file, then traffic_logcat writes to the standard output (stdout).

For example, to convert a binary log file to an ASCII file, you can use the traffic_logcat command with eitherof the following options below:

traffic_logcat binary_file > ascii_filetraffic_logcat -o ascii_file binary_file

The binary log file is not modified by this command.

See Also

tail(1)

traffic_logstats

Description

Options

-f FILE, --log_file FILE

-o LIST, --origin_list LIST

-O FILE, --origin_file FILE

-M COUNT, --max_origins COUNT

-u COUNT, --urls COUNT

-U COUNT, --show_urls COUNT

--A, --as_object

-i, --incremental

-S FILE, --statetag FILE

-t, --tail

-s, --summary

-j, --json

-c, --cgi

-m, --min_hits

-a, --max_age



-l COUNT, --line_len COUNT

-T TAGS, --debug_tags TAGS



traffic_manager

Description

--aconfPort PORT

--action TAGS

--clusterMCPort PORT

--clusterRSPort PORT

--debug TAGS

--groupAddr ADDRESS

--help

--nosyslog

--path FILE

--proxyBackDoor PORT

--proxyOff

--proxyPort PORT

--recordsConf FILE

--tsArgs ARGUMENTS

--version

Environment

MGMT_ACONF_PORT

MGMT_CLUSTER_MC_PORT

MGMT_CLUSTER_RS_PORT

MGMT_GROUP_ADDR

Signals

SIGHUP This signal causes a reconfiguration event, equivalent to running traffic_ctl config reload.

See also

traffic_ctl(8)



traffic_server

Description

Options

-n COUNT, --net_threads COUNT

-Z COUNT, --cluster_threads COUNT

-U COUNT, --udp_threads COUNT

-a, --accepts_thread

-b, --accept_till_done

-p PORT, --httpport PORT

-P PORT, --cluster_port PORT

-f, --disable_freelist

In order to improve performance, traffic_server caches commonly used data structures in a set of free objectlists. This option disables these caches, causing traffic_server to use malloc(3) for every allocation. Thoughthis option should not commonly be needed, it may be beneficial in memory-constrained environments or where theworking set is highly variable.

-o LEVEL, --dprintf_level LEVEL

-R LEVEL, --regression LEVEL

-r TEST, --regression_rest TEST

-T TAGS, --debug_tags TAGS

-B TAGS, --action_tags TAGS

-i COUNT, --interval COUNT

-M, --remote_management

-C CMD, --command CMD

-k, --clear_hostdb

-K, --clear_cache

-c CORE, --read_core CORE

--accept_mss MSS

-t MSECS, --poll_timeout MSECS



Environment

PROXY_REMOTE_MGMT

PROXY_AUTO_EXIT



traffic_top

Description

Options

-s COUNT

traffic_via

Synopsis

traffic_via [OPTIONS] [VIA ...]

Description

traffic_via decodes the Traffic Server Via header codes. Via header strings are passed as command-line options.The enclosing square brackets are optional.

If the argument is ‘-‘, traffic_via will filter standard input for Via headers. This modes supports only supportsVia headers that are enclosed by square brackets.

Options


-V, --versionPrint version.

Examples

Decode the Via header from command-line arguments:

$ traffic_via "[uScMsEf p eC:t cCMi p sF]"Via header is uScMsEf p eC:t cCMi p sF, Length is 24Via Header Details:Request headers received from client :simple request (not→˓conditional)Result of Traffic Server cache lookup for URL :miss (a cache "MISS")Response information received from origin server :error in responseResult of document write-to-cache: :no cache write performedProxy operation result :unknownError codes (if any) :connection to server failedTunnel info :no tunnelingCache Type :cacheCache Lookup Result :cache miss (url not in cache)ICP status :no icpParent proxy connection status :no parent proxy or unknownOrigin server connection status :connection open failed

Decode the Via header from a curl request, using the X-Debug plugin:



$ curl -H "X-Debug: Via" -I http://test.example.com | traffic_via -Via header is uScMsSf pSeN:t cCMi p sS, Length is 24Via Header Details:Request headers received from client :simple request (not→˓conditional)Result of Traffic Server cache lookup for URL :miss (a cache "MISS")Response information received from origin server :connection opened successfullyResult of document write-to-cache: :no cache write performedProxy operation result :served or connection opened→˓successfullyError codes (if any) :no errorTunnel info :no tunnelingCache Type :cacheCache Lookup Result :cache miss (url not in cache)ICP status :no icpParent proxy connection status :no parent proxy or unknownOrigin server connection status :connection opened successfully

traffic_wccp

Description

Front end to the wccp client library. It is a stand alone program that speaks the client side of the WCCP cache protocol.

It can be used instead of the built in WCCP feature in Traffic Server. This can be beneficial if you have multipleprograms running on the same computer that are relying on WCCP to redirect traffic from the router to the computer.

Since it relies on the wccp library, traffic_wccp is only built if Traffic Server is configured with--enable-wccp.

The overall Apache Traffic Server WCCP configuration documentation is at WCCP Configuration

Options

--address IP address to bind.

--router Booststrap IP address for routers.

--service Path to service group definitions.

--debug Print debugging information.

--daemon Run as daemon.

You need to run at least with the --service arguments. An example service definition file,service-nogre-example.config, is included in the cmd/traffic_wccp directory. In this file you de-fine your MD5 security password (highly recommended), and you define your service groups. The details of theservice file are defined at WCCP Service Configuration.

Limitations

The current WCCP implementation associated with ATS only supports one cache client per service group per router.The cache assignment logic currently assigns the current cache client to all buckets.



tspush

Description

Options

-f FILE, --file FILE

-u URL, --url URL

-v, --verbose


tsxs

Description

Options

-q VAR, --query VAR

-I PATH

-L PATH

-l LIB

-o FILE

-i

-c FILE [FILE ...]

-C FILE [FILE ...]

-v, --verbose


Frequently Asked Questions

How do you create a raw disk for the cache if all your disks have mounted filesystems?

Create a large file on filesystem (with dd(1)) and mount it as loopback device. This is accomplished withlosetup(8) on Linux, lofiadm(1m) on Solaris and Illumos, and mdconfig(8) on FreeBSD.

How do disk I/O errors affect the cache and what does Traffic Server do when acache disk fails?

If a disk drive fails five successive I/O operations, then Traffic Server considers the drive inaccessible and removes theentire disk from the cache. Normal cache operations continue for all other Traffic Server disk drives.

5.2. Frequently Asked Questions 505


If a client disconnects during the time that Traffic Server is downloading a largeobject, is any of the object saved in the cache?

When a client disconnects during an HTTP operation, Traffic Server can continue to down-load the object from the origin server, using the background fill feature. It will con-tinue downloading based on the :ts:cv:‘proxy.config.http.background_fill_active_timeout‘ and:ts:cv:‘proxy.config.http.background_fill_completed_threshold‘ settings.

Can Traffic Server cache Java applets, JavaScript programs, or other applicationfiles like VBScript?

Yes, Traffic Server can store and serve Java applets, JavaScript programs, VBScripts, and other executable objectsfrom its cache according to the freshness and cacheability rules for HTTP objects. Traffic Server does not execute theapplets, scripts, or programs. These objects run entirely client-side (the system which originated the request for theobjects), and do not execute on the server.

In Squid- and Netscape-format log files, what do the cache result codes mean?

This is described in detail in the Squid documentation.

What is recorded by the cqtx field in a custom log file?

• In forward proxy mode, the cqtx field records the complete client request in the log file (for example, GEThttp://www.company.com HTTP/1.0).

• In reverse proxy mode, the cqtx field records the hostname or IP address of the origin server because TrafficServer first remaps the request as per map rules in the remap.config file.

Does Traffic Server refresh entries in its host database after a certain period of timeif they have not been used?

By default, the Traffic Server host database observes the time-to-live (ttl) values set by name servers. You canreconfigure Traffic Server to ignore the ttl set by name servers and use a specific Traffic Server setting instead.Alternatively, you can configure Traffic Server to compare the ttl value set by the name server with the ttl valueset by Traffic Server, and then use either the lower or the higher value.

Refer to :ts:cv:‘proxy.config.hostdb.ttl_mode‘ for more info.

Can you improve the look of your custom response pages by using images, ani-mated .gifs, and Java applets?

No, because Traffic Server can only respond to clients with a single text or HTML document. As a workaround,however, you can provide references on your custom response pages to images, animated .gifs, Java applets, or objectsother than text which are located on a web server. Add links in the body_factory template files in the same way youwould for any image in an HTML document (i.e., with the full URL in the SRC attribute).



Can Traffic Server run in forward proxy and reverse proxy modes at the same time?

Yes. When you enable reverse proxy mode, Traffic Server remaps incoming requests according to the map rules inremap.config. All other requests that do not match a map rule are served in forward proxy mode.

If you want to run in reverse proxy only mode (wherein Traffic Server does not serve requests that fail to match amap rule), then you must set the configuration variable :ts:cv:‘proxy.config.url_remap.remap_required‘ to 1 inrecords.config.

How do I enable forward proxy mode

Please refer to the Forward Proxy documentation.

How do I interpret the Via: header code?

The Via header string can be decoded with the Via Decoder Ring.

The Via header is an optional HTTP header added by Traffic Server and other HTTP proxies. If a request goes throughmultiple proxies, each one appends its Via header content to the end of the existing Via header. Via header content isfor general information and diagnostic use only and should not be used as a programmatic interface to Traffic Server.

The form of the Via header is

Via: <protocol> <proxyname> (<product/version> [<via-codes>])

Value Meaning<protocol> the scheme and version of the HTTP request<proxyname> the configured name of the proxy server<prod-uct/version>

the Traffic Server product name and version

<via-codes> a string of alphabetic codes presenting status information about the proxy handling of theHTTP request

For example: Via: HTTP/1.0 storm (Traffic-Server/4.0.0 [cMs f ])

• [u lH o f pS eN] cache hit

• [u lM oS fF pS eN] cache miss

• [uN l oS f pS eN] no-cache origin server fetch

The short status code shows the cache-lookup, server-info and cache-fill information as listed in the full status codedescription below. The long status code list provided in older, commercial versions of Traffic Server can be restored bysetting the verbose_via_str config variable. The character strings in the via-code show [<request results>:<proxy op>]where <request results> represents status information about the results of the client request and <proxy op> representsome information about the proxy operations performed during request processing. The full via-code status format is

[u<client-info> c<cache-lookup> s<server-info> f<cache-fill> p<proxy-info> e<error-codes> : t<tunnel-info>c<cache-type><cache-lookup-result> i<icp-conn-info> p<parent-proxy> s<server-conn-info>]

u client-info

Request headers received from client. Value is one of:




Value MeaningC cookieE error in requestI If Modified Since (IMS)N no-cacheS simple request (not conditional)

c cache-lookup

Result of Traffic Server cache lookup for URL. Value is one of:

Value MeaningA in cache, not acceptable (a cache “MISS”)H in cache, fresh (a cache “HIT”)M miss (a cache “MISS”)R in cache, fresh RAM hit (a cache “HIT”)S in cache, stale (a cache “MISS”)blank no cache lookup performed

s server-info

Response information received from origin server. Value is one of:

Value MeaningE error in responseN not-modifiedS servedblank no server connection needed

f cache-fill

Result of document write to cache. Value is one of:

Value MeaningD cached copy deletedU updated old cache copyW written into cache (new copy)blank no cache write performed

p proxy-info

Proxy operation result. Value is one of:

Value MeaningN not-modifiedR origin server revalidatedS served

e error-codes

Value is one of:



Value MeaningA authorization failureC connection to server failedD dns failureF request forbiddenH header syntax unacceptableN no errorR cache read errorM moved temporarilyS server related errorT connection timed out

: = Separates proxy request result information from operation detail codes

t tunnel-info

Proxy-only service operation. Value is one of:

Value MeaningA tunnel authorizationF tunneling due to a header field (such as presence of If-Range header)M tunneling due to a method (e.g. CONNECT)N tunneling due to no forwardO tunneling because cache is turned offU tunneling because of url (url suggests dynamic content)blank no tunneling

c cache-type and cache-lookup

cache result values (2 characters)

cache-type character value is one of

Value MeaningC cacheI icpL cluster, (not used)P parentS serverblank cache miss or no cache lookup

cache-lookup-result character value is one of:

Value MeaningC cache hit, but config forces revalidateD cache hit, but method forces revalidated (e.g. ftp, not anonymous)H cache hitI conditional miss (client sent conditional, fresh in cache, returned 412)K cookie missM cache miss (url not in cache)N conditional hit (client sent conditional, doc fresh in cache, returned 304)S cache hit, but expiredU cache hit, but client forces revalidate (e.g. Pragma: no-cache)blank no cache lookup



i icp-conn-info

ICP status

Value MeaningF connection open failedS connection opened successfullyblank no icp

p parent-proxy

parent proxy connection status

Value MeaningF connection open failedS connection opened successfullyblank no parent proxy

s server-conn-info

origin server connection status

Value MeaningF connection open failedS connection opened successfullyblank no server connection

Support for HTTP Expect: Header

Traffic Server currently does not handle Expect: request headers according to the HTTP/1.1 spec.

Clients such as cURL automatically send Expect: for POST requests with large POST bodies, with a 1 second timeoutif a 100 Continue response is not received. To avoid the timeout when using cURL as a client to Traffic Server, youcan turn off the Expect: header:

curl -H"Expect:" http://www.example.com/

Or with the C (libcurl) library from within your own applications:

struct curl_slist *header_list=NULL;header_list = curl_slist_append(header_list, "Expect:");curl_easy_setopt(my_curlp, CURLOPT_HTTPHEADER, header_list);

Or with the PHP cURL library:

curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:'));



Troubleshooting Tips

The throughput statistic is inaccurate

Traffic Server updates the throughput statistic after it has transferred an entire object. For larger files, the byte countincreases sharply at the end of a transfer. The complete number of bytes transferred is attributed to the last 10-secondinterval, although it can take several minutes to transfer the object. This inaccuracy is more noticeable with a lightload. A heavier load yields a more accurate statistic.

You are unable to execute Traffic Line commands

Traffic Line commands do not execute under the following conditions:

When the traffic_manager process is not running Check to see if the traffic_manager process is running byentering the following command:

pgrep -l traffic_manager

If the traffic_manager process is not running, then enter the following command from the Traffic Serverbin directory to start it:

./traffic_manager

When you are not executing the command from $TSHome/bin If the Traffic Server bin directory is not in yourpath, then prepend the Traffic Line commands with ./ (for example, ./traffic_ctl -h).

When multiple Traffic Server installations are present and you are not executing the Traffic Line commandfrom the active Traffic Server path specified in ‘‘/etc/trafficserver‘‘

You observe inconsistent behavior when one node obtains an object from anothernode in the cluster

As part of the initial system preparation, you must synchronize the clocks on all nodes in your cluster. Minor timedifferences do not cause problems, but differences of more than a few minutes can affect Traffic Server operation.

You should run a clock synchronization daemon such as xntpd. To obtain the latest version of xntpd, go to http://www.eecis.udel.edu/~ntp/.

Web browsers display an error document with a ‘data missing’ message

A message similar to the following might display in web browsers:

Data Missing

This document resulted from a POST operation and has expired from the cache. You can→˓repost the form data to recreate the document by pressing the Reload button.

This is a Web browser issue and not a problem specific to (or caused by) Traffic Server. Because Web browsersmaintain a separate local cache in memory and/or disk on the client system, messages about documents that haveexpired from cache refer to the browser’s local cache and not to the Traffic Server cache. There is no Traffic Servermessage or condition that can cause such messages to appear in a web browser.

5.3. Troubleshooting Tips 511

http://www.eecis.udel.edu/~ntp/

http://www.eecis.udel.edu/~ntp/


Traffic Server does not resolve any websites

The browser indicates that it is contacting the host and then times out with the following message:

The document contains no data; Try again later, or contact the server's Administrator.→˓..

Make sure the system is configured correctly and that Traffic Server can read the name resolution file:

• Check if the server can resolve DNS lookups by issuing the nslookup command (for example, nslookupwww.myhost.com).

• Check if the resolv.conf(5) file contains valid IP addresses for your DNS servers.

• On some systems, if the resolv.conf(5) file is unreadable or has no name server entry, then the operatingsystem uses localhost as a name server. Traffic Server, however, does not use this convention. If you wantto use localhost as a name server, then you must add a name server entry for 127.0.0.1 or 0.0.0.0 inthe resolv.conf(5) file.

• Check that the Traffic Server user account has permission to read the resolv.conf(5) file. If it does not,then change the file permissions to rw-r--r-- (644).

‘Maximum document size exceeded’ message in the system log file

The following message appears in the system log file:

WARNING: Maximum document size exceeded

A requested object was larger than the maximum size allowed in the Traffic Server cache, so Traffic Server providedproxy service for the oversized object but did not cache it. To set the object size limit for the cache, modify the:ts:cv:‘proxy.config.cache.max_doc_size‘ variable in the records.config file. If you do not want to limit the size ofobjects in the cache, then set the document size to 0 (zero).

‘DrainIncomingChannel’ message in the system log file

The following messages may appear in the system log file:

Feb 20 23:53:40 louis traffic_manager[4414]: ERROR ==> [drainIncomingChannel] Unknown→˓message: 'GET http://www.telechamada.pt/ HTTP/1.0'Feb 20 23:53:46 louis last message repeated 1 timeFeb 20 23:53:58 louis traffic_manager[4414]: ERROR ==> [drainIncomingChannel] Unknown→˓message: 'GET http://www.ip.pt/ HTTP/1.0'

These error messages indicate that a browser is sending HTTP requests to one of the Traffic Server cluster ports, eitherrsport (default port 8088) or mcport (default port 8089). Traffic Server discards these requests. This error doesnot cause any Traffic Server problems. The misconfigured browser must be reconfigured to use the correct proxy port.Traffic Server clusters should ideally be configured to use a separate network interface and cluster on a private subnet,so that client machines have no access to the cluster ports.

‘No cop file’ message in the system log file

The following message appears repeatedly in the system log file:

traffic_cop[16056]: encountered "var/trafficserver/no_cop" file...exiting



The file var/trafficserver/no_cop acts as an administrative control that instructs the traffic_cop pro-cess to exit immediately without starting traffic_manager or performing any health checks. The no_cop fileprevents Traffic Server from starting automatically when it has been stopped with the option:trafficserver stop com-mand. Without this static control, Traffic Server would restart automatically upon system reboot. The no_cop controlkeeps Traffic Server off until it is explicitly restarted with:

trafficserver start

Warning in the system log file when manually editing vaddrs.config

If you manually edit vaddrs.config as a non-root user, then Traffic Server issues a warning message in the systemlog file similar to the following:

WARNING: interface is ignored: Operation not permitted

You can safely ignore this message as Traffic Server will still apply your configuration edits.

Traffic Server is running but no log files are created

Traffic Server only writes event log files when there is information to record. If Traffic Server is idle, then it’s possiblethat no log files exist.

If Traffic Server is not idle, and you still do not see log files being generated, verify the following:

• Make sure you’re looking in the correct directory. By default, Traffic Server creates log files in the logsdirectory. This can be modified by changing :ts:cv:‘proxy.config.log.logfile_dir‘ in records.config.

• Check that the log directory has read/write permissions for the Traffic Server user account. If the log directorydoes not have the correct permissions, then the traffic_server process will be unable to open or createlog files.

• Check that logging is enabled by checking the value of the :ts:cv:‘proxy.config.log.logging_enabled‘ variablein records.config.

• Check that a log format is enabled. In records.config, select the standard or custom format by editingvariables in the Logging Config section.

Traffic Server shows an error indicating too many network connections

By default, Traffic Server supports 8000 network connections. Half of this number is allocated for client connectionsand the remaining half is for origin server connections. A connection throttle event occurs when either client or originserver connections reach 90% of half the configured total limit (3600 by default). When a connection throttle eventoccurs, Traffic Server continues processing all existing connections but will not accept new client connection requestsuntil the connection count falls below the limit.

Connection throttle events can occur under the following conditions:

Connection Spike Too many client requests (enough to exceed your configured maximum connections) all reachTraffic Server at the same time. Such events are typically transient and require no corrective action if yourconnection limits are already configured appropriately for your Traffic Server and origin resources.

Service Overload Client requests are arriving at a rate faster than that which Traffic Server can service them. Thismay indicate network problems between Traffic Server and origin servers or that Traffic Server may requiremore memory, CPU, cache disks, or other resources to handle the client load.

5.3. Troubleshooting Tips 513


If necessary, you can adjust the maximum number of connections supported by Traffic Server by editing:ts:cv:‘proxy.config.net.connections_throttle‘ in records.config.

: Do not increase the connection throttle limit unless the system has adequate memory to handle the client connectionsrequired. A system with limited RAM might need a throttle limit lower than the default value. Do not set this variablebelow the minimum value of 100.

Low memory symptoms

Under heavy load, the Linux kernel can run out of RAM. This low memory condition can cause slow performanceand a variety of other system problems. In fact, RAM exhaustion can occur even if the system has plenty of free swapspace.

Symptoms of extreme memory exhaustion include the following messages in the system log files (/var/log/messages):

WARNING: errno 105 is ENOBUFS (low on kernel memory), consider a memory upgrade

kernel: eth0: can't fill rx buffer (force 0)!

kernel: recvmsg bug: copied E01BA916 seq E01BAB22

To avoid memory exhaustion, add more RAM to the system or reduce the load on Traffic Server.

Config checker

Traffic Server supports the below command to validate the config offline, in order to allow the config to be pre-checkedfor possible service disruptions due to synatx errors:

traffic_server -Cverify_config -D<config_dir>

<config_dir> is the location of the config files to be validated.

Connection timeouts with the origin server

By default, Traffic Server will timeout after 30 seconds when contacting origin servers. If you cannot avoid such time-outs by otherwise addressing the performance on your origin servers, you may adjust the origin connection timeout inTraffic Server by changing :ts:cv:‘proxy.config.http.connect_attempts_timeout‘ in records.config to a largervalue.

Glossary

alternate A variant of a cache object. This was originally created to handle the VARY mechanism but has since beenused for additional purposes. All alternates of an object must be equivalent in some manner. That is, they arealternate forms of the same stream. The most common example is having normal and compressed versions ofthe stream.

cache fragment The unit of storage in the cache. All reads from the cache always read exactly one fragment.Fragments may be written in groups, but every write is always an integral number of fragments. Each fragmenthas a corresponding directory entry which describes its location in the cache storage.




cache ID A 128 bit value used as a fixed sized identifier for an object in the cache. This is computed from the cachekey using the MD5 hashing function.

cache key A byte sequence that is a globally unique identifier for an object in the cache. By default the URL for theobject is used.

cache object The minimal self contained unit of data in the cache. Cache objects are the stored version of equivalentcontent streams from an origin server. A single object can have multiple variants called alternates.

cache partition A subdivision of the cache storage in Traffic Server which is dedicated to objects for specific proto-cols, origins, or other rules. Defining and managing cache partitions is discussed in Partitioning the Cache.

cache span The physical storage described by a single line in storage.config.

cache stripe A homogenous, persistent store for the cache in a single cache span. A stripe always resides entirely ona single physical device and is treated as an undifferentiated span of bytes. This is the smallest independent unitof storage.

cache tag The bottom few bits (12 currently) of the cache ID. This is used in the cache directory for a preliminaryidentity check before going to disk.

cache volume A user defined unit of persistent storage for the cache. Cache volumes are defined in volume.config. A cache volume is by default spread across cache spans to increase robustness. Each section of acache volume on a specific cache span is a cache stripe.

continuation A callable object that contains state. These are are mechanism used by Traffic Server to implementcallbacks and continued computations. Continued computations are critical to efficient processing of trafficbecause by avoiding any blocking operations that wait on external events. In any such case a continuation isused so that other processing can continue until the external event occurs. At that point the continuation isinvoked to continue the suspended processing. This can be considered similar to co-routines.

directory bucket A contiguous fixed sized group of directory entries. This is used for hash bucket maintenanceoptimization.

directory entry An in memory entry that describes a cache fragment.

directory segment A contiguous group of buckets. Each cache stripe has a set of segments all of which have thesame number of buckets, although the number of buckets per segment can vary between cache stripes. Segmentsare administrative in purpose to minimize the size of free list and hash bucket pointers.

fresh The state of a cache object which can be served directly from the the cache in response to client requests.Fresh objects have not met or passed their origin server defined expiration time, nor have they reached thealgorithmically determined stale age.

global plugin A plugin which operates on all transactions. Contrast with remap plugin.

object store The database of cache objects.

origin server An HTTP server which provides the original source of content being cached by Traffic Server.

remap plugin A plugin which operates only on transactions matching specific remap rules as defined in remap.config. Contrast with global plugin.

revalidation Verifying that a currently cached object is still valid. This is usually done using an If-Modified-Sincerequest which allows the origin server to validate the content without resending the content.

session A single connection from a client to Traffic Server, covering all requests and responses on that connection.A session starts when the client connection opens, and ends when the connection closes.

stale The state of a cache object which is not yet expired, but has reached an algorithmically determined age at whichthe origin server will be contacted to revalidate the freshness of the object. Contrast with fresh.

storage unit Obsolete term for cache span.

5.4. Glossary 515

http://www.openssl.org/docs/crypto/md5.html



transaction A client request and response, either from the origin server or from the cache. A transaction begins whenTraffic Server receives a request, and ends when Traffic Server sends the response.

write cursor The location in a cache stripe where new data is written.

HTTP Status Codes

The following standard HTTP response messages are provided for your information.

Code Description200 OK202 Accepted204 No Content206 Partial Content300 Multiple Choices301 Moved Permanently302 Found303 See Other304 Not Modified400 Bad Request401 Unauthorized; retry403 Forbidden404 Not Found405 Method Not Allowed406 Not Acceptable408 Request Timeout500 Internal Server Error501 Not Implemented502 Bad Gateway504 Gateway Timeout

More detail can be found in RFC2616 which defines the full range of standardized HTTP/1.1 response codes.


http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

CHAPTER 6

Indices and tables

• genindex

• search

517


518 Chapter 6. Indices and tables

AAFTER_BODY (C ), 384AFTER_HEADER (C ), 384alternate, 272, 514

Ccache directory, 269cache fragment, 514cache ID, 269, 515cache key, 515cache object, 515cache partition, 515cache span, 515cache stripe, 515cache tag, 515cache volume, 515CacheControlResult (C++ ), 283CacheHTTPInfo (C++ ), 283CacheHTTPInfoVector (C++ ), 283CacheVC (C++ ), 283CacheVC::openReadStartEarliest (C++ ), 283CacheVC::openReadStartHead (C++ ), 283CacheVol (C++ ), 284continuation, 515

DDEPRECATED (C ), 390Dir (C++ ), 275DIR_BLOCK_SIZE (C++ ), 285DIR_DEPTH, 276DIR_SIZE_WITH_BLOCK (C++ ), 285directory bucket, 515directory entry, 269, 515directory segment, 515Doc (C++ ), 284Doc::checksum (C++ ), 285Doc::Doc::first_key (C++ ), 284Doc::Doc::flen (C++ ), 284Doc::Doc::ftype (C++ ), 284Doc::Doc::hlen (C++ ), 284Doc::Doc::key (C++ ), 284

Doc::Doc::len (C++ ), 284Doc::Doc::magic (C++ ), 284Doc::Doc::sync_serial (C++ ), 285Doc::Doc::total_len (C++ ), 284Doc::Doc::write_serial (C++ ), 285Doc::pinned (C++ ), 285

EEvacuationBlock (C++ ), 284

Ffragment, 269fresh, 515

Gglobal plugin, 515

HHTTPCacheAlt (C++ ), 283HTTPInfo (C++ ), 283HttpTunnel (C++ ), 283

Iindex buckets, 276index segment, 276ink_hrtime (C ), 459

NNO_CALLBACK (C ), 384

Oobject store, 515OpenDir (C++ ), 283origin server, 515

Ppinned, 274port descriptors, 237

Rremap plugin, 515

519


revalidation, 515RFC

RFC 6961, 58

Ssession, 515stale, 515storage unit, 515

Ttransaction, 516TS_AIO_EVENT_DONE (C ), 383TS_CACHE_DATA_TYPE_HTTP (C ), 380TS_CACHE_DATA_TYPE_NONE (C ), 380TS_CACHE_DATA_TYPE_OTHER (C ), 380TS_CACHE_ERROR_DOC_BUSY (C ), 380TS_CACHE_ERROR_NO_DOC (C ), 380TS_CACHE_ERROR_NOT_READY (C ), 380TS_CACHE_LOOKUP_HIT_FRESH (C ), 381TS_CACHE_LOOKUP_HIT_STALE (C ), 381TS_CACHE_LOOKUP_MISS (C ), 381TS_CACHE_LOOKUP_SKIPPED (C ), 381TS_CACHE_SCAN_RESULT_CONTINUE (C ), 381TS_CACHE_SCAN_RESULT_DELETE (C ), 381TS_CACHE_SCAN_RESULT_DELETE_ALL_ALTERNATES

(C ), 381TS_CACHE_SCAN_RESULT_DONE (C ), 381TS_CACHE_SCAN_RESULT_RETRY (C ), 381TS_CACHE_SCAN_RESULT_UPDATE (C ), 381TS_CONFIG_HTTP_ACCEPT_ENCODING_FILTER_ENABLED

(C ), 392TS_CONFIG_HTTP_ANONYMIZE_INSERT_CLIENT_IP

(C ), 391TS_CONFIG_HTTP_ANONYMIZE_REMOVE_CLIENT_IP

(C ), 391TS_CONFIG_HTTP_ANONYMIZE_REMOVE_COOKIE

(C ), 391TS_CONFIG_HTTP_ANONYMIZE_REMOVE_FROM

(C ), 391TS_CONFIG_HTTP_ANONYMIZE_REMOVE_REFERER

(C ), 391TS_CONFIG_HTTP_ANONYMIZE_REMOVE_USER_AGENT

(C ), 391TS_CONFIG_HTTP_BACKGROUND_FILL_ACTIVE_TIMEOUT

(C ), 392TS_CONFIG_HTTP_BACKGROUND_FILL_COMPLETED_THRESHOLD

(C ), 392TS_CONFIG_HTTP_CACHE_CACHE_RESPONSES_TO_COOKIES

(C ), 391TS_CONFIG_HTTP_CACHE_CACHE_URLS_THAT_LOOK_DYNAMIC

(C ), 391TS_CONFIG_HTTP_CACHE_CLUSTER_CACHE_LOCAL

(C ), 391

TS_CONFIG_HTTP_CACHE_FUZZ_MIN_TIME (C ),392

TS_CONFIG_HTTP_CACHE_FUZZ_PROBABILITY(C ), 392

TS_CONFIG_HTTP_CACHE_FUZZ_TIME (C ), 392TS_CONFIG_HTTP_CACHE_GUARANTEED_MAX_LIFETIME

(C ), 391TS_CONFIG_HTTP_CACHE_GUARANTEED_MIN_LIFETIME

(C ), 391TS_CONFIG_HTTP_CACHE_HEURISTIC_LM_FACTOR

(C ), 392TS_CONFIG_HTTP_CACHE_HEURISTIC_MAX_LIFETIME

(C ), 391TS_CONFIG_HTTP_CACHE_HEURISTIC_MIN_LIFETIME

(C ), 391TS_CONFIG_HTTP_CACHE_HTTP (C ), 391TS_CONFIG_HTTP_CACHE_IGNORE_AUTHENTICATION

(C ), 391TS_CONFIG_HTTP_CACHE_IGNORE_CLIENT_CC_MAX_AGE

(C ), 391TS_CONFIG_HTTP_CACHE_IGNORE_CLIENT_NO_CACHE

(C ), 391TS_CONFIG_HTTP_CACHE_IGNORE_SERVER_NO_CACHE

(C ), 391TS_CONFIG_HTTP_CACHE_IMS_ON_CLIENT_NO_CACHE

(C ), 391TS_CONFIG_HTTP_CACHE_MAX_OPEN_READ_RETRIES

(C ), 392TS_CONFIG_HTTP_CACHE_MAX_STALE_AGE (C

), 391TS_CONFIG_HTTP_CACHE_OPEN_READ_RETRY_TIME

(C ), 392TS_CONFIG_HTTP_CACHE_RANGE_LOOKUP (C ),

392TS_CONFIG_HTTP_CACHE_RANGE_WRITE (C ),

392TS_CONFIG_HTTP_CACHE_REQUIRED_HEADERS

(C ), 391TS_CONFIG_HTTP_CACHE_WHEN_TO_REVALIDATE

(C ), 390TS_CONFIG_HTTP_CHUNKING_ENABLED (C ),

390TS_CONFIG_HTTP_CHUNKING_SIZE (C ), 392TS_CONFIG_HTTP_CONNECT_ATTEMPTS_MAX_RETRIES

(C ), 391TS_CONFIG_HTTP_CONNECT_ATTEMPTS_MAX_RETRIES_DEAD_SERVER

(C ), 391TS_CONFIG_HTTP_CONNECT_ATTEMPTS_RR_RETRIES

(C ), 391TS_CONFIG_HTTP_CONNECT_ATTEMPTS_TIMEOUT

(C ), 392TS_CONFIG_HTTP_DEFAULT_BUFFER_SIZE (C ),

392TS_CONFIG_HTTP_DEFAULT_BUFFER_WATER_MARK

520


(C ), 392TS_CONFIG_HTTP_DOC_IN_CACHE_SKIP_DNS (C

), 392TS_CONFIG_HTTP_DOWN_SERVER_ABORT_THRESHOLD

(C ), 392TS_CONFIG_HTTP_DOWN_SERVER_CACHE_TIME

(C ), 392TS_CONFIG_HTTP_FLOW_CONTROL_ENABLED

(C ), 392TS_CONFIG_HTTP_FLOW_CONTROL_HIGH_WATER_MARK

(C ), 392TS_CONFIG_HTTP_FLOW_CONTROL_LOW_WATER_MARK

(C ), 392TS_CONFIG_HTTP_FORWARD_PROXY_AUTH_TO_PARENT

(C ), 391TS_CONFIG_HTTP_GLOBAL_USER_AGENT_HEADER

(C ), 392TS_CONFIG_HTTP_INSERT_AGE_IN_RESPONSE

(C ), 392TS_CONFIG_HTTP_INSERT_REQUEST_VIA_STR

(C ), 391TS_CONFIG_HTTP_INSERT_RESPONSE_VIA_STR

(C ), 391TS_CONFIG_HTTP_INSERT_SQUID_X_FORWARDED_FOR

(C ), 391TS_CONFIG_HTTP_KEEP_ALIVE_ENABLED_IN (C

), 390TS_CONFIG_HTTP_KEEP_ALIVE_ENABLED_OUT

(C ), 390TS_CONFIG_HTTP_KEEP_ALIVE_NO_ACTIVITY_TIMEOUT_IN

(C ), 391TS_CONFIG_HTTP_KEEP_ALIVE_NO_ACTIVITY_TIMEOUT_OUT

(C ), 391TS_CONFIG_HTTP_KEEP_ALIVE_POST_OUT (C ),

390TS_CONFIG_HTTP_NEGATIVE_CACHING_ENABLED

(C ), 390TS_CONFIG_HTTP_NEGATIVE_CACHING_LIFETIME

(C ), 390TS_CONFIG_HTTP_NEGATIVE_REVALIDATING_ENABLED

(C ), 392TS_CONFIG_HTTP_NEGATIVE_REVALIDATING_LIFETIME

(C ), 392TS_CONFIG_HTTP_NORMALIZE_AE_GZIP (C ), 392TS_CONFIG_HTTP_ORIGIN_MAX_CONNECTIONS

(C ), 391TS_CONFIG_HTTP_POST_CHECK_CONTENT_LENGTH_ENABLED

(C ), 392TS_CONFIG_HTTP_POST_CONNECT_ATTEMPTS_TIMEOUT

(C ), 392TS_CONFIG_HTTP_REQUEST_HEADER_MAX_SIZE

(C ), 392TS_CONFIG_HTTP_RESPONSE_HEADER_MAX_SIZE

(C ), 392

TS_CONFIG_HTTP_RESPONSE_SERVER_ENABLED(C ), 391

TS_CONFIG_HTTP_RESPONSE_SERVER_STR (C ),392

TS_CONFIG_HTTP_SEND_HTTP11_REQUESTS (C), 391

TS_CONFIG_HTTP_SERVER_SESSION_SHARING_MATCH(C ), 390

TS_CONFIG_HTTP_SERVER_SESSION_SHARING_POOL(C ), 390

TS_CONFIG_HTTP_SERVER_TCP_INIT_CWND (C), 391

TS_CONFIG_HTTP_TRANSACTION_ACTIVE_TIMEOUT_OUT(C ), 391

TS_CONFIG_HTTP_TRANSACTION_NO_ACTIVITY_TIMEOUT_IN(C ), 391

TS_CONFIG_HTTP_TRANSACTION_NO_ACTIVITY_TIMEOUT_OUT(C ), 391

TS_CONFIG_LAST_ENTRY (C ), 392TS_CONFIG_NET_SOCK_OPTION_FLAG_OUT (C ),

390TS_CONFIG_NET_SOCK_PACKET_MARK_OUT (C

), 392TS_CONFIG_NET_SOCK_PACKET_TOS_OUT (C ),

392TS_CONFIG_NET_SOCK_RECV_BUFFER_SIZE_OUT

(C ), 390TS_CONFIG_NET_SOCK_SEND_BUFFER_SIZE_OUT

(C ), 390TS_CONFIG_NULL (C ), 390TS_CONFIG_SSL_HSTS_INCLUDE_SUBDOMAINS

(C ), 392TS_CONFIG_SSL_HSTS_MAX_AGE (C ), 392TS_CONFIG_URL_REMAP_PRISTINE_HOST_HDR

(C ), 390TS_ERROR (C ), 396TS_EVENT_CACHE_CLOSE (C ), 383TS_EVENT_CACHE_DELETE (C ), 382TS_EVENT_CACHE_LOOKUP (C ), 382TS_EVENT_CACHE_LOOKUP_COMPLETE (C ), 383TS_EVENT_CACHE_LOOKUP_READY (C ), 383TS_EVENT_CACHE_OPEN_READ (C ), 382TS_EVENT_CACHE_OPEN_READ_FAILED (C ), 382TS_EVENT_CACHE_OPEN_WRITE (C ), 382TS_EVENT_CACHE_OPEN_WRITE_FAILED (C ),

382TS_EVENT_CACHE_READ (C ), 382TS_EVENT_CACHE_READ_COMPLETE (C ), 383TS_EVENT_CACHE_READ_READY (C ), 383TS_EVENT_CACHE_REMOVE (C ), 382TS_EVENT_CACHE_REMOVE_FAILED (C ), 382TS_EVENT_CACHE_SCAN (C ), 382TS_EVENT_CACHE_SCAN_DONE (C ), 382TS_EVENT_CACHE_SCAN_FAILED (C ), 382

521


TS_EVENT_CACHE_SCAN_OBJECT (C ), 382TS_EVENT_CACHE_SCAN_OPERATION_BLOCKED

(C ), 382TS_EVENT_CACHE_SCAN_OPERATION_FAILED

(C ), 382TS_EVENT_CACHE_WRITE (C ), 382TS_EVENT_CACHE_WRITE_HEADER (C ), 383TS_EVENT_CONTINUE (C ), 382TS_EVENT_ERROR (C ), 381TS_EVENT_HOST_LOOKUP (C ), 382TS_EVENT_HTTP_CACHE_LOOKUP_COMPLETE

(C ), 383TS_EVENT_HTTP_CONTINUE (C ), 383TS_EVENT_HTTP_ERROR (C ), 383TS_EVENT_HTTP_OS_DNS (C ), 383TS_EVENT_HTTP_POST_REMAP (C ), 383TS_EVENT_HTTP_PRE_REMAP (C ), 383TS_EVENT_HTTP_READ_CACHE_HDR (C ), 383TS_EVENT_HTTP_READ_REQUEST_HDR (C ), 383TS_EVENT_HTTP_READ_RESPONSE_HDR (C ), 383TS_EVENT_HTTP_REQUEST_TRANSFORM (C ),

383TS_EVENT_HTTP_RESPONSE_TRANSFORM (C ),

383TS_EVENT_HTTP_SELECT_ALT (C ), 383TS_EVENT_HTTP_SEND_REQUEST_HDR (C ), 383TS_EVENT_HTTP_SEND_RESPONSE_HDR (C ), 383TS_EVENT_HTTP_SSN_CLOSE (C ), 383TS_EVENT_HTTP_SSN_START (C ), 383TS_EVENT_HTTP_TXN_CLOSE (C ), 383TS_EVENT_HTTP_TXN_START (C ), 383TS_EVENT_IMMEDIATE (C ), 381TS_EVENT_INTERNAL_1200 (C ), 383TS_EVENT_INTERNAL_206 (C ), 382TS_EVENT_INTERNAL_207 (C ), 382TS_EVENT_INTERNAL_208 (C ), 382TS_EVENT_INTERNAL_209 (C ), 382TS_EVENT_INTERNAL_210 (C ), 382TS_EVENT_INTERNAL_211 (C ), 382TS_EVENT_INTERNAL_212 (C ), 382TS_EVENT_INTERNAL_60200 (C ), 383TS_EVENT_INTERNAL_60201 (C ), 383TS_EVENT_INTERNAL_60202 (C ), 383TS_EVENT_LIFECYCLE_CACHE_READY (C ), 383TS_EVENT_LIFECYCLE_CLIENT_SSL_CTX_INITIALIZED

(C ), 383TS_EVENT_LIFECYCLE_PORTS_INITIALIZED (C ),

383TS_EVENT_LIFECYCLE_PORTS_READY (C ), 383TS_EVENT_LIFECYCLE_SERVER_SSL_CTX_INITIALIZED

(C ), 383TS_EVENT_MGMT_UPDATE (C ), 383TS_EVENT_NET_ACCEPT (C ), 382TS_EVENT_NET_ACCEPT_FAILED (C ), 382

TS_EVENT_NET_CONNECT (C ), 382TS_EVENT_NET_CONNECT_FAILED (C ), 382TS_EVENT_NONE (C ), 381TS_EVENT_TIMEOUT (C ), 381TS_EVENT_VCONN_ACTIVE_TIMEOUT (C ), 382TS_EVENT_VCONN_EOS (C ), 382TS_EVENT_VCONN_INACTIVITY_TIMEOUT (C ),

382TS_EVENT_VCONN_PRE_ACCEPT (C ), 383TS_EVENT_VCONN_READ_COMPLETE (C ), 382TS_EVENT_VCONN_READ_READY (C ), 382TS_EVENT_VCONN_WRITE_COMPLETE (C ), 382TS_EVENT_VCONN_WRITE_READY (C ), 382TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK (C

), 385TS_HTTP_LAST_HOOK (C ), 385TS_HTTP_OS_DNS_HOOK (C ), 384TS_HTTP_POST_REMAP_HOOK (C ), 385TS_HTTP_PRE_REMAP_HOOK (C ), 385TS_HTTP_READ_CACHE_HDR_HOOK (C ), 384TS_HTTP_READ_REQUEST_HDR_HOOK (C ), 384TS_HTTP_READ_RESPONSE_HDR_HOOK (C ), 384TS_HTTP_REQUEST_TRANSFORM_HOOK (C ), 384TS_HTTP_RESPONSE_CLIENT_HOOK (C ), 385TS_HTTP_RESPONSE_TRANSFORM_HOOK (C ),

384TS_HTTP_SELECT_ALT_HOOK (C ), 384TS_HTTP_SEND_REQUEST_HDR_HOOK (C ), 384TS_HTTP_SEND_RESPONSE_HDR_HOOK (C ), 384TS_HTTP_SSN_CLOSE_HOOK (C ), 385TS_HTTP_SSN_START_HOOK (C ), 384TS_HTTP_STATUS_ACCEPTED (C ), 385TS_HTTP_STATUS_ALREADY_REPORTED (C ), 385TS_HTTP_STATUS_BAD_GATEWAY (C ), 386TS_HTTP_STATUS_BAD_REQUEST (C ), 386TS_HTTP_STATUS_CONFLICT (C ), 386TS_HTTP_STATUS_CONTINUE (C ), 385TS_HTTP_STATUS_CREATED (C ), 385TS_HTTP_STATUS_EXPECTATION_FAILED (C ),

386TS_HTTP_STATUS_FAILED_DEPENDENCY (C ),

386TS_HTTP_STATUS_FORBIDDEN (C ), 386TS_HTTP_STATUS_GATEWAY_TIMEOUT (C ), 386TS_HTTP_STATUS_GONE (C ), 386TS_HTTP_STATUS_HTTPVER_NOT_SUPPORTED

(C ), 387TS_HTTP_STATUS_IM_USED (C ), 385TS_HTTP_STATUS_INSUFFICIENT_STORAGE (C ),

387TS_HTTP_STATUS_INTERNAL_SERVER_ERROR (C

), 386TS_HTTP_STATUS_LENGTH_REQUIRED (C ), 386TS_HTTP_STATUS_LOCKED (C ), 386

522


TS_HTTP_STATUS_LOOP_DETECTED (C ), 387TS_HTTP_STATUS_METHOD_NOT_ALLOWED (C ),

386TS_HTTP_STATUS_MOVED_PERMANENTLY (C ),

386TS_HTTP_STATUS_MOVED_TEMPORARILY (C ),

386TS_HTTP_STATUS_MULTI_STATUS (C ), 385TS_HTTP_STATUS_MULTIPLE_CHOICES (C ), 385TS_HTTP_STATUS_NETWORK_AUTHENTICATION_REQUIRED

(C ), 387TS_HTTP_STATUS_NO_CONTENT (C ), 385TS_HTTP_STATUS_NON_AUTHORITATIVE_INFORMATION

(C ), 385TS_HTTP_STATUS_NONE (C ), 385TS_HTTP_STATUS_NOT_ACCEPTABLE (C ), 386TS_HTTP_STATUS_NOT_EXTENDED (C ), 387TS_HTTP_STATUS_NOT_FOUND (C ), 386TS_HTTP_STATUS_NOT_IMPLEMENTED (C ), 386TS_HTTP_STATUS_NOT_MODIFIED (C ), 386TS_HTTP_STATUS_OK (C ), 385TS_HTTP_STATUS_PARTIAL_CONTENT (C ), 385TS_HTTP_STATUS_PAYMENT_REQUIRED (C ), 386TS_HTTP_STATUS_PERMANENT_REDIRECT (C ),

386TS_HTTP_STATUS_PRECONDITION_FAILED (C ),

386TS_HTTP_STATUS_PRECONDITION_REQUIRED (C

), 386TS_HTTP_STATUS_PROXY_AUTHENTICATION_REQUIRED

(C ), 386TS_HTTP_STATUS_REQUEST_ENTITY_TOO_LARGE

(C ), 386TS_HTTP_STATUS_REQUEST_HEADER_FIELDS_TOO_LARGE

(C ), 386TS_HTTP_STATUS_REQUEST_TIMEOUT (C ), 386TS_HTTP_STATUS_REQUEST_URI_TOO_LONG (C

), 386TS_HTTP_STATUS_REQUESTED_RANGE_NOT_SATISFIABLE

(C ), 386TS_HTTP_STATUS_RESET_CONTENT (C ), 385TS_HTTP_STATUS_SEE_OTHER (C ), 386TS_HTTP_STATUS_SERVICE_UNAVAILABLE (C ),

386TS_HTTP_STATUS_SWITCHING_PROTOCOL (C ),

385TS_HTTP_STATUS_TEMPORARY_REDIRECT (C ),

386TS_HTTP_STATUS_TOO_MANY_REQUESTS (C ),

386TS_HTTP_STATUS_UNAUTHORIZED (C ), 386TS_HTTP_STATUS_UNPROCESSABLE_ENTITY (C

), 386TS_HTTP_STATUS_UNSUPPORTED_MEDIA_TYPE

(C ), 386TS_HTTP_STATUS_UPGRADE_REQUIRED (C ), 386TS_HTTP_STATUS_USE_PROXY (C ), 386TS_HTTP_STATUS_VARIANT_ALSO_NEGOTIATES

(C ), 387TS_HTTP_TXN_CLOSE_HOOK (C ), 384TS_HTTP_TXN_START_HOOK (C ), 384TS_HTTP_TYPE_REQUEST (C ), 387TS_HTTP_TYPE_RESPONSE (C ), 387TS_HTTP_TYPE_UNKNOWN (C ), 387TS_IOBUFFER_SIZE_INDEX_128 (C ), 388TS_IOBUFFER_SIZE_INDEX_128K (C ), 388TS_IOBUFFER_SIZE_INDEX_16K (C ), 388TS_IOBUFFER_SIZE_INDEX_1K (C ), 388TS_IOBUFFER_SIZE_INDEX_1M (C ), 388TS_IOBUFFER_SIZE_INDEX_256 (C ), 388TS_IOBUFFER_SIZE_INDEX_256K (C ), 388TS_IOBUFFER_SIZE_INDEX_2K (C ), 388TS_IOBUFFER_SIZE_INDEX_2M (C ), 388TS_IOBUFFER_SIZE_INDEX_32K (C ), 388TS_IOBUFFER_SIZE_INDEX_4K (C ), 388TS_IOBUFFER_SIZE_INDEX_512 (C ), 388TS_IOBUFFER_SIZE_INDEX_512K (C ), 388TS_IOBUFFER_SIZE_INDEX_64K (C ), 388TS_IOBUFFER_SIZE_INDEX_8K (C ), 388TS_LIFECYCLE_CACHE_READY_HOOK (C ), 436TS_LIFECYCLE_CACHE_READY_HOOK (C ), 388TS_LIFECYCLE_CLIENT_SSL_CTX_INITIALIZED_HOOK

(C ), 388TS_LIFECYCLE_LAST_HOOK (C ), 388TS_LIFECYCLE_PORTS_INITIALIZED_HOOK (C ),

436TS_LIFECYCLE_PORTS_INITIALIZED_HOOK (C ),

388TS_LIFECYCLE_PORTS_READY_HOOK (C ), 436TS_LIFECYCLE_PORTS_READY_HOOK (C ), 388TS_LIFECYCLE_SERVER_SSL_CTX_INITIALIZED_HOOK

(C ), 388TS_LOOKUP_HOST_NONE (C ), 389TS_LOOKUP_ICP_SUGGESTED_HOST (C ), 389TS_LOOKUP_INCOMING_ROUTER (C ), 389TS_LOOKUP_ORIGIN_SERVER (C ), 389TS_LOOKUP_PARENT_PROXY (C ), 389TS_LOOKUP_UNDEFINED_LOOKUP (C ), 389TS_MILESTONE_CACHE_OPEN_READ_BEGIN (C

), 390TS_MILESTONE_CACHE_OPEN_READ_END (C ),

390TS_MILESTONE_CACHE_OPEN_WRITE_BEGIN (C

), 390TS_MILESTONE_CACHE_OPEN_WRITE_END (C ),

390TS_MILESTONE_DNS_LOOKUP_BEGIN (C ), 390TS_MILESTONE_DNS_LOOKUP_END (C ), 390

523


TS_MILESTONE_LAST_ENTRY (C ), 390TS_MILESTONE_NULL (C ), 389TS_MILESTONE_SERVER_BEGIN_WRITE (C ), 389TS_MILESTONE_SERVER_CLOSE (C ), 390TS_MILESTONE_SERVER_CONNECT (C ), 389TS_MILESTONE_SERVER_CONNECT_END (C ),

389TS_MILESTONE_SERVER_FIRST_CONNECT (C ),

389TS_MILESTONE_SERVER_FIRST_READ (C ), 389TS_MILESTONE_SERVER_READ_HEADER_DONE

(C ), 389TS_MILESTONE_SM_FINISH (C ), 390TS_MILESTONE_SM_START (C ), 390TS_MILESTONE_UA_BEGIN (C ), 389TS_MILESTONE_UA_BEGIN_WRITE (C ), 389TS_MILESTONE_UA_CLOSE (C ), 389TS_MILESTONE_UA_READ_HEADER_DONE (C ),

389TS_PARSE_CONT (C ), 393TS_PARSE_DONE (C ), 393TS_PARSE_ERROR (C ), 393TS_PARSE_OK (C ), 393TS_RECORDACCESS_NO_ACCESS (C ), 393TS_RECORDACCESS_NULL (C ), 393TS_RECORDACCESS_READ_ONLY (C ), 393TS_RECORDCHECK_INT (C ), 394TS_RECORDCHECK_IP (C ), 394TS_RECORDCHECK_NULL (C ), 394TS_RECORDCHECK_STR (C ), 394TS_RECORDDATATYPE_COUNTER (C ), 394TS_RECORDDATATYPE_FLOAT (C ), 394TS_RECORDDATATYPE_INT (C ), 394TS_RECORDDATATYPE_MAX (C ), 394TS_RECORDDATATYPE_NULL (C ), 394TS_RECORDDATATYPE_STAT_CONST (C ), 394TS_RECORDDATATYPE_STAT_FX (C ), 394TS_RECORDDATATYPE_STRING (C ), 394TS_RECORDMODE_CLIENT (C ), 395TS_RECORDMODE_NULL (C ), 395TS_RECORDMODE_SERVER (C ), 395TS_RECORDMODE_STAND_ALONE (C ), 395TS_RECORDP_NON_PERSISTENT (C ), 395TS_RECORDP_NULL (C ), 395TS_RECORDP_PERSISTENT (C ), 395TS_RECORDTYPE_ALL (C ), 396TS_RECORDTYPE_CLUSTER (C ), 395TS_RECORDTYPE_CONFIG (C ), 395TS_RECORDTYPE_LOCAL (C ), 395TS_RECORDTYPE_NODE (C ), 395TS_RECORDTYPE_NULL (C ), 395TS_RECORDTYPE_PLUGIN (C ), 395TS_RECORDTYPE_PROCESS (C ), 395TS_RECORDUPDATE_DYNAMIC (C ), 396

TS_RECORDUPDATE_NULL (C ), 396TS_RECORDUPDATE_RESTART_TC (C ), 396TS_RECORDUPDATE_RESTART_TM (C ), 396TS_RECORDUPDATE_RESTART_TS (C ), 396TS_SDK_VERSION_2_0 (C ), 397TS_SDK_VERSION_3_0 (C ), 397TS_SERVER_SESSION_SHARING_MATCH_BOTH

(C ), 397TS_SERVER_SESSION_SHARING_MATCH_HOST

(C ), 397TS_SERVER_SESSION_SHARING_MATCH_IP (C ),

397TS_SERVER_SESSION_SHARING_MATCH_NONE

(C ), 397TS_SERVER_SESSION_SHARING_POOL_GLOBAL

(C ), 397TS_SERVER_SESSION_SHARING_POOL_THREAD

(C ), 397TS_SRVSTATE_ACTIVE_TIMEOUT (C ), 398TS_SRVSTATE_BAD_INCOMING_RESPONSE (C ),

398TS_SRVSTATE_CONGEST_CONTROL_CONGESTED_ON_F

(C ), 398TS_SRVSTATE_CONGEST_CONTROL_CONGESTED_ON_M

(C ), 398TS_SRVSTATE_CONNECTION_ALIVE (C ), 398TS_SRVSTATE_CONNECTION_CLOSED (C ), 398TS_SRVSTATE_CONNECTION_ERROR (C ), 398TS_SRVSTATE_INACTIVE_TIMEOUT (C ), 398TS_SRVSTATE_OPEN_RAW_ERROR (C ), 398TS_SRVSTATE_PARSE_ERROR (C ), 398TS_SRVSTATE_STATE_UNDEFINED (C ), 398TS_SRVSTATE_TRANSACTION_COMPLETE (C ),

398TS_SSL_CERT_HOOK (C ), 385TS_SSL_FIRST_HOOK (C ), 385TS_SSL_HOOK_OP_DEFAULT (C ), 398TS_SSL_HOOK_OP_LAST (C ), 398TS_SSL_HOOK_OP_TERMINATE (C ), 398TS_SSL_HOOK_OP_TUNNEL (C ), 398TS_SSL_LAST_HOOK (C ), 385TS_SSL_SNI_HOOK (C ), 385TS_SUCCESS (C ), 396TS_THREAD_POOL_CLUSTER (C ), 399TS_THREAD_POOL_DEFAULT (C ), 399TS_THREAD_POOL_DNS (C ), 399TS_THREAD_POOL_NET (C ), 399TS_THREAD_POOL_REMAP (C ), 399TS_THREAD_POOL_SSL (C ), 399TS_THREAD_POOL_TASK (C ), 399TS_THREAD_POOL_UDP (C ), 399TS_VC_CLOSE_ABORT (C ), 399TS_VC_CLOSE_NORMAL (C ), 399TS_VCONN_PRE_ACCEPT_HOOK (C ), 385

524


TSAction (C ), 459TSActionCancel (C ), 401TSActionDone (C ), 402TSAssert (C ), 406TSCacheDataType (C ), 380TSCacheError (C ), 380TSCacheKey (C ), 459TSCacheLookupResult (C ), 380TSCacheRead (C ), 402TSCacheRemove (C ), 402TSCacheScanResult (C ), 381TSCacheUrlSet (C ), 285TSCacheWrite (C ), 403TSConfig (C ), 459TSConfigDataGet (C ), 403TSConfigDestroyFunc (C ), 459TSConfigDirGet (C ), 435TSConfigGet (C ), 403TSConfigRelease (C ), 403TSConfigSet (C ), 404TSCont (C ), 459TSContCall (C ), 404TSContCreate (C ), 404TSContDataGet (C ), 404TSContDataSet (C ), 404TSContDestroy (C ), 405TSContMutexGet (C ), 405TSContSchedule (C ), 405TSDebug (C ), 405TSDebugSpecific (C ), 405TSError (C ), 405TSEvent (C ), 381, 459TSEventFunc (C ), 459TSfclose (C ), 473TSFetchWakeUpOptions (C ), 384TSfflush (C ), 473TSfgets (C ), 473TSFile (C ), 459TSfopen (C ), 473TSfread (C ), 474TSfree (C ), 475TSfwrite (C ), 474TSHandleMLocRelease (C ), 438TSHostLookup (C ), 407TSHostLookupResult (C ), 459TSHostLookupResultAddrGet (C ), 407TSHRTime (C ), 459TSHttpConnect (C ), 407TSHttpConnectWithPluginId (C ), 407TSHttpEventNameLookup (C ), 406TSHttpHdrClone (C ), 408TSHttpHdrCopy (C ), 409TSHttpHdrCreate (C ), 409TSHttpHdrDestroy (C ), 409

TSHttpHdrHostGet (C ), 410TSHttpHdrLengthGet (C ), 410TSHttpHdrMethodGet (C ), 410TSHttpHdrMethodSet (C ), 410TSHttpHdrParseReq (C ), 419TSHttpHdrParseResp (C ), 419TSHttpHdrPrint (C ), 411TSHttpHdrReasonGet (C ), 411TSHttpHdrReasonLookup (C ), 411TSHttpHdrReasonSet (C ), 411TSHttpHdrStatusGet (C ), 411TSHttpHdrStatusSet (C ), 412TSHttpHdrTypeGet (C ), 412TSHttpHdrTypeSet (C ), 412TSHttpHdrUrlGet (C ), 412TSHttpHdrUrlSet (C ), 412TSHttpHdrVersionGet (C ), 413TSHttpHdrVersionSet (C ), 413TSHttpHookAdd (C ), 413TSHttpHookID (C ), 384, 460TSHttpHookNameLookup (C ), 406TSHttpIsInternalRequest (C ), 415TSHttpIsInternalSession (C ), 415TSHttpParser (C ), 460TSHttpParserClear (C ), 418TSHttpParserCreate (C ), 418TSHttpParserDestroy (C ), 419TSHttpServerStateNameLookup (C ), 406TSHttpSsn (C ), 460TSHttpSsnClientFdGet (C ), 420TSHttpSsnDebugGet (C ), 405TSHttpSsnDebugSet (C ), 405TSHttpSsnHookAdd (C ), 413TSHttpSsnIsInternal (C ), 425TSHttpSsnReenable (C ), 420TSHttpStatus (C ), 385, 460TSHttpTxn (C ), 460TSHttpTxnCachedReqGet (C ), 421TSHttpTxnCachedRespGet (C ), 421TSHttpTxnCacheLookupStatusGet (C ), 420TSHttpTxnCacheLookupUrlGet (C ), 285, 420TSHttpTxnCacheLookupUrlSet (C ), 285, 421TSHttpTxnClientFdGet (C ), 421TSHttpTxnClientPacketDscpSet (C ), 422TSHttpTxnClientPacketMarkSet (C ), 422TSHttpTxnClientPacketTosSet (C ), 422TSHttpTxnClientReqGet (C ), 423TSHttpTxnClientRespGet (C ), 423TSHttpTxnConfigFind (C ), 416TSHttpTxnConfigFloatGet (C ), 415TSHttpTxnConfigFloatSet (C ), 415TSHttpTxnConfigIntGet (C ), 415TSHttpTxnConfigIntSet (C ), 415TSHttpTxnConfigStringGet (C ), 416

525


TSHttpTxnConfigStringSet (C ), 416TSHttpTxnDebugGet (C ), 405TSHttpTxnDebugSet (C ), 405TSHttpTxnErrorBodySet (C ), 423TSHttpTxnHookAdd (C ), 413TSHttpTxnIncomingAddrGet (C ), 424TSHttpTxnInfoIntGet (C ), 424TSHttpTxnIntercept (C ), 425TSHttpTxnIsInternal (C ), 425TSHttpTxnMilestoneGet (C ), 426TSHttpTxnNextHopAddrGet (C ), 428TSHttpTxnOutgoingAddrGet (C ), 428TSHttpTxnParentProxySet (C ), 429TSHttpTxnReenable (C ), 429TSHttpTxnReqCacheableSet (C ), 285TSHttpTxnServerAddrGet (C ), 429, 430TSHttpTxnServerFdGet (C ), 430TSHttpTxnServerIntercept (C ), 430TSHttpTxnServerPacketDscpSet (C ), 431TSHttpTxnServerPacketMarkSet (C ), 432TSHttpTxnServerPacketTosSet (C ), 432TSHttpTxnServerReqGet (C ), 432TSHttpTxnServerRespGet (C ), 433TSHttpTxnSsnGet (C ), 433TSHttpTxnTransformedRespCache (C ), 433TSHttpTxnTransformRespGet (C ), 433TSHttpTxnUntransformedRespCache (C ), 433TSHttpType (C ), 387, 460TSInstallDirGet (C ), 435TSIOBuffer (C ), 460TSIOBufferBlock (C ), 460TSIOBufferBlockReadStart (C ), 434TSIOBufferCopy (C ), 434TSIOBufferCreate (C ), 434TSIOBufferDestroy (C ), 434TSIOBufferProduce (C ), 434TSIOBufferReader (C ), 460TSIOBufferSizedCreate (C ), 434TSIOBufferSizeIndex (C ), 460TSIOBuffersSizeIndex (C ), 387TSIOBufferWaterMarkGet (C ), 434TSIOBufferWaterMarkSet (C ), 434TSIOBufferWrite (C ), 434TSIsDebugTagSet (C ), 405TSLifecycleHookAdd (C ), 436TSLifeCycleHookID (C ), 388TSLifecycleHookID (C ), 460TSLookingUpType (C ), 389TSmalloc (C ), 474TSMBuffer (C ), 460TSMBufferCreate (C ), 438TSMBufferDestroy (C ), 438TSMgmtCounter (C ), 460TSMgmtCounterGet (C ), 439

TSMgmtFloat (C ), 460TSMgmtFloatGet (C ), 439TSMgmtInt (C ), 460TSMgmtIntGet (C ), 439TSMgmtString (C ), 460TSMgmtStringGet (C ), 439TSMgmtUpdateRegister (C ), 440TSMilestonesType (C ), 426TSMiletonesType (C ), 389TSMimeHdrClone (C ), 440TSMimeHdrCopy (C ), 440TSMimeHdrCreate (C ), 441TSMimeHdrDestroy (C ), 441TSMimeHdrFieldAppend (C ), 441TSMimeHdrFieldClone (C ), 441TSMimeHdrFieldCopy (C ), 442TSMimeHdrFieldCopyValues (C ), 442TSMimeHdrFieldCreate (C ), 442TSMimeHdrFieldDestroy (C ), 442TSMimeHdrFieldFind (C ), 443TSMimeHdrFieldGet (C ), 443TSMimeHdrFieldLengthGet (C ), 443TSMimeHdrFieldNameGet (C ), 444TSMimeHdrFieldNameSet (C ), 444TSMimeHdrFieldNext (C ), 444TSMimeHdrFieldNextDup (C ), 444TSMimeHdrFieldRemove (C ), 444TSMimeHdrFieldsClear (C ), 449TSMimeHdrFieldsCount (C ), 449TSMimeHdrFieldValueAppend (C ), 445TSMimeHdrFieldValueDateGet (C ), 446TSMimeHdrFieldValueDateInsert (C ), 445TSMimeHdrFieldValueDateSet (C ), 445TSMimeHdrFieldValueInt64Get (C ), 446TSMimeHdrFieldValueIntGet (C ), 446TSMimeHdrFieldValueIntSet (C ), 446TSMimeHdrFieldValuesClear (C ), 448TSMimeHdrFieldValuesCount (C ), 448TSMimeHdrFieldValueStringGet (C ), 446TSMimeHdrFieldValueStringInsert (C ), 447TSMimeHdrFieldValueStringSet (C ), 448TSMimeHdrFieldValueUintGet (C ), 446TSMimeHdrFieldValueUintInsert (C ), 448TSMimeHdrFieldValueUintSet (C ), 448TSMimeHdrLengthGet (C ), 449TSMimeHdrParse (C ), 449TSMimeHdrPrint (C ), 450TSMimeParser (C ), 460TSMimeParserClear (C ), 450TSMimeParserCreate (C ), 450TSMimeParserDestroy (C ), 451TSMLoc (C ), 460TSMutex (C ), 460TSMutexCreate (C ), 451

526


TSMutexDestroy (C ), 451TSMutexLock (C ), 451TSMutexLockTry (C ), 451TSMutexUnlock (C ), 452TSNetAccept (C ), 452TSNetAcceptNamedProtocol (C ), 452TSNetConnect (C ), 453TSOverridableConfigKey (C ), 390, 415TSParseResult (C ), 393, 460TSPluginDirGet (C ), 435TSPluginInit (C ), 453TSPluginRegister (C ), 453TSPluginRegistrationInfo (C ), 460TSrealloc (C ), 474TSRecordAccessType (C ), 393TSRecordCheckType (C ), 393TSRecordDataType (C ), 394, 461TSRecordModeType (C ), 394TSRecordPersistType (C ), 395TSRecordType (C ), 395TSRecordUpdateType (C ), 396TSReleaseAssert (C ), 406TSRemapDeleteInstance (C ), 454TSRemapDone (C ), 454TSRemapDoRemap (C ), 454TSRemapInit (C ), 454TSRemapInterface (C ), 461TSRemapNewInstance (C ), 454TSRemapOSResponse (C ), 454TSRemapRequestInfo (C ), 461TSRemapStatus (C ), 461TSReturnCode (C ), 396, 461TSSDKVersion (C ), 396, 461TSServerSessionSharingMatchType (C ), 397TSServerSessionSharingPoolType (C ), 397TSServerState (C ), 398, 461TSSslContext (C ), 455TSSslContextFindByAddr (C ), 455TSSslContextFindByName (C ), 455TSSslVConnection (C ), 470TSSslVConnOp (C ), 398TSSslVConnReenable (C ), 469TSstrdup (C ), 474TSStringPercentDecode (C ), 465TSStringPercentEncode (C ), 465TSstrlcat (C ), 474TSstrlcpy (C ), 474TSstrndup (C ), 474TSTextLogObject (C ), 461TSTextLogObjectCreate (C ), 456TSTextLogObjectDestroy (C ), 456TSTextLogObjectFlush (C ), 456TSTextLogObjectHeaderSet (C ), 456TSTextLogObjectRollingEnabledSet (C ), 456

TSTextLogObjectRollingIntervalSecSet (C ), 456TSTextLogObjectRollingOffsetHrSet (C ), 456TSTextLogObjectRollingSizeMbSet (C ), 456TSTextLogObjectWrite (C ), 456TSThread (C ), 461TSThreadCreate (C ), 456TSThreadDestroy (C ), 456TSThreadFunc (C ), 461TSThreadInit (C ), 457TSThreadPool (C ), 399, 461TSThreadSelf (C ), 457TSTrafficServerVersionGet (C ), 457TSTrafficServerVersionGetMajor (C ), 457TSTrafficServerVersionGetMinor (C ), 457TSTrafficServerVersionGetPatch (C ), 457TSTransformCreate (C ), 459TSTransformOutputVConnGet (C ), 459TSUrlClone (C ), 461TSUrlCopy (C ), 461TSUrlCreate (C ), 461TSUrlDestroy (C ), 462TSUrlFtpTypeGet (C ), 463TSUrlFtpTypeSet (C ), 463TSUrlHostGet (C ), 463TSUrlHostSet (C ), 464TSUrlHttpFragmentGet (C ), 463TSUrlHttpFragmentSet (C ), 464TSUrlHttpParamsGet (C ), 463TSUrlHttpParamsSet (C ), 464TSUrlHttpQueryGet (C ), 463TSUrlHttpQuerySet (C ), 464TSUrlLengthGet (C ), 466TSUrlParse (C ), 461TSUrlPasswordGet (C ), 463TSUrlPasswordSet (C ), 464TSUrlPathGet (C ), 463TSUrlPathSet (C ), 464TSUrlPercentEncode (C ), 465TSUrlPortGet (C ), 463TSUrlPortSet (C ), 464TSUrlPrint (C ), 466TSUrlSchemeGet (C ), 463TSUrlSchemeSet (C ), 464TSUrlStringGet (C ), 466TSUrlUserGet (C ), 463TSUrlUserSet (C ), 464TSVConn (C ), 461TSVConnAbort (C ), 467TSVConnCacheObjectSizeGet (C ), 467TSVConnClose (C ), 467TSVConnClosedGet (C ), 467TSVConnCloseFlags (C ), 399TSVConnFdCreate (C ), 467TSVConnIsSsl (C ), 468

527


TSVConnRead (C ), 468TSVConnReadVIOGet (C ), 468TSVConnShutdown (C ), 469TSVConnSslConnectionGet (C ), 469TSVConnTunnel (C ), 470TSVConnWrite (C ), 470TSVConnWriteVIOGet (C ), 470TSVIO (C ), 461TSVIOBufferGet (C ), 470TSVIOContGet (C ), 471TSVIOMutexGet (C ), 471TSVIONBytesGet (C ), 471TSVIONBytesSet (C ), 471TSVIONDoneGet (C ), 471TSVIONDoneSet (C ), 472TSVIONTodoGet (C ), 472TSVIOReaderGet (C ), 472TSVIOReenable (C ), 472TSVIOVConnGet (C ), 472

VVol (C++ ), 284Vol::evac_range (C++ ), 284Vol::len (C++ ), 284Vol::Vol::buckets (C++ ), 284Vol::Vol::evacuate (C++ ), 284Vol::Vol::segments (C++ ), 284VolHeaderFooter (C++ ), 285

Wwrite cursor, 516

528