Denial of Service anD DiStributeD Denial of Service ProtectionBusiness white paper

2

Table of contents

Introduction 3What is a Denial of Service attack? 3HP TippingPoint solution 4Seven common DDoS attack methods 4Method 1—Vulnerabilities 4The HP TippingPoint solution for vulnerabilities 4

Method 2—Zombie recruitment 5The HP TippingPoint solution for zombie recruitment 6

Method 3—Attack tools 6The HP TippingPoint solution for DDoS tools 6

Method 4—Bandwidth attacks 6The HP TippingPoint solution for bandwidth attacks 6

Method 5—SYN flood 7SYN flood attack 7Mitigating SYN flood attacks with proxy server 8The HP TippingPoint solution for SYN floods 8

Method 6—Established connection flood 8The HP TippingPoint solution for Established connection floods 8

Method 7—Connections per second floods 9The HP TippingPoint solution for CPS floods 9

Case study—eNom 9IPS “must-haves” 10Conclusion 11

3

IntroductionThe degraded service and lost business from a Denial of Service (DoS) attack can lead to staggering costs both during and after an attack. For an e-commerce site like eBay or Buy.com, one day of downtime due to a DoS attack can cost tens of millions of dollars in terms of lost revenue. The SQL Slammer worm, a DoS attack that made mission-critical Microsoft® SQL servers inaccessible, cost corporations billions of dollars worldwide. The need to maintain an accessible Web presence for valid users while preventing DoS and DDoS attacks continues to be essential as these threats continue to block the legitimate use of websites. Beyond the immediate costs, the lasting effects of a successful DoS attack include lost customers, loss of faith in the service’s dependability, and damage to the corporate brand.

A recent trend in Distributed DoS (DDoS) attacks reveals a new twist in the spiraling costs to companies and organizations. The evolution of DoS attacks began with hackers that targeted larger websites for the thrill of hacking.1 However, as opportunities increase, organized crime has set its sight on companies with more to lose in their businesses and reputations, such as online banks, lenders, and service providers. In recent years, the objectives of organizers’ have become more menacing, such as, industrial and political warfare.

Organized crime syndicates extort money from online companies to keep them from receiving severe DDoS attacks. If a company does not meet the demands, the attackers bombard the company’s systems with constant and overwhelming DDoS attacks from thousands of zombies, placing their e-commerce businesses into gridlock.

What is a Denial of Service attack?DoS attacks are network-based attacks that prevent access to a service. DoS attacks disable a network service by flooding connections, crashing servers or programs running on the servers, exhausting server resources, or otherwise preventing legitimate clients from accessing the network service. DoS attacks range from single packet attacks that crash servers to coordinated packet floods from multiple hosts. In single packet attacks, a carefully crafted packet that exploits a known operating system or application vulnerability is sent through the network to disable a server and/or any associated services it performs. The Slammer worm exploited one such vulnerability.

In a flood attack, server or network resources are corrupted or exhausted by a flood of packets. Since a single site launching a flood can be identified and isolated fairly easily, a more sophisticated approach, called a DDoS attack, is the tool of choice for many flood attacks. In a DDoS attack, an attacker uses multiple machines to assault a target. Some attacks are simple in design, such as sending a relentless stream of data to flood the network connection to the server. Other attacks, such as SYN floods, use carefully crafted packets to exhaust critical server resources in order to prevent legitimate clients from connecting to the server. Regardless of the specifics, a DDoS attack utilizes a significant number of machines in a coordinated manner. These machines, known as zombies, are machines that have been previously compromised and are under the attackers’ control. Hackers often boast about the number of zombies that they have under their control. By sending commands to the zombies over covert communication channels, hackers can stage large coordinated attacks. As the attack is originating from a large number of PCs spread across a wide network, simple identification and isolation techniques do not work. In many cases, it is extremely difficult to separate legitimate traffic from attack traffic. As more PCs gain broadband access from homes, the field of potential zombies increases. Experts estimate that one-third of home user’s PCs on the Internet have been compromised. The sophistication required and barrier to launch these DDoS attacks has been greatly reduced through the availability of packaged tools (for example, Tribe Flood Network and Stacheldraht) that are freely available on the Internet.

“Denial of Service (DoS) attacks are on the rise. Denial of Service protection is a natural extension for intrusion prevention systems because they are in-line and have the ability to deeply inspect and classify traffic, then take action accordingly.” Richard Stiennon, Vice President, Gartner Research

1 Naftali Bennett, CEO of U.S. Internet security company Cyota, quoted by Robin Arnfield in “Credit-Card Processor Hit by DDoS Attack” for NewsFactor.

4

HP TippingPoint solutionIn response to the evolving nature of DoS and DDoS attacks, HP TippingPoint has developed a protection mechanisms corresponding to the methods attackers employ. The HP TippingPoint Intrusion Prevention System (IPS) operates in line to protect a network and the hosts connected to it by examining every bit of traffic that passes through it and filtering unwanted traffic. HP TippingPoint has two primary classes of protection: standard DoS protection and advanced DDoS protection. Standard DoS protection provides a base level of protection against vulnerabilities, attack tools, and traffic anomalies. Advanced DDoS protection guards against SYN flood, established connection flood, and connections per second flood attacks.

HP TippingPoint provides standard DoS protection through all its IPS products:•Vulnerability protection: It protects against DoS attacks

that crash servers by exploiting known vulnerabilities.•Zombie recruitment protection: It protects

against zombie recruitment of systems through Trojan programs.

•Attack tool protection: It blocks the covert channels used by well-known DDoS attack programs including TFN, Loki, and Stacheldraht.

•Bandwidth protection: It protects against packet floods like ICMP, TCP, or UDP that can consume network bandwidth or server resources causing legitimate packets to be dropped. These filters baseline and throttle traffic when it goes beyond a set percentage.

Advanced DDoS protection provides the following additional protection:•SYN proxy: An attacker floods a server with malicious

connection requests (TCP SYNs) with spoofed source IP addresses, preventing legitimate clients from accessing the server.

•Established connection flood: An attacker uses a zombie army to establish a large number—potentially millions—of malicious TCP connections to a server, preventing it from accepting new requests from legitimate clients.

Standard and advanced DoS/DDoS protection work together to stop surgical and brute force DoS attacks and prevent the recruitment of new zombies.

Seven common DDoS attack methodsHackers have an arsenal of methods to enact DDoS attacks. The following seven sections highlight the extent of the dilemma faced by organizations trying to combat the DDoS threat. HP TippingPoint provides solutions to combat these common methods of DDoS attacks:•Vulnerabilities•Zombie recruitment•Attack tools•Bandwidth attacks•SYN floods•Established connection floods•Connections per second floods

Method 1—VulnerabilitiesAttackers can attempt to crash a service or underlying operating system directly through a network. These attacks disable services by exploiting buffer overflows and other implementation loopholes that exist in unprotected servers. Vulnerability attacks do not require extensive resources or bandwidth to perpetrate; attackers only need to know of the existence of a vulnerability to be able to exploit it and cause extensive damage.

Once an attacker has control of a vulnerable service, application, or operating system, they abuse the opening to disable systems and ultimately crash an entire network from within.

The HP TippingPoint solution for vulnerabilitiesHP TippingPoint provides a powerful engine that detects and blocks attempts to exploit vulnerabilities for all incoming and outgoing traffic. The HP TippingPoint security team simultaneously develops attack filters to address discovered vulnerabilities in network services and operating systems and incorporates these filters into digital vaccines. Digital vaccines are delivered to customers every week, or immediately when critical vulnerabilities emerge, and can be deployed automatically without user interaction for automatic protection.

To perpetrate an attack using a large number of hosts that attack simultaneously, attackers infect hosts with a “zombie” or agent program, which connects to a pre-defined master host. Once connected, the attacker can send the command across the entire zombie network. HP TippingPoint protects against zombie attacks by detecting and blocking the viruses used to introduce the zombie agent.

5

Method 2—Zombie recruitmentThe same vulnerabilities that are used to crash a server allow hackers to transform vulnerable PCs into DDoS zombies. Once hackers exploit the vulnerability to gain control of the system, they plant a backdoor into the system for later use in perpetrating DDoS attacks. The Trojan or similar infection provides a path into the system. Once attackers have the path, they remotely control the network, making the server a “zombie” that waits for the given attack command. Using these

zombies, attackers can send a multitude of DoS and DDoS attacks with anonymity.

Viruses can also be used for zombie recruitment. For instance, the MyDoom virus was designed to convert PCs into zombies that attacked SCO and Microsoft at a predetermined time programmed into the virus. Other viruses install backdoors that allow hackers to launch coordinated attacks, increasing the distribution of the attacks across networks around the globe. The following figures detail how attackers create and launch these attacks against a network.

Figure 1: Attacker builds pool of zombies The attacker builds a pool of zombies by compromising unprotected computers.

Figure 2: Attacker launches the attack The attacker launches an attack against a server/network using zombie computers. The attack cripples performance and blocks the network from receiving legitimate traffic.

Hacker

Unprotected PC

Zombie PC Zombie PC New zombie PC

a) Break into computerb) Download rootkitc) Add to pool of zombies

Hacker controlled zombies

Zombie PC

Windows® Server

Zombie PC

Zombies flood target, renderingit slow or inaccessible by legitimate client

Hacker sends target IP andattack parameters to zombies

Hacker

Zombie PC

6

The HP TippingPoint solution for zombie recruitmentIn addition to the previously described vulnerability protection, HP TippingPoint IPS includes filters to detect and block viruses. The combined effects of virus and vulnerability filters make it virtually impossible for hackers to recruit new zombies.

Method 3—Attack toolsThrough zombie recruitment, hackers use covert communication channels to contact and control their zombie army. They can select from hundreds of off-the-shelf backdoor programs and custom tools from websites. These tools and programs initiate these attacks to infiltrate and control networks as zombie armies to enact further attacks from within. Once they have the zombie systems, they can use other tools to send a single command to all zombies simultaneously. In some cases, commands are carried in ICMP or UDP packets that can bypass firewalls. In other cases, the zombie “phones home” by creating a TCP connection to the master. Once the connection is created, the master can control the zombie.

The tools used to attack and control systems include:•Tribe Flood Network (TFN): Focuses on Smurf, UDP,

SYN, and ICMP echo request floods.•Tribe Flood Network 2000 (TFN2K): The updated

version of TFN.•Trinoo: Focuses on UDP floods. Sends UDP packets to

random destination ports. The size is configurable.•Stacheldraht: Software tool that focuses on TCP, ACK,

TCP NULL, HAVOC, DNS floods, and TCP packet floods with random headers. DDoS tools are maturing both in terms of covert channel implementation and in DDoS flooding techniques. New tools utilize arbitrary port numbers or work across IRC. Further, smarter tools intelligently disguise flooding packets as legitimate service requests and/or introduce a high degree of randomness. These enhancements make it increasingly difficult for a port-filtering device to separate attack packets from legitimate traffic.

The HP TippingPoint solution for DDoS toolsHP TippingPoint offers hundreds of filters that accurately detect and block the covert communication channels, disrupting the command and control network of the hackers’ DDoS army. When combined with virus and vulnerability protection, HP TippingPoint prevents recruitment of new zombies, blocks communications to existing zombies, and gives the administrator detailed information needed to clean the infected system.

Method 4—Bandwidth attacksWhen a DDoS attack is launched, it can often be detected as a significant change in the statistical composition of the network traffic. For example, a typical network might consist of 80 percent TCP and a 20 percent mix of UDP and ICMP. A change in the statistical mix can be a signal of a new attack. For instance, the Slammer worm resulted in a surge of UDP packets, whereas the Welchi worm created a flood of ICMP packets. Such surges can be DDoS attacks or so-called zero-day attacks—attacks that exploit undisclosed vulnerabilities.

The HP TippingPoint solution for bandwidth attacksThe HP TippingPoint IPS provides statistical anomaly filters to detect packet floods and rate-shaping to mitigate their effects. HP TippingPoint provides both protocol and application traffic threshold filters. Protocol traffic threshold filters can be created for TCP, UDP, ICMP, and other IP protocols. Application traffic threshold filters monitor traffic to specific TCP and UDP ports. Both types of statistical anomaly filters create baseline of normal levels for one traffic type and alert if the traffic of that type surges above a user-defined level. For example, you can create a protocol traffic threshold filter that creates a baseline of the normal level for ICMP traffic and alerts if the ICMP traffic levels exceed 300 percent of the normal.

In addition to alerting, HP TippingPoint IPS can prevent the monitored traffic from exceeding or consuming more than a preset amount of network bandwidth. For example, if ICMP traffic exceeds 500 percent of the normal, it can be rate-limited so that it uses no more than 3 Mbps. This powerful capability controls excessive bandwidth consumption of non mission-critical applications and facilitates bandwidth availability for mission-critical traffic. The aggressive propagation traffic produced by recent worms has resulted in DoS attacks against routers, firewalls, and other network infrastructure elements. Limiting this traffic to a capped bandwidth keeps the network running and stifles the attack. Traffic threshold filters are edge-triggered. These filters fire when the threshold is exceeded and again when the threshold is no longer being exceeded. These triggers provide information on the duration of each change in traffic patterns.

7

Method 5—SYN floodOne of the most common types of DoS attacks is the SYN flood. This attack can be launched from one or more attacker machines to disable access to a target server. The attack exploits the mechanism used to establish a TCP connection. Every TCP connection requires the completion of a three-way handshake before it can pass data:

•Connection request: First packet (SYN) sent from the requester to the server, starting the three-way handshake

•Request acknowledgement: Second packet (SYN+ACK) sent from the server to the requester

•Connection complete: Third packet (ACK) sent from the requester back to the server, completing the three-way handshake

The attack consists of a flood of invalid SYN packets with spoofed source IP addresses. The spoofed source address causes the target server to respond to the SYN with a SYN-ACK to an unsuspecting or nonexistent source machine. The target then waits for an ACK packet from the source to complete the connection. The ACK never comes and ties up the connection table with a pending connection request that never completes. The table will quickly fill up and consume all available

resources with invalid requests. While the number of connection entries may vary from one server to another, tables may fill up with only hundreds or thousands of requests. The result is a denial of service since, once a table is full; the target server is unable to service legitimate requests.

The difficulty with SYN attacks is that each request in isolation looks benign. An invalid request is very difficult to distinguish from a legitimate one.

SYN floods are one of the oldest DoS attacks in existence. Any knowledgeable person can launch a TCP SYN flood, making this attack one of the most common. Without proper protection, SYN floods can place an entire organization at risk.

As DoS attacks bombard a network, the requests quickly fill up the connection table of most network security devices.

HP TippingPoint 100E removes DoS attack traffic from the network. It drops the requests immediately from the connection table, as in the case of a TCP SYN flood.

SYN flood attackThe SYN flood attack using spoofed IPs prevents a valid requester from accessing a server due to lack of connections.

Figure 3: A SYN flood attack

Server

Valid requester

Failed link

SYN flood requests with spoofed IP

Replies lost(spoofed IP recipient)

Server busyAttackers took all TCP connections

Attacker

Attacker

Attacker

8

Mitigating SYN flood attacks with proxy serverThe addition of an HP TippingPoint 100E with advanced DDoS protection (including SYN proxy filters) prevents the SYN flood attack from consuming all TCP connections on the server. A valid request can complete a three-way handshake.

The HP TippingPoint solution for SYN floodsHP TippingPoint 100E uses advanced methods to detect and protect enterprise networks against SYN flood. The IPS acts as a proxy, synthesizing and sending the SYN/ACK packet back to the originator, and waiting for the final ACK packet. After the IPS receives the ACK packet from the originator, the IPS “replays” the three-figure sequence to the receiver.

The full attack and response scenario is as follows.•The attacker sends a SYN packet to the target.

HP TippingPoint 100E intercepts the SYN and determines if HP TippingPoint IPS protects the target.

•If so, the IPS generates SYN-ACK on behalf of the target.

•If the IPS receives the final ACK of the three-way handshake, the IPS validates the ACK by utilizing advanced algorithms to verify that this packet is in response to a SYN-ACK generated by the IPS. If so, the IPS creates a connection with the target.

•Once both connections are established, HP TippingPoint maintains the data and connection, ensuring safe traffic. If the originator of the attack does not complete the three-way handshake, no packets are

sent to the target and no state is maintained on the HP TippingPoint IPS.

In the case of a SYN flood, respondent is fully protected from the attack as HP TippingPoint 100E scans, detects, and block the SYN flood.

HP TippingPoint allows the user to designate clients as trusted. Connections from trusted sources are never proxied.

Method 6—Established connection floodAn Established connection is an evolution of the SYN flood attack that employs a multiplicity of zombies to perpetrate a DDoS attack on a target. Zombies establish seemingly legitimate connections to the target server. By using a large number of zombies, each creating a large number of connections to the target, an attacker can create so many connections that the target is no longer able to accept legitimate connection requests. For example, if a thousand zombies create a thousand connections to a target server, the server must manage a million open connections. The effect is similar to a SYN flood attack in that it consumes server resources, but is even more difficult to detect.

The HP TippingPoint solution for Established connection floodsHP TippingPoint Established connection flood filters track the number of connections each source has made to a protected server. When a source attempts to create more than a specified number of connections to a protected

Figure 4: Mitigating a SYN flood attack

Servers

Valid requester

HP TippingPoint 100E

Valid requester receives service

Completes three-wayhandshakes

Advanced DDoSincluding SYN proxy

SYN flood requests with spoofed IP

Replies lost(spoofed IP recipient)

Attacker

Attacker

Attacker

9

server, new connections are blocked until the source closes some connections. For example, HP TippingPoint can facilitate that no single source can create more than 10 open connections to a server. Thus, a thousand zombies can create no more than ten thousand connections to a protected server.

When HP TippingPoint detects a DoS attack, it enacts a series of actions and notifications according to customized settings. Administrators can set the system to block, permit, or generate notifications for the system, users, and logs.

Every filter in the IPS provides protection against a wide variety of attacks. Network administrators can customize the settings for filters, including the following:

•Actions for attack responses•Notification contacts for alert messages•Exceptions for specific IP addresses

Method 7—Connections per second floodsCPS flood attacks flood servers with a high rate of connections from a seemingly valid source. In these attacks, an attacker or army of zombies attempts to exhaust server resources by quickly setting up and tearing down TCP connections, possibly initiating a request on each connection. For example, an attacker might use his zombie army to repeatedly fetch the home page from a target Web server. The resulting load makes the server extremely sluggish.

The HP TippingPoint solution for CPS floodsHP TippingPoint enables network administrators to create CPS filters. Each filter limits the average number of connections that a client may open to a particular server per second. Each filter includes a threshold setting of the calculated average number of connections per second to allow from a particular client. The network administrator can create a CPS filter for both port A->B and port B->A traffic. The flexible settings allow customizations for incoming and outgoing traffic and attack detection based on network traffic needs.

HP TippingPoint computes the average of a 10 second window to allow for normal fluctuations of traffic. A common traffic pattern is a Web browser that opens 10 connections to download a complex page, then sits idle while the user reads. To accommodate this pattern, the filters scan and detect against the amount of new connections averaged over a 10 second period. For example, if a filter specifies a maximum

of 3.5 connections per second, browsers can open up to 35 connections in a second. However, after making these connections, the browser is unable to open any new connections for 9 more seconds. As a result, over the 10 second period, the browser has averaged 3.5 connections permitted per second. Used in conjunction with established connection filters, CPS flood protection can provide powerful detection and protection of a network.

Established connection flood attacks can be some of the most difficult to detect and block. These attacks originate from an IP address that is checked and accepted by a proxy server through a complete three-way handshake.

Once an established connection flood attack enters a network, it strikes against the proxy server, intending to crash it. Once the proxy crashes, access to systems and servers behind the proxy server is blocked.

CPS flood filters working in conjunction with established connection flood filters and SYN proxy filters can provide dynamic and powerful protection for your network traffic.

Case study—eNomFounded in 1997, eNom, Inc. is one of the largest ICANN-accredited domain name registrars with over four million names. The company suffered from continual DoS attacks against their servers and customers. According to eNom, their systems suffered DDoS attacks 15 days a month for each month, January to August 2004. In reviewing their network traffic, the eNom servers received 6000 to 7000 attack SYNs/second. Peak attacks against the systems included approximately 40,000 attack SYNs/second.

To protect their customers and network systems, the company sought an IPS to detect and block attacks without interrupting legitimate traffic. Facing a difficult and costly problem, eNom sought out a group of vendors of IPS systems powered with DoS protection. The following list includes the vendors they considered for their company’s network protection and security:•HP TippingPoint•Radware•Top Layer•NAI•NetScreen

10

Table 1: Vendor comparison list

eNom evaluated the HP TippingPoint 100E IPS system with Advanced DoS protection. The enhanced DoS protection coupled with best-of-breed network protection, digital vaccine updates, and outstanding technical support provided the solution they needed to provide continued service for their customers. The Advanced DoS protection blocked a variety of DoS and DDoS attacks including SYN floods, connection floods, packet floods, and difficult-to-detect attacks originating from spoofed and non-spoofed sources.

IPS “must-haves”For the most comprehensive protection for networks, an IPS solution should have a set of core capabilities. The vendor comparison list details these attributes according to IPS companies. Of these must-have categories, HP TippingPoint provides them all with award winning products and services.

2 Rarely deployed inline, usually as IDS

Attributes HP TippingPoint McAfee ISS Juniper Radware Top Layer

Custom ASICs Y 8 Celeron® Software Software Y Y

50 Mb/s–5 Gb/s 5 Gb/s 2 Gb/s 1 Gb/s 500M 3 Gb/s 2 Gb/s

Switch-like latency Y N N N Y Y

Inline attack blocking Y Y2 Y2 Y Limited N

Bandwidth management Y N N N Y N

DDoS SYN flood protection Y N N Y Y Y

DDoS connection rate limits Y N N N N Y

Filter method: signature Y Y Y Y Y N

Filter method: protocol Y Y Y Y N Limited

Filter method: vulnerability Y Y Y Limited N N

Filter method: traffic anomaly Y Y N N N Limited

VoIP protection Y N N N N N

11

ConclusionTo obtain full protection for DoS attacks, organizations typically need to purchase multiple proxy servers, network security devices, intrusion preventions systems, as well as software packages, updates, and expanded licenses as an organization grows.

HP TippingPoint provides the answer in a single system. HP TippingPoint IPS is an easy, affordable,

and scalable solution, equipped with a broad range of protection mechanisms including, application anomaly filters, protocol anomaly filters, exploit signature filters, threshold rate-shaping filters, and advanced DoS/DDoS filters for detecting and blocking attacks.

Attacks continue to evolve and increase in sophistication. The flexibility of HP TippingPoint platform offers state-of-the-art protection against current attacks and the power to protect against future ones.

It’s time to obtain full protection against Denial of Service attacks. HP TippingPoint is the answer. Learn more about it at www.hp.com/networking/security.

Get connectedwww.hp.com/go/getconnected

Get the insider view on tech trends, alerts, and HP solutions for better business outcomes

© Copyright 2011 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

Microsoft and Windows are U.S. registered trademarks of Microsoft Corporation. Celeron is a trademark of Intel Corporation in the U.S. and other countries.

4AA1-1987ENW, Created June 2011; Updated November 2011, Rev. 1

IntroductionWhat is a Denial of Service attack?HP TippingPoint solutionSeven common DDoS attack methodsMethod 1—VulnerabilitiesThe HP TippingPoint solution for vulnerabilities

Method 2—Zombie recruitmentThe HP TippingPoint solution for zombie recruitment

Method 3—Attack toolsThe HP TippingPoint solution for DDoS tools

Method 4—Bandwidth attacksThe HP TippingPoint solution for bandwidth attacks

Method 5—SYN floodSYN flood attackMitigating SYN flood attacks with proxy serverThe HP TippingPoint solution for SYN floods

Method 6—Established connection floodThe HP TippingPoint solution for Established connection floods

Method 7—Connections per second floodsThe HP TippingPoint solution for CPS floods

Case study—eNomIPS “must-haves”Conclusion