go with the flow-v2

Go with the Flow …

Md. Zobair Khan AS58587

kzobair

One name to rule them all … :P

Network Flow

A flow is a set of packets with common characteristics within a given time frame and a given direction. In packet switching networks, traffic flow, packet flow or network flow is a sequence of packets from a source computer to a destination, which may be another host, a multicast group, or a broadcast domain. RFC 2722 defines traffic flow as "A TRAFFIC FLOW is an artificial logical equivalent to a call or connection, belonging to a (user-specified) METERED TRAFFIC GROUP."

Network Flow

NetFlow

NetFlow was introduced in Cisco routers first to get the traffic information from one or many source/s to one or many destination/s. Also supported by Juniper, Mikrotik etc.

Jflow or cflowd for Juniper Networks NetStream for 3Com/HP NetStream for Huawei Technologies Cflowd for Alcatel-Lucent Rflow for Ericsson AppFlow Citrix Traffic Flow MikroTik sFlow vendors include: Alcatel Lucent, Cisco, Dell, D-Link, Fortinet, Hewlett-Packard, Huawei, IBM, Juniper, NEC, Netgear, ZTE etc

NetFlow

Version Comment

v1 First implementation, now obsolete, and restricted to IPv4 (without IP mask and AS Numbers).

v2 Cisco internal version, never released.



v5 Most common version, available on many routers from different brands, but restricted to IPv4 flows.

v6 No longer supported by Cisco.

v7 Like version 5 with a source router field. Used on Cisco Catalyst switches.

v8 Several aggregation form, but only for information that is already present in version 5 records

v9 Template Based, available on some recent routers. Mostly used to report flows like IPv6, MPLS, or even plain IPv4 with BGP nexthop.

v10 Used for identifying IPFIX - IP Flow Information Export.

Cisco Configuration

ip flow-export version 5 origin-as ip flow-export source Loopback0 ip flow-export destination [ServerIP] 3000 interface TenGigabitEthernet1/0/0 ip flow ingress ip flow egress

Juniper Configuration

set firewall filter test-flow term 1 then sample set firewall filter test-flow term 1 then accept set interfaces ge-0/0/0 unit 0 family inet filter input test-flow set interfaces ge-0/0/0 unit 0 family inet filter output test-flow set forwarding-options sampling input rate 1000 set forwarding-options sampling family inet output flow-server [ServerIp] port 3000 set forwarding-options sampling family inet output flow-server [ServerIp] version 5

Server

First Check if you are receiving the flows or not tcpdump -i eth0 port 3000 17:30:19.248072 IP InterfaceName.53344 > ServerName.3000: UDP, length 1464 17:30:19.248079 IP InterfaceName.53344 > ServerName.3000: UDP, length 1272 17:30:19.248853 IP InterfaceName.53344 > ServerName.3000: UDP, length 1464 17:30:19.248887 IP InterfaceName.53344 > ServerName.3000: UDP, length 1464 17:30:19.248894 IP InterfaceName.53344 > ServerName.3000: UDP, length 1272 17:30:19.249385 IP InterfaceName.60532 > ServerName.3000: UDP, length 1416

Now What !!!

Yes…. The Flow exporter is exporting the flows and the Flow Collector is receiving them. So now we can start analyzing them to understand the traffic pattern of our network. Can be done in many ways with many tools. We will discuss one of the most basic tools which can be run on bash easily with little resources and required output.

Flow-Tools

Flow-tools is a library and a collection of programs used to collect, send, process, and generate reports from NetFlow data. Supports NetFlow version upto 8. Best output with NetFlow version 5. Included Programs are flow-capture , flow-cat, flow-stats and many more. http://linux.die.net/man/1/flow-tools

Advantages

1. Gives detail information on each & every particular conversation without sniffing.

2. No problem for encrypted data. For any incident, traffic source & Destinations are visible.

3. Historical data of Flows can help operator to improve quality.

4. Data can be fetched from anywhere in network as needed in a customized way.

5. If you are multihomed, these information are important to make sure that your clients are getting quality service.

6. NFSEN does the same work but needs bigger resources compared to Flow-Tools.

7. Ideal for startup ISPs, small enterprise, office IT network, campus network etc.

Flow-Tools

apt-get install flow-tools Or get it from here and install it https://flow-tools.googlecode.com/files/flow-tools-0.68.5.1.tar.bz2 Make a directory to store your flows mkdir /var/flows/

Flow-Tools

Edit the flow-capture.conf file at /etc/flow-tools/ , comment all and use the below line. -V 5 -E 5G -N 3 -w /var/flows 0.0.0.0/ServerIP/3000 Which means – NetFlow Version will be 5 Expire the total stored flow files as per the given space – here we will set 5 G Nesting level for sorting flow files Working directory will be /var/flows Allow any IP as analyzer and ServerIP as exporter with port 3000

Flow-Tools

We can now start capturing flows with the following command. flow-capture -w /var/flows -E5G -S3 0/0/3000 Which means – Flow capture will start with working directory /var/flows Total size of all the flow files will not exceed 5 G Emit a stat log message every 3 minutes Allow any IP as analyzer and exporter with port 3000

Flow-Tools

Now if you go to /var/flows/2015/2015-10/2015-10-27/ to see the flow files. Filenames begining with tmp which are typically in-progress flow files from flow-capture are not processed. cd /var/flows/2015/2015-10/2015-10-27/ [3 level nesting for sorting files] ls -lah total 259M drwxr-xr-x 2 root root 4.0K Oct 27 17:07 . drwxr-xr-x 3 root root 4.0K Oct 27 15:24 .. -rw-r--r-- 1 root root 36M Oct 27 16:43 ft-v05.2015-10-27.163000+0600 -rw-r--r-- 1 root root 1022K Oct 27 16:45 ft-v05.2015-10-27.164438+0600 -rw-r--r-- 1 root root 26M Oct 27 16:54 ft-v05.2015-10-27.164500+0600 -rw-r--r-- 1 root root 2.6M Oct 27 16:55 ft-v05.2015-10-27.165435+0600 -rw-r--r-- 1 root root 12M Oct 27 17:00 ft-v05.2015-10-27.165558+0600 -rw-r--r-- 1 root root 21M Oct 27 17:07 ft-v05.2015-10-27.170000+0600 -rw-r--r-- 1 root root 16M Oct 27 17:13 tmp-v05.2015-10-27.170753+0600

Flow-Tools

We are ready to see some outputs finally… 0 Overall Summary 1 Average packet size distribution 2 Packets per flow distribution 3 Octets per flow distribution 4 Bandwidth per flow distribution 5 UDP/TCP destination port 6 UDP/TCP source port 7 UDP/TCP port 8 Destination IP 9 Source IP 10 Source/Destination IP 11 Source or Destination IP 12 IP protocol 13 octets for flow duration plot data 14 packets for flow duration plot data 15 short summary 16 IP Next Hop

17 Input interface 18 Output interface 19 Source AS 20 Destination AS 21 Source/Destination AS 22 IP ToS 23 Input/Output Interface 24 Source Prefix 25 Destination Prefix 26 Source/Destination Prefix 27 Exporter IP 28 Engine Id 29 Engine Type 30 Source Tag 31 Destination Tag 32 Source/Destination Tag

Flow-Tools

To view output in bash we need to use the below command remaining at the flow files directory which is /var/flows/2015/2015-10/2015-10-27/ flow-cat -p ft-v05.2015-10-27.170000+0600 | flow-stat -f11 -P -p -S4 | head -30 Meaning – Concatenate flow file named ft-v05.2015-10-27.170000+0600 The headers are preloaded for this file containing the metadata. Flow-stat will provide function 11 (Source or Destination IP) with preloaded headers and Percentage to the total amount for 4 minutes duration of flows.

Flow-Tools : Case Study


Viewing most talkative Source or Destination IP: flow-cat -p ft-v05.2015-10-27.170000+0600 | flow-stat -f11 -P -p -S4 | head -30 # --- ---- ---- Report Information --- --- --- # # Fields: Percent Total # Symbols: Disabled # Sorting: Descending Field 4 # Name: Source or Destination IP # # Args: flow-stat -f11 -P -p -S4 # # IPaddr flows octets packets # 122.102.32.106 5.809 7.793 8.062 103.253.38.11 2.577 10.787 9.039 122.102.32.90 2.791 5.509 5.384 103.26.112.170 3.132 3.334 3.175 103.26.112.17 1.849 1.865 1.767


Finding most talkative Source ASNs: flow-cat -p ft-v05.2015-10-27.170000+0600 | flow-stat -f19 -P -p -S4 | head -30 # --- ---- ---- Report Information --- --- --- # # Fields: Percent Total # Symbols: Disabled # Sorting: Descending Field 4 # Name: Source AS # # Args: flow-stat -f19 -P -p -S4 # # # src AS flows octets packets # 45245 3.764 0.045 0.236 15169 0.738 0.071 0.066 7565 0.792 0.021 0.056 56138 0.985 0.010 0.062


Finding Top Source/Destination ASN: flow-cat -p ft-v05.2015-10-27.170000+0600 | flow-stat -f21 -P -p -S4 | head -30 # --- ---- ---- Report Information --- --- --- # # Fields: Percent Total # Symbols: Disabled # Sorting: Descending Field 4 # Name: Source/Destination AS # # Args: flow-stat -f21 -P -p -S4 # # # src AS dst AS flows octets packets # 45245 32934 1.023 0.012 0.061 45245 15169 0.505 0.008 0.032 15169 7565 0.230 0.030 0.025 15169 45245 0.277 0.025 0.024


Finding DoS/DDoS Source/Destination: flow-cat -p ft-v05.2015-10-28.133000+0600 | flow-stat -f10 -P -p -S3 | head -40 # --- ---- ---- Report Information --- --- --- # src IPaddr dst IPaddr flows octets packets # 178.79.215.39 43.231.22.2 0.001 0.793 0.622 62.210.157.218 122.102.32.90 0.000 0.713 0.560 87.248.221.154 43.231.22.2 0.001 0.555 0.436 87.240.163.54 122.102.32.90 0.001 0.523 0.410 37.59.33.100 210.1.249.136 0.001 0.510 0.400 163.53.140.168 103.52.134.18 0.017 0.461 0.365 31.14.252.94 210.1.253.68 0.000 0.455 0.357 62.210.103.45 122.102.32.90 0.001 0.450 0.353 188.138.1.11 122.102.32.106 0.001 0.448 0.351 4.26.227.253 103.230.105.13 0.004 0.420 0.330 31.3.245.242 122.102.32.106 0.002 0.418 0.328 86.146.198.23 122.102.32.106 0.000 0.408 0.229 95.55.44.11 103.230.105.26 0.000 0.405 0.222 163.53.140.179 103.52.134.18 0.011 0.387 0.308 163.53.140.178 103.230.105.14 0.027 0.382 0.222


Finding DoS/DDoS Source/Destination: flow-cat -p ft-v05.2015-10-28.134500+0600 | flow-stat -f10 -P -p -S3 | head -40 # --- ---- ---- Report Information --- --- --- # src IPaddr dst IPaddr flows octets packets # 103.7.251.45 173.205.124.100 12.517 6.555 5.006 95.211.191.89 103.253.38.11 0.003 4.548 2.901 69.16.175.42 103.253.38.5 0.005 3.779 2.486 46.28.51.252 103.253.38.5 0.002 2.850 1.818 89.46.103.119 103.253.38.5 0.018 2.454 1.575 69.16.175.42 103.253.38.11 0.006 2.440 1.610 91.121.4.51 103.253.38.11 0.003 2.437 1.558 195.154.194.38 103.253.38.5 0.011 2.047 1.310 203.223.95.78 50.97.95.98 0.008 2.037 0.960 87.248.212.205 103.253.38.11 0.002 1.955 2.144 176.205.18.194 103.253.38.5 0.003 1.504 1.332 188.165.200.34 103.253.38.11 0.002 1.440 0.919 94.242.246.118 103.253.38.11 0.005 1.374 0.881 205.185.216.42 103.253.38.11 0.002 0.973 0.676 178.149.93.10 103.253.38.11 0.000 0.965 0.443

Any Query ???

Thank You …

go with the flow-v2

Documents