IBM QRadar – Fundamentals of Flows—Asia Pacific Threat Management Team

• Jenson John

• Ashish Kothekar

• Deepankar Panda

• Boudhayan Chakrabarty (Bob)

Presenters and Panelists

What are Flows ?

• Flows provide information about network traffic and can be sent to QRadar in various formats

• Can accept multiple flow formats simultaneously

• QFlow Collectors provide full application detection of network traffic regardless of the port on which the application is operating

Magistrate(Console)

QFlowCollectors

FlowProcessors

Flow Sources

Flow Collection and Processing

• A flow is a record of the communication between two machines

• QFlow Collectors read packets from the wire

• QFlow Collectors can also receive flow information from routers, firewalls, etc.

• QFlow Collectors convert all gathered network data to flows similar normalized events

• They include such details as when, who, how much, protocols, and options

3

© COPYRIGHT IBM CORPORATION 2015

Data Collection – Events vs. Flows

How do Flows and Events differ from each other?

• A flow is different from an event, in that flows (for the most part) will have a start and end time, or, a life of multiple seconds. For example, when you connect to a website, the communication will include HTML files, images, flash files, longer file downloads, etc., and may take some time to transfer the data.

• An Event, in contrast, represents a single event on the network, such as the login action of a VPN session or a firewall deny by someone trying to connect to a network.

• Events and flows are treated as separate entities within the Event Collector (EC) but are treated as the same entity within the Event Processor (EP).

Event Collector component

l The Event Collector component completes a number of flow processing functions for ECS.

l Flow deduplication: Flow deduplication is a process that removes duplicate flows when multiple QFlow collectors are providing data to flow processor appliances.

l Asymmetric recombination: Responsible for combining two sides of each flow when data is provided asymmetrically. This process can recognize flows from each side and combine them in to one record. However, sometimes both sides of the data do not exist.

- External flow sources such as NetFlow that may only report ingress or egress traffic.

- Instances where span traffic enters a network from a single point, and exists via another, creating asymmetric reporting of data to flow collectors.

l Throttle: Monitors the number of incoming events & flows to the system to manage input queues and licensing.

l Forwarding: Applies routing rules for the system, such as sending data to offsite targets, external Syslog systems, JSON systems, other SIEMs, etc.

Why do I need flows over the Events?

1. The key to QFlow is you gain visibility you cannot get with typical logs from firewalls routers and switches.

2. It has several key advantages like feeding the data into the SIEM, so that it aware of the environment and allows auto discovery of assets and protocols in use.

3. It also build a passive database of all assets and what ports are open on them, this allows rules to be more selective in what they alert you to.

4. Also we have the capability to see the anomaly detection on top of flows.

5. And lastly forensics where AV platform will detect an infection it is unable to clean, the firewall will detect activity out of the network, the firewall log stops there. Flows allow you to dig into the packet and see what was leaving.

IBM Security / © 2019 IBM Corporation 7

QRadar Network Insights ( QNI )

QRadar Network Packet Capture

QRadar Incident Forensics Processor

Analyst

QRadar Flow Deployment – Explanation

IBM Security / © 2019 IBM Corporation 8

Internal Flows : Sources that include packet data.

External Flows : Third party flow sources.

QRadar Flow Processor : QFlow + Flows written to Ariel DB

QRadar Flow Collector : QFlow

QRadar Network Insights : Real-time in-depth visibility in network communication

QRadar Network Packet Capture : Recording of raw network data for forensic analysis

QRadar Incident Forensics Processor : Retrieves packet captures for an incident and reconstructs sessions for forensic analysis.

Another view of the deployment

9

QRadar Console

QRadar Incident Forensics Processor

Aggregation switch

TAP or Mirror port

TAP or Mirror port

TAP or Mirror port

TAP or Mirror port

Flow Collector / ProcessorQRadar Network Insights

QRadar Network Packet Capture

IBM Security / © 2019 IBM Corporation 10

Network flow from routers/switches

QFlow Collector software

QRadar Network Insights appliance

Includes basic network traffic info Yes Yes Yes

Includes application info No Yes Yes

Includes user info No No Yes

Includes deep content visibility No No Yes

Includes attack/exploit identification No No No

Can inspect SSL traffic No No Inbound and outbound(with keys)

Can block traffic No No No

Deployment modes TAP / SPAN port TAP / SPAN port TAP / SPAN port

Comparing QFlow Collector with QNI capabilities

QRadar can collect several types of flow data: QFlow, NetFlow, SFlow, JFlow, IPFIX, and Packeteer.

We differentiate these into two categories:

• Internal/Passive flows: packet based collection (Qflow & Packeteer)

• External/Active flows: sources from routers or switches that generate their own session statistics (NetFlow, SFlow, and JFlow)

Types of flows:

11

Data available by flow types:

• QFlow or Packeteer – Layer 7 visibility, provides details on application communication, URLs, usernames, etc.

• NetFlow (Cisco), Jflow (Juniper), IPFIX (IBM XGS), and SFlow – Layer 3 and Layer 4 visibility. SRC/DST IP, Ports, etc.

12

Types of flows:

Standard flow: A single standard flow record.

• Type A Superflow (Network scans): One source to many destination IPs.This is a unidirectional flow, which has the same source, but multiple destinations.

• Type B Superflow (DDoS): Multiple sources to a single destination IP.

• This is a unidirectional flow, which has the multiple sources, but has a single destination.

• Type C Superflow (Port scans): One-to-one source and destination with many ports. This is a one-to-one flow with different source or destination ports.

Types of Super flows:

13

Example of flow representation

Flow representation:

14

Superflow Type B

Standard Flow

Like events, since multiple QFlow Collectors can connect to a single Flow Processor, flow licensing is enforced in both the ECS-EC and the ESC-EP components.

The primary distinctions between flow licensing and event licensing are:

• Flows are calculated per minute (rather than per second).

• The flow licensing check in the ECS-EC is performed after asymmetric recombination and de-duplication, while this is performed prior to coalescing for events.

• No licensing checks are performed by the qflow component, or on the qflow collector.

Flow licensing:

15

16

Flow licensing:No licensing checks are performed by the qflowcomponent or on the qflowcollector.

QRadar has two separate metrics used to measure the amount of flows coming into an appliance, Flow Limits and Flow Licensing. These should not be confused with each other.

• The QFlow component’s flow limits are measured and enforced prior to entering the ECS Pipeline. These limits are used purely for managing the amount of incoming traffic the appliance can process. When a flow limit is reached, an overflow record will be created. No flows will be dropped at the QFlow component.

• The FPM license enforcement is applied once the flows enter the ECS since multiple QFlowinstances could be feeding into a single Processor.

Limits and Licenses Overview:

17

18

Limits and Licenses Overview:

Flow limits are applied at the Qflow component prior to entering the ECS.

Licensing is enforced at the ECS-EC level, and then again at the ECS-EP

There are 3 types of flow limits applied to the receiving side of the flow pipeline by the Qflowcomponent.

• Deployment flow limit - sum of all flows per minute (FPM) licenses across a deployment, not the individual FPM that might be allocated to a connected flow processor appliance.

• User flow limit (QF Governor) - a user-configurable limit of flows:- If the user flow limit is set, it will be used, provided it is less than the

deployment flow limit- If no user flow limit is set, the minimum of the hardware flow limit and

deployment flow limit will be used

• Hardware flow limit - the recommended number of flows calculated based on the available CPUs and memory (7.3.1+)

Flow Limits:

19

Over Flow record: Created when license limits are exceeded

• When a QFlowcollector hits its flow license limit, it begins creating over flow records.

• Over flow records have a source IP of 127.0.0.4 and a destination IP of 127.0.0.5 with one flow created per protocol (icmp, udp, tcp, etc).

• When the license limit is reached, QFlow rolls the rest of the traffic for the protocol within the interval in to a single record.

• All bytes and packet counts are totaled up and added to these "overflow records".

Over Flows:

20

Examples of flow rules:

21

QFlow

l The Flow Process

- QFlow sends it’s flow information to the "FlowSource" queue within the ecs-ec, which attempts to put the event on a blocking queue. If that queue is full, the event is dropped and the TotalDropped is updated in the MBean.

l QFlow has it’s own service “qflow”:

- stopping/starting/restart/status of qflow: l 7.3.x: systemctl <option> qflowl 7.2.x: service qflow <option>

l Example systemctl status qflow

QFlow Troubleshooting

• Grepping for “qflow” in the qradar.log can help gather additional information about the process and flow sources and other information.

• As each netflow device is seen, qflow will report the netflow activity on the flow source

- Aug 30 14:24:10 qradar3105 [QRadar] [30604] qflow0: [INFO] default_Netflow: Receiving V5 data from: 192.168.1.1:0,0

• Verify data is being written (every minute) to /store/ariel/flows/records/yyyy/mm/dd/hh/* for flow data

• Monitor burst handling by checking in /store/transient/spillover/queue/ecs-ec.ecs-ec/

- ls -lrt FlowGovernerQueue_*.dat | wc -l (if near 50 the queue is close to full, if 1 then the queue is clean)

QFlow Troubleshooting – Internal Flow Sources

• Internal Sources come in on built-in ethernet cards or napatech cards

• Confirm ethernet link:

- Built-in interfacel ethtool <interface>

• Confirm data via statistics:

- Built-in interfacel ifconfig <interface>l Watch the RX and TX packets

• Watch traffic:

- Built-in interfacel tcpdump -nnAs0 -c 100 -i <interface>

- Collect 100 packets and then stop

QFlow Troubleshooting - NetFlow

• To validate if QRadar is listening on the proper port for the flow source

- When Interface is Anyl netstat -nap | grep 2055

- When Interface is a specific one

l ifconfig <interface>

Notice that the interface is in PROMISC mode. This means it is listening for all traffic coming into the interface.

QFlow Troubleshooting - NetFlow

• To validate if flow data is hitting a specific interface (for instance netflow on port 2055)

l tcpdump -p -nn -i <interface> port 2055

• (this will display data as it arrives on eth0 – port 2055). If no data shows, there could be firewall (iptables) or other issues that need further investigation. For instance is the “sender” setup correctly?

• NetFlow v9 requires a template be sent from source, without a template packet, qflow will not be able to interpret the flows. Use this tcpdump command to confirm it is receiving a template packet:

l tcpdump -i <interface> -nneXXs90 port <port#> and host <ipaddress> and 'ether[62:2] = 0x0000'

• Where <port#> is the port QRadar is receiving the NetFlow on and <ipaddress> is the device sending the flows

© Copyright IBM Corporation 2019. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. Any statement of direction represents IBM’s current intent, is subject to change or withdrawal, and represent only goals and objectives. IBM, the IBM logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.

All names and references for organizations and other business institutions used in this deliverable’s scenarios are fictional. Any match with real organizations or institutions is coincidental.

Statement of Good Security Practices: IT system security involves protecting systems and information through prevention, detection and response to improper access from within and outside your enterprise. Improper access can result in information being altered, destroyed, misappropriated or misused or can result in damage to or misuse of your systems, including for use in attacks on others. No IT system or product should be considered completely secure and no single product, service or security measure can be completely effective in preventing improper use or access. IBM systems, products and services are designed to be part of a lawful, comprehensive security approach, which will necessarily involve additional operational procedures, and may require other systems, products or services to be most effective. IBM does not warrant that any systems, products or services are immune from, or will make your enterprise immune from, the malicious or illegal conduct of any party.

Follow us on:

Thank you

xforce.ibmcloud.com

twitter.com/AskIBMSecurity

youtube/user/ibmsecuritysupport

securityintelligence.com

securitylearningacademy.com

IBM Security Community

LinkedIn - IBM Security Client Success