bro: a system for detecting network intruders in real-time
DESCRIPTION
Bro: A System for Detecting Network Intruders in Real-Time. Presented by Abdulmajid , Ameen, Yaseen ,mohammed Lecturer : DR.Nur Izura. Bro Overview. - PowerPoint PPT PresentationTRANSCRIPT
Bro: A System for Detecting Network
Intruders in Real-Time
Presented by Abdulmajid , Ameen, Yaseen ,mohammed
Lecturer :DR.Nur Izura
Bro Overview
Bro is an open-source, Unix-based Network Intrusion Detection System (NIDS) that passively monitors network traffic and looks for suspicious activity.
Bro was developed by Vern Paxson in the Network Research Group at Lawrence Berkley National Lab, and by the International Computer Science Institute.
Bro == Big Brother (“watching and taking care of you” and all your actions).
Original Goals
High-speed, large volume monitoring:
No packet filter drops (Low packet loss rate)
Would allow intruder to Avoid the NIDS.
Real-time notification
The monitor will be attacked
Mechanism separate from policy:
Bro Features
Rich Application-Layer Analysis
Custom Scripting Language
Pre-written Policy Scripts
Powerful Signature Matching Facility
Network Traffic Analysis
Detection Followed by Action
Snort Compatibility Support (snort2bro)
Target EnvironmentsBro is intended for use by sites requiring flexible, highly customizable intrusion detection.
Development of bro is primarily driven by research for intrusion detection and traffic analysis
Bro is specifically well-suited for scientific environments
• Extremely useful in networks with liberal (“default allow”) policies
• Supports intrusion prevention schemes
• High-performance on commodity hardware
• Runs on Unix-based systems (e.g., Linux, FreeBSD, MacOS)
• Open-source (BSD license)
Target Environments(con`t)
Bro is designed for use by Unix experts who place a premium on the ability to extend an intrusion detection system with new functionality as needed, which can greatly aid with tracking evolving attacker techniques as well as inevitable changes to a site's environment and security policy requirements.
Some sites may wish to run a commercial IDS as their front-line of defense, and then also run Bro as a way to:
• Verify the results of the commercial IDS / defense-in-depth
• Attain richer forensics capabilities
• Provide policy-checking capabilities not facilitated by the commercial IDS
• Experiment with new approaches and incorporate leading-edge research
Bro Deployment
Structure of the system
• Bro is layered into three major components . It uses the now-standard libpcap packet-capture library developed by LBNL's Network Research Group to filter the packet stream in the kernel for high performance.
• Bro is conceptually divided into an “event engine” that reduces a stream of (filtered) packets to a stream of higher-level network events.
• An interpreter for a specialized language that is used to express a site's security policy.
• The lower-most layers process the greatest volume of data, and hence must limit the work performed to a minimum.
• As we go higher up through the layers, the data stream diminishes, allowing for more processing per data item.
• This basic design reflects the need to conserve processing as much as possible, in order to meet the goals of monitoring high-speed, large volume traffic flows without dropping packets
Bro Architecture
Network
Policy Interpreter
Event Engine
libpcapKernel filters down high-
volume stream via packet capture library.
Distills filtered stream into high-level events.
“Policy Scripts” in Bro Policy language processes event stream ...
Tap network link passively, send up a copy of all traffic.
... and takes action!
10
Architecture
• Taps network link passively, sends up a copy of all network traffic.Network
11
Architecture
• Kernel filters down high-volume stream via standard libpcap packet capture library.
Network
libpcap
Packet Stream
Filtered PacketStream
TcpdumpFilter
12
Architecture
• “Event engine” distills filtered stream into high-level, policy-neutral events reflecting underlying network activity
• E.g., connection_attempt, http_reply, user_logged_in
• These span a range of semantic levels
• Currently about 300 different types
Network
libpcap
Event Engine
Packet Stream
Filtered PacketStream
TcpdumpFilter
EventStream
EventControl
13
Architecture
• “Policy script” processes event stream, incorporates:
• Context from past events
• Site’s particular policies
Network
libpcap
Event Engine
Policy Script Interpreter
Packet Stream
Filtered PacketStream
TcpdumpFilter
EventStream
EventControl
Real-time NotificationRecord To Disk
PolicyScript
14
Architecture
• “Policy script” processes event stream, incorporates:
• Context from past events
• Site’s particular policies
Network
libpcap
Event Engine
Policy Script Interpreter
Packet Stream
Filtered PacketStream
TcpdumpFilter
EventStream
EventControl
Real-time NotificationRecord To Disk
PolicyScript
… and takes action: Records to disk
Generates alerts via syslog or paging
Sends events to other Bro’sExecutes programs as a form of
response
Packet Filtering
•capture via pcap API (libpcap)
Filters expressed in same format as tcpdump(8).Also support off-line traces
port finger or port ftp or (tcp[13] & 7 != 0)
• Using libpcap gains significant advantages: it isolates Bro from details of the network link technology (Ethernet, FDDI, SLIP, etc.Network .
it greatly aids in porting Bro to different Unix variants (which also makes it easier to upgrade to faster hardware as it becomes available). Berkeley Packet Filter (BPF), found on some Unix, it allows libpcap to bring Bro’s network filter into the kernel (filters aren’t in userland!).
FreeBSD has a Zero-copy Buffer Extension, which makes it Bro’s OS of choice!
Event Engine
Written in C++.
Distills filtered stream into high-level, policy-neutral events reflecting underlying network activity!
E.g., connection_attempt, http_reply, user_logged_in, etc.
Protocols supported: 30 protocols
ARP, IP, ICMP, TCP, UDP
DCE-RPC, DNS, FTP, Finger, Gnutella, HTTP, IRC, Ident, NCP, NFS, NTP, NetBIOS, POP3, Portmapper, RPC, Rsh, Rlogin, SMB, SMTP, SSH, SSLCiphers, SunRPC, Telenet
Event Engine, cont.
Those protocol analyzers generate over 300 events!
new_connection(c: connection), new_packet(c: connection, p: pkt_hdr), connection_established(c: connection), tcp_contents(c: connection, is_orig: bool, seq: count, contents: string), icmp_echo_reply(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string), ...
Originally, event engine analyzers were written in C++ Class libraries.
Now BinPAC (Binary Protocol Analyzer Compiler) is used.
binpac generates the C++ code.
Event Engine, cont.• When the packets arrive, the Event Engine checks if the
headers of these packets are well formed or not. Packages with not well formed headers are discarded and events of this problem are created.
• The packets with well formed headers gets a connection state which can be connection_attempt,connection_established, connection_rejected or connection_finished for TCP and for UDP the state is defined as udp_request or udp_reply.
• The packets are then collected to their corresponding connection. These collections use different protocols e.g. HTTP, FTP, SMTP and is therefore handled by the analyzers specialized for each protocol.
• Whether or not the connection are handled by any analyzer depends on which analyzers that are enabled in the system.
Event Engine, cont.
• The analyzer for each protocol then examines the content of the connection.
• When this procedure is done, the Event Engine has created high-level events describing the activity and these are now ready for the Policy Script Interpreter.
• As analyzers observe activity, generate events
• Event = name + typed parameter list
• Queued for asynchronous execution
• All events triggered by a given packet executed prior to processing next packet (Along with timers that expire at that point).
Policy Script Interpreter
• The Policy Script Interpreter takes the high-level events generated by the Event Engine and compares these with the policy scripts in the system.
• The events are sorted in a FIFO list which means the first that comes along are the first that is processes.
• Policy Script Interpreter takes action if it detects any suspicious and dangerous actions or it discards other events not defined in the policy scripts.
• Traffic that seems like attacks but aren’t (false negatives), can be detected at this point, but if the policy scripts are good, this will be minimal.
Policy LanguagePolicy analysis scripts which track network activity written in Bro’s language
Procedural You write event handlers = functions asynchronously executed that don’t return a value).
Strongly typed (to detect inconsistencies at compile-time).
Rich in types:
USUAL : bool, int, count, double, string, time, interval, port, addr, hostname, pattern.
• Domain-specific types, such as addresses, ports, subnets
Aggregates: record, table, set, file.
Real-time communication with other Bro instances and Extensive state management support (Timers; automatic expiration; persistence
Policy LanguagePolicy analysis scripts written in Bro’s language:
Procedural (you provide event-handlers that are executed asynchronously and that don’t return a value).
Strongly typed (to detect inconsistencies at compile-time).
Rich in types:
bool, int, count, double, string, time, interval, port, addr, hostname, pattern.
Aggregates: record, table, set, file.
Flex-like regular expressions built into language (pattern)!
Language, cont.
Operators (+, -, *, /, %, !, &&, ||, ?:, relationals, infix, --, ++).
Static, global & const scoping of variables.•const allowed_services: set[addr, port] = {
•[ftp_serv, [ftp, smtp, 20/tcp]], •[nttp.lbl.gov, nttp]•};
• Statements (if-then-else, when, return, print, add, delete, and function).
•No for loops! But recursion through functions is allowed.
Notices & Alarms
Convention is to express much detection in terms of “notices”
• I.e., “I’ve noticed the following activity”
• Notices do not necessarily convey a problem
• Distributed scripts detect 78 different types of notices
• Scripts provide number of ways to decide when to promote a notice to an “alarm” These can include syslog, sending email, paging
• When you see in a script something like NOTICE([$note=OutboundTFTP, $conn=u, $msg=fmt("outbound TFTP: %s -> %s", src, dst)]);it’s just a call to a function named “NOTICE”, using the language’s mechanism for constructing a record
• Note: Entire mechanism is add-on to language, not built in
Sample Bro Policy
• Using the Bro language, sites can write custom policy scripts to generate alarms on any policy violation.
• For example, if a site only allows external http and mail to a small, controlled lists of hosts, they could do this:
const web_servers = { www.lbl.gov, www.bro-ids.org, };
const mail_servers = { smtp.lbl.gov, smtp2.lbl.gov, };
redef allow_services_to: set[addr, port] += {
[mail_servers, smtp],
[web_servers, http], };
• Bro can then generate an Alarm or even terminate the connection for policy violations:
if ( service !in allow_services) NOTICE([$note=SensitiveConnection, $conn=c,]);
if ( inbound && service in terminate_successful_inbound_service )
terminate_connection(c);
Sample Bro Alarms
FTP buffer overflow attack
Bro in Practical Use• Primary IDS for LBNL/NERSC since 1996
• Primary IDS for SC00-03 conferences
• No specialized hardware needed
• Low cost allows for multiple deployment
• Requirements
• FreeBSD
• Intel platform
• Fiber tap
• Disk space to archive data
Sites using Bro (LBNL)
Use multiple Bros across a 10Gb/s link, where each Bro is only executing a minimal set of polices.
Operational 247 since 1996. Monitors traffic for suspicious behavior or policy violations: incoming/outgoing/internal
In conjunction with blocking routers, Bro acts as a dynamic and intelligent firewall
Blocks access from offending IP addresses
Blocks high risk ports
Blocks known high-risk activity
Terminates connections and/or sends alarms
Locates site policy violations (e.g.: Kazaa and gnutella)
Sites using Bro• IDS controls a blocking router
• IDS blocks dynamically when an intrusion attempt is detected or alerts upon suspicious activity
• Router blocks statically like a firewall “Intrusion Prevention”
November 20, 2003 SC2003, Poenix, AZ
Use of Bro Within NERSC
ESNet
NERSC
Filtering BorderRouter
Network Traffic
Tapped Traffic
Multiple Bro Systems
ACL Insertion
• Real Time Analysis• Redundant Backup
• Test Box• Bulk Traffic Recorder
Multiple IDS
• Snort• Bro Heavyweight Protocol Analysis
• Bro GRID / SSL AnalysisTapped Traffic
• Internal Traffic Bro Monitor• Wireless Network Bro Monitor
Wireless Network
Tapped Traffic
Bro at (NERSC)•Tied into a paging system for on-call security person
•Bro check pointed at set intervals (Reports generated whenever checkpointed)
• Clears out ‘orphaned’ sessions
• Allows for offline data analysis
Data archiving
• Maintain traffic data for about 3 months
• Maintain network
• Automatic ACL injection has very low false positive rate
•At NERSC average about 1 every 6 months
NERSC Network Traffic3 Week Period
Type of Traffic Number of
Connections
Overall Percentage
of Traffic
Bulk Data Transfer 666,529 83.73%
Grid Services 74,178 7.19%
Web Related 288,3754 5.30%
Database 620,1730 .27%
Mail 200,484 .04%
System Services 185,272 .04%
Interactive 116 <.1%
Hardware Support Significant Bro research effort these days focuses on
hardware to accelerate network analysis• Note, quite different problem than fast signature matching!
• Approach #1: Bro Cluster• Set of PCs use inter-Bro coordination for joint analysis
• Scales to x23 performance for 24 nodes (then flattens)
• Approach #2: Shunting• Custom (simple) FPGA provides intelligent front-end
• Per-flow decisions on forward/block/divert through IDS/IPS
• 1 Gbps prototype working, undergoing upgrading
• Future: Multicore Bro• Will require extensive analysis to understand/tune memory
locality & intra-analysis messaging
Bro Future Directions
Grid related technologies
Ability to detect Grid related protocols
X.509 Certificate Analyzer
• SSL Analyzer
• Verify certificates are legitimate
Router Shunting
Primary bottleneck in moving packets into user space
• Leverage router based hardware filtering to analyze “packets of interest”
• Proof of concept demo at SC01-03
1)Utilizing Bro and Juniper router
2) Hardware based BPF to filter traffic
Port Mirroring
External Network
JuniperGigE
Interface Bro
InternalNetwork
Mirrored Traffic
Filter-based Forwarding
Bro
Juniper
GigE Interface
Filtered Traffic
External Network
InternalNetwork
Filter
Pros
Wow, where do we begin?
Separation of event generation and event handling (mechanism vs. policy).
Code is/was written for a hostile environment.
Bro’s language (to write policy scripts) is quite tractable.
Resilience in the presence of flooding(Via connection compressor)
An active community (mailing list: [email protected]).
Reaffirms that open-source solution are possible!
Support for plugging in external analyses
Pros : Bro Noteworthy Features
Broccoli = Bro Client Communications Library
• C interface for external programs to transmit & receive values and events
• E.g., syslog reader sends log messages to Bro
Dynamic adaptation to load
• Via policy/load-levels.bro
Changes current packet filter to activate more/less analysis
User-level state management
• Entries in tables, sets automatically expire T sec. after Creation / last read / last write
• Advanced detection algorithms (Backdoors: protocols running off-port (obviated by DPD)
• )
ConsEncrypted Traffic is Bro’s bane:
Act like a SSL proxy (keep *all* necessary certificates?)
IPv6 ... I think it (finally) made it into development code and asymmetry .
Pretty complex, script-based system
• Requires understanding of the network
• No GUI, just ASCII logs
Only partially documented
• Lacking resources to fully polish the system
Thank You
The End