high availability. module objectives by the end of this module participants will be able to:...
TRANSCRIPT
High Availability
Module Objectives
• By the end of this module participants will be able to:• Identify the components in a FortiGate high availability
cluster
• Describe the FortiGate HA modes of operation
• Describe the use of the FortiGate Clustering Protocol
• Define the failover methods used in FortiGate HA
• Configure session synchronization
• Configure a FortiGate HA cluster
High Availability
Two or more FortiGate units
operate as a cluster
If one cluster unit fails, another in the cluster replaces it
High Availability
If one cluster unit fails, another in the cluster replaces it
• FortiGate HA is implemented by configuring two or more FortiGate units to operate as an HA cluster• The cluster appears to function as a single
FortiGate unit
• Provides enhanced reliability and increased performance
• Cluster units share state and configuration information• If one unit fails, the other unit in the
cluster replaces it
Cluster Membership
• Identical hardware model• Identical firmware versions• Same hard disk configuration• Same operating mode
HA cluster
Cluster Units
Primary unit
Subordinate units
Cluster
Cluster Units
• Every cluster contains one primary (master) unit and one or more subordinate (slave) units• The primary unit controls how the cluster operates• Synchronizes session information with
subordinates
• Synchronizes cluster configuration with subordinates
• Synchronizes cluster routing table
• Tracks status of subordinates
• Subordinates are always waiting to become primary
Viewing Cluster Members
Change hostname of the FortiGate unit to simplify
administration
High Availability Modes of Operation
Primary unit
Active-Passive
Primary unit processes traffic
Subordinates run in standby mode
Configuration of primary is synchronized with
subordinates
If primary fails, a subordinate immediately
takes its place
Subordinate units
Click here to read more about HA modes of operation
High Availability Modes of Operation
Active-Passive
• An active-passive cluster provides hot standby failover protection• In an active-passive cluster, the primary unit processes all traffic, while the subordinate units run in standby mode• The configuration of primary is
synchronized with subordinates
• If primary unit fails, a subordinate will resume processing traffic• Synchronized state information provides
transparent failoverClick here to read more about HA modes of operation
High Availability Modes of Operation
Active-Active
Primary unit processes traffic
Subordinate units also process traffic
If primary fails, a subordinate immediately
takes its place
Primary unit load balances sessions with subordinates
High Availability Modes of Operation
Active-Active
• An active-active cluster balances communication session and provides failover protection• In an active-active cluster, both the primary and subordinate units process traffic• The primary unit load balances sessions
• If the primary unit fails, a subordinate will resume operations as the primary
FortiGate Clustering Protocol
• FortiGate Clustering Protocol (FGCP) is used to discover other FortiGate units configured for high availability and to negotiate the creation of a cluster• FGCP shares communication and synchronization information among cluster members• Referred to as HA heartbeat
• The cluster uses FGCP to select the primary unit and provide device and link failover
Click here to read more about FortiGate Clustering Protocol
Virtual MAC Addresses
Virtual MAC addresses assigned to each interface
Original device
Failover device
Same virtual MAC addresses assigned to each interface
Virtual MAC Addresses
Virtual MAC addresses assigned to each interface
Original device
Failover device
Same virtual MAC addresses assigned to each interface
• FGCP assigns virtual MAC addresses to each primary interface• If a failover occurs, the new unit interfaces
will have the same MAC addresses as the failed unit
• Allows the network to recover more quickly since attached network devices do not have to learn new MAC addresses before they can communicate with the cluster
FGCP Heartbeat
Cluster members
FGCP Heartbeat
Cluster members
• The FGCP heartbeat keeps cluster units communicating with each other• Hello packets are sent at regular intervals
by the heartbeat interface of all cluster units
• Describes the state of the units and keeps all unit synchronized
•Operates on TCP port 703•Default time interval between heartbeats is 200ms
Heartbeat Interfaces
• For redundancy purposes, two interfaces should be assigned as heartbeat interfaces• Default heartbeat interfaces depend on model
• The heartbeat interface with the highest priority is used for all HA heartbeat communications• If two interface have same priority, interface highest in
list used
• If communications are interrupted and the FortiGate device cannot failover to second heartbeat interface, the cluster stops processing traffic
Heartbeat Interface IP Addresses
• Cluster assigns virtual IP to interfaces processing traffic• Primary: 169.254.0.1
• Subordinates: 169.254.0.2, 169.254.0.3 and 169.254.0
• If both units boot up and join cluster at the same time, FGCP will assign 169.254.0.1 to the FortiGate unit with the highest serial number
Heartbeat Interface IP Addresses
• Sample output for master with IP address of 169.254.0.2:diag sys ha status
HA information
Statistics
traffic.local = s:20871 p:78602 b:32853886
traffic.total = s:20980 p:78602 b:32853886
activity.fdb = c:0 q:0
Model=50, Mode=2 Group=12 Debug=0
nvcluster=1, ses_pickup=0
HA group member information: is_manage_master=1.
FG50BH3G09600554, 1. Master:128 FG50BH3G09600554
FG50BH3G09600577, 0. Slave:128 FG50BH3G09600577
vcluster 1, state=work, master_ip=169.254.0.2, master_id=0:
FG50BH3G09600554, 0. Master:128 FG50BH3G09600554(prio=0, rev=0)
FG50BH3G09600577, 1. Slave:128 FG50BH3G09600577(prio=1, rev=1)
HA Configuration Synchronization
Primary unit
Subordinate unit
Config
Config
The change is synchronized on the
subordinate unitA configuration change is made on
primary unit
Incremental synchronization
HA Configuration Synchronization
Another configuration change is made on
primary unit
Primary unit
Subordinate unit
This change is synchronized on the
subordinate unit
Incremental synchronization
HA Configuration Synchronization
Another configuration change is made on
primary unit
Primary unit
Subordinate unit
Config
Config
The change is synchronized on the
subordinate unit
This change is synchronized on the
subordinate unit
Incremental synchronization
• FGCP uses synchronization to ensure that the configurations of all cluster units are identical•With incremental synchronization, changes made to the primary unit are immediately made to the subordinate• Includes dynamic information such as
DHCP leases, routing table updates etc
• Synchronization is silent• No log message unless level is set to
Information
HA Configuration Synchronization
Checksum values are compared on cluster
members
Primary unit
Subordinate unit
Config
Config
The change is synchronized on the
subordinate unit
A configuration change is made on
primary unit
Periodic synchronization
Checksum
Checksum
Checksum
Checksum
HA Configuration Synchronization
Primary unit
Subordinate unit
This change is synchronized on the
subordinate unit
Another configuration change is made on
primary unit
Checksum
Checksum
Checksum values are compared on cluster
members
Periodic synchronization
Checksum
Checksum
HA Configuration Synchronization
Primary unit
Subordinate unit
Config
Config
The change is synchronized on the
subordinate unit
Periodic synchronization
Checksum
Checksum
Checksum
Checksum
Another configuration change is made on
primary unit
Checksum
Checksum
• Period synchronization is a mechanism that looks for and fixes synchronization problems• The checksum value of the configuration file on each cluster member is compared• If checksum values match, cluster units are
consider synchronized
• If there is not a match, the subordinate will retrieve the configuration from the primary
Load Balancing
1. dstMAC 09-01-01, srcMAC X, TCP ACK dport 802. dstMAC 0b-a4-8c, srcMAC 0b-a1-c0, TCP ACK dport 803. dstMAC 09-01-03, srcMAC Y, TCP SYN ACK sport 804. dstMAC 0b-a4-8e, srcMAC 0b-a1-c2, TCP SYN ACK sport 80 5. dstMAC Y, srcMAC 0b-a4-8e, TCP ACK dport 80
Click here to read more about FortiGate HA load balancing
Load Balancing
1. dstMAC 09-01-01, srcMAC X, TCP ACK dport 802. dstMAC 0b-a4-8c, srcMAC 0b-a1-c0, TCP ACK dport 803. dstMAC 09-01-03, srcMAC Y, TCP SYN ACK sport 804. dstMAC 0b-a4-8e, srcMAC 0b-a1-c2, TCP SYN ACK sport 80 5. dstMAC Y, srcMAC 0b-a4-8e, TCP ACK dport 80
Click here to read more about FortiGate HA load balancing
session info: proto=6 proto_state=11 expire=3599 timeout=3600 flags=00000000 av_idx=4 use=5bandwidth=0/sec guaranteed_bandwidth=0/sec traffic=0/sec prio=0 logtype=session ha_id=0
hakey=49729tunnel=/state=redir log local may_dirty statistic(bytes/packets/err): org=1253/21/0 reply=1503/19/0 tuples=3orgin->sink: org pre->post, reply pre->post oif=3/5 gwy=192.168.11.254/10.0.1.1hook=post dir=org act=snat 10.0.1.1:2287->193.1.193.64:21(192.168.11.101:2287)hook=pre dir=reply act=dnat 193.1.193.64:21->192.168.11.101:2287(10.0.1.1:2287)hook=post dir=reply act=noop 193.1.193.64:21->10.0.1.1:2287(0.0.0.0:0)pos/(before,after) -233083355/(0,8), 0/(0,0)misc=20004 domain_info=0 auth_info=0 ftgd_info=0 ids=0x0 vd=0 serial=00005ae5 tos=ff/ff
session info: proto=6 proto_state=11 expire=3595 timeout=3600 flags=00000000 av_idx=4 use=6bandwidth=0/sec guaranteed_bandwidth=0/sec traffic=0/sec prio=0 logtype=session ha_id=1
hakey=49729tunnel=/state=redir log may_dirty statistic(bytes/packets/err): org=999/21/0 reply=1921/19/0 tuples=3orgin->sink: org pre->post, reply pre->post oif=3/5 gwy=192.168.11.254/10.0.1.1hook=post dir=org act=snat 10.0.1.1:2291->193.1.193.64:21(192.168.11.101:2291)hook=pre dir=reply act=dnat 193.1.193.64:21->192.168.11.101:2291(10.0.1.1:2291)hook=post dir=reply act=noop 193.1.193.64:21->10.0.1.1:2291(0.0.0.0:0)pos/(before,after) 1555340173/(8,16), 0/(0,0)misc=20004 domain_info=0 auth_info=0 ftgd_info=0 ids=0x0 vd=0
serial=00005b07 tos=ff/ff
Cluster ID of device
handing session
AV scan enabled for FTP
Load Balancing Master Session Table
Failover
• FGCP provides transparent device and link failover• Can be caused by hardware failure, software failure, or
even a network cable disconnected
•When failover occurs, cluster detects and takes steps so network can operate without interruption• Internal operation of cluster changes• Components outside of cluster notice little or no change
• Cluster records log messages• Also send SNMP trap or alert email
Click here to read more about FortiGate HA failover
Device Failover
• If the FortiGate device fails, another device automatically takes its place•Does not maintain communication sessions• Session must be restarted
• HA can be configured to support session failover
• Subordinate units sends heartbeat packets to detect primary failure• If a failure is detected, a subordinate unit will assume the primary role
•New primary unit has same network identity as failed primary unit• Configuration synchronization insures that new primary unit has same
configuration as the failed primary unit
Link Failover
• If a monitored interface fails, the cluster reorganizes to reestablish link to the network• Continue operation with minimal or no disruption
• The cluster monitors each unit to determine if the monitored interfaces are operating and connected• Each cluster unit stores link state information for all monitored units in link state database
Session Failover
• Cluster maintains active networks sessions after device or link failover• Must enable session pick-up
• Only sessions not being handled by a proxy can failover
• FGCP maintains a session table for most communication sessions being process by cluster• Information available to cluster members to resume
sessions being processed by failed unit
Session Synchronization
PrimaryFortiGate
unit
SecondaryFortiGate
unit
Syncmanagement
Relies on external networking device for
traffic redirection
Click here to read more about FortiGate HA session synchronization
Session Synchronization
PrimaryFortiGate
unit
SecondaryFortiGate
unit
Syncmanagement
Relies on external networking device for
traffic redirection
• This mechanism provides an alternative to an active-passive HA configuration for session synchronization• Two units operating in standalone mode
• Configurations synched between the two units
• An external networking device (router or load-balancer) is responsible for traffic redirection
Click here to read more about FortiGate HA session synchronization
Configuring Session Synchronization
•On primary FortiGate unit:config globalconfig system interfaceedit "port2" set vdom "root" set ip 192.168.8.3 255.255.255.0 set allowaccess ping set type physical next
.../...
config system session-sync edit 1 set peerip 192.168.8.4 set peervd "root" set syncvd "VDT1" nextend
Configuring Session Synchronization
•On secondary FortiGate unit:config globalconfig system interfaceedit "port2" set vdom "root" set ip 192.168.8.4 255.255.255.0 set allowaccess ping set type physical next
.../...
config system session-sync edit 1 set peerip 192.168.8.3 set peervd "root" set syncvd "VDT1" nextend
Virtual Clustering
Domain A Domain B Domain C Domain A Domain D Domain E
Active-passive HA
Primary Subordinate
Click here to read more about FortiGate HA virtual clustering
Virtual Clustering
Domain A Domain B Domain C Domain A Domain D Domain E
Active-passive HA
Primary Subordinate
• Virtual clustering provides failover between two instances of virtual domains operating on two different cluster units• Operates in active-passive mode
• Can also be configured to provide load balancing• The primary unit processes all traffic for the virtual domain
Click here to read more about FortiGate HA virtual clustering
Upgrades
•Upgrading or downgrading cluster firmware is similar to upgrading or downgrading a standalone FortiGate firmware. • The firmware is uploaded once to the primary unit and
the cluster automatically upgrades or downgrades all cluster units in one operation with minimal or no service interruption
• The firmware upgrade takes place without interrupting communication through the cluster
Upgrades
• To upgrade the firmware without interrupting communication through the cluster: • The administrator uploads a new firmware image from Web
Config or CLI
• If the cluster is operating in active-active mode, load balancing is turned off
• The cluster upgrades the firmware running on all of the subordinate units
• Once the subordinate units have been upgraded, a new primary unit is selected. This primary unit will be running the new upgraded firmware.
• The cluster now upgrades the firmware of the former primary unit.
Full Mesh HA
Full Mesh HA
• Full mesh HA is a method of reducing the number of single points of failure on a network that includes an HA cluster• Available on certain FortiGate models
•Uses aggregate and redundant interfaces to include redundant connections between all network components
High Availability Lab Topology
Labs
Lab – High Availability• Configuring the Student FortiGate device as the Master• Configuring the Remote FortiGate device as the Slave• Verifying HA synchronization and failover
Click here for step-by-step instructions on completing this lab
Student Resources
Click here to view the list of resources used in this module