fcip pt. 1
DESCRIPTION
FCIP FundamentalsTRANSCRIPT
FCIP The Beginning February 10, 2015 by Jason Lunde. FCIP is notably a part of the CCIE Data Center lab exam blueprint. It is also a sticking point for a lot of candidates who have not done a whole lot on the storage networking side. Luckily FCIP has many correlations to the modern-day Ethernet networking that we all know and love, as its really just another tunneling technology! After some thought, I have decided to break this down into 2 blog posts. This one will cover FCIP basics, and another that will cover some more advanced FCIP options that you might have to use during the CCIE lab examination.FCIP is used for extending a Fibre Channel (FC) network over an IP backbone. It encapsulates FC in IP so that SCSI and non-SCSI FC frames can be sent over an IP network. Normally most organizations are not going to do this simply for the sake of extending their FC network (why extend a lossless network over a lossy medium?), but rather for backup or replication jobs that need to occur between storage systems that are across some geographical distance. A typical deployment scenario is shown below:
Here we have two SANs separated by an IP network. Now, the MDSs currently in scope for the CCIE DC lab exam are MDS 9222is, which have 4 Gigabit interfaces native to the system. They also have the needed SAN_EXTN_OVER_IP license shipped natively with the system. The MDS 9222i can support up to 3 FCIP tunnels per gigabit interface, giving us a maximum of 12 FCIP tunnels available by default. So we can easily take one of these Gigabit interfaces, and create an FCIP tunnel across the IP network using its addressing as our tunnel source.The configuration of an FCIP tunnel is actually really simple. There are a couple of housekeeping items however, that we must take care of prior to beginning the configuration. 1 We must put an IP address on our Gig interfaces, and 2 We must make sure we have IP reachability to the other side (where we wish to terminate the tunnel). This may seem like a no-brainer, but its often the simple things that trip us up in our exams! It must be noted that the MDSs do not support dynamic routing either. So if we have to extend past a L3 boundary, we must put in static routes. Lets assign some IP addressing to our sample topology:
MDS1MDS1(config)# int gig 1/1MDS1(config-if)# ip address 192.168.10.1 255.255.255.0MDS1(config-if)# no shutMDS1(config-if)# show int gig 1/1GigabitEthernet1/1 is upHardware is GigabitEthernet, address is 000d.bd85.4a88Internet address is 192.168.10.1/24MTU 1500 bytesPort mode is IPSSpeed is 1 GbpsBeacon is turned offAuto-Negotiation is turned on5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec0 packets input, 0 bytes0 multicast frames, 0 compressed0 input errors, 0 frame, 0 overrun 0 fifo1 packets output, 42 bytes, 0 underruns0 output errors, 0 collisions, 0 fifo0 carrier errorsMDS2MDS2(config-if)# int gig 1/1MDS2(config-if)# ip address 192.168.10.2 255.255.255.0MDS2(config-if)# no shutMDS2(config-if)# show int gig 1/1GigabitEthernet1/1 is upHardware is GigabitEthernet, address is 0017.5ab5.2f58Internet address is 192.168.10.2/24MTU 1500 bytesPort mode is IPSSpeed is 1 GbpsBeacon is turned offAuto-Negotiation is turned on5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec0 packets input, 0 bytes0 multicast frames, 0 compressed0 input errors, 0 frame, 0 overrun 0 fifo1 packets output, 42 bytes, 0 underruns0 output errors, 0 collisions, 0 fifo0 carrier errorsNow that we have placed our addressing, and verified that the interfaces are up, we want to establish that we have basic IP reachability. The best method to verify this is via a PING!MDS1(config-if)# ping 192.168.10.2PING 192.168.10.2 (192.168.10.2) 56(84) bytes of data.64 bytes from 192.168.10.2: icmp_seq=2 ttl=255 time=0.397 ms64 bytes from 192.168.10.2: icmp_seq=3 ttl=255 time=0.390 ms64 bytes from 192.168.10.2: icmp_seq=4 ttl=255 time=0.380 ms64 bytes from 192.168.10.2: icmp_seq=5 ttl=255 time=0.385 ms--- 192.168.10.2 ping statistics ---5 packets transmitted, 4 received, 20% packet loss, time 3999msrtt min/avg/max/mdev = 0.380/0.388/0.397/0.006 msWe are good to go! Again, had we needed to pass across a L3 boundary, we would have needed a static route. The basic FCIP configuration can be broken down into 2 very easy components:1 The FCIP Profile configurationThis consists of declaring our tunnel source.2 The FCIP interface configurationWe will reference our FCIP profile (for our tunnel source), and we will declare a tunnel destination.It really is that simple! An FCIP tunnel is ALWAYS an E port, or rather a virtual E (vE) port. So it will be extending the fabric in exactly the same manner as a traditional E-port, even trunking VSANs if you wish it to do so. So zoning changes, principle switch elections, etcwill all be extended across this IP boundary between the storage area networks.So, in our current demo topology I have setup the JBODs VSAN 10. Each MDS has a locally attached JBOD, and some local FLOGI entries. There is no other connection between the MDSs, so the only entries in the FCNS database, are the local entries also present in the FLOGI DB:MDS1(config-if)# show flogi database-----------------------------------------------------------------------INTERFACE VSAN FCID PORT NAME NODE NAME-----------------------------------------------------------------------fc1/5 10 0x0b00e1 21:00:00:11:c6:a6:3c:72 20:00:00:11:c6:a6:3c:72fc1/5 10 0x0b00ef 21:00:00:14:c3:a0:68:ed 20:00:00:14:c3:a0:68:edTotal number of flogi = 2.MDS1(config-if)# show fcns databaseVSAN 10:--------------------------------------------------------------------FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE--------------------------------------------------------------------0x0b00e1 NL 21:00:00:11:c6:a6:3c:72 scsi-fcp:target0x0b00ef NL 21:00:00:14:c3:a0:68:ed scsi-fcp:targetTotal number of entries = 2MDS2(config-if)# show flogi database-----------------------------------------------------------------------INTERFACE VSAN FCID PORT NAME NODE NAME-----------------------------------------------------------------------fc1/5 10 0x1600da 22:00:00:11:c6:a6:25:78 20:00:00:11:c6:a6:25:78fc1/5 10 0x1600e2 22:00:00:14:c3:a0:68:ee 20:00:00:14:c3:a0:68:eeTotal number of flogi = 2.MDS2(config-if)# show fcns databaseVSAN 10:--------------------------------------------------------------------FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE--------------------------------------------------------------------0x1600da NL 22:00:00:11:c6:a6:25:78 scsi-fcp:target0x1600e2 NL 22:00:00:14:c3:a0:68:ee scsi-fcp:targetTotal number of entries = 2So when we get the FCIP tunnel up and running, one of our first verifications steps will be to check the FCNS database, ensure that the fabric has merged, and that we see 4 entries (2 per side).Our first configuration step is to enable the feature, and create our FCIP profiles. Again, this is fairly straight-forward. We simply want to create the profile, with any number (its locally significant), and bind it to our LOCAL IP address. Remember, this is our tunnel source.MDS1MDS1(config-if)# feature fcipMDS1(config)# fcip profile 1MDS1(config-profile)# ip address 192.168.10.1MDS2MDS2(config-if)# feature fcipMDS2(config)# fcip profile 1MDS2(config-profile)# ip address 192.168.10.2Now we want to create our FCIP interfaces. These are logical interfaces, and again can be any number as they are only locally significant. We will reference our FCIP profile, which we want to use as our tunnel source, and we will declare a tunnel destination.MDS1MDS1(config-profile)# int fcip1MDS1(config-if)# use-profile 1MDS1(config-if)# peer-info ipad 192.168.10.2MDS1(config-if)# no shutMDS2MDS2(config-profile)# int fcip1MDS2(config-if)# use-profile 1MDS2(config-if)# peer-info ipad 192.168.10.1MDS2(config-if)# no shutBelieve it or not, thats all we really need in order to get FCIP up and going! Lets verify that our FCIP interfaces came up.MDS2(config-if)# show int fcip1fcip1 is trunkingHardware is GigabitEthernetPort WWN is 20:10:00:0d:ec:34:67:40Peer port WWN is 20:10:00:0d:ec:10:52:40Admin port mode is auto, trunk mode is onsnmp link state traps are enabledPort mode is TEPort vsan is 1Speed is 1 GbpsTrunk vsans (admin allowed and active) (1,10,301-303)Trunk vsans (up) (1,10,301-303)Trunk vsans (isolated) ()Trunk vsans (initializing) ()Interface last changed at Fri Dec 26 15:53:16 2014This is awesome! Our interface is up and trunking, and our VSAN is even in the UP state! Lets see if the fabrics have merged, and if we see the FCNS database populated properly.MDS1(config-if)# show fcns databaseVSAN 10:--------------------------------------------------------------------FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE--------------------------------------------------------------------0x0b00e1 NL 21:00:00:11:c6:a6:3c:72 scsi-fcp:target0x0b00ef NL 21:00:00:14:c3:a0:68:ed scsi-fcp:target0x1600da NL 22:00:00:11:c6:a6:25:78 scsi-fcp:target0x1600e2 NL 22:00:00:14:c3:a0:68:ee scsi-fcp:targetTotal number of entries = 4MDS2(config-if)# show fcns databaseVSAN 10:--------------------------------------------------------------------FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE--------------------------------------------------------------------0x0b00e1 NL 21:00:00:11:c6:a6:3c:72 scsi-fcp:target0x0b00ef NL 21:00:00:14:c3:a0:68:ed scsi-fcp:target0x1600da NL 22:00:00:11:c6:a6:25:78 scsi-fcp:target0x1600e2 NL 22:00:00:14:c3:a0:68:ee scsi-fcp:targetTotal number of entries = 4That, my friends, is winning! We see now that we have all 4 entries in our FCNS database, meaning that our fabrics have, in fact, merged properly. There are a couple more things that we can verify here however. Lets take a look at the full output of the command show interface fcip1.MDS2(config-if)# show int fcip1fcip1 is trunkingHardware is GigabitEthernetPort WWN is 20:10:00:0d:ec:34:67:40Peer port WWN is 20:10:00:0d:ec:10:52:40Admin port mode is auto, trunk mode is onsnmp link state traps are enabledPort mode is TEPort vsan is 1Speed is 1 GbpsTrunk vsans (admin allowed and active) (1,10,301-303)Trunk vsans (up) (1,10,301-303)Trunk vsans (isolated) ()Trunk vsans (initializing) ()Interface last changed at Fri Dec 26 15:53:16 2014Using Profile id 1 (interface GigabitEthernet1/1)Peer InformationPeer Internet address is 192.168.10.1 and port is 3225Write acceleration mode is configured offTape acceleration mode is configured offTape Accelerator flow control buffer size is automaticFICON XRC Accelerator is configured offFicon Tape acceleration configured off for all vsansIP Compression is disabledMaximum number of TCP connections is 2QOS control code point is 0QOS data code point is 0TCP Connection Information2 Active TCP connectionsControl connection: Local 192.168.10.2:3225, Remote 192.168.10.1:65508Data connection: Local 192.168.10.2:3225, Remote 192.168.10.1:6551018 Attempts for active connections, 6 close of connectionsTCP ParametersPath MTU 1500 bytesCurrent retransmission timeout is 200 msRound trip time: Smoothed 2 ms, Variance: 3 Jitter: 150 usAdvertized window: Current: 33 KB, Maximum: 25 KB, Scale: 5Peer receive window: Current: 29 KB, Maximum: 29 KB, Scale: 5Congestion window: Current: 14 KB, Slow start threshold: 112 KBCurrent Send Buffer Size: 25 KB, Requested Send Buffer Size: 0 KBCWM Burst Size: 50 KBMeasured RTT : 0 us Min RTT: 0 us Max RTT: 0 us5 minutes input rate 2680 bits/sec, 335 bytes/sec, 2 frames/sec5 minutes output rate 2632 bits/sec, 329 bytes/sec, 2 frames/sec808 frames input, 100688 bytes792 Class F frames input, 98872 bytes16 Class 2/3 frames input, 1816 bytes0 Reass frames0 Error frames timestamp error 0814 frames output, 99900 bytes798 Class F frames output, 98084 bytes16 Class 2/3 frames output, 1816 bytes0 Error framesThis is one of the best troubleshooting and verification commands available for FCIP! We can immediately note a couple of things here. The default TCP port for FCIP is 3225, and we have 2 TCP streams by default. One of these streams is for control traffic, and the other for data traffic. We will see in the next post, that we can assign certain DSCP values to these streams, so that they can have QoS policies give them certain treatment throughout the network. We can also tell here, that MDS1 initiated the connection to MDS2, as MDS2s local port is 3225, while MDS1s port is a high-numbered ephemeral port. This is another item that we will learn to control, and manipulate, in the next blog post!If you have any questions or comments please feel free to leave them here on the blog, or shoot me an email directly at [email protected]. I look forward to hearing from you, and please check back soon for the second post in this series where we will tweak some nerd knobs with regards to FCIP!