cable test at server end

4
Identifying a failed Ethernet cable on an Oracle Database Machine (Exadata) (Doc ID 1994662.1) APPLIES TO: Oracle Exadata Hardware Information in this document applies to any platform. PURPOSE The symptoms of a cable failure are many. Some of the more obvious is the link light on the switch or the NIC will not light. Another might be a degraded network bandwidth. TimeDomain Reflectometer (TDR) needs to be executed from the switch and can't be done from the host. TDR requires specific circuitry that is not typically available on the network interface card. To use TDR, you will need “enable” privileges on the switch to issue the commands needed. Ethtool is another tool to used to validate TDR's output. Ethtool shows the settings on the NIC so one can compare them to the settings on the switch. For more information on Cisco's implementation of TDR see http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst6500/ios/12 2SX/configuration/guide/book/intrface.html#wpmkr1066782 TROUBLESHOOTING STEPS Before covering the commands used, let's review the construction of Twisted Pair cabling to properly understand the switch diagnostic output. Category 5 or Category 6 Twisted Pair Ethernet cable consists of four pairs of twisted copper wire that connect in a specific way into an RJ45 plug. The TDR diagnostic runs tests on each twisted pair to determine how long the overall cable is, or how far from the switch a potential break is located. The diagnostics also lists the network speed of the NIC. During the test, connectivity between the switch and host is terminated while the test is being run, so it is best to run the test during a system outage period. NOTE: The information in this note was acquired using a Cisco Catalyst 4948 switch running Version 15.0(2)SG8 of the CIOS. It should be valid for other Cisco equipment that has TDR available. If a cable failure is suspected, the Linux OS utility ethtool should not detect a link: [root@dm01adm01 ~]# ethtool eth0 | grep Link Link detected: yes Another symptom might be the link speed is not what the NIC is rated for. For instance, if the NIC is supposed to be 1Gb/s Ethernet, it might only come up as 100Mb/s Ethernet: [root@dm01adm01 ~]# ethtool eth0 | grep Speed Speed: 100Mb/s NOTE: Keep in mind that the iLOM's' in Exadata are only rated for 100Mb/s. With this information, one can validate if it is due to a cable failure by going to the Cisco switch and issue the TDR test. This will require to get to the "enable" prompt in the switch. dm01swadm0#test cablediagnostics tdr interface GigabitEthernet1/3

Upload: vidhu-ranjan-gopal

Post on 16-Sep-2015

216 views

Category:

Documents


2 download

DESCRIPTION

Testing the cable/LAN cable and their status from RHEL 7

TRANSCRIPT

  • 24/04/2015 DocumentDisplay

    https://support.oracle.com/epmos/faces/SearchDocDisplay?_adf.ctrlstate=16sjohapiu_9 1/4

    IdentifyingafailedEthernetcableonanOracleDatabaseMachine(Exadata)(DocID1994662.1)

    APPLIESTO:

    OracleExadataHardwareInformationinthisdocumentappliestoanyplatform.

    PURPOSE

    Thesymptomsofacablefailurearemany.SomeofthemoreobviousisthelinklightontheswitchortheNICwillnotlight.Anothermightbeadegradednetworkbandwidth.TimeDomainReflectometer(TDR)needstobeexecutedfromtheswitchandcan'tbedonefromthehost.TDRrequiresspecificcircuitrythatisnottypicallyavailableonthenetworkinterfacecard.

    TouseTDR,youwillneedenableprivilegesontheswitchtoissuethecommandsneeded.EthtoolisanothertooltousedtovalidateTDR'soutput.EthtoolshowsthesettingsontheNICsoonecancomparethemtothesettingsontheswitch.

    FormoreinformationonCisco'simplementationofTDRseehttp://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst6500/ios/122SX/configuration/guide/book/intrface.html#wpmkr1066782

    TROUBLESHOOTINGSTEPS

    Beforecoveringthecommandsused,let'sreviewtheconstructionofTwistedPaircablingtoproperlyunderstandtheswitchdiagnosticoutput.Category5orCategory6TwistedPairEthernetcableconsistsoffourpairsoftwistedcopperwirethatconnectinaspecificwayintoanRJ45plug.TheTDRdiagnosticrunstestsoneachtwistedpairtodeterminehowlongtheoverallcableis,orhowfarfromtheswitchapotentialbreakislocated.ThediagnosticsalsoliststhenetworkspeedoftheNIC.Duringthetest,connectivitybetweentheswitchandhostisterminatedwhilethetestisbeingrun,soitisbesttorunthetestduringasystemoutageperiod.

    NOTE:TheinformationinthisnotewasacquiredusingaCiscoCatalyst4948switchrunningVersion15.0(2)SG8oftheCIOS.ItshouldbevalidforotherCiscoequipmentthathasTDRavailable.

    Ifacablefailureissuspected,theLinuxOSutilityethtoolshouldnotdetectalink:

    [root@dm01adm01~]#ethtooleth0|grepLinkLinkdetected:yes

    AnothersymptommightbethelinkspeedisnotwhattheNICisratedfor.Forinstance,iftheNICissupposedtobe1Gb/sEthernet,itmightonlycomeupas100Mb/sEthernet:

    [root@dm01adm01~]#ethtooleth0|grepSpeedSpeed:100Mb/s

    NOTE:KeepinmindthattheiLOM's'inExadataareonlyratedfor100Mb/s.

    Withthisinformation,onecanvalidateifitisduetoacablefailurebygoingtotheCiscoswitchandissuetheTDRtest.Thiswillrequiretogettothe"enable"promptintheswitch.

    dm01swadm0#testcablediagnosticstdrinterfaceGigabitEthernet1/3

  • 24/04/2015 DocumentDisplay

    https://support.oracle.com/epmos/faces/SearchDocDisplay?_adf.ctrlstate=16sjohapiu_9 2/4

    Thecommanddoesnotreturnanythingasyoujuststartedthetest.Itwillbeatthispointtheportbecomesnonresponsivetothehost.Tofindtheoutputorstatusofthetest,runtheshowcommand:

    dm01swadm0#showcablediagnosticstdrinterfaceGigabitEthernet1/3TDRtestisinprogressoninterfaceGi1/3

    dm01swadm0#showinterfaceGigabitEthernet1/3|includelineprotocolGigabitEthernet1/3isdown,lineprotocolisdown(notconnect)

    Oncethetestiscompleted,toviewtheresultsusetheshowcommand:

    dm01swadm0#showcablediagnosticstdrinterfaceGigabitEthernet1/3InterfaceSpeedLocalpairCablelengthRemotechannelStatusGi1/31Gbps12N/AUnknownTerminated36N/AUnknownTerminated45N/AUnknownTerminated78N/AUnknownTerminated

    Inthiscase,thecablewasfineasitsStatusshowstheallthetwistedpairswereproperlyterminated.Terminatedmeanstheimpedanceonthewireisasexpected.ItshowsthattheratedSpeedoftheinterfaceis1Gpbs,whichiswhatwouldbeexpected.Letstryonethathasafailure:

    dm01swadm0#testcablediagnosticstdrinterfaceGigabitEthernet1/1dm01swadm0#showcablediagnosticstdrinterfaceGigabitEthernet1/1InterfaceSpeedLocalpairCablelengthRemotechannelStatusGi1/1100Mbps12N/AUnknownTerminated36N/AUnknownTerminated45N/AUnknownTerminated78N/AUnknownTerminated

    InanormalExadatainstallation,theDescriptionfieldintheswitchisfilledoutwiththehostnameofthenodepluggedintotheport.Toseethisname:

    dm01swadm0#showinterfaceGigabitEthernet1/1|includeDescriptionDescription:dm01cel14NET0

    Tovalidatetheconnection,gathertheMACaddressfromtheExadatasystem:

    [root@dm01celadm14~]#iplinkshoweth0|greplinklink/ether00:14:4f:ca:ef:cebrdff:ff:ff:ff:ff:ff

    ToseewhattheswitchseesastheMACaddressoftheNICpluggedintotheport:

    dm01swadm0#showmacaddresstableinterfaceGigabitEthernet1/1|includeGigabitEthernet30580014.4fcb.1bc0dynamicipGigabitEthernet1/1

    Wellthisisfun,thetwodon't'match.Thisisatestsysteminalab,soitmaynothavebeenleftinanormalproductionstate,whichisgoodforthisnote,badifthisisaproductionsystemandits3AMinthemorning.TofindtheproperMACaddress,sshintoeveryhostandissue"iplinkshoweth0|greplink".

    root@dm01adm08link/ether00:14:4f:cb:1b:c0brdff:ff:ff:ff:ff:ff

    OncetheproperMACaddressisfound,onthathostcheckthestatusoftheNICusingethtool:

  • 24/04/2015 DocumentDisplay

    https://support.oracle.com/epmos/faces/SearchDocDisplay?_adf.ctrlstate=16sjohapiu_9 3/4

    [root@dm01adm08~]#ethtooleth0Settingsforeth0:Supportedports:[TP]Supportedlinkmodes:10baseT/Half10baseT/Full100baseT/Half100baseT/Full1000baseT/FullSupportedpauseframeuse:SymmetricSupportsautonegotiation:YesAdvertisedlinkmodes:10baseT/Half10baseT/Full100baseT/Half100baseT/Full1000baseT/FullAdvertisedpauseframeuse:SymmetricAdvertisedautonegotiation:YesSpeed:100Mb/sDuplex:FullPort:TwistedPairPHYAD:1Transceiver:internalAutonegotiation:onMDIX:offSupportsWakeon:pumbgWakeon:gCurrentmessagelevel:0x00000007(7)drvprobelinkLinkdetected:yes

    Soitshowsthesameastheswitch,thatthespeedoftheNICis100Mbps.YetthisNICisabletoadvertise1Gbps.Theswitchportisconfiguredfor:

    dm01swadm0#showinterfaceGigabitEthernet1/1|includeHardwareHardwareisGigabitEthernetPort,addressis0024.9723.52c0(bia0024.9723.52c0)dm01swadm0#showinterfaceGigabitEthernet1/1|includemediaFullduplex,100Mb/s,linktypeisauto,mediatypeis10/100/1000TX

    NOTE:Don'tmindthehardwareaddresshereisdifferent,asitistheswitchport'shardwareaddressnottheNICintheExadatasystem.

    ChangetheNICspeedusingethtool:

    [root@dm01adm08~]#ethtoolseth0speed1000Cannotadvertisespeed1000

    Checktoseeifthechangetookusingethtool:

    [root@dm01adm08~]#ethtooleth0|grepSpeedSpeed:1000Mb/s

    Checktheswitchportconfiguration:

    dm01swadm0#showinterfaceGigabitEthernet1/1|includemediaFullduplex,1000Mb/s,linktypeisauto,mediatypeis10/100/1000TX

    Itlookslikethingsarefixed.Sohowdiditchange?TryTDRagainandseewhatitknowsabout.

    dm01swadm0#testcablediagnosticstdrinterfaceGigabitEthernet1/1dm01swadm0#showcablediagnosticstdrinterfaceGigabitEthernet1/1

  • 24/04/2015 DocumentDisplay

    https://support.oracle.com/epmos/faces/SearchDocDisplay?_adf.ctrlstate=16sjohapiu_9 4/4

    InterfaceSpeedLocalpairCablelengthRemotechannelStatusGi1/1100Mbps12N/AUnknownTerminated36N/AUnknownTerminated45N/AUnknownTerminated78N/AUnknownTerminated

    Wait,itwentbackto100Mbpsandethtoolagrees:

    [root@dm01adm08~]#ethtooleth0|grepSpeedSpeed:100Mb/s

    ItappearsthereissomekindofproblemwheretheTDRtestiscausingthespeedtobeknockeddownto100Mbps.Theredoesnotseemtobeacablefault,butseemstobesomekindoffaultineithertheCiscoswitchortheNICastheinformation,atthispoint,isinconclusive.ThisnoteisnotaboutfindingabugineithertheCiscoswitchorNIC,buthowtouseTDR.

    AnotherexampleshowsanactualfaultconditionusingTDR.Iftherebeenafaultinthecable,theStatusintheoutputoftheTDRcommandwouldshow"Fault".OnthisswitchIexaminedoneportandfoundthis:

    dm01swadm0#testcablediagnosticstdrinterfaceGigabitEthernet1/2dm01swadm0#showcablediagnosticstdrinterfaceGigabitEthernet1/2InterfaceSpeedLocalpairCablelengthRemotechannelStatusGi1/2100Mbps12N/AUnknownTerminated36N/AUnknownTerminated451+/10mUnknownFault781+/10mUnknownFault

    Eventhoughthisisa1Gbpsportintheswitch,a100MbpsNIConlyusesthefirsttwotwistedpairs.Herethespeedis100Mpbsandthefirsttwotwistedpairsareproperlyterminated.Thecable"fault"isabouttheproperlengthof1meter(eventhoughithasanuncertaintyof+/10m)whichshowstheTDRsignalbouncingbackfromtwopairsofa1mcable.MorethenlikelytheNICinthiscasewasaniLOM,whichisonlyratedat100Mbps.

    REFERENCES

    http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst6500/ios/122SX/configuration/guide/book/intrface.html#wpmkr1066782

    Didn'tfindwhatyouarelookingfor?