unified communications troubleshooting

84
1 © 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential UC Troubleshooting 1 UC Troubleshooting Instructor: Goldman Lee, Triple CCIE #8269 (RS/SP/Sec) company email: [email protected] facebook email: [email protected] add me today! :)

Upload: mandeepmails

Post on 22-Oct-2015

170 views

Category:

Documents


1 download

DESCRIPTION

A very good understanding presentation to cisco Unified Communications Troubleshooting

TRANSCRIPT

Page 1: Unified Communications Troubleshooting

1© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting 1

UC TroubleshootingInstructor: Goldman Lee, Triple CCIE #8269 (RS/SP/Sec)company email: [email protected] email: [email protected] me today! :)

Page 2: Unified Communications Troubleshooting

2© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Agenda

What this Course is?

An environment where student can learn by doing (25% brief concise theory, 75% hands on labs)

An advanced course, enterprise-focused

What this Course is Not?

A course that teaches fundamental

A course that teaches design in detail

Page 3: Unified Communications Troubleshooting

3© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Troubleshooting Signaling Protocol Overview

Protocol Description

H.323ITU standard protocol for interactive conferencing; evolved from H.320 ISDN standard; flexible, complex

MGCP IETF standard for PSTN gateway control; thin device control

SIPIETF protocol for interactive and noninteractive conferencing; simpler, but less mature, than H.323

SCCP or “Skinny”Cisco proprietary protocol used between Cisco Unified Communications Manager and Cisco VoIP phones

Page 4: Unified Communications Troubleshooting

4© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Application of H.323

H.323 suite:–Peer-to-peer protocol

–Gateway configuration necessary because gateway must maintain dial plan and route pattern.

–Examples: Cisco VG224 Analog Phone Gateway (FXS only) and, Cisco 2800 Series and, Cisco 3800 Series routers.

Q.931

Q.921H.323

PSTN

Page 5: Unified Communications Troubleshooting

5© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Application of MGCP

MGCP:–Works in a client/server architecture–Simplified configuration –Cisco Unified Communications Manager maintains the dial plan–Examples: Cisco VG224 Analog Phone Gateway (FXS only) and, Cisco 2800 Series and , Cisco 3800 Series routers

–Cisco Catalyst operating system MGCP example: Cisco Catalyst 6000 WS-X6608-T1 and Catalyst 6000 ws-X6608-E1

Q.931

Q.921MGCP

PSTN

Page 6: Unified Communications Troubleshooting

6© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Application of SIP

SIP:–Peer-to-peer protocol.

–Gateway configuration is necessary because the gateway must maintain a dial plan and route pattern.

–Examples: Cisco 2800 Series and Cisco 3800 Series routers.

Q.931

Q.921SIP

PSTN

Page 7: Unified Communications Troubleshooting

7© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Application of SCCP

SCCP–Works in a client/server architecture.

–Simplified configuration.

–Cisco Unified Communications Manager maintains a dial plan and route patterns.

–Examples: Cisco VG224 (FXS only) and, Cisco VG248 Analog Voice Gateways, Cisco ATA 186, and Cisco 2800 Series with routers FXS ports.

FXSSCCP

PSTN

SCCP Endpoint

Page 8: Unified Communications Troubleshooting

8© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

MGCP Registration

The gateway boots up.

A TCP connection is opened.

Restart in Progress (RSIP) informs Cisco Unified CallManager that

gateway is being broughtinto service.

Acknowledgment (ACK)

Cisco Unified CallManager sends an Audit Endpoint

(AUEP) per endpoint.

RSIP

Endpoints are now registered in Cisco Unified CallManager

OK

TCP

AUEP

OKAcknowledgment (ACK)

TCP

RQNT

OK

Cisco Unified CallManager sends a Request Notify (RQNT) per endpoint.

Acknowledgment (ACK)

A TCP connection is opened.

Page 9: Unified Communications Troubleshooting

9© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

MGCP Gateway Registration Issues

Cisco IOS MGCP gateway registration issues:–Network connectivity issues–Gateway issues–Cisco Unified CallManager issues

Page 10: Unified Communications Troubleshooting

10© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Rundown of Cisco IOS MGCP Gateway Communications

Understanding the following Cisco IOS MGCP gateway behaviors is critical to effective troubleshooting:

–Endpoints on the gateway will register with Cisco Unified CallManager server.

–Cisco Unified CallManager must have the gateway added to its configuration database.

–Endpoints are under the control of Cisco Unified CallManager.

–MGCP configuration on the gateway must be performed initially.

–The ability to download configuration files to the MGCP gateway enables adds, moves, and changes to MGCP endpoints to be performed in the Cisco Unified CallManager web interface.

Page 11: Unified Communications Troubleshooting

11© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Verify MGCP Gateway Status

Pod1HQ#show ccm-managerMGCP Domain Name: Pod1HQPriority Status Host============================================================Primary Registered 10.1.1.2First Backup Backup Ready 10.1.1.1Second Backup None

Current active Call Manager: 10.1.1.2Backhaul/Redundant link port: 2428Failover Interval: 30 secondsKeepalive Interval: 15 secondsLast keepalive sent: 10:52:49 UTC Jun 17 2006 (elapsed time: 1:41:10)Last MGCP traffic time: 12:33:37 UTC Jun 17 2006 (elapsed time: 00:00:22)Last failover time: 12:33:37 UTC Jun 17 2006 from (10.1.1.2)Last switchback time: 12:33:07 UTC Jun 17 2006 from (10.1.1.1)Switchback mode: GracefulMGCP Fallback mode: Not SelectedLast MGCP Fallback start time: NoneLast MGCP Fallback end time: NoneMGCP Download Tones: Disabled Configuration Error History:FAX mode: cisco

Page 12: Unified Communications Troubleshooting

12© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Verify MGCP Gateway Status

To validate the configuration of an MGCP gateway:

–Choose Device > Gateway > Find.

–Click See Endpoints on the desired gateway.

–Select the desired endpoint to validate the configuration and status.

Page 13: Unified Communications Troubleshooting

13© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Viewing Cisco Unified CallManager Event Log Messages

Enum Definitions -Device type. [Optional]

120 MODEL_MGCP_STATION

121 MODEL_MGCP_TRUNK

125 MODEL_TRUNK

254MODEL_UNKNOWN_MGCP_GATEWAY

255 MODEL_UNKNOWN

Enum Definitions -Reason Code [Optional].

Value Definition

1 Unknown

2 NoEntryInDatabase

3 DatabaseConfigurationError

4 DeviceNameUnresolveable

5 MaxDevRegExceeded

6 ConnectivityError

7 InitializationError

8 DeviceInitiatedReset

9 CallManagerReset

10 DeviceUnregistered

Page 14: Unified Communications Troubleshooting

14© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

show ccm-manager

Pod1HQ#

Displays a list of Cisco Unified CallManager servers and their current status and availability

show mgcp endpoint

Pod1HQ#

Displays information for endpoints that are controlled using MGCP

show mgcp connection

Pod1HQ#

Displays information for active connections that are controlled using MGCP

Cisco IOS MGCP Gateway Commands

Page 15: Unified Communications Troubleshooting

15© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

show mgcp statistics

Pod1HQ#

Displays MGCP statistics regarding received and transmitted network messages

show mgcp srtp {detail |summary}

Pod1HQ#

Displays information for active SRTP connections controlled using MGCP

debug mgcp [all | errors | endpoint endpoint-name | media | nas | packets | parser | src | voipcac]

Pod1HQ#

Enables debugging messages for MGCP errors, endpoints, events, media, packets, parser, and CAC

Cisco IOS MGCP Gateway Commands (Cont.)

Page 16: Unified Communications Troubleshooting

16© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

MGCP IOS Gateway Troubleshooting

Problem report:–When deploying a new Cisco IOS MGCP gateway, it shows up as not registered in Cisco Unified CallManager.

Consider the possibilities:–Network connectivity problems.

–IP addressing problems with DHCP or manual settings.

–Cisco Unified CallManager service is not running.

–Cisco IOS Software version is not compatible with Cisco Unified CallManager version.

–Missing or incorrect configuration in Cisco Unified CallManager configuration.

–Missing or incorrect configuration on the MGCP gateway.

Page 17: Unified Communications Troubleshooting

17© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Cisco IOS MGCP Gateway Troubleshooting Topology

PRI

Cisco Unified CallManager NodesFirst Node/TFTP = 10.1.1.1

Second Node = 10.1.1.2

MGCP GatewayEthernet IP = 10.1.1.101Loopback = 10.1.250.101

Hostname = HQ-1

Page 18: Unified Communications Troubleshooting

18© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Pod1HQ#ping 10.1.1.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 10.1.1.1, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/3/8 ms

Cisco IOS MGCP Gateway Troubleshooting

Pod1HQ#show version

Cisco IOS Software, 3700 Software (C3725-IPVOICEK9-M), Version 12.4(6)T5, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2006 by Cisco Systems, Inc.

Compiled Fri 06-Oct-06 20:16 by kellythw

Page 19: Unified Communications Troubleshooting

19© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Cisco IOS MGCP Gateway Troubleshooting (Cont.)

Verify settings in Cisco Unified CallManager under the gateway.

Host Name of Router

Slot 0/Subunit0/Port 0

Page 20: Unified Communications Troubleshooting

20© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

HQ-1#show run | begin mgcpmgcpmgcp call-agent 10.1.1.2mgcp dtmf-relay voip codec all mode out-of-bandmgcp rtp unreachable timeout 1000 action notifymgcp modem passthrough voip mode ciscomgcp package-capability rtp-packagemgcp package-capability sst-packageno mgcp timer receive-rtcp!ccm-manager redundant-host 10.1.1.1ccm-manager mgcpccm-manager config server 10.1.1.1ccm-manager config!controller T1 0/1/0 framing esf linecode b8zs pri-group timeslots 1-24 service mgcp!interface Serial0/1/0:23 no ip address isdn switch-type primary-ni isdn incoming-voice voice isdn T306 30000 isdn bind-l3 ccm-manager no cdp enable!dial-peer voice 999200 pots service mgcpapp port 2/0/0

Cisco IOS MGCP Gateway Troubleshooting (Cont.)

First choice Cisco Unified CM

Second choice Cisco Unified CM

Enabled MGCP for the analog port

Enables PRI backhauling of the Q931 signaling

TFTP server to get configuration from

Enables MGCP for PRI

Hostname of gateway

Page 21: Unified Communications Troubleshooting

21© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

HQ-1#Dec 10 01:45:49.343: MGCP Packet sent to 10.1.1.1:2427--->RSIP 604725784 *@HQ-1 MGCP 0.1RM: graceful<---

HQ-1#Dec 10 01:47:02.287: MGCP Packet sent to 10.1.1.1:2427--->RSIP 604725786 *@HQ-1 MGCP 0.1RM: restart<---

Dec 10 01:47:02.487: MGCP Packet received from 10.1.1.1:2427--->500 604725786 <---

Cisco IOS MGCP Gateway Troubleshooting (Cont.)

debug mgcp packets

Page 22: Unified Communications Troubleshooting

22© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Cisco IOS MGCP Gateway Troubleshooting (Cont.)

MGCPRestartInProgress | (1,100,120,1).31-(*:10.1.1.101) | *@HQ-1 portNum= 0 RSIP 604725826 - RM: restart

MGCPRestartInProgress | (1,100,120,1).32-(*:10.1.1.101) | [R:NP - HP: 0, NP: 0, LP: 0, VLP: 0, LZP: 0 DBP: 0]*@HQ-1 portNum= 0 RSIP 604725826 - RM: restart

MGCPRestartInProgress | (1,100,120,1).32-(*:10.1.1.101) | [R:NP - HP: 0, NP: 0, LP: 0, VLP: 0, LZP: 0 DBP: 0]*@HQ-1 portNum= 0 RSIP 604725826 - RM: restart

MGCPRestartInProgress | (1,100,120,1).32-(*:10.1.1.101) | *@HQ-1 portNum= 0 RSIP 604725826 - RM: restart

| Destination process is not Active:[1,100,3,1]->[1,100,117,11] - [MGCPKeepAliveTimer]

| Destination process is not Active:[1,100,3,1]->[1,100,117,11] - [MGCPocTimer]

| Destination process is not Active:[1,100,117,11]->[1,100,3,1] - [SdlProcessNE]

| Destination process is not Active:[1,100,117,11]->[1,100,3,1] - [SdlProcessNE]

* Output has been edited to fit on the screen

Detailed Trace Output for MGCP

Page 23: Unified Communications Troubleshooting

23© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Point to Note H.323/SIP Gateway

Understanding the following H.323/SIP gateway behaviors is critical to effective troubleshooting:

–H.323/SIP gateways will never register to Cisco Unified CallManager.

–In Cisco Unified CallManager, their status will always show as “unknown”.

–Cisco Unified CallManager requires destinations reached through these gateways to be included in the dial plan.

–The H.323/SIP gateway requires that dial peers be configured to create the dial plan on the gateway.

Page 24: Unified Communications Troubleshooting

24© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Gather Facts: Network Connectivity

Verify connectivity between the endpoints and the gatekeeper.

Endpoint

Gatekeeper

Endpoint

Page 25: Unified Communications Troubleshooting

25© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Gather Facts: Verify Gatekeeper Configuration That Can Affect Registration

gatekeeper zone local HQ-1 cisco.com 10.1.250.101 zone remote DGK cisco.com 10.0.0.99 1719 no zone subnet HQ-1 default enable zone subnet HQ-1 10.1.250.0/24 enable zone prefix HQ-1 1... zone prefix HQ-1 501* zone prefix DGK * gw-type-prefix 1#* default-technology bandwidth interzone default 256 bandwidth session default 256 bandwidth session zone HQ-1 256 no shutdown

Verify the gatekeeper configuration.

Verify the zone name and IP address to use on the local gatekeeper. If the endpoint settings do not match, registration can fail.

Verify that the endpoint is allowed to register for the zone.

Verify the technology prefix. Registration issues can result if this does not match on the endpoint.

Verify that the gatekeeper is enabled with a no shutdown command.

Page 26: Unified Communications Troubleshooting

26© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Will send an RRQ every 60 seconds as a form of keepalive

The configured gatekeeper to use when sending call across the trunk

Wait 5 minutes on a failed registration attempt to try to register again

Gather Facts: Verify Gateway Configuration That Can Affect Registration

IP address of the gatekeeper

Device > Gatekeeper

Device > Trunk

Technology prefix matches technology prefix on gatekeeper

Zone name must match configured zone on gatekeeper; case matters

Terminal type should be Gateway

Page 27: Unified Communications Troubleshooting

27© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Gather Facts: Verify Gateway Configuration That Can Affect Registration (Cont.)

interface Loopback 0 ip address 10.1.250.101 255.255.255.0

h323-gateway voip interface h323-gateway voip id HQ-1 ipaddr 10.1.250.101 1718

h323-gateway voip h323-id HQGateway-1 h323-gateway voip tech-prefix 1# h323-gateway voip bind srcaddr 10.1.250.101 ! dial-peer voice 1 voip destination-pattern .... session target ras ! dial-peer voice 2 pots destination-pattern 1611 port 1/1/1 no register e164 gateway

Verify that the technology prefix matches the gatekeeper.

Verify that the gateway function is enabled.

Verify that the IP address of the gatekeeper is correct.

Verify IP address that H.323 messages will be sourced from is allowed to register with the gatekeeper.

Page 28: Unified Communications Troubleshooting

28© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Gather Facts: Debugging Gatekeeper Discovery

debug h225 asn1

Gatekeeper# or Gateway#

Displays RAS messages with ASN1 content

value RasMessage ::= registrationReject :

{

requestSeqNum 5

protocolIdentifier { 0 0 8 2250 0 4 }

rejectReason discoveryRequired : NULL

}.

Page 29: Unified Communications Troubleshooting

29© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Gather Facts: Debugging Gatekeeper Registration

Reject reasons:–DuplicateAlias: duplicate E.164 or H323 ID–TerminalExcluded: not authorized to register–SecurityDenial: security parameters do not match–InvalidAlias: zone name does not match or is not configured

value RasMessage ::= registrationReject :

{

requestSeqNum 2829

protocolIdentifier { 0 0 8 2250 0 3 }

rejectReason REJECT REASON:Duplicate Alias

gatekeeperIdentifier {“HQ-1"}

}

Page 30: Unified Communications Troubleshooting

30© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Gather Facts: Cisco Unified CallManager Trace

IP address in Hex = 10.1.1.1

Digits dialed

ARQ

Page 31: Unified Communications Troubleshooting

31© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Problem Report: Endpoint Calls Fail

Problem report: Endpoints are registered but not able to make calls.

GatekeeperEndpoint

Endpoint

Xgatekeeper#show gatekeeper endpoint

GATEKEEPER ENDPOINT REGISTRATION

================================

CallSignalAddr Port RASSignalAddr Port Zone Name Type Flags

--------------- ----- --------------- ----- --------- ---- -----

10.1.250.101 1720 10.1.250.101 50890 HQ-1 VOIP-GW

H323-ID: HQGateway-1

10.1.1.1 1720 10.1.1.1 50890 HQ-1 VOIP-GW

H323-ID: CCM1_1

10.1.1.2 1720 10.1.1.2 50890 HQ-1 VOIP-GW

H323-ID: CCM1_2

Total number of active registrations = 3

Page 32: Unified Communications Troubleshooting

32© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Possible Causes: Endpoint Calls Fail

Possible causes of failed calls when the endpoints are registered:

–Connectivity between endpoints

–Insufficient bandwidth due to CAC

–Dial plan misconfiguration

–Incorrect IP address returned

–Called party is not registered

Gather facts:–Verify connectivity between endpoints

–show gatekeeper endpoint

–show gatekeeper zone status

–debug h225 asn1

–debug ras

Page 33: Unified Communications Troubleshooting

33© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Gather Facts: Debugging Gatekeeper Admission

Reject reasons:–calledPartyNotRegistered: The called party either has never registered or has timed out

–requestDenied: Bandwidth is not sufficient for the call

value RasMessage ::= admissionReject :

{

requestSeqNum 34

rejectReason REJECT REASON : NULL

}

Page 34: Unified Communications Troubleshooting

34© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Where to start?

1.

2.

3.

Obtain Detailed Problem Description: Time of events, calling/called#, IP phone’s Device name(SEPMacAddress)

In the CCM trace, use the MAC or IP address to find the IP phone’skeepalive message, this will yield the TCP handle for this device.12/29/2003 14:49:01.602 CCM|StationInit - InboundStim - KeepAliveMessage -Send KeepAlive to Device Controller. DeviceName=SEP003094C37F47,TCPHandle=000000046, IPAddr=172.16.240.11, Port=49888, DeviceController=[1,89,40]|<CLID::EDPUB-Cluster><NID::172.16.240.119><CT::1,100,90,1.85832><IP::172.16.240.11><DEV::SEP003094C37F47>

FYI: StationInit is an inbound skinny msg to CM, while StationD isan outbound msg from CM.

Follow the TCP handle throughout the CCM trace for all events onthis phone.

Page 35: Unified Communications Troubleshooting

35© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Common Skinny Call States

OffHook = 1

OnHook = 2

RingOut = 3

RingIn = 4

Connected = 5

Busy = 6

Congestion = 7

Hold = 8

CallWaiting = 9

CallTransfer = 10

CallPark = 11

Proceed = 12

CallRemoteMultiline = 13

InvalidNumber = 14

Page 36: Unified Communications Troubleshooting

36© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Common Skinny Tones

InsideDialTone=0x21,

Silence = 0,

Dtmf1 = 1,

Dtmf2 = 2,

Dtmf3 = 3,

Dtmf4 = 4,

Dtmf5 = 5,

Dtmf6 = 6,

Dtmf7 = 7,

Dtmf8 = 8,

Dtmf9 = 9,

Dtmf0 = 0xa,

DtmfStar = 0xe,

DtmfPound = 0xf,

OutsideDialTone=0x22,

LineBusyTone=0x23,

AlertingTone=0x24,

ReorderTone=0x25,

RecorderWarningTone=0x26,

RecorderDetectedTone=0x27,

RevertingTone=0x28,

ReceiverOffHookTone=0x29,

PartialDialTone=0x2A,

NoSuchNumberTone=0x2B,

BusyVerificationTone=0x2C,

CallWaitingTone=0x2D,

ConfirmationTone=0x2E,

CampOnIndicationTone=0x2F,

RecallDialTone=0x30,

Page 37: Unified Communications Troubleshooting

37© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Skinny Softkey Events

Redial = 1

NewCall = 2

Hold = 3

Trnsfer = 4

CFwdAll = 5

CFwdBusy = 6

CFwdNoAnswer = 7

BackSpace = 8

EndCall = 9

Resume = 10

Answer = 11

Info = 12

Confrn = 13

Park = 14

Join = 15

MeetMeConfrn = 16

CallPickUp = 17

GrpCallPickUp = 18

Page 38: Unified Communications Troubleshooting

38© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Softkey Indexes (Part I)

0. Call State = Onhook, Key Set =00. Redial

1. NewCall

1. Call State = Connected, Key Set =10. Hold

1. EndCall

2. Trnsfer

3. Park

4. Confrn

2. Call State = OnHold, Key Set =20. Resume

1. NewCall

Page 39: Unified Communications Troubleshooting

39© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Softkey Indexes (Part II)

3. Call State = Ring In, Key Set =30. Answer

4. Call State = Offhook, Key Set = 40. Redial

1. EndCall

2. CFwdAll

3. MeetMeConfrn

4. CallPickUp

5. GrpCallPickUp

5. Call State = Connected w/ Transfering, Key Set = 50.

1. EndCall

2. Transfer

Page 40: Unified Communications Troubleshooting

40© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Softkey Indexes (Part III)

6. Call State = “More Digits”, Key Set =60. BackSpace

1. EndCall

7. Call State = Connected w/ Conferencing, Key Set = 70.

1. EndCall

2. Confrn

8. Call State = Ring Out, Key Set = 80.

1. EndCall

Page 41: Unified Communications Troubleshooting

41© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Skinny Media Payload Types

Media_Payload_G729AnnexB = 15,

Media_Payload_NonStandard = 1,

Media_Payload_G711Alaw64k = 2,

Media_Payload_G711Alaw56k = 3,

Media_Payload_G711Ulaw64k = 4,

Media_Payload_G711Ulaw56k = 5,

Media_Payload_G722_64k = 6,

Media_Payload_G722_56k = 7,

Media_Payload_G722_48k = 8,

Media_Payload_G7231 = 9,

Media_Payload_G728 = 10,

Media_Payload_G729 = 11,

Media_Payload_G729AnnexA = 12,

Media_Payload_Is11172AudioCap =13,

Media_Payload_Is13818AudioCap =14,

Media_Payload_G729AnnexAwAnnexB =16,

Media_Payload_GSM_Full_Rate = 18,

Media_Payload_GSM_Half_Rate = 19,

Media_Payload_GSM_Enhanced_Full_Rate= 20,

Media_Payload_Wide_Band_256k = 25,

Media_Payload_Data64 = 32,

Media_Payload_Data56 = 33,

Media_Payload_GSM = 80,

Media_Payload_ActiveVoice = 81,

Media_Payload_G726_32K = 82,

Media_Payload_G726_24K = 83,

Media_Payload_G726_16K = 84,

Page 42: Unified Communications Troubleshooting

42© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Call Flow #1

2000 calls 2001 and the call established.

2001 puts 2000 on hold.

2000 hears MOH.

2001 takes 2000 off hold.

2001 blind transfers the call to 2002.

2002 answers and the call is established.

Page 43: Unified Communications Troubleshooting

43© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

46 44

42Call

Call Flow #1 (Cont’d)

CCM

x2000

x2002

Hold

Resume

Transfer

MOH

x2001

Page 44: Unified Communications Troubleshooting

44© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CiscoCM

Basic Skinny Call Flows

Station OffHookStation Display Text

Station Set Lamp (Steady)

Station Start Tone (Dial Tone)

Station Keypad Button

Station Stop ToneStation Keypad Button

Station Keypad Button

Cisco IPPhone 2

Cisco IPPhone 1

IPT Bootcamp Day 5 – e 96

Station Call Info

Station Set Lamp (Blink)Station Set Ringer

Station Off Hook

Station Set Ringer (Off)

Station Set Lamp (Steady)

Station Open Receive Channel

Station Start Media Xmission

Station Open Receive Channel Ack

Station Keypad Button

Station Call Info

Station Start Tone (Ringback)

Station Open Receive ChannelStation Stop Tone

Station Call InfoStation Open Receive Channel Ack

Station Start Media Xmission

User Information Exchange

Station On HookStation Set Lamp (Off)

Station Close Receive Channel

Station Stop Media Xmission

Page 45: Unified Communications Troubleshooting

45© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1

Cisco

CallManager

Cisco IP

PhoneCisco IP

Phone

Page 46: Unified Communications Troubleshooting

46© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1

- IP phone device SEP003094C37F47 and DN 2000 went offhook, CM instructs the phone to displayappropriate info, and play a dialtone. The TCP Handle for this phone is 000000046

StationInit: 000000046 OffHook.|<CLID::EDPUB-Cluster><NID::172.16.240.119><CT::1,100,90,1.85834><IP::172.16.240.11><DEV::SEP003094C37F47>

StationD: 000000046 SetRinger ringMode=1(RingOff)StationD: 000000046 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=2(LampOn)StationD: 000000046 CallState callState=1 lineInstance=1 callReference=16777264StationD: 000000046 DisplayPromptStatus timeOutValue=0 promptStatus='€ ' content='Enter Number'

lineInstance=1 callReference=16777264 ver=0x80000004StationD: 6991ad8 ActivateCallPlanelineInstance=1.

StationD: 000000046 SelectSoftKeys instance=1 reference=16777264 softKeySetIndex=4 validKeyMask=-1StationD: 000000046 ActivateCallPlane lineInstance=1StationD: 000000046 StartTone tone=33(InsideDialTone)User dials and CM informs the ip phone to stop the dialtone and change the softkeys.StationInit: 000000046 KeypadButton kpButton=2StationD: 000000046 StopToneStationD: 000000046 SelectSoftKeys instance=1 reference=16777264 softKeySetIndex=6 validKeyMask=-1

Page 47: Unified Communications Troubleshooting

47© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1

Cisco

CallManager

Cisco IP

Phone

Page 48: Unified Communications Troubleshooting

48© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1,continued…

- 2000 continues to dial and CM does digit analysis to rule out potentialmatches. Note pss= are the partitions this DN has access to:

Digit analysis: match(fqcn="2000", cn="2000", pss="Line 1:Line2:Local_Dial:Long_Distance_Dial:International_Dial:Internal", dd="2")

Digit analysis: potentialMatches=PotentialMatchesExist

StationInit: 000000046 KeypadButton kpButton=0

Digit analysis: match(fqcn="2000", cn="2000", pss="Line 1:Line2:Local_Dial:Long_Distance_Dial:International_Dial:Internal", dd="20")

Digit analysis: potentialMatches=PotentialMatchesExist

StationInit: 000000046 KeypadButton kpButton=0.

Digit analysis: match(fqcn="2000", cn="2000", pss="Line 1:Line2:Local_Dial:Long_Distance_Dial:International_Dial:Internal", dd="200")

Digit analysis: potentialMatches=PotentialMatchesExist

StationInit: 000000046 KeypadButton kpButton=1

Digit analysis: match(fqcn="2000", cn="2000", pss="Line 1:Line2:Local_Dial:Long_Distance_Dial:International_Dial:Internal", dd="2001")

Page 49: Unified Communications Troubleshooting

49© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Traces for Call Flow #1,continued…

- Last digit came in the CM, Digit analysis found amatch. NoPotentialMatchesExist means an exactmatch was found.

|CallingPartyNumber=2000

|DialingPartition=Line 1

|DialingPattern=2001|DialingRoutePatternRegularExpression=(2001)|DialingWhere=|PatternType=Enterprise|PotentialMatches=NoPotentialMatchesExist|DialingSdlProcessId=(1,34,20)|IndexOfAnalyzedPattern=0|PretransformDigitString=2001|PretransformTagsList=SUBSCRIBER|PretransformPositionalMatchList=2001|CollectedDigits=2001|UnconsumedDigits=|TagsList=SUBSCRIBER

|PositionalMatchList=2001

|VoiceMailboxMask=

|VoiceMailCallingSearchSpace=

|VoiceMailPilotNumber=

|DisplayName=

|RouteBlockFlag=RouteThisPattern

|InterceptPartition=

|InterceptPattern=

|InterceptWhere=

|InterceptSdlProcessId=(0,0,0)

|InterceptSsType=0

|InterceptSsKey=0

Page 50: Unified Communications Troubleshooting

50© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Traces for Call Flow #1,continued…

Cisco

CallManagerCisco IP

Phone

Cisco IP

Phone

Page 51: Unified Communications Troubleshooting

51© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1,continued…

- CM located the device with extension 2001 with TCP handle 000000044; instructsthe phone to ring and more.

StationD:

StationD:

StationD:

000000044 CallState callState=4 lineInstance=1 callReference=16777265

000000044 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=5(LampBlink)

000000044 SetRinger ringMode=2(InsideRing)

StationD: 000000044 DisplayPromptStatus timeOutValue=0 promptStatus='€ 2000'content='From 2000' lineInstance=1 callReference=16777265 ver=0x80000004

StationD: 000000044 SelectSoftKeys instance=1 reference=16777265 softKeySetIndex=3validKeyMask=-1

- In the mean time, CM tells the calling phone (2000) to change softkeys, playringback tone, and more.

StationD: 000000046 CallState callState=12 lineInstance=1 callReference=16777264

StationD: 000000046 CallInfo callingPartyName='' callingParty=2000 cgpnVoiceMailbox=

StationD: 000000046 DialedNumber dialedNumber=2001 lineInstance=1 callReference=16777264

StationD: 000000046 StartTone tone=36(AlertingTone), direction=0

StationD: 000000046 CallState callState=3 lineInstance=1 callReference=16777264

StationD: 000000046 SelectSoftKeys instance=1 reference=16777264 softKeySetIndex=8validKeyMask=-1

Page 52: Unified Communications Troubleshooting

52© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1,continued…

Cisco

CallManager

Cisco IP

Phone

Cisco IP

Phone

Page 53: Unified Communications Troubleshooting

53© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1,continued…

- 2001 went offhook.StationInit: 000000044 OffHook

StationD: 000000044 ClearNotify

StationD: 000000044 SetRinger ringMode=1(RingOff)

StationD: 000000044 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=2(LampOn)

StationD: 000000044 CallState callState=1 lineInstance=1 callReference=16777265

StationD: 000000044 ActivateCallPlane lineInstance=1

- CM tells 2000 to stop ringback and 2001 to stop ringing. Then sendsOpenReceiveChannel to both phones.

StationD: 000000046 StopTone

StationD: 000000046 OpenReceiveChannel conferenceID=0 passThruPartyID=1000201millisecondPacketSize=20 compressionType=4(Media_Payload_G711Ulaw64k) qualifierIn=?.myIP: bf010ac (172.16.240.11)

StationD: 000000044 StopTone

StationD: 000000044 OpenReceiveChannel conferenceID=0 passThruPartyID=1000211millisecondPacketSize=20 compressionType=4(Media_Payload_G711Ulaw64k) qualifierIn=?.myIP: 57f010ac (172.16.240.87)

Page 54: Unified Communications Troubleshooting

54© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1,continued…

- CM changes the call state phones to “Connected” and changes softkeys and diaplays.StationD: 000000044 CallState callState=5 lineInstance=1 callReference=16777265StationD: 000000044 SelectSoftKeys instance=1 reference=16777265 softKeySetIndex=1

validKeyMask=-1StationD: 000000044 DisplayPromptStatus timeOutValue=0 promptStatus='€ ' content='Connected'

lineInstance=1 callReference=16777265 ver=0x80000004StationD: 000000046 CallState callState=5 lineInstance=1 callReference=16777264StationD: 000000046 SelectSoftKeys instance=1 reference=16777264 softKeySetIndex=1

validKeyMask=-1StationD: 000000046 DisplayPromptStatus timeOutValue=0 promptStatus='€ ' content='Connected'

lineInstance=1 callReference=16777264 ver=0x80000004

- CM receives OpenReceiveChannelAck from both phones, cuts through audio rtpstreams between the phones.

StationInit: 000000046 OpenReceiveChannelAck Status=0, IpAddr=0xbf010ac, Port=27686,PartyID=16777729

StationD: 000000044 StartMediaTransmission conferenceID=0 passThruPartyID=1000211remoteIpAddress=bf010ac(172.16.240.11) remotePortNumber=27686 milliSecondPacketSize=20compressType=4(Media_Payload_G711Ulaw64k) qualifierOut=?. myIP: 57f010ac (172.16.240.87)

StationInit: 000000044 OpenReceiveChannelAck Status=0, IpAddr=0x57f010ac, Port=30432,PartyID=16777745

StationD: 000000046 StartMediaTransmission conferenceID=0 passThruPartyID=1000201remoteIpAddress=57f010ac(172.16.240.87) remotePortNumber=30432 milliSecondPacketSize=20compressType=4(Media_Payload_G711Ulaw64k) qualifierOut=?. myIP: bf010ac (172.16.240.11)

Page 55: Unified Communications Troubleshooting

55© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1,continued…

- 2001 puts the call on hold, CM tears down the rtp media stream on both phones.

StationInit: 000000044 SoftKeyEvent softKeyEvent=3(Hold) lineInstance=1 callReference=16777265StationD: 000000046 CloseReceiveChannel conferenceID=0 passThruPartyID=1000201. myIP:

bf010ac (172.16.240.11)StationD: 000000046 StopMediaTransmission conferenceID=0 passThruPartyID=1000201. myIP:

bf010ac (172.16.240.11)StationD: 000000044 CloseReceiveChannel conferenceID=0 passThruPartyID=1000211. myIP:

57f010ac (172.16.240.87)StationD: 000000044 StopMediaTransmission conferenceID=0 passThruPartyID=1000211. myIP:

57f010ac (172.16.240.87)- CM instructs 2001 to change call state, softkeys, displays, etc.StationD: 000000044 CallState callState=8 lineInstance=1 callReference=16777265StationD: 000000044 SelectSoftKeys instance=1 reference=16777265 softKeySetIndex=2

validKeyMask=-1StationD: 000000044 DisplayPromptStatus timeOutValue=0 promptStatus='€ ' content='Hold'

lineInstance=1 callReference=16777265 ver=0x80000004- CM tries to allocated MOH to stream music to held phone with DN 2000.MediaResourceManager::waiting_MrmAllocateMohResourceReqMediaResourceCdpc::sortDeviceGivenList Name=MOH_EDPUBDeviceManager::findDpidGivenKey - Name=MOH_EDPUB Pid=(1,66,2)MohDControl - AllocateMohResourceReq successfully - AudioSourceID and MuticastFlag = 1MRM::updateMohCounter MRL allocateCounter=1

Page 56: Unified Communications Troubleshooting

56© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1,continued…

- CM sets up the music rtp stream between 2000 and MOH server.StationD: 000000046 OpenReceiveChannel conferenceID=0 passThruPartyID=1000202

millisecondPacketSize=20 compressionType=4(Media_Payload_G711Ulaw64k) qualifierIn=?. myIP:bf010ac (172.16.240.11)

StationInit: 000000046 OpenReceiveChannelAck Status=0, IpAddr=0xbf010ac, Port=28264,PartyID=16777730

MohDControl - stationOutputStartMediaTransmission tcpHandle=0x8 myIP: 77f010ac (172.16.240.119)

MohDControl - RemoteIpAddr: bf010ac (172.16.240.11) RemoteRtpPortNumber: 28264 msecPacketSize: 20compressionType: 4

- 2001 takes the call off hold by pressing the “Resume” softkey.StationInit: 000000044 SoftKeyEvent softKeyEvent=10(Resume) lineInstance=1 callReference=16777265

- CM tears down the music stream from MOH server to 2000StationD: 000000046 CloseReceiveChannel conferenceID=0 passThruPartyID=1000202. myIP: bf010ac

(172.16.240.11)

MohDControl - stationOutputStopMediaTransmission tcpHandle=0x8 myIP: 77f010ac (172.16.240.119)

MediaResourceManager::waiting_MrmDeallocateMohResourceReq

MRM::updateMohCounter MRL allocateCounter=0

Page 57: Unified Communications Troubleshooting

57© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1,continued…

- CM reconnects the IP phones

StationD: 000000046 OpenReceiveChannel conferenceID=0 passThruPartyID=1000203millisecondPacketSize=20 compressionType=4(Media_Payload_G711Ulaw64k) qualifierIn=?. myIP:bf010ac (172.16.240.11)

StationD: 000000044 OpenReceiveChannel conferenceID=0 passThruPartyID=1000212millisecondPacketSize=20 compressionType=4(Media_Payload_G711Ulaw64k) qualifierIn=?. myIP:57f010ac (172.16.240.87)

StationInit: 000000046 OpenReceiveChannelAck Status=0, IpAddr=0xbf010ac, Port=29378,PartyID=16777731

StationD: 000000044 StartMediaTransmission conferenceID=0 passThruPartyID=1000212remoteIpAddress=bf010ac(172.16.240.11) remotePortNumber=29378 milliSecondPacketSize=20compressType=4(Media_Payload_G711Ulaw64k) qualifierOut=?. myIP: 57f010ac (172.16.240.87)

StationInit: 000000044 OpenReceiveChannelAck Status=0, IpAddr=0x57f010ac, Port=32122,PartyID=16777746

StationD: 000000046 StartMediaTransmission conferenceID=0 passThruPartyID=1000203remoteIpAddress=57f010ac(172.16.240.87) remotePortNumber=32122 milliSecondPacketSize=20compressType=4(Media_Payload_G711Ulaw64k) qualifierOut=?. myIP: bf010ac (172.16.240.11)

Page 58: Unified Communications Troubleshooting

58© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

StationD:

StationD:

StationD:

000000044 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=3(LampWink)

000000044 CallState callState=8 lineInstance=1 callReference=16777265

000000044 SelectSoftKeys instance=1 reference=16777265 softKeySetIndex=2 validKeyMask=-1

CCM Trace for Call Flow #1,continued…

- 2001 initiated a Transfer by pressing the “Transfer” softkey

StationInit: 000000044 SoftKeyEvent softKeyEvent=4(Trnsfer) lineInstance=1 callReference=16777265.

- CM puts the call on 2001’s line 1 on hold first.

StationD: 000000044 DisplayPromptStatus timeOutValue=0 promptStatus='€ ' content='Hold' lineInstance=1callReference=16777265 ver=0x80000004

- CM tears down the media stream between the phones again.

StationD: 000000046 CloseReceiveChannel conferenceID=0 passThruPartyID=1000203. myIP: bf010ac(172.16.240.11)

StationD: 000000046 StopMediaTransmission conferenceID=0 passThruPartyID=1000203. myIP: bf010ac(172.16.240.11)

StationD: 000000044 CloseReceiveChannel conferenceID=0 passThruPartyID=1000212. myIP: 57f010ac(172.16.240.87)

StationD: 000000044 StopMediaTransmission conferenceID=0 passThruPartyID=1000212. myIP: 57f010ac(172.16.240.87)

Page 59: Unified Communications Troubleshooting

59© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1,continued…

- CM sets up MOH to 2000 again.

StationD: 000000046 OpenReceiveChannel conferenceID=0 passThruPartyID=1000204millisecondPacketSize=20 compressionType=4(Media_Payload_G711Ulaw64k) qualifierIn=?. myIP:bf010ac (172.16.240.11)

StationInit: 000000046 OpenReceiveChannelAck Status=0, IpAddr=0xbf010ac, Port=29672,PartyID=16777732

MohDControl - stationOutputStartMediaTransmission tcpHandle=0x8 myIP: 77f010ac (172.16.240.119)

MohDControl - RemoteIpAddr: bf010ac (172.16.240.11) RemoteRtpPortNumber: 29672 msecPacketSize: 20compressionType: 4

- CM opens the 2nd call on 2001’s line 1 to transfer the call, provides dial-tone.

StationD: 000000044 CallState callState=1 lineInstance=1 callReference=16777268

StationD: 000000044 DisplayPromptStatus timeOutValue=0 promptStatus='€ ' content='Enter Number'lineInstance=1 callReference=16777268 ver=0x80000004

StationD:

StationD:

StationD:

000000044 SelectSoftKeys instance=1 reference=16777268 softKeySetIndex=9 validKeyMask=-1

000000044 ActivateCallPlane lineInstance=1

000000044 StartTone tone=33(InsideDialTone), direction=0

Page 60: Unified Communications Troubleshooting

60© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1,continued…

- User started to input number (2002) to transfer the call followed by DigitAnalysis Results.

StationInit: 000000044 KeypadButton kpButton=2StationD: 000000044 StopToneStationD: 000000044 SelectSoftKeys instance=1 reference=16777268 softKeySetIndex=6

validKeyMask=-1Digit analysis: match(fqcn="2001", cn="2001", pss="Internal:Line 2:Line 1:Local_Dial:Long_Distance_Dial",

dd="2")Digit analysis: potentialMatches=PotentialMatchesExistStationInit: 000000044 KeypadButton kpButton=0Digit analysis: match(fqcn="2001", cn="2001", pss="Internal:Line 2:Line 1:Local_Dial:Long_Distance_Dial",

dd="20")Digit analysis: potentialMatches=PotentialMatchesExistStationInit: 000000044 KeypadButton kpButton=0Digit analysis: match(fqcn="2001", cn="2001", pss="Internal:Line 2:Line 1:Local_Dial:Long_Distance_Dial",

dd="200")Digit analysis: potentialMatches=PotentialMatchesExistStationInit: 000000044 KeypadButton kpButton=2Digit analysis: match(fqcn="2001", cn="2001", pss="Internal:Line 2:Line 1:Local_Dial:Long_Distance_Dial",

dd="2002")

Page 61: Unified Communications Troubleshooting

61© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1,continued…

Digit analysis: analysis results||CallingPartyNumber=2001

|DialingPartition=Line 1|DialingPattern=2002|DialingRoutePatternRegularExpression=(2002)|DialingWhere=|PatternType=Enterprise|PotentialMatches=NoPotentialMatchesExist|DialingSdlProcessId=(1,34,17)|IndexOfAnalyzedPattern=0|PretransformDigitString=2002|PretransformTagsList=SUBSCRIBER|PretransformPositionalMatchList=2002|CollectedDigits=2002|UnconsumedDigits=|TagsList=SUBSCRIBER|PositionalMatchList=2002|VoiceMailboxMask=|VoiceMailCallingSearchSpace=

|VoiceMailPilotNumber=

|DisplayName=

|RouteBlockFlag=RouteThisPattern

|InterceptPartition=

|InterceptPattern=

|InterceptWhere=

|InterceptSdlProcessId=(0,0,0)

|InterceptSsType=0

|InterceptSsKey=0

|WithTags=

|WithValues=

|CgpnPresentation=NotSelected

|CallManagerDeviceType=UserDevice

|AlternateMatches= Information Not Available

Page 62: Unified Communications Troubleshooting

62© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1,continued…

- CM locates the phone with DN of 2002, rings it.StationD: 000000042 CallState callState=4 lineInstance=1 callReference=16777269

StationD: 000000042 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=5(LampBlink)

StationD: 000000042 SetRinger ringMode=2(InsideRing)

StationD: 000000042 DisplayPromptStatus timeOutValue=0 promptStatus='€ 2001' content='From 2001'lineInstance=1 callReference=16777269 ver=0x80000004

StationD: 000000042 SelectSoftKeys instance=1 reference=16777269 softKeySetIndex=3 validKeyMask=-1

- CM changes call state, tone, softkeys, on 2001.StationD: 000000044 SelectSoftKeys instance=1 reference=16777265 softKeySetIndex=2 validKeyMask=-3

StationD: 000000044 CallState callState=12 lineInstance=1 callReference=16777268

StationD: 000000044 DialedNumber dialedNumber=2002 lineInstance=1 callReference=16777268

StationD: 000000044 StartTone tone=36(AlertingTone), direction=0

StationD: 000000044 CallState callState=3 lineInstance=1 callReference=16777268

StationD: 000000044 DisplayPromptStatus timeOutValue=0 promptStatus='€ ' content='Ring Out'lineInstance=1 callReference=16777268 ver=0x80000004

- 2001 completes the transfer by pressing the “transfer” softkey a second time.StationInit: 000000044 SoftKeyEvent softKeyEvent=4(Trnsfer) lineInstance=1 callReference=16777268

Page 63: Unified Communications Troubleshooting

63© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1,continued...

- CM tears down MOH to 2000, then tells it to play ringback tone.StationD: 000000046 CloseReceiveChannel conferenceID=0 passThruPartyID=1000204. myIP: bf010ac

(172.16.240.11)MohDControl - stationOutputStopMediaTransmission tcpHandle=0x8 myIP: 77f010ac (172.16.240.119)MediaResourceManager::waiting_MrmDeallocateMohResourceReqStationD: 000000046 StartTone tone=36(AlertingTone), direction=0

- CM clears the 1st and 2nd call on line 1 of 2001.

StationD:StationD:StationD:StationD:StationD:StationD:

000000044 ClearPromptStatus lineInstance=1 callReference=16777265000000044 CallState callState=2 lineInstance=1 callReference=16777265000000044 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=2(LampOn)000000044 ClearPromptStatus lineInstance=1 callReference=16777268000000044 CallState callState=2 lineInstance=1 callReference=16777268000000044 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=1(LampOff)

- Finally, 2002 accepts the incoming call by pressing the “answer” softkey.StationInit: 000000042 OffHook

- The media setup between 2000 and 2002 is similar to the media setup shownearlier in the initial call between 2000 and 2001.

Page 64: Unified Communications Troubleshooting

64© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

CCM Trace for Call Flow #1,continued…

- 2000 hangs up to disconnect the call so CM sends a close receivechannel and stop media transmission then changes the display on thephone.

StationInit: 000000046 OnHook.

StationD: 000000046 CloseReceiveChannel conferenceID=0 passThruPartyID=1000205. myIP:bf010ac (172.16.240.11)

StationD: 000000046 StopMediaTransmission conferenceID=0 passThruPartyID=1000205. myIP:bf010ac (172.16.240.11)

StationD: 000000046 ClearPromptStatus lineInstance=1 callReference=16777264.StationD: 000000046 SelectSoftKeys instance=0 reference=0 softKeySetIndex=0 validKeyMask=-1StationD: 000000046 DisplayPromptStatus timeOutValue=0 promptStatus='€ ' content='Your

current options' lineInstance=0 callReference=0 ver=0x80000004

- CM sends similar messages to 2002.StationD: 000000042 CloseReceiveChannel conferenceID=0 passThruPartyID=1000241. myIP:

5bf010ac (172.16.240.91)StationD: 000000042 StopMediaTransmission conferenceID=0 passThruPartyID=1000241. myIP:

5bf010ac (172.16.240.91)StationD: 000000042 ClearPromptStatus lineInstance=1 callReference=16777269StationD: 000000042 SelectSoftKeys instance=0 reference=0 softKeySetIndex=0 validKeyMask=-1StationD: 000000042 DisplayPromptStatus timeOutValue=0 promptStatus='€ ' content='Your

current options' lineInstance=0 callReference=0 ver=0x80000004

Page 65: Unified Communications Troubleshooting

65© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

QUIZZ TIME!

Question 1.

Which Call Trace provide more detailed information?

A) SDI

B) SDL

Question 2.

Gateway send RSIP to CM to ….

C) Update the status idle, busy and out of service

D) Notify its endpoints are remove or back in service

Page 66: Unified Communications Troubleshooting

66© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

QUIZZ TIME!

Question 3.

Which is the command to troubleshoot gatekeeper?

A) Debug h225 asn1

B) Debug mgcp event

Question 4.

How to identify Skinny Message send to CCM via?

C) StationInit

D) StationD

Page 67: Unified Communications Troubleshooting

67© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Page 68: Unified Communications Troubleshooting

68© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Appendix: Further Reading Call Flow #2

Inbound call from the PSTN over PRI on an H323gateway. The calling number is (408) 555-1212 andthe called number is 2001 (IP Phone).

2001’s TCP handle is 000000007.

Make note of the callref so you can follow the H225messages in the trace. Note: the call referenceincreases by 8 depending on the direction (ie.inbound is 0x0008 so outbound is 0x8008 orinbound is 00 07 and outbound is 80 07).

Page 69: Unified Communications Troubleshooting

69© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

H.323 Client to Skinny Client CallFlow

H.323Gateway

CiscoCallManager

Cisco IPPhone

Station Set Lamp (Blink)

Station Call Info

H.225 Setup

H.225 Setup Ack

Station Set Ringer (On)

Station Off Hook

Station Set Lamp (Steady)

Station Set Ringer (Off)

Station Start Media Reception

Station Start Media Transmission

H.225 Alerting

H.225 Connect

H.245 Master/Slave Determination

H.245 Master/Slave Determination Ack

H.245 Terminal Capabilities Set

H.245 Terminal Capabilities Set Ack

H.245 Open Logical Channel

H.245 Open Logical Channel Ack

ConversationStation On Hook

Station Stop Media Transmission

Station Stop Media Reception

Station Set Lamp (Off)

Page 70: Unified Communications Troubleshooting

70© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

H.225 Setup

H.225 Setup Ack

Trace and Debug for Call Flow #2

H.323Gateway

CiscoCallManager

Cisco IPPhone

Page 71: Unified Communications Troubleshooting

71© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Trace and Debug for Call Flow #2

- Inbound Q931 setup message on the IOS voice gateway:00:42:12: ISDN Se0/0:23: RX <- SETUP pd = 8 callref = 0x0008

00:42:12:

00:42:12:

00:42:12:

00:42:12:

00:42:12:

Bearer Capability i = 0x9090A2

Channel ID i = 0xA98397

Progress Ind i = 0x8183 - Origination address is non-ISDN

Calling Party Number i = 0x00, 0x80, '4085551212', Plan:Unknown, Type:Unknown

Called Party Number i = 0x81, '2001', Plan:ISDN, Type:Unknown

- In the CCM trace, we see the H225 setup message:

In Message -- H225SetupMsg -- Protocol= H225Protocol

Ie - H225BearerCapabilityIe -- IEData= 04 03 90 90 A3

Ie - H225CallingPartyIe -- IEData= 6C 0C 00 80 34 30 38 35 35 35 31 32 31 32

Ie - Q931CalledPartyIe -- IEData= 70 05 81 32 30 30 31

IsdnMsgData1= 08 02 00 07 05 04 03 90

Page 72: Unified Communications Troubleshooting

72© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Trace and Debug for Call Flow #2,cont…

- CM then performs digit analysis on the incoming call:Digit analysis: match(fqcn="", cn="4085551212", pss="", dd="2001")

|CallingPartyNumber=4085551212|DialingPartition=|DialingPattern=2001|DialingRoutePatternRegularExpression=(2001)|DialingWhere=|PatternType=Enterprise|PotentialMatches=NoPotentialMatchesExist|DialingSdlProcessId=(1,34,3)|PretransformDigitString=2001|PretransformTagsList=SUBSCRIBER|PretransformPositionalMatchList=2001|CollectedDigits=2001|UnconsumedDigits=|TagsList=SUBSCRIBER|PositionalMatchList=2001

|VoiceMailbox=

|VoiceMailCallingSearchSpace=

|VoiceMailPilotNumber=

|DisplayName=

|RouteBlockFlag=RouteThisPattern

|InterceptPartition=

|InterceptPattern=

|InterceptWhere=

|InterceptSdlProcessId=(0,0,0)

|InterceptSsType=0

|InterceptSsKey=0

|WithTags=

|WithValues=

Page 73: Unified Communications Troubleshooting

73© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Trace and Debug for Call Flow #2, cont…H.323

GatewayCisco

CallManagerCisco IPPhone

Station Set Lamp (Blink)Station Set Ringer (On)

Station Call Info

H.225 Setup

H.225 Setup Ack

H.225 Alerting

Page 74: Unified Communications Troubleshooting

74© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

StationD: 000000007 CallState callState=4 lineInstance=1 callReference=16777218

Trace and Debug for Call Flow #2,cont…

- CM rings phone with DN 2001 and changes the display.

StationD: 000000007 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=5(LampBlink)StationD: 000000007 SetRinger ringMode=3(OutsideRing)StationD: 000000007 DisplayPromptStatus timeOutValue=0 promptStatus='€ 4085551212'

content='From 4085551212' lineInstance=1 callReference=16777218 ver=0x84000005StationD: 000000007 SelectSoftKeys instance=1 reference=16777218 softKeySetIndex=3

validKeyMask=-1

- CM sends a call proceeding message.Out Message -- H225CallProceedingMsg -- Protocol= H225ProtocolIsdnMsgData2= 08 02 80 07 02 7E

- This is followed by an alerting message from CM.Out Message -- H225AlertMsg -- Protocol= H225ProtocolIsdnMsgData2= 08 02 80 07 01

- On the IOS voice gateway, we see the following Q931 messages.00:42:12: ISDN Se0/0:23: TX -> CALL_PROC pd = 8 callref = 0x800800:42:12: Channel ID i = 0xA9839700:42:12: ISDN Se0/0:23: TX -> ALERTING pd = 8 callref = 0x8008

Page 75: Unified Communications Troubleshooting

75© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Trace and Debug for Call Flow #2,cont…

H.323Gateway

CiscoCallManager

Cisco IPPhone

Station Set Lamp (Blink)Station Set Ringer (On)

Station Off Hook

Station Set Lamp (Steady)

Station Set Ringer (Off)

Station Call Info

H.225 Setup

H.225 Setup Ack

H.225 Alerting

H.225 Connect

H.245 Master/Slave Determination

H.245 Master/Slave Determination Ack

H.245 Terminal Capabilities Set

H.245 Terminal Capabilities Set Ack

Page 76: Unified Communications Troubleshooting

76© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Trace and Debug for Call Flow #2,cont…

- IP phone with DN 2001 goes offhook to answer the call and CM changes the display.

StationInit: 000000007 OffHookStationD: 000000007 SetRinger ringMode=1(RingOff)StationD: 000000007 SetLamp stimulus=9(Line) stimulusInstance=1 lampMode=2(LampOn)StationD: 000000007 CallState callState=1 lineInstance=1 callReference=16777218StationD: 000000007 ActivateCallPlane lineInstance=1StationD: 000000007 SelectSoftKeys instance=1 reference=16777218 softKeySetIndex=1

validKeyMask=-1StationD: 000000007 DisplayPromptStatus timeOutValue=0 promptStatus='€ ' content='Connected'

lineInstance=1 callReference=16777218 ver=0x84000005- Connect message is send to the PSTN.Out Message -- H225ConnectMsg -- Protocol= H225ProtocolIsdnMsgData2= 08 02 80 07 07 7E- The following Q931 messages are seen on the IOS voice gateway.00:42:19: ISDN Se0/0:23: CALL_PROGRESS: CALL_CONNECTED call id 0x8, bchan 22, dsl 000:42:19: ISDN Se0/0:23: TX -> CONNECT pd = 8 callref = 0x800800:42:19: ISDN Se0/0:23: RX <- CONNECT_ACK pd = 8 callref = 0x0008- At this point, the capabilities exchange occurs (codec, silence suppression, etc.). I have not

included the full capabilities exchanges.H245ASN - TtPid=(1,100,108,1) -Incoming -value MultimediaSystemControlMessageH245ASN - TtPid=(1,100,108,1) -Outgoing -value MultimediaSystemControlMessage

Page 77: Unified Communications Troubleshooting

77© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Trace and Debug for Call Flow #2,cont…

H.323Gateway

CiscoCallManager

Cisco IPPhone

Station Set Lamp (Blink)

Station Call Info

H.225 Setup

H.225 Setup Ack

Station Set Ringer (On)

Station Off Hook

Station Set Lamp (Steady)

Station Set Ringer (Off)

Station Start Media Reception

Station Start Media Transmission

H.225 Alerting

H.225 Connect

H.245 Master/Slave Determination

H.245 Master/Slave Determination Ack

H.245 Terminal Capabilities Set

H.245 Terminal Capabilities Set Ack

H.245 Open Logical Channel

H.245 Open Logical Channel Ack

Conversation

Page 78: Unified Communications Troubleshooting

78© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Trace and Debug for Call Flow #2,cont…

- CM and Gateway send Open Logical Channel to the gateway and OpenReceiveChannel tothe IP phone to establish RTP between the two devices.

H245ASN - TtPid=(1,100,108,1) -Outgoing -value MultimediaSystemControlMessage ::= request :openLogicalChannel :

H245ASN - TtPid=(1,100,108,1) -Incoming -value MultimediaSystemControlMessage ::= request :openLogicalChannel :

StationD: 000000007 StopTone

StationD: 000000007 OpenReceiveChannel conferenceID=0 passThruPartyID=11 millisecondPacketSize=20compressionType=11(Media_Payload_G729) qualifierIn=?. myIP: 6c00000a (10.0.0.108)

H245ASN - TtPid=(1,100,108,1) -Incoming -value MultimediaSystemControlMessage ::= response :openLogicalChannelAck :

H245Interface(1) OLC outgoing confirm ip = 300010a, port = 18590

StationInit: 000000007 OpenReceiveChannelAck Status=0, IpAddr=0x6c00000a, Port=28816, PartyID=17

StationD: 000000007 StartMediaTransmission conferenceID=0 passThruPartyID=11remoteIpAddress=300010a(10.1.0.3) remotePortNumber=18590 milliSecondPacketSize=20compressType=11(Media_Payload_G729) qualifierOut=?. myIP: 6c00000a (10.0.0.108)

H245ASN - TtPid=(1,100,108,1) -Outgoing -value MultimediaSystemControlMessage ::= response :

openLogicalChannelAck :

H245Interface(1) paths established ip = 6c00000a, port = 28816

Page 79: Unified Communications Troubleshooting

79© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Trace and Debug for Call Flow #2,cont…

H.323Gateway

CiscoCallManager

Cisco IPPhone

Station Set Lamp (Blink)

Station Call Info

H.225 Setup

H.225 Setup Ack

Station Set Ringer (On)

Station Off Hook

Station Set Lamp (Steady)

Station Set Ringer (Off)

Station Start Media Reception

Station Start Media Transmission

H.225 Alerting

H.225 Connect

H.245 Master/Slave Determination

H.245 Master/Slave Determination Ack

H.245 Terminal Capabilities Set

H.245 Terminal Capabilities Set Ack

H.245 Open Logical Channel

H.245 Open Logical Channel Ack

ConversationStation On Hook

Station Stop Media Transmission

Station Stop Media Reception

Station Set Lamp (Off)

Page 80: Unified Communications Troubleshooting

80© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Trace and Debug for Call Flow #2,cont…

- IP phone 2001 hangs up so CM tears down the call and changes the phone display.StationInit: 000000007 OnHookStationD: 000000007 CallState callState=2 lineInstance=1 callReference=16777218StationD: 000000007 SelectSoftKeys instance=0 reference=0 softKeySetIndex=0 validKeyMask=-1StationD: 000000007 DisplayPromptStatus timeOutValue=0 promptStatus='€ ' content='Your current

options' lineInstance=0 callReference=0 ver=0x84000005StationD: 000000007 CloseReceiveChannel conferenceID=0 passThruPartyID=11. myIP: 6c00000a

(10.0.0.108)StationD: 000000007 StopMediaTransmission conferenceID=0 passThruPartyID=11. myIP: 6c00000a

(10.0.0.108)

- CM sends a H245 request to close logical channel to the voice gateway and receives aclose logical channel ack.

H245ASN - TtPid=(1,100,108,1) -Outgoing -value MultimediaSystemControlMessage ::= request :closeLogicalChannel :

H245ASN - TtPid=(1,100,108,1) -Incoming -value MultimediaSystemControlMessage ::= response :closeLogicalChannelAck :

- CM sends a H245 disconnect.H245ASN - TtPid=(1,100,108,1) -Outgoing -value MultimediaSystemControlMessage ::= command :

endSessionCommand : disconnect :H245ASN - TtPid=(1,100,108,1) -Incoming -value MultimediaSystemControlMessage ::= command :

endSessionCommand : disconnect :

Page 81: Unified Communications Troubleshooting

81© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Trace and Debug for Call Flow #2,cont…

- CM then sends out a ReleaseComplete message.Out Message -- H225ReleaseCompleteMsg -- Protocol= H225Protocol

IsdnMsgData2= 08 02 80 07 5A

- The following Q931 message is seen on the IOS gateway.00:42:24: ISDN Se0/0:23: TX -> DISCONNECT pd = 8 callref = 0x8008

00:42:24: Cause i = 0x8290 - Normal call clearing

00:42:25: ISDN Se0/0:23: RX <- RELEASE pd = 8 callref = 0x0008

00:42:25: ISDN Se0/0:23: TX -> RELEASE_COMP pd = 8 callref = 0x8008

Page 82: Unified Communications Troubleshooting

82© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Forwarding Calls

What do I look for when calls are forwarded to another extension?- Look for ForwardManager activities, right after the initial digit analysis of the

original called number. Then you should see what kind of forwarding it is(CFA, CFNA, CFB) with forwarded number. Lastly you should see CMperforming digit analysis again for the forwarded number.

In this case, 2000 called 2001, 2001 has CFA set to 2002.Digit analysis: match(fqcn="2000", cn="2000", pss="Line 1:Line

2:Local_Dial:Long_Distance_Dial:International_Dial:Internal", dd="2001")ForwardManager - wait_SsInterceptInd Received - Party= 0x1000038, InterceptKey= 0x2ForwardManager - findInterceptTableEntry(ssKey) - Found Intercept table entry for dn= 2001:Line 1,

InterceptKey= 0x2,0x2ForwardManager::findActivationEntryBySsPartyForwardManager::getNextForwardActiveCallIndexForwardManager - wait_SsInterceptInd - New Forward or Call Pickup started for Dn= 2001:Line 1. Party=

0x1000038, InterceptKey= 0x2, CallKey= 0x1Forwarding::callForwardAllSetForwarding - awaitForwardInitiation_SsInterceptInd - 1 CFA attempted. CallKey= 0x1Forwarding::callForwardAllDestinationAvailableForwarding::redirectCallToForwardAllDestinationForwarding - redirectCallToForwardAllDestination - Last Forward - Dn= 2002, CallKey= 0x1

Page 83: Unified Communications Troubleshooting

83© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting

Forwarding Calls… cont

- Digit analysis is then performed on the forwarded call. The traceis similar to call flow 1 presented earlier.

Digit analysis: match(fqcn="2000", cn="2000", pss="Internal:Line 2:Line1:Local_Dial:Long_Distance_Dial", dd="2002")

Page 84: Unified Communications Troubleshooting

84© 2010 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialUC Troubleshooting