sum405: troubleshooting and debugging receiver for ios and android christian suarez sr. escalation...

SUM405: Troubleshooting and Debugging Receiver for iOS and Android

Christian Suarez

Sr. Escalation Engineer

May 21, 2013

© 2013 Citrix

Why Mobility?

2

© 2013 Citrix

Why Now?

3

© 2013 Citrix

A fast growing, rapidly changing mobile market

Source: Gartner (September 2011)Source: Gartner (January 2012)

© 2013 Citrix

How do you find the problem?

© 2013 Citrix

Devices Network Web Resources

Access Gateway

Third PartyVPN

StoreFront

Web Interface

Citrix Mobile Infrastructure

01011 01011

01011 01011

© 2013 Citrix

Agenda

8

NEW!

Network and ICA Connection Troubleshooting

© 2013 Citrix

Isolate the Issue

10

Detailed Reproduction Steps•XenApp / XenDesktop?

Resources

© 2013 Citrix

Isolate the Issue

11

Reproducible Environment•XenApp / XenDesktop•StoreFront / Web Interface / Services Site

Web

StoreFront

Web InterfaceServices Site

© 2013 Citrix

Isolate the Issue

12

Reproducible Environment•XenApp / XenDesktop•StoreFront / Web Interface / Services Site•User-based or Device-based•Frequency of the Issue

© 2013 Citrix

Detailed Reproduction

13

Repro Steps•Clear Sequential Repro Steps with timing

(Start Tracing at 6:09pm on 4/2)1.Logon to WI from Browser at 6:102.Launch XenApp Desktop at 6:11

•Expected Behavior3.Hear Windows logon sound

•Unexpected Behavior3.Cannot hear Windows logon sound at 6:12

© 2013 Citrix

External Reproduction

14

Complex backend or proprietary software•Provide Store Account configuration•Test user account•Security Token

requirements present

a challenge

© 2013 Citrix

Test Other Citrix Receivers

15

•Previous Receiverᵒ Possible Regression

•Beta / EAP Receiverᵒ Already Fixed

•From Browser or Receiver•Android vs iOS•Windows Receiver

© 2013 Citrix

Network Troubleshooting

16

Checking the Network•Netscaler AG- VPN or ICA Proxy?

ᵒ Other VPN

•Local Users or External Users only

•3/4G or Wireless Network

•Single or Dual Authentication

•Public or Private CA Certificates

•Intermediate Certificate on the Gateway

Network

Access Gateway

Third PartyVPN

© 2013 Citrix

Network Troubleshooting

17

Identify Network devices within the connection•Look for unique or custom configurations or design•Check for third party logs or tracing•Identify Closest Working Scenario

Collect Working and Non-working Network Traces

© 2013 Citrix

ICA Enumeration and Connection

18

• ICA is Citrix’s primary TCP Protocol - 1494 ᵒ ICA wrapped in CGP / Session Reliability – 2598

•Determine Enumeration or Connectionᵒ Authentication in Enumerationᵒ Session Launch in Connection

•Create Account/Store in Mobile to Enumerateᵒ Enumerate via Web Interface each time

•Launch App or Desktop from Browser or Receiver

© 2013 Citrix

Network Traces Analysis Tips

•Filter by Regular Expressionsᵒ tcp.port == 1494 || tcp.port == 2598ᵒ ip.addr == 10.12.33.105 && ip.addr == 10.12.26.144

Finding the ICA Session

© 2013 Citrix


•Filter by Regular Expressionsᵒ tcp.port == 1494 || tcp.port == 2598ᵒ ip.addr == 10.12.33.105 && ip.addr == 10.12.26.144

• Add Name Resolution• Use Display Time Format


© 2013 Citrix


•Find TCP Handshake – SYN, SYN-ACK, ACK•Use Follow TCP Stream•Verify the ICA Sounder


© 2013 Citrix


•Time issue occurred during trace•Use Expert Info Features •Look for Network Saturation•Look for Retransmissions and Resets•Find which side of conversation initiates the problem•Compare with a working trace

Identifying Issues

© 2013 Citrix

Network Traces with Netscaler AG

•nstrace.sh command with nsroot access ᵒ Secure Shell (SSH) access with tool like PuTTY or

Secure Consoleᵒ Secure Copy (SCP) download with tool like WinSCP or

SFTP

•Netscaler’s Web Consoleᵒ Start new trace under System-> Diagnostics

•Decrypt a SSL Trace with Private Key

© 2013 Citrix

Other Data for ICA Connection Issues

24

•Event Logs of XenApp or XenDesktop hostᵒ Licensing Issuesᵒ Authentication Issues

•Event Logs of Authenticating Serverᵒ Web Interface or XML Broker

•STA Logs in CtxSta.config for Ticketing Issues•Enable Web Interface tracing in web.config file • IIS Logs for Web Site issues

© 2013 Citrix

Citrix Diagnostic Facility (CDF) Traces

25

•Key Citrix Diagnostic Data requires parsing•CDF is based off Event Trace for Windows (ETW)ᵒ Providers, Controllers and Consumers

• Identify which XenApp Server to Traceᵒ Citrix XenApp Server and XenDesktop Host

• Session Launching issuesᵒ Zone Data Collector and XML Broker

• Application Enumeration or Load-Balancing issues

© 2013 Citrix

Capturing CDF Traces

26

•Steps to gather CDF Tracesᵒ Scout - CTX130147ᵒ CDF Control – CTX111961

•Use Circular Traces for Intermittent issues•Provide Citrix Technical Support with .etl trace file

Citrix Diagnostic Facility

http://support.citrix.com/article/CTX130147


CDF Trace Demo

© 2013 Citrix

Data Collected

•Network Traces – .pcap, .cap•Network IP Info

© 2013 Citrix

Data Collected

•Network Traces – .pcap•Network IP Info•CDF Traces - .etl•Repro Info

© 2013 Citrix

Data Collected

•Network Traces – .pcap•Network IP Info•CDF Traces - .etl•Repro Info•Event Logs - .csv

LongCommandLine parameter fails for iOS and Android Case Study #1

© 2013 Citrix


33

•Troubleshootingᵒ Occurs on Receivers for iOS, Android and Mac not Windowsᵒ Occurs only when session sharingᵒ Happens from Local, External, Web Interface or Storefront

ICA FILE…[IE] Address=10.12.33.101:1494 InitialProgram=#IE LongCommandLine=http://www.citrix.com/lang/English/home.asp ClientAudio=On AudioBandwidthLimit=1

© 2013 Citrix


34

•Debuggingᵒ Obtained CDF Traces with successful and failed launchᵒ Found launch attempts with Enter:HostLaunchRequest()ᵒ Successful launch

© 2013 Citrix


35

•Debuggingᵒ Obtained CDF Traces with successful and failed launchᵒ Found launch attempts with Enter:HostLaunchRequest()ᵒ Failed launch

© 2013 Citrix


36

•Resultsᵒ Discovered defect in all non-windows Receiversᵒ Fixed for iOS, Android, Linux and Mac

ICA FILE…[IE] Address=10.12.33.101:1494 InitialProgram=#IE LongCommandLine=http://www.citrix.com/lang/English/home.asp ClientAudio=On AudioBandwidthLimit=1

Advanced Logs forReceiver for iOS

© 2013 Citrix

Introduction to Advanced Logs on iOS

38

•Advanced Logs introduced in Receiver for iOS 5.7•AeTracing – Advanced Extensible Tracing•Plain Text Log File •Time and Date Stamp - dd-mm-yyyy hh:mm:ss•Message Types- INFO, WARN, VERBOSE, ERROR

© 2013 Citrix

Enabling Advanced Logs

39

•Enable Advanced Logs in the Receiver Settings

© 2013 Citrix

Two Ways of Gathering Advanced Logs

41

• Send Feedback and Request Help from Support

• From Synced iTunes under File Sharing

© 2013 Citrix

Send Feedback and Request Help from Support

42

© 2013 Citrix

Email

43

© 2013 Citrix

From Synced iTunes under File Sharing

44

© 2013 Citrix

Advanced Logs

45

•Advanced Logs disabled after Send Feedback•Find Issue within all compressed Advanced Logs•Keywords to search for issuesᵒ ERRORᵒ NOTᵒ FAIL

© 2013 Citrix

Supportability for Citrix Receivers

See the Citrix Receiver Feature Matrix

http://www.citrix.com/content/dam/citrix/en_us/documents/products/citrix-receiver-feature-matrix.pdf




© 2013 Citrix

Supportability for iOS

iOS Receiver Admin Troubleshooting•Workspace control feature is not supported•Connecting with a proxy is not supported•Applications might open in different sessions

Limited Support for Session Reliability

✗ 2598/CGP ✗ Client Buffer ✔ Session Freeze

Advanced Logs Messages

© 2013 Citrix

Discover Store config by email

VERBOSE: -[CRLaunchPadViewController changeStore:]

Advanced Logs Analysis

© 2013 Citrix

Enumerating Apps from Web Interface

INFO: -[StoreManager getMobileAppEnumForStore


© 2013 Citrix

Select Published Desktop icon

VERBOSE: -[CRLaunchPadViewController appClicked:]


© 2013 Citrix

Published Desktop ICA File

INFO: -The ICA: [Encoding]


© 2013 Citrix

ICA Session begins

INFO: setSessionViewWasShown


© 2013 Citrix

ICA Session Display Settings

INFO: Session Width:


© 2013 Citrix

User Notification Messages

INFO: updateLaunchStatus: statusMessage:


© 2013 Citrix

Session successfully loaded

INFO: Settings succesfully loaded for session view


© 2013 Citrix

Receiver returns to main screen

INFO: updateVideoOut


© 2013 Citrix

User Disconnect Session

VERBOSE: closeSession:


© 2013 Citrix

iOS Receiver crash dump

• iOS uses CrashReporter• .crash report includes Stack Trace•Located on Synced iTunes machineᵒ Mac - ~\Library\Logsᵒ Win - \AppData\Roaming\Apple computer\Logs\CrashReporter\

© 2013 Citrix

iOS Data Collected

•Advanced Logs - .zip of .txt•Repro timing info

© 2013 Citrix

iOS Data Collected

•Advanced Logs - .zip of .txt•Repro timing info•Crash Reporter - .crash and .log

iPad users are unable to connect through Access GatewayCase Study #2

© 2013 Citrix


65

•Troubleshootingᵒ Occurs with iOS but not Windows Receiversᵒ Only through Access Gateway not for Internal Usersᵒ SHA2 Certificates are on the device

Access Gateway

© 2013 Citrix


66

•Debuggingᵒ No connection attempt from VDA, so no CDF Traces ᵒ Advanced Logs captured during the failure

© 2013 Citrix


67

•DebuggingYou have not chosen to trust "Entrust Certification Authority - L1C", the issuer of the server's security certificate.

© 2013 Citrix


68

•Resultsᵒ Checked the Citrix Client Feature Matrixᵒ FIPS/SHA2 Security is a Limitation in the Receiver for iOS

iOS Receiver disconnects issues after timeoutCase Study #3

© 2013 Citrix


70

•Troubleshootingᵒ Occurs only on Receivers for iOSᵒ Only happens with Access Gateway connections

© 2013 Citrix


71

•Debuggingᵒ Obtained Advanced Logs from iOS Receiver

This app is no longer available from the server or you are no longer permitted to access it

© 2013 Citrix


72

•Resultsᵒ Discovered defect caching the credentials on the Receiverᵒ Fixed the code and the user now returns to logon screen

after timeout

Logcat with Receiver for Android

© 2013 Citrix

Introduction to LogCat on Android

74

•Android uses LogCat to dump system debug output ᵒ Using Android’s Log class

•Android Receiver always sends output to LogCat•Plain Text Log File •Time and Date Stamp - dd-mm-yyyy hh:mm:ss•Problematic Keywords:ᵒ ERROR, FATAL EXCEPTION, FAIL

© 2013 Citrix

Logcat Message Tags

75

V/ - verbose

D/ - debug

I/ - informational

W/ - warning

E/ - error

F/ - fatal

S/ - silent

© 2013 Citrix

Two Ways of Gathering LogCat on Android

76

•Send Feedback and Request Help from Support•From Third Party Utility

© 2013 Citrix

Collect LogCat from Receiver for Android

77

•Send Feedback and Request Help from Support

From Receiver for Android

© 2013 Citrix

aLogcat on Android

82

•aLogcat is a third party app for collecting LogCat logs•Regular expression filteringᵒ com.citrix&E/

•Citrix Receiver filterᵒ com.citrix

•Send logs from aLogcat ᵒ Email, ShareFile and more

Logcat utility

© 2013 Citrix

Supportability of Receiver for Android

Android Receiver Admin Troubleshooting•Color depth limitation for sessions•Workspace control feature is not supported•Connecting with a proxy is not supported

Full Support for Session Reliability

✔ 2598/CGP ✔ Client Buffer ✔ Session Freeze

LogCat Messages

Unknown Error from Android connecting through AG the second timeCase Study #3

© 2013 Citrix


94

•Troubleshootingᵒ Occurs only on Receivers for Androidᵒ Occurs after one successful connection

Access Gateway

© 2013 Citrix


95

•Debuggingᵒ Network Traces checked outᵒ Obtained LogCat from Android Receiver

Receiver 403 response from gateway…

© 2013 Citrix


96

•Results• Defect discovered in Android Receiver• Resolved in Receiver for Android 3.1.2

Access Gateway

Wrap it up!

© 2013 Citrix

Citrix Connection Troubleshooting

98

• Isolate Reproduction for Data Collection•Separate failing components•Gather and Investigate Diagnostic Data ᵒ Event Logs, Network Tracesᵒ Citrix CDF Tracing

•Note the IP info during Network Traces•Focus on Working vs. Non-working Data

© 2013 Citrix

Logging for iOS and Android

99

•Mobile Receiver logging is quick and easy•Readable text messages•Search logs for ICA Session Starting Pointᵒ iOS- VERBOSE & CRLaunchPadViewControllerᵒ Android- CTX|CITRIX|RECEIVER.COM

© 2013 Citrix

Logging for iOS and Android

100

•Search logs for problematic keywordsᵒ iOS- ERROR, NOT, FAILᵒ Android- ERROR, FAIL, FATAL EXCEPTION

•Search questionable messages in working logs•Provide Concise Repro Steps•Collect details and data for Citrix Technical Support

© 2013 Citrix

Resources

101

Troubleshooting Citrix Receiver for Mobile Devices


How to Record a Network Packet Trace on a NetScaler Appliance


Case Study: Troubleshooting ICA Session Disconnection with CDF and Network Traces


How to Enable and Collect Advanced Logs for Receiver for iOS


Case Study: Unable to Launch Applications from Android Device












© 2013 Citrix

Before you leave…

• Recommended related breakout sessions: ᵒ SUM212: How to create a fabulous user experience with Citrix Receiver

ᵒ SUM604: Enterprise mobility planning and design workshop

• Conference surveys are available online at www.citrixsynergy.com starting Friday, May 24 at 9:00 a.m. PTᵒ Provide your feedback by 4:00 p.m. PT that day and you’ll receive a $30

Amazon.com gift card via email

• Download presentations starting Monday, June 3, from your My Conference Planning tool located within the My Account section

http://www.citrixsynergy.com/

Questions & Answers

Work better. Live better.Work better. Live better.

sum405: troubleshooting and debugging receiver for ios and android christian suarez sr. escalation...

Documents

issue slide

citrix network troubleshooting

citrix test

challenge slide

citrix receivers

citrix agenda

ica session slide

citrix ica enumeration