citrix internals: tracing, debugging & troubleshooting

Citrix Internals: Tracing,

Debugging &

TroubleshootingDenis Gundarev

Senior Consultant

Entisys Solutions

About me

C:\>whoami /all

USER INFORMATION

----------------

User Name Twitter E-Mail

============== ============ ==================

ENTISYS\denisg @fdwl DenisG@entisys.com

GROUP INFORMATION

-----------------

Group Name Type SID

======================================== ================ =================

BUILTIN\Geeks Mandatory group S-1-5-32-540

Mandatory Label\Crazy Russian Label S-1-16-8192

COMMUNITY\Bay Area Citrix User Group Well-known group S-1-5-32-544

COMMUNITY\Citrix Technology Professional Well-known group S-1-5-32-545

Not a basic “Restart the computer” stuff

Do not restart the computer if you going to troubleshoot the issue

Not a CDF tracing tutorial

Search for TechEdge presentations on citrix.com

Not a crash dump analysis workshop

Check Dmitry Vostokov’s http://www.dumpanalysis.org/

What’s the topic?

Shaman’s Guide to Troubleshooting

Phase 1 Phase 2 Phase 3

Identify

the Issue

Profit

StoreFront Troubleshooting

250+ articles about Web Interface

2000+ articles about XenApp

59 articles about XenDesktop 7.5

13 articles about StoreFront 2.5

support.citrix.com

Generic Errors in StoreFront

Using abracadabra, alakazam

and sim sala bim for Citrix troubleshooting

Step 1: Check event logs

StoreFront (as well as most of the XenDesktop components) is based on Windows

Communication Foundation (WCF)

WCF services communicate with each other using endpoints

Endpoint configuration and listening address defined in the .config file

Storefront Internals

Step 2: Search files for text

PS C:\>Get-ChildItem -Recurse *.config|Select-String "net.pipe://localhost/Citrix/Authentication/Win32"

Select-Object -Property Path, LineNumber

|

|Out-GridView -OutputMode Multiple |%{notepad $_.Path}

Some service is causing this issue. Which

one?

Step 3: Search for service by executable path

PS C:\>Get-WmiObject -Class Win32_Service -Filter "pathname like '%defaultdomainservices%'"

Select-Object -Property Displayname, PathName, Started

|

|Out-GridView

Generic Errors in StoreFront

Step 1: Check event logs

Step 2: Search files for text

PS C:\>Get-ChildItem -Recurse *.config|Select-String "loadbalancer.corp.itbubble.ru"

Select-Object -Property Path, LineNumber

|

|Out-GridView -OutputMode Multiple |%{notepad $_.Path}

Step 2: Search files for text

PS C:\>Get-ChildItem -Recurse *.config|Select-String "loadbalancer.corp.itbubble.ru"

Select-Object -Property Path, LineNumber

|

|Out-GridView -OutputMode Multiple |%{notepad $_.Path}

Fiddler

Generic Errors in StoreFront

Fiddler

Set-Cookie: NSC_ttmx=ffffffffc3a01fa645525d5f4f58455e445a4a423660;expires=Sun, 14-Jul-2013 13:46:47 GMT;path=/;secure;httponly

Set-Cookie: NSC_ttmx=ffffffffc3a01fa245525d5f4f58455e445a4a423660;expires=Sun, 14-Jul-2013 13:47:15 GMT;path=/;secure;httponly

Set-Cookie: NSC_ttmx=ffffffffc3a01fa645525d5f4f58455e445a4a423660;expires=Sun, 14-Jul-2013 13:47:15 GMT;path=/;secure;httponly

Set-Cookie: NSC_ttmx=ffffffffc3a01fa245525d5f4f58455e445a4a423660;expires=Sun, 14-Jul-2013 13:47:15 GMT;path=/;secure;httponly

Fiddler

Receiver for Web

Receiver for Web

Reverse Engineering

Assembly browsing

IL Disassembly

Decompilation to C#

Decompilation to VB

Find usage of field/method

Available at http://ILSpy.net

Free

ILSpy

Unhandled exception error when browsing for an application in Delivery Services Console

or AppCenter

System.IO.DirectoryNotFoundException: The system cannot find the path specified. (Exception from HRESULT: 0x80070003)

at IWshRuntimeLibrary.IWshShortcut.Save()

at Citrix.CMI.PSE.Utils.FileBrowsing.CreateServerLinks(FileDialog dialog, ServerSet serverSet, ModifyShareNameCallback

modifyShareNameCallback)

at Citrix.CMI.PSE.Cmo.App.PropertyPages.InstalledICALocationPage.u_browseCommandLineButton_Click(Object sender, EventArgs e)

at System.Windows.Forms.Control.OnClick(EventArgs e)

at System.Windows.Forms.Button.OnClick(EventArgs e)

at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)

at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)

at System.Windows.Forms.Control.WndProc(Message& m)

at System.Windows.Forms.ButtonBase.WndProc(Message& m)

at System.Windows.Forms.Button.WndProc(Message8; m)

at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message&; m)

at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)

at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

Cause:

Customer have Folder redirection implemented via GPO and "Links" special folder is

redirected.

Citrix Management console doesn't handle folder redirection and assume that

%USERPROFILE%\Links folder exists.

private static readonly string FavoriteLink =

Path.Combine(Path.Combine(Environment.GetEnvironmentVariable("USERPROFILE"), "Links"), Services.ResString.GetGlobal("$Profile_Links_Shortcut_Name") + ".lnk");

Search Files for Text

PS C:\>Get-ChildItem -Recurse *.* -ErrorAction SilentlyContinue|Select-String "XenDesktop is not installed"

Select-Object -Property Path

|

C:\Windows\assembly\NativeImages_v2.0.50727_32\Citr

ix.GroupPolicy.#\a4a323e135cabd1324a46585b70ee461

\Citrix.GroupPolicy.Filters.ni.dll

CTX_DLL_Citrix.VirtApp.PowerShellSnapIn:8:5:Exit: Get-CtxAppVServer::DecodeAccount

CTX_DLL_Citrix.VirtApp.PowerShellSnapIn:8:5:Exit: Get-CtxAppVServer::ProcessRecord

CTX_DLL_Citrix.VirtApp.PowerShellSnapIn:8:5:Entry: Get-CtxAppVApplication::ProcessRecord

CTX_DLL_CtxAppVStudio:8:5:Entry: AppVManager::AddServer 'av01.corp.itbubble.ru'

CTX_DLL_CtxAppVStudio:8:5:Entry: AppVServerTimestamp::AppVServerTimestamp for Server: av01.corp.itbubble.ru and IsManagementServer: True

CTX_DLL_CtxAppVStudio:8:5:Entry: AppVServerDataManager::AppVServerDataManager for Server : av01.corp.itbubble.ru and IsManagementServer: True

CTX_DLL_CtxAppVCommon:8:5:Entry: ServerPoshManager::ServerPoshManager

CTX_DLL_CtxAppVCommon:8:5:Exit: ServerPoshManager::ServerPoshManager

CTX_DLL_CtxAppVCommon:8:5:Entry: ServerPoshManager::Initialize

App-V integration in XD

$a=get-appvserverpackage $output = @() $cnenabledpackage = 0 for($i=0;$i -lt $a.count;$i++) { if($a[$i].Enabled -eq $false -or $a[$i].Entitlements.GroupName.Count -eq 0 -or $a[$i].applications.count -eq 0) { #skip disabled apps. continue } $pakagedata = New-Object PSObject | select Id, PackageName, PackageGuid , VersionGuid, Enabled, PackageLocation , AppName, AppId, Users , DisabledAppIds $output += $pakagedata $output[$cnenabledpackage].AppName= $a[$i].applications.name $output[$cnenabledpackage].PackageGuid = $a[$i].PackageGuid $output[$cnenabledpackage].PackageName = $a[$i].Name $output[$cnenabledpackage].AppId= $a[$i].applications.appid $output[$cnenabledpackage].Users = $a[$i].Entitlements.GroupName $output[$cnenabledpackage].PackageLocation = $a[$i].PackageUrl $output[$cnenabledpackage].Id = $a[$i].Id $output[$cnenabledpackage].VersionGuid = $a[$i].VersionGuid $output[$cnenabledpackage].Enabled = $a[$i].Enabled

Java Decompiler - http://jd.benow.ca/

In any event, you may not modify, translate, reverse engineer, decompile, disassemble, create derivative works based on, or copy the SOFTWARE, except as specifically licensed herein or to the extent such foregoing restriction is expressly prohibited by applicable law.

Is it Legal?

“

”

Citrix XenDesktop 7 EULA

Are there things I’m not allowed to do with the software? Yes. Because the software is licensed, not sold, Microsoft reserves all rights ..... not expressly granted in this agreement. In particular, this license does not give you any right to, and you may not: ..... reverse engineer, decompile, or disassemble the software, except if the laws where you live permit this even when our agreement does not. In that case, you may do only what your law allows. .....

Microsoft Windows 8 EULA

In any event, you may not modify, translate, reverse engineer, decompile, disassemble, create derivative works based on, or copy the SOFTWARE, except as specifically licensed herein or to the extent such foregoing restriction is expressly prohibited by applicable law.

Are there things I’m not allowed to do with the software? Yes. Because the software is licensed, not sold, Microsoft reserves all rights ..... not expressly granted in this agreement. In particular, this license does not give you any right to, and you may not: ..... reverse engineer, decompile, or disassemble the software, except if the laws where you live permit this even when our agreement does not. In that case, you may do only what your law allows. .....

In any event, you may not modify, translate, reverse engineer, decompile, disassemble, create derivative works based on, or copy the SOFTWARE, except as specifically licensed herein or to the extent such foregoing restriction is expressly prohibited by applicable law.

Are there things I’m not allowed to do with the software? Yes. Because the software is licensed, not sold, Microsoft reserves all rights ..... not expressly granted in this agreement. In particular, this license does not give you any right to, and you may not: ..... reverse engineer, decompile, or disassemble the software, except if the laws where you live permit this even when our agreement does not. In that case, you may do only what your law allows. .....

Sec. 103(f) of the DMCA (17 U.S.C. § 1201 (f)): (f) Reverse Engineering.—

(1) Notwithstanding the provisions of subsection (a)(1)(A), a person who has lawfully obtained the right to use a copy of a computer program may circumvent a technological measure that effectively controls access to a particular portion of that program for the sole purpose of identifying and analyzing those elements of the program that are necessary to achieve interoperability of an independently created computer program with other programs, and that have not previously been readily available to the person engaging in the circumvention, to the extent any such acts of identification and analysis do not constitute infringement under this title.

(2) Notwithstanding the provisions of subsections (a)(2) and (b), a person may develop and employ technological means to circumvent a technological measure, or to circumvent protection afforded by a technological measure, in order to enable the identification and analysis under paragraph (1), or for the purpose of enabling interoperability of an independently created computer program with other programs, if such means are necessary to achieve such interoperability, to the extent that doing so does not constitute infringement under this title.

(3) The information acquired through the acts permitted under paragraph (1), and the means permitted under paragraph (2), may be made available to others if the person referred to in paragraph (1) or (2), as the case may be, provides such information or means solely for the purpose of enabling interoperability of an independently created computer program with other programs, and to the extent that doing so does not constitute infringement under this title or violate applicable law other than this section.

(4) For purposes of this subsection, the term 「interoperability」means the ability of computer programs to exchange information, and of such programs mutually to use the information which has been exchanged.

Article 6 of the 1991 EU Computer Programs Directive (15) The unauthorised reproduction, translation, adaptation or transformation of the form of the code in which a copy of a computer program has been made available constitutes an infringement of the

exclusive rights of the author. Nevertheless, circumstances may exist when such a reproduction of the code and translation of its form are indispensable to obtain the necessary information to achieve the interoperability of an independently created program with other programs. It has therefore to be considered that, in these limited circumstances only, performance of the acts of reproduction and translation by or on behalf of a person having a right to use a copy of the program is legitimate and compatible with fair practice and must therefore be deemed not to require the authorisation of the rightholder. An objective of this exception is to make it possible to connect all components of a computer system, including those of different manufacturers, so that they can work together. Such an exception to the author's exclusive rights may not be used in a way which prejudices the legitimate interests of the rightholder or which conflicts with a normal exploitation of the program.

Is it Legal?

“

”

…for the sole purpose of identifying and analyzing

those elements of the program that are necessary to

achieve interoperability of an independently created

computer program with other programs…

…to obtain the necessary information to achieve

the interoperability of an independently created

program with other programs….

Use PowerShell for searching

Analyze config files

Use Fiddler to sniff StoreFront traffic

Use ILSpy

Conclusion

@fdwl

denisg@entisys.com

http://meetup.com/BayCUG

http://blog.itbubble.ru

Q&A