little known vc++ debugging tricks

Little KnownNative Debugging Tricks

Ofek ShilonAlgorithms Team Leader @ Sarin Technologies

www.thetweaker.wordpress.com

DISCLAIMER

VS AS AN INTERPRETER

Expression Evaluator

• Scope–Watch, QuickWatch,– Immediate,– BP conditions, ‘when hit’

• Call Functions from the debugger!• Demo

Expression Evaluator

• Context Operator– Documentation is wrong.– VS2010 blocked WIN-API, but eased usage of

others.

• Thread context• Access insensitive• Blind to inlined functions

Expression Evaluator

• Things to try:– Very explicit casting,– Explicit addresses.

Edit and Continue

• Demo• Stale code

Edit and Continue• Preparation: /ZI, /INCREMENTAL

Edit and Continue

• Limitations:– Inconsistent with /RELEASE, /OPT:REF,

/OPT:ICF, /ORDER, /FORCE,– Does not invoke custom build steps,– Isn’t supported on attached processes,– Can’t change object layout,– Doesn’t regenerate IDL files,– Can’t change resource files,– Does not build static libs.

CLICK SAVERS

StepOver

• Demo

StepOver

• VC6: autoexp section [ExecutionControl]• VC7: Reg key -

HKCU\Software\Microsoft\VisualStudio\7.0\NativeDE\StepOver

• VC8: HKLM\Software\Microsoft\VisualStudio\8.0\NativeDE\StepOver

• VC10: HKCU\Software\Microsoft\VisualStudio\10.0_Config\NativeDE\StepOverDropped ‘=NoStepInto’ syntax

StepOver

• Warning!–When 2 or more instances of VS2010 are active,

occasionally it would create and use a duplicate reg key:HKCU\Software\Microsoft\VisualStudio\10.0_Config_%PID%(e.g., …\VisualStudio\10.0_Config_6172) … and all ’10.0_Config’ modifications would have no effect !

• Credit: Andy Pennell http://blogs.msdn.com/b/andypennell/archive/2004/02/06/69004.aspx

autoexp

• How’s the debugger ‘aware’ of STL containers?

• autoexp.dat– By default at

%VS folder %\Common7\Packages\Debugger– Provide alternate path in environment variable:

_vcee_autoexp

autoexp

• Demo

• Specialized syntax for common containers: Array, List, Map, Tree.

• Re-parsed on every debugging session• Extremely fragile, yet excellent investment

autoexp

• Status limbo:

autoexp

• Bypass locally: xxx,!• Bypass globally in Tools/Options:

autoexp

• Resources:– Avery Lee

http://virtualdub.org/blog/pivot/entry.php?id=120

– Stephan T. Lavavej BoostCon presentationhttp://filetolink.com/d/?h=a456fc7fd3d35644edcbe5e764d1ba63&t=1315076450&f=17ad36ef

–My blog: complete MFC containers visualization, Matrix 2D visualization..www.thetweaker.wordpress.org

Sound Alerts

• Win7:Control Panel \ Hardware and Sounds \ Sound \ Change system sounds

• WinXP:Control Panel \Sounds and Audio... \Program Events – Microsoft Developer

TRACING ERRORS

GetLastError Interactively

• GetLastError implementation:

• (int*)($fs)+0x34• (int*)($tib)+0x34• $err !• Credit: msdn

http://msdn.microsoft.com/en-us/library/dtw169z6.aspx

Extensions• Customizing ‘,hr’ display in autoexp.dat:

• Other TEB data: (_TEB*)$tib

Breaking on Errors

• BP location: (int*)($tib)+0x34• Specific error: BP condition– Slooooooooow

• g_dwLastErrorToBreakOn !– (int*){,,ntdll.dll}_g_dwLastErrorToBreakOn– Seems to fail for Win7.

• Credit: Daniel Pearson, http://blogs.msdn.com/b/danpear/archive/2007/04/06/2033100.aspx

INVESTIGATING CODE

Breaking on all class methods

• Break at function: Class::*

• Credit: Habib Heydarianhttp://blogs.msdn.com/b/habibh/archive/2009/09/10/class-breakpoint-how-to-set-a-breakpoint-on-a-c-class-in-the-visual-studio-debugger.aspx

Naming Native Threads

• Demo

• ‘Documented hack’Credit: MSDNhttp://msdn.microsoft.com/en-us/library/xcb2z8hs.aspx

Breaking on Data Read

• Write-to BPs are implemented in hardware – dedicated x86 debug registers.

• Read-from debugs registers are available too, but were never exposed in VS.

• Some code required to set the register values to the desired address.

• Links to 4 implementations:http://thetweaker.wordpress.com/2011/05/22/breaking-on-data-read/

Breaking on Data Read

• Usage:

• Interactive:

Static Object Layout

• /d1reportSingleClassLayout• /d1reportAllClassLayout

• Credit: Andy Richhttp://blogs.msdn.com/b/vcblog/archive/2007/05/17/diagnosing-hidden-odr-violations-in-visual-c-and-fixing-lnk2022.aspx

ENHANCED DIAGNOSIS

Useful Pseudo-registers

• $tid-> very useful as a BP condition.

• $env=0-> view debugee environment variables

• $cmdline = 0-> view launching command line

Useful Pseudo-registers

• $handles-> narrows down handle leaks

• $clk-> rough profiling

• $user-> view permissions info

• Credits: Steve Steiner, Gregg Miskelly, et. al.http://blogs.msdn.com/b/stevejs/http://blogs.msdn.com/b/greggm/

Searching Through Memory

• Credit: msdnhttp://msdn.microsoft.com/en-us/library/ms171363.aspx

THANK YOU.

www.thetweaker.wordpress.comofekshilon@gmail.com