exchange issue

Issue 1:System Attendant service is not starting.

Solution.

Check for the event ids in the application log.

Check whether the account used to start the service is a “Local system account” by looking at the properties of SA service from the event viewer.(Eventvwr)

Check whether the account used to login is a “Domain administrator account” by using “SET” or “SET l” command from the DOS Prompt. The account should have the full permissions on the Exchange organization.

Check whether the server object (Exchange server name) has full control permissions on the server container object in the Active directory using Adsiedit console. http://support.microsoft.com/kb/297295

Check whether we have Exchange Enterprise servers group and Exchange Domain servers group listed in the “Users” container from the ADUC console.

Check whether the Exchange domain servers group is a member of Exchange Enterprise servers group.

Check whether the Exchange server is a member of Exchange Domain servers group.

Check whether the Enterprise Exchange server group is a member of “Manage auditing and Security log right” using ADUC. http://support.microsoft.com/kb/314294

Right click on Domain controllers -> properties -> Group policy tab -> Edit - > Computer configuration -> Security - > local policies -> user rights assignment -> Managing audit and security log -> Properties.

Run DomainPrep command from the Exchange CD to reset the permissions and to recreate the security groups for the Exchange in the Active directory.

Setup.exe /domainprep from the run command.

If the problem persists, we need to check the functionality of the

Global catalog servers and the Domain controllers. We need to have atleast one GC server in the site to start the SA service successfully.

To start the Exchange services DNS and Active directory should be inplace.

Use the following commands and tools to test the functionality of Domain controllers and DNS.

Dcdiag /s > output file name to run the command on a member server). Just type Dcdiag if running on a DC. We need to execute the command for each DC. If the output contains any replication error or DNS errors then the issue is with the Active directory.

Check for the event id’s 2101 or 2102 in the application or system log for Topology discovery errors. (If the topology discovery is successful an event id: 2080 will be generated in the app log).

Verify for the FSMO roles on the Domain controllers using the following command.

Netdom query FSMO

Nltest /DCLIST: - Get list of DC's for

To test the connectivity between Exchange and DC’s execute the following command.

Netdiag > output file name

Run the following command to test the DNS functionality:

Netdiag /test: DNS

To check the Exchange site run the following command:

Nltest /dsgetsite

To check the DC and the GC functionality:

Nltest /dsgetdc :< FQDN>

Copy “Policytest” tool from the Exchange CD and execute the command “Policytest.exe” from the DOS prompt. http://support.microsoft.com/kb/314294

If failed to found the privileges to the DC then run the command “Domainprep”

From the Exchange server CD. http://support.microsoft.com/?kbid=328662

Use “replmon.exe” GUI tool to troubleshoot the replication problems. Also we can use “repadmin” to troubleshoot the replication problems of DControllers.

Articles for reference:================

http://support.microsoft.com/?kbid=821907







http://support.microsoft.com/?id=822590

Overview of nltest command:

http://support.microsoft.com/default.aspx?scid=kb;EN-US;158148

Overview of netdiag and dcdiag commands:

http://support.microsoft.com/kb/321708


========================================================================Issue 2:

Mailbox store and Public folder stores not mounting (Database not mounting).

Possible causes:

Exchange services are not started.

Stores are in Inconsistent state.

Insufficient permissions to mount the stores.

Exchange databases reached 16 GB limitation (Standard edition)

Insufficient Disk space causing the stores to mount.

Solution:

Check whether the Exchange services are started successfully. (System Attendant service and Information store service)If the System Attendant service is not started perform the troubleshooting steps stated for Issue 2.

If the SA is started and Information store is not started, the follow the below troubleshooting steps:

• Check the application log for Event ids.

• Check whether we are using local system account to start the service.

• Check out whether the Exchsrvr folder and the Inetsrv folder is excluded from the “Anti-virus Exclusions list”.

• Check whether the Anti virus is scanning the Exchange databases in the back ground. If possible stop the Anti virus service or disable the background scanning.

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MSExchangeIS\VirusScan (In the Value Data field, change the value to 0.) 0 means disabled, 1 means enabled.

If the problem persists, check the “STATE” field of the service. Whether the service is in starting state or the service itself is not starting. If the service is “Crashing or is in Hanging mode” then follow the troubleshooting steps for Issue 5.

If the Information Store service started successfully then we need to verify the permissions and the consistency of the Exchange databases.

Checking the permissions and the consistency of the Exchange databases.

Minimum permissions required for mailbox store and public folder store: (check on the mail box and public folder store from ESM)

• Administrators group - Full Control

• Authenticated Users group - Read and Execute, List Folder Contents, and Read

• Creator Owner - None

• Server Operators group - Modify, Read and Execute, List Folder Contents, Read, and Write

• System account - Full Control

Check whether the Exchange computer has full control permissions on the mail box store.

Check for the above permissions on the MDBDATA folder.

Check whether the inherited permission check box is enabled on the mail box store.

Check for the consistency of the Exchange databases (Priv1.edb and Pub1.edb) by using below commands:

C:\Program files\Exchsrvr\Bin> eseutil /mh

C:\Program files\Exchsrvr\Bin> eseutil /mh

Note: We need to run the “Eseutil” command from the BIN folder.

• Check for the value in “State” field /mh command. Either it should show “Consistent - Clean shutdown” or “Inconsistent - Dirty shutdown”.

• If the database is “Consistent” then verify for the permissions listed above or look for the event ids in the application log. Also, try removing or stopping the anti-virus software.

• If the database is “In-consistent” then verify for the “Log required field” in the output of /mh command.

• Find the check point value of the last log file by using the below command: (check point file tells the last log file committed to the database)

C:\Program files\Exchsrvr\Bin> eseutil /mk

If all the required log files are present in the log files folder:

• If the required log files are available in the Log files folder then do a soft recovery to play the log files using eseutil command.

• Before doing soft recovery check the integrity of the log files whether all the log files are in sequence and the files are not corrupted by using the below command:

C:\Program files\Exchsrvr\Bin> eseutil /ml (Do not mention .log in the command path”

• Keep only the required log files in the log files folder and move all the log files to the temp folder.

• Performing soft recovery to play the required log files in the log files folder using below command:

http://www.microsoft.com/technet/prodtechnol/exchange/Analyzer/f8faaa6d-7213-4271-9ba3-a2c4274f091d.mspx

The complete syntax for the Eseutil.exe soft recovery function, listing all possible switches, is:

ESEUTIL /r enn /L[path to log files] /s[path to checkpoint file] /d[path to database file] /i

Example: ESEUTIL /r e01 /Lf:\mdbdata /sc:\exchsrvr\mdbdata /dg:\mdbdata /i

We can use the switch “/t” with the eseutil command to ignore the log files available 0n the hard drive (In MDBDATA) folder.

Note: In the Application event log, an Extensible Storage Engine (ESE) event ID 204 information message is logged to indicate the start of recovery, and then a series of ESE event ID 301 information messages are logged. These information messages indicate the exact log numbers that are recovered. Finally, an ESE event ID 205 information message is logged to indicate that the recovery process is complete.



Note: We can also dismount and remount the mail box store to replay the required log files in the log file folder.

If any of the required log file is missing:

Verify any log file is quarantined by the anti virus software. If any log file is quarantined restore the log file to the original location and perform soft recovery.

Restore the databases from the backup and play the log files from Restore.env file. (By default log files are restored to a TEMP folder with the file name “Restore.env”). After playing the log files from Restore.env file we can also play the log files available on hard disk.

To see the contents of the Restore.env file, execute the below command:

C:\Program files\Exchsrvr\Bin> eseutil /cm (Do not mention the file name “Restore.env” in the command line.

To play the log files present in Restore.env file, execute the below command:

C:\Program files\Exchsrvr\Bin> eseutil /cc (Do not mention the file name “Restore.env” in the command line.

Performing hard repair if the backup is not available

If backup is not available, then we need to do a hard repair on the Exchange databases. We need to follow the below steps to do hard repair on the databases followed by defragmentation and running Isinteg command.

Description of Eseutil /p command:


C:\Program files\Exchsrvr\Bin> eseutil /p

C:\Program files\Exchsrvr\Bin> eseutil /p

Description of Eseutil /d command:


C:\Program files\Exchsrvr\Bin> eseutil /d

C:\Program files\Exchsrvr\Bin> eseutil /d

Note: After doing offline defragmentation we need to move all the log files in the log file folder to a Temp folder. Mount the stores and dismount the stores to run Isinteg command. We need to take the backup of the Exchange databases after performing the offline defragmentation because the signature of the databases changes after defragmentation.

Description of Isinteg command:


C:\Program files\Exchsrvr\Bin> isinteg –s -fix –test alltests

How to run Eseutil on a computer without Exchange Server:


If the database is reached 16 GB limitation follow the steps given in the below KB article:


For more information on 16 GB limitation proceed to Issue 6.

Articles for reference:================

Top 10 Database Mounting Issues and Their Solutions:

http://www.microsoft.com/technet/prodtechnol/exchange/2003/hd.mspx

Information store does not mount with 0xfffff745 and -2235 errors:


Information Stores Do Not Mount After You Use the "/disasterrecovery" Switch:


How Circular Logging Affects the Use of Transaction Logs:


How and when to manipulate Exchange transaction logs in disaster recovery:


How the Restore in Progress Registry Key Works (Exchange 5.5):


=========================================================================

Issue 3:

Information store service is not starting or the IS service is crashing or in hanging state.

Possible causes:

Anti virus scanning the Exchange databases.

Active directory issue.

Solution:

• Check the application log for Event ids.

• Check whether we are using local system account to start the service.

• Check out whether the Exchsrvr folder and the Inetsrv folder is excluded from the “Anti-virus Exclusions list”.

• Check whether the Anti virus is scanning the Exchange databases in the back ground. If possible stop the Anti virus service or disable the background scanning.

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MSExchangeIS\VirusScan (In the Value Data field, change the value to 0.) 0 means disabled, 1 means enabled.

If the problem persists, check the “STATE” field of the service. Whether the service is in starting state or the service itself is not starting. If the service is “Crashing or is in Hanging mode” then follow the troubleshooting steps.

If the Information store service is crashing or hanging(starting state):

To assist in troubleshooting Exchange 2000 Server information store issues, you can enable information store logging. This writes verbose logging information to a Store.log file that is created in the Exchsrvr\Bin folder.

This logging is enabled by default in debug versions of the Store.exe file, but it is disabled in customer retail (RTL) versions.

MORE INFORMATION

To enable information store tracing capabilities in retail versions of the Store.exe file you need to add a new key to the registry. 1. Copy \\exutils\exes\storeitp\store.ttd to the Exchsrvr\Bin folder.

2. Start the Registry Editor utility (Regedt32.exe).

3. Locate the following key in the registry:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MSExchangeIS\ParametersSystem

4. On the Edit menu, click Add Value, and then add the following registry value:

Value Name: Enable Tracing

Data Type: REG_DWORD

Value: 0x1

5. Quit the Registry Editor.

The Information Store will begin to log the debug traces in the Store.log file in the \Exchsrvr\Bin folder (this may take up to one minute to begin its logging mechanism). You do not have to stop and restart the information store for these changes to take effect.

NOTE: This pre-made Store.ttd file contains the generic tracing

information. You can create a custom Store.ttd file on your own, but this requires you to install the debug build of Store.exe, or use the StartITP.exe utility to enable the specific tracing that you require.

For additional information about how to use StartITP.exe, click the article number below to view the article in the Microsoft Knowledge Base:

250913 How to Open a Store ITP Window against a Retail Build of Store.exe (Internal KB)

The debug build of the Store.exe file is located at: Standard version: file://exrel/RELEASE/boneyard/Ex60RTM/server/standard/std/dbg/USA/setup/i386/exchange/bin

Enterprise Version: file://exrel/RELEASE/boneyard/Ex60RTM/server/enterprise/ent/dbg/USA/setup/i386/exchange/bin

For example, start the debug build of the Store.exe file from the command prompt, click Debug , click Trace Info , and then, under TAG Numbers , click to select the 1 and 2 check boxes.

The information store creates a TTD file in the Exchsrvr\Bin folder.

You can also copy the TTD file from a different server.

The information store begins to log the traces in the Store.log file in the Bin folder. You do not have to stop and restart the information store.

You can limit store tracing for individual users. To do so:

1. Locate the following key in the registry: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MSExchangeIS\ParametersSystem

2. On the Edit menu, click Add Value, and then add the following registry key: Value Name: Trace User LegacyDN Data Type: REG_SZ Value: legacyExchangeDN of the user that you want to trace

Per-User Tracing Feature Added to Exchange 2000 Server Service Pack 2 (SP2):

http://support.microsoft.com/default.aspx?scid=kb;en-us;296624

Enabling store tracing:=======================

1. Make sure that the Microsoft Exchange Information Store service is stopped. 2. Click Start, click Run, type regedit, and then click OK. 3. In Registry Editor, locate the following subkey: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS\ParametersSystem

4. On the Edit menu, point to New, click DWORD Value, and then follow these steps: a. In the Value Name box, type Enable Tracing . b. In the Data Type box, type REG_DWORD . c. In the Data Value box, type 1 .

5. Try to start the Microsoft Exchange Information Store service again. We can also use ADPLUS tool to take the dump of store.exe process. Below is the detailed action plan for taking the dump of Store.exe for IS service crashing or hanging using ADPLUS tool.

Before the crash occurs

Step I: Download and install the Microsoft Debuggers

NOTE: You do not have to install the debug tools on your Exchange server. It can be installed on a workstation then copied to the the Exchange server.

• Go to http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx

• In the Installing Debugging Tools for Windows section, follow steps 1 and 2.

• In the File Download dialog box, Select Save and save file to the local hard drive.

• Once downloaded, navigate to the folder and double click the executable to install.


Step 2: Run Adplus.vbs on the Exchange server console (not through Terminal Services).

• Open a command prompt by going to Start, Run and enter the syntax: CMD

• Navigate to the debuggers folder (Default is c:\Program Files\Debugging Tools For Windows):

• Enter the syntax:

cscript adplus.vbs –crash(-hang) -pn store.exe -NoDumpOnFirst -o c:\store _DumpFile

• Click "OK" to the Adplus Warning stating "An _NT_SYMBOLS_PATH environment variable is not set..."

• At this point, Adplus will attach a debugger to the process and wait for an Access Violation.

• Once an Access Violation occurs, the debugger will be invoked and a dump file will be created.

• The folder Store_DumpFile will be automatically created and all data files will be placed in the folder.

Visit the following web site for instructions and to download the ADPLUS tool:


Articles for Reference:=============

How to Collect Diagnostic Data to Help Troubleshoot Information Store Issues:


How to Collect Diagnostic Data for Information Store Troubleshooting:


Information Store Does Not Start with Service-Specific Error 0 (Zero)


Per-User Tracing Feature Added to Exchange 2000 Server Service Pack 2 (SP2):


The Information Store service and Outlook 2003 crash intermittently when Outlook 2003 clients connect to a mailbox on an Exchange 5.5 server http://support.microsoft.com/?kbid=834466

Exchange Server 5.5 Post-SP3 Information Store Fixes Available:


=========================================================================¬Issue 4:

Logs files are not purged after taking full backup.

Possible causes:

Log files may get corrupted.

Selecting wrong backup type (Differential, Copy or Daily)

Either Information store or Public folder store is not selected during backup.

Solution:

Try taking the backup using NTBackup software.

• Select Information store and Public folder store during backup.

• Select the option full backup in the backup software.

Once the backup completes all the log files get deleted automatically. If the log files are not purged during full online backup then move the log files to the Temp folder. Try sending few e-mails to create new log files; to isolate the issue whether the problem is with the old log files get corrupted.

If the problem persists, verify whether the backup was successful. Also check the application and system logs for any event ids and

errors and proceed with the further troubleshooting steps as necessary.

=========================================================================Issue 5:

Problem replicating Public folders from one Exchange server to other server.

Possible Causes:

Mail flow is not working between the servers.

Public folder store is not stamped with the proxy address.

Replicas are not added to the public folders.

Network connectivity issues.

Solution:

Follow the steps given in the below KB to troubleshoot the public folder replication issue:


If the problem persists after performing the steps given in the above article, turn on the diagnostic logging on both the Exchange servers and check for event ids in the application log and proceed accordingly with further troubleshooting steps. The above article mention on how to turn the diagnostic logging for public folder replication.Articles for Reference:TechNet Support WebCast: Troubleshooting public folder replication:http://support.microsoft.com/?id=907304

How to Send Replication Status Request Messages in Exchange 2000 Server:


Replication does not occur for one Exchange server in the organization:



Public Folder Hierarchy and Content Is Not Being Replicated Across Routing Groups:


Description of the public folder referral functionality in Exchange 2000 Server and in Exchange Server 2003:


Public Folder Replication Troubleshooting:

http://www.microsoft.com/technet/prodtechnol/exchange/guides/WorkingE2k3Store/f92870f5-ad69-4b45-96cd-4a32a03537a7.mspx

Controlling Exchange Server 2003 Public Folder Replication:


=========================================================================Issue 6:

Problem Expanding Public Folder Hierarchy from the Exchange System Manager.

Possible causes:

Incorrect IIS settings.

IIS database is corrupted.

IIS lockdown tool is installed.

Anti virus scanning the Exchange databases in the background.

Solution:

Check for the event ids in the application log.

The errors are separated by graphical images of the actual error themselves, the associated possible causes and the troubleshooting steps to follow.






Error 80040e19

Possible Causes

• Improper IP address binding in IIS

• Host Header for the default web site is incorrect in IIS

• Incorrect Path name listed in IIS under the Home Directory tab

• Metabase can be corrupted

• URLScan may be installed on IIS blocking specific extensions or verbs

• Non working public folder store is selected in ESM

• Invalid folderpathname set for the http public folder in Active Directory.

• Other 3rd party ISAPI filters installed in IIS

• This can occur if the public folder hierarchy on the exchange server is associated with the public folder store with another server.

Troubleshooting

• Verify proper IP address, host header and port in the properties of the Default Web Site. IP address by default is set to all unassigned with no host header bound to port 80. Set back to defaults for testing purposes. If multiple IP addresses are bound to the network card due to the “All in one” syndrome with running multiple websites on an exchange server, try using the first bound IP address. Add host header if necessary to get around this. Change port if necessary. If this error is occurring on a cluster server, then you may need to add a host header for the Exchange cluster virtual name to the website to get this working

• Check to make sure that URLScan is not installed on the server. This can be found by getting the master WWW properties in IIS and selecting the ISAPI filters tab. If URLScan is listed, then the best thing to do to see if URLScan is causing the problem is to check the urlscan.log file that is created in the \winnt\system32\inetsrv\urlscan folder.


IIS lockdown and URLscan configurations in an Exchange environment:


NOTE: Once a setting in the urlscan.ini is changed for use with URLScan, the IISAdmin service will need to be restarted for the change to take effect. Keep in mind that this will stop the Information Store which will affect Outlook and OWA clients.

• Connect to another Public Folder store, if available, to see if the issue persists. To do this, right click on “Public Folders” and then select “Connect to…” and then select another PF store.

• Check the properties in IIS on the public and exadmin folders whether they are pointing to the proper Home Directory paths. Be default, exadmin should point to \\.\BackOfficeStorage and public should point to m:\domain.com\public folders. Having incorrect directory paths will cause this issue even if it is off by one letter. For example, if Public Folders was spelled Public Folder.

• Verify the correct folderpathname is listed for the http public folder. See SOX031125700026 for more information.

• Take a netmon trace when the client is attempting to connect to the public folders in the ESM, this will tell you what is being returned from IIS. The client, in this case, would be an ESM on another computer besides the Exchange server itself so that we can see the proper network traffic.

Error 80070005

Possible Causes

• There are instances where you are prompted for your network password and after 3 tries, you may receive this Access Denied error. This usually occurs when HTTP keep-alives and not configured for the default website.

Troubleshooting

• Get properties of the Default website and then put a check box in the “Enable HTTP Keep- Alives”

Error 80070057

Possible Causes

• Anonymous only authentication is set for the exadmin virtual directory. This normally occurs if the DS2MB replication process if broken and an administrator changed the access permissions for this vdir to anonymous only.

Troubleshooting

Check off the box to enable Integrated Authentication and then find out why the DS2MB process is failing to replicate this information from Active Directory in to the Metabase.

Error 80072030

Possible Causes

• This can occur if you are attempting to propagate permissions in the ESM for a top level folder in a Mixed Mode organization in which you have no local Exchange 5.5 replicas on this server.

Troubleshooting

You Receive an Error Message When You Propagate Public Folder http://support.microsoft.com/?id=317675

Error 80090301

Possible Causes

• This can occur if you are requiring SSL for the exadmin virtual directory under the Default website, but there is no certificate bound to the website. • Incorrect Host header is configured for the Default Web Site.

Troubleshooting

• To resolve this, open the ISM and then get properties of the exadmin vdir and select the Directory Security tab. From there, select the Edit Button under Secure Communications. Uncheck “Require 128-bit encryption” and then uncheck “Require secure channel (SSL). Click OK and then expand Public Folders again.

• If you do not have access to change this information in the ISM, then you can manually remove this.

o Open a command prompt and then change in to the \inetpub\adminscripts folder.

o Run “cscript adsutil.vbs enum w3svc/1/root/exadmin” to view the settings for that virtual directory. You should see the following if you are experiencing this error.

AccessSSL : (BOOLEAN) True

AccessSSL128 : (BOOLEAN) True

To change this to not require SSL, perform the following command.

cscript adsutil.vbs set w3svc/1/root/exadmin/AccessSSL False

cscript adsutil.vbs set w3svc/1/root/exadmin/AccessSSL128 False

• Remove any incorrect host headers from the website.

Error c007274c

Possible Causes

• Incorrect Host Headers defined on clustered virtual server

Troubleshooting

• Add or remove the correct host headers to the Exchange VS in IIS.

Error c1030af0

Possible Causes

• This can occur if you have multiple IP addresses bound to the network card on the Exchange server and you have the default website bound to this secondary IP address.

• An invalid host header or IP address has been set for the default website

• Incorrect IP addresses or host header is set on the cluster virtual server.

Troubleshooting

• Try setting the default web site to “All Unassigned” and then reopen the ESM and try to expand Public Folders.

• Correct the host header value or remove it altogether.

Error c1030af1

Possible Causes

• You will see this error message if you go to the properties of the Default Website > select the Directory Security Tab, click the Edit button of IP address and domain name restrictions > and Deny your Exchange server’s IP address or subnet.

• Network card bindings are set incorrectly.

• Read access on the exadmin virtual directory is unchecked

Error c1030af2

Possible Causes

• Davex In-Process ISAPI application is not loaded or missing on the Master WWW properties.

• ExchangeApplicationPool has been deleted in IIS. See SOX040609700142 on how to recreate this.

• Default website is listening on a port other than port 80.

Troubleshooting

• Check the IIS logs to see if all http requests are connecting over port 80. If any other port is listed, then you will get this error.

• Use Metaedit or MBExplorer to add this information back. A Metaedit sample view is shown below. Once adding this back in, Public Folders should be able to expand immediately. The path to get to this is LM/W3SVC and then double-click on the InProcessIsapiApps to get the below view. Note: Davex is also explicitly set at the vdir level, but that is not used for administration, only the viewing of Public folders through OWA.

• If all else fails, try the following (Taken from SOX040413700052)

o Perform a backup of the IIS Metabase. Right-click the servername in

IIS Manager, select All Tasks and then Backup/Restore.

o Remove the Exadmin virtual directory.

o Open up a command prompt and navigate to the following directory: C:\Inetpub\AdminScripts

o Run "cscript.exe adsutil delete ds2mb" (this deletes the ds2mb in the Metabase)

o Restart the Exchange System Attendant (this will restart the Information Store and the MTA Stacks and recreate the Exadmin virtual directory)

o Verify the Exadmin folder had been recreated in IIS Manager.

o Test to see if we are able to expand the public folder tree.

IIS logging properties reference –

http://www.microsoft.com/resources/documentation/WindowsServ/2003/standard/proddocs/enus/Default.asp?url=/resources/documentation/windowsserv/2003/standard/proddocs/en-us/ref_we_logging.asp

Error c1030af3

Possible Causes

• Improper authentication method set in IIS for the public and exadmin vdir’s

• DS2MB process has not populated the proper entries in the IIS metabase

• IE security settings are incorrect.

• Permissions in Active Directory are set incorrectly

• Realtime virus software is scanning the M: Drive

• The msExchDS2MBOptions for the exadmin virtual directory is set incorrectly.




• There is an incorrect path listed for the properties of the exadmin vdir in IIS. This value should be \\.\BackOfficeStorage.

Troubleshooting

• Ensure that a proper authentication method is selected for the HTTP Public Vdir in the ESM. If none is listed, then you will receive the above error. If the correct authentication method shows up in ESM, open the Internet Service Manager to see if the same authentication method is listed for the Public vdir. Also check the exadmin vdir for the same authentication method. If this authentication method is not present, then there is a problem with DS2MB replicating this information from Active Directory to the IIS metabase. Check the event log for additional information regarding the failing DS2MB process.

• Open the ESM on another computer or server. If this works, then this is machine specific.

• Set your IE security settings to a lower setting such as Med-Low.

• Check permissions on the Top level Public Folder using ADSIEdit or the ESM to ensure no specific denies are set. Correct if necessary.

• Exclude the M: Drive or hidden EXIFS mapping from any AV File level scanning. Restart server to release any locked or open files.

• If msExchDS2MBOptions is set to 0 for the exadmin vdir, the above error will occur. Reset this back to the default of 66 and allow time for replication to the metabase.

Error c1030af4

Possible Causes

• The exadmin vdir local path has a trailing backslash in the path such as the following \\.\BackOfficeStorage\

Troubleshooting

• Remove the trailing backslash for the vdir properties in IIS and then refresh the ESM.

Error c1030af6

The operation failed because of an HTTP error 501 (Not implemented). Verify that the ExAdmin virtual root exists on the destination server.

ID no: c1030af6 Exchange System Manager.

Possible Causes

• The cause of this is similar to c1030af7

Error c1030af7

Possible Causes

• This can occur if you have multiple Web sites configured and the default website is currently stopped. This website includes the associated vdirs (exadmin, exchange, public, exchweb) which are not available when that site is stopped.

• The exadmin vdir is missing within IIS

Troubleshooting

• Check to see if the exadmin directory exists in IIS under the default website. If it does not, from a command prompt in the \inetpub\adminscripts folder, enter “cscript adsutil.vbs delete ds2mb” and hit enter. This will remove the information from the metabase. Restart the System Attendant service to get this information repopulated in IIS.

Error c1030af8

The operation failed due to HTTP error 501 ID: c1030af8

Possible Causes

• This can occur there is another service taking up port 80.

Troubleshooting

• Run “fport.exe 80” to find the offending service and either disable it or change its port number.

Error c0072af9

Possible Causes

• You are unable to resolve the DNS name for the server you are trying to connect to in the ESM.

Troubleshooting

• Try pinging the server by both the netbios name and the FQDN name.

• If the servers name is not registered in DNS, go ahead and register it by opening a command prompt on the remote server and typing “ipconfig /registerdns”. Verify in the DNS manager that this was registered properly.

Error c1038a21

Possible Causes

• RUS is not stamping public folders with proxy addresses

• The Microsoft Exchange System Objects folder is not located in the root of the domain

• The folder in question is not mail enabled.

• The ExchangeApplicationPool has been deleted from IIS.

Error c0072afc

Possible Causes

• You are unable to resolve the DNS name for the server you are trying to connect to in the ESM.

Troubleshooting

• Try pinging the server by both the netbios name and the FQDN name.

• If the servers name is not registered in DNS, go ahead and register it by opening a command prompt on the remote server and typing “ipconfig /registerdns”. Verify in the DNS manager that this was

registered properly.

Error c0072741

Possible Causes

This can occur if the msExchSecureBindings or msExchServerBindings attribute is populated with invalid information. For example: if port 443 is listed and there is not a SSL certificate bound to the default website.

Troubleshooting

To resolve the above issue, do the following.

• Go to Start - Programs - Windows 2000 Support Tools - Tools and launch ADSI Edit.

• In the left side pane expand the Configuration container.

• Next expand CN=Configuration

• Then CN=Services CN=Microsoft Exchange CN= CN=Administrative Groups CN=First Administrative Group CN=Servers CN=Protocols CN=HTTP CN=1

• Right Click on CN=Exadmin and choose Properties.

• In the Properties dialog box you will see 2 drop-down lists. drop down the top list and select "Both". Drop down the second list and scroll down to the attribute "msExchSecureBindings" and double click on it.

• If this attribute is set to 443 or any other value really, click the 443 value to select it and click the "Remove" button. Then click "Apply" and then "OK"

• Close out of ADSI Edit, close and reopen Exchange System Manager and test Public Folder access again.

Error c103b401

Possible Causes

• This can occur if the default website (Default) is not configured to use port 80 or an incorrect host header is configured.

• Default Website is stopped in IIS causing the exadmin vdir to be unavailable.

• Website is configured to use an external IP address, not internal.

• msExchServerBindings attribute in Active Directory for the exadmin container has an incorrect port number populated.

Troubleshooting

• Change the Default website to listen on port 80 and remove any host header entries.

• Start the Default Website on the Exchange server that you are attempting to connect to in the ESM.

• Try binding the website to “All Unassigned” or an internal IP address.

• Verify that msExchServerBindings port number for the exadmin vdir matches the port bound to the Default Web Site.

• Try connecting to another Exchange public folder to see if the same issue occurs on another server.

Error c103b404

Possible Causes

• This can occur if you have a certificate bound to the default website, you have required SSL on the exadmin virtual directory, and the certificate common name does not contain the FQDN of the server name that IIS is installed on.

Troubleshooting

• To resolve this issue, remove the certificate from the default website

and request a new certificate that contains a common name of the FQDN of the server.

Error c103b405

Possible Causes

• The exadmin vdir local path is set to a value other than \\.\BackOfficeStorage

This can occur if the following are true.

o Multiple IP Addresses are bound to the NIC card

o Multiple websites are configured on the Exchange server

o The default website is bound to the secondary IP address that is bound to the NIC card

o No DNS mapping for the second IP that maps to the FQDN of the server

Enter Network Password

Possible Causes

• HTTP keep-alives and not configured for the default website.

Troubleshooting

• Get properties of the Default website and then put a check box in the “Enable HTTP Keep-Alives”

Articles for Reference:

Troubleshooting Microsoft Exchange System Manager Public Folder Expansion Problems:

http://www.microsoft.com/technet/prodtechnol/exchange/2003/insider/publicfolders.mspx

Please see the web version of this document (will also be the most updated one) here:

http://msweb/personal/mikelag/Shared%20Documents/ESM%20PF%20Troubleshooting/default.htm

=========================================================================

Issue 7:

Problem downloading Offline address book

====================================

Possible Causes

• Check for the Exchange server (Home server) responsible for generating the OAB.

• In the Default Offline Address List properties, confirm that the Default Global Address List is in the Address List list. Add any other address lists that are necessary for offline access. Check whether the replica’s are added for the OAB system folder.

• Check for the OAB system folders (OAB Version 2a for Exchange 2000 and Version 3a for Exchange 2003).

• Check for the public folder tree listed on public folder store.

• Compare the value of the Distinguish Name on the public folder store with the value of site folder server on the Administrative group using ADSIEDIT.

• If the mailbox store has not been associated with a valid offline Address Book.

• Check whether outlook is configured for Cached-Exchange mode.

Troubleshooting

=============

If the mailbox store has not been associated with a valid offline Address Book:

1. Start Exchange System Manager.

2. Expand Organization, and then expand Administrative Groups.

If "Administrative Groups" does not appear under Organization, make sure that you have configured Exchange System Manager to display Administrative Groups. To do so, right-click Organization, click Properties, and then click Display administrative groups.

3. Expand the administrative group to which the server belongs.

4. Expand Servers, expand the server that you want to configure, and then expand First Storage Group.

5. Right-click Mailbox Store, and then click Properties.

6. In the Offline Address Book section, make sure that a valid offline Address Book is listed.

If one is not listed, click Browse, and then click a valid offline Address Book. You can use Default Offline Address Book as the default setting.

Replicate the Offline Address List to Other Servers

If there are multiple sites in the Exchange organization, replicate the default offline address list to the servers in those sites to improve download performance. To replicate the offline address list to other servers:

1. Open the properties of the OAB Version2 object, and then click the Replication tab.

2. Add the other servers to the Replicate content to these public stores list.

3. Click OK to save the changes.

4. The replication of the OAB Version2 folder follows the replication schedule of the public folder store by default. To force replication to occur immediately, rebuild the default offline address list.

Enable Diagnostics Logging

To troubleshoot activity that occurs when the offline address list is rebuilt, increase the diagnostic logging to maximum on the following object in the offline address list server properties:

Service: MSExchangeSA

Category: OAL Generator

To turn the diagnostic logging, follow the below steps:

Open the ESM -> Right click on the Exchange server name -> properties -> diagnostic logging tab

Monitor the Application event log for events that can assist you in troubleshooting issues with offline address list generation.

Rebuild the Offline Address List

If the OAB Version2 folder is missing from the Public Folders container (to locate this container, click the Administrative Groups container, click the Servers container, click the server_name container, and then click the First Storage Group container), rebuild the default offline address list in Exchange System Manager. To rebuild the default offline address list:

1. Expand the Recipients container, and then expand the Offline Address Lists container. Delete and re-create the Default Offline Address List by using a different name.

Note: If you do not use a different name, you will not be able to download the offline address list.

2. Open the Default Offline Address List properties, and then set the Update Interval to Always.

3. In the Default Offline Address List properties, confirm that the Default Global Address List is in the Address List list. Add any other address lists that are necessary for offline access.

4. Click OK to save the settings.

5. Right-click the Default Offline Address List, and then click Rebuild. Allow time for the system to rebuild the offline address list and replicate the information among the domain controllers.

6. Verify that the OAB Version2 folder was created successfully in the Public Folders container under the Administrative Groups\Servers\server_name\First Storage Group containers. If the OAB Version2 folder was not created, view the Application event log to determine which errors occurred.

7. If the OAB Version2 folder was successfully created, verify that a MAPI client can download the offline address list.

If the system folders (OAB version 2a or OAB version 3a) then we need to reset the system folders by following the steps listed in the below website:

http://support.microsoft.com/kb/275171/

Articles for Reference:=================TechNet Support WebCast: How to troubleshoot Offline Address Book issues in Microsoft Exchange Server: http://support.microsoft.com/kb/905482/en-us

How to Reset System Folders: http://support.microsoft.com/kb/275171/

=========================================================================Issue 8:

Troubleshooting Recipient Update Service, RUS failed to stamp proxy addresses on User accounts.

Possible Causes:

• Incorrect Domain RUS settings i.e.: Not pointing to the correct servers.

• Missing permissions (Recipient Update Service does not have the necessary access permissions to update the objects)

Incorrect Domain RUS settings

We need to check whether the Domain RUS is pointing to the respective Exchange server and to the Domain Controller by viewing the properties page of the Domain RUS.

1) Click Start – Programs – Microsoft Exchange – System Manager

2) Expand Recipient container – Recipient Update Service

The following objects are displayed:

• Recipient Update Service (Enterprise Configuration)

• Recipient Update Service (Domain_name)

3) Right click the Domain RUS – Click properties.

The domain, the Exchange server, and the Windows domain controller that are involved in running this instance of the Recipient Update Service are displayed. The Update interval setting is also displayed. This is set to Always Run by default.

4) To change the Exchange computer on which an instance of the Recipient Update Service runs, click Browse next to Exchange computer, locate the Exchange server that you want the Recipient Update Service to run on, and then click OK.

5) To change the Windows domain controller that this instance of the Recipient Update Service connects to when it updates objects in Active Directory, click Browse next to Windows Domain Controller, locate the domain controller that you want the Recipient Update Service to connect to, and then click OK.

Important Points

• Each instance of the Recipient Update Service associates one Exchange server (where the Recipient Update Service runs) with one Windows 2000 or Windows Server 2003 domain controller (on which the Active Directory objects are updated). Only one Recipient Update Service object can be associated with one Active Directory domain controller.

• We can have multiple instances of the Recipient Update Service on an Exchange server, but each domain controller in a domain can only participate in one Recipient Update Service. If you want multiple Recipient Update Services for a domain, that domain must have more

than one domain controller.

• Creating a RUS, the Exchange server is granted rights to modify Exchange Server attributes on recipients that are in the domain.

• When you are creating a Recipient Update Service, you cannot select the domain controller. You can only select the domain that you want the new Recipient Update Service to be associated with. After you have finished creating the Recipient Update Service, you can edit the properties of the Recipient Update Service and select the domain controller that you want. (Vimp)

UPDATE and REBUILD================== • If you perform an update operation, proxy e-mail addresses are generated immediately for all new users. This operation forces address book membership to be recalculated for recently modified Exchange recipients. Perform an update operation if you are running the Recipient Update Service on a schedule and you want to complete mailbox creation before the next duty cycle runs.(UPDATE)

• • If you perform a rebuild operation, all proxy e-mail addresses are recalculated and all address list memberships are verified. Perform a rebuild operation if you make a change to organizational policy on SMTP addressing (for example, if you change the DNS domain suffix from .com to .net or if you change the DNS domain name). A rebuild operation may take several hours. We recommend that you perform this operation during a period of low network activity. (REBUILD)

Missing Permissions

Four possible reasons for this behavior:

• Administrator or user of built-in Active Directory security group name or log on does not resolve.

• Inheritable permissions from parent are not propagated to object check box on the Active Directory organizational unit that the accounts reside in.

• Exchange Enterprise Servers group is missing required permissions at the domain level.

• Group has the hideDLMembership attribute set to True. (KB253828)

SOL: To resolve the permissions issue, run DomainPrep on the

Domain controller or on the Exchange server.

:\I386\Setup.exe /domainprep

For detailed information on required permissions for RUS, Visit the following web site:


=========================================================================Issue: 9

Troubleshooting Virtual Defragmentation Problems

Possible Causes:

• Exchange server is running on low virtual memory.

• Anti virus may reduce the virtual memory to a level below the 32 MB address space. (Where an antivirus program that is configured to scan the messaging databases reduces the virtual memory block to less than 32 MB).

Note: Event ID 9582 or 12800(Performance issues) in the application log indicates the problem of Virtual Defragmentation. When you scale a server to handle more users and larger loads, the server may run low on virtual memory in the Microsoft Exchange Information Store process (Store.exe). When this issue occurs, event ID 9582 events are logged to the application event log. You can use a monitoring tool that generates an administrative alert each time an event ID 9582 message is logged.

When an Exchange server has less than 32 MB of free contiguous virtual

address space, the following warning message is logged to the application event log:

Source: MSExchangeIS

Category: Performance

ID: 9582

Type: Warning

Description:

The virtual memory necessary to run your Exchange server is fragmented in such a way that performance may be affected. It is highly recommended that you restart all Exchange services to correct this issue.

For more information, click http://search.support.microsoft.com/search/?adv=1

When this warning message is logged, follow these steps:

1. Prepare and perform the steps to shut down and then restart the server in the next 36 to 72 hours.

2. To determine the rate of decay, use the Performance Logs and Alerts tool to monitor the following counter for the MSExchangeIS performance object :

• VM Total Large Free Block Bytes

Use this data to help you plan an appropriate time (in the next 36 to 72 hours) to shut down and then restart the server.

When an Exchange server has less than 16 MB of free contiguous virtual address space, the following error message is logged to the application event log:

Source: MSExchangeIS

Category: Performance

ID: 9582

Type: Error

Description:

The virtual memory necessary to run your Exchange server is fragmented in such a way that performance may be affected. It is highly recommended that you restart all Exchange services to correct this issue.

For more information, click

http://search.support.microsoft.com/search/?adv=1

At this level of virtual memory fragmentation, the Store.exe process cannot create additional heaps and cannot correctly mount and dismount storage groups. If the VM Largest Block Size counter is below 10 MB, the storage groups do not mount. When an event ID 9582 error message is logged, prepare to shut down and restart the server. When you shut down and then restart the server to clear virtual memory fragmentation, there are additional considerations when Exchange 2000 Server is configured in a clustered environment. When you move cluster resources from one node to another node, this process does not ensure a "clean" virtual memory address space. If cluster resources are owned by the destination cluster node, and the cluster resources are moved to the passive node (without first restarting the destination node), you may experience virtual memory fragmentation on the passive node. To avoid this situation, and to clear memory fragmentation in an Exchange 2000 Server clustered environment, follow these steps:

1. Restart the passive node before you move cluster resources to it.

This step helps to make sure that the cluster resources are moved to a server that has a "clean" virtual memory address space.

2. Move the cluster resources to the passive node.

3. Restart the node that previously owned the cluster resources.

Note: Exchange Server 2003 restarts the Store.exe service automatically after the resource records have been moved to a different node in the cluster to reset the Store.exe address space on that node. Therefore, the next time that the Exchange virtual server is moved back to the passive node, the Store.exe is operating with a "clean" address space.

Event ID: 9665 Warning messages

Exchange 2003 performs an optimal memory configuration check when the Store.exe process starts. If the memory settings are not optimal, an event ID 9665 warning message is logged to the application event log of Event Viewer. This warning message is logged when any one of the following conditions is true:

• Exchange is installed on a computer that is running any version of Microsoft Windows 2000 Server, and the SystemPages value in the registry is set outside the range of 24000 to 31000.

• Exchange is installed on a computer that is running Microsoft Windows 2000 Advanced Server or Microsoft Windows 2000 Datacenter Server, and the server has 1 gigabyte (GB) or more of physical memory (RAM) installed but does not have the /3GB switch set in the Boot.ini file.

• Exchange is installed on a computer that is running Microsoft Windows Server 2003 Standard Edition, Microsoft Windows Server 2003 Enterprise Edition, or Microsoft Windows Server 2003 Datacenter Edition, and the SystemPages value in the registry is set to a value other than 0.

• Exchange is installed on a computer that is running Windows Server 2003 Standard Edition, Windows Server 2003 Enterprise Edition, or Windows Server 2003 Datacenter Edition, the server has 1 GB or more of RAM installed, and the /3GB switch is set, but the /userva switch is either not present in the Boot.ini file or is set outside the range of 3030 to 2970.

• Exchange is installed on a computer that is running any version of Windows 2000 Server or Windows Server 2003, and the HeapDeCommitFreeBlockThreshold value in the registry is set to a value other than 0x00040000.

When an event ID 9665 warning message is logged, follow these steps

1. Check the SystemPages setting and the HeapDeCommitFreeBlockThreshold setting in the registry.

2. Check the /3GB switch and the /userva switch in the Boot.ini file.

ote: If you want to turn off the memory configuration check, add the Suppress Memory Configuration Notification DWORD value to the following registry key, and then set the value to 1:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS\ParametersSystem

The following counter is the most important counter to monitor for virtual memory fragmentation in the Store.exe process in Exchange 2003 and Exchange 2000:

Performance object: MSExchangeIS

Counter: VM Largest Block Size

This counter displays the size (in bytes) of the largest free block of virtual memory. This counter appears as a line that slopes downward as virtual memory is used. If this counter drops below 32 MB, Exchange logs an event ID 9582 warning message in the application event log. If this counter drops below 16 MB, Exchange logs an event ID 9582 error message in the application event log. If the largest free block is small (less than 10 MB), the server is approaching a critical state where message operations may start to fail and event ID 12800 error messages are repeatedly logged. (Refer to the web site to add more counters for monitoring the VM issues: http://support.microsoft.com/kb/296073/)

Setting /3GB switch:

Important points:

• Windows 2000 server does not support /3GB switch.

• Adding /3GB switch increases the Virtual Address Space.

• One of the effects of using the /3GB switch is a significant reduction in the number of system pages that are available to the kernel. Setting the /3GB switch in the Boot.ini file on Exchange servers is to modify the default settings and to increase the number of system pages that are allocated.

• When you set the /3GB in the Boot.ini file on a Windows Server 2003-based computer, set the /userva switch in the Boot.ini file to a value between 2970 and 3030. The recommended value is 3030 (this value is the equivalent to the Windows 2000 SystemPages value of 31000).

On Windows 2003, the /userva switch is to be used instead of the SystemPages registry key. They should not be used in conjunction. If the value for the /userva switch is not set between 2970 and 3030 in the SystemPages registry entry, and the /3GB switch is set, Exchange 2003 logs Event ID 9665 to the application event log. This event ID indicates that virtual memory on the server is not configured to use the optimal memory settings.

To set the SystemPages registry value on a computer that is running Windows 2000 Server, follow these steps:

1. Click Start, and then click Run.

2. In the Open box, type regedit, and then click OK.

3. Locate and then click the following registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management

4. In the right pane, double-click SystemPages.

5. In the Value data box, type a value between 24000 and 31000, and then click OK.

6. Quit Registry Editor.

For more information on the above registry key, refer to the below web site:


Setting HeapDeCommitFreeBlockThreshold registry value:

The HeapDeCommitFreeBlockThreshold registry value is the minimum size of a free block that the heap decommits. The default setting is 0 (zero). This means that the heap manager decommits each 4 KB page that becomes available. Decommit operations can cause additional virtual memory fragmentation. You can set the HeapDeCommitFreeBlockThreshold registry entry in the following registry key to a higher value to help reduce virtual memory fragmentation:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager

The recommended value to use for the HeapDeCommitFreeBlockThreshold registry entry is 0x00040000

For more information on the above registry key, visit the below web site:


Note: The HeapDeCommitFreeBlockThreshold registry entry is independent of the /3GB switch.

Articles to Refer:

How to troubleshoot virtual memory fragmentation in Exchange

Server 2003 and Exchange 2000 Server:


A description of the 4 GB RAM Tuning feature and the Physical Address Extension switch:


Exchange 2000 requires /3GB switch with more than 1 gigabyte of physical RAM


The "HeapDecommitFreeBlockThreshold" registry key


How to modify the Store Database maximum cache size in Exchange 2000 Server:


Monitoring for Exchange 2000 memory fragmentation:


Virtual Memory Fragmentation Occurs When You Fail Over an Exchange 2000 Virtual Server:


=========================================================================Issue 10:

Error message “Mailboxes still exist in the mailbox store” while removing the Exchange server from the site or organization.

Solution:

By default we have three mail boxes listed in the mail box store when we install the Exchange server.

• SMTP mail box. (when PF replication happens through mail that time SMTP mailbox uses)

• System mail box. (it uses for system generated mail reports or NDR )

• System Attendant mail box. (It’s only one for each exchange server, when user access mail through OWA and create appointment it uses SA mailbox to publish system site folder.)

To check whether we have any other mail boxes in the mail box store we need to verify an attribute “MSExch HomeMDBBL or HomeMDBBL” on the mail box store using ADSIEDIT in the Active directory. We need to install the Windows support tools to open ADSIEDIT console.

Expand configuration container -> services -> Microsoft Exchange -> Organization name -> Administrative groups -> Administrative Group name -> First storage group -> Information store -> Mail box store -> Properties.

Look for the above attribute. If found any mail boxes, search for the users in the AD and delete the mail boxes for the users or move the mail boxes to the other Exchange server in the site or organization.=========================================================================Issue 11:

Changing the database paths and log file path locations to the different drive from Active directory. We can also change the paths from ESM.

Solution:

Open Adsiedit.msc from Run command.

We need to change the value for the following attributes:

MSEXCHEDB file on the mail box store for (.EDB files)

MSEXCHSLV file on the mail box store for (.STM files)

MSEXCHLogfilePath on the storage group for (Log files location)

Note : We need to dismount the stores before changing the paths.

How to Move Exchange Databases and Logs in Exchange 2000 Server 257184

How to move Exchange databases and logs in Exchange Server 2003 821915

=========================================================================Issue 12:

Understand -1018, -1019 and -1022 database errors in the application log.

Possible causes:

If the databases get corrupted.

Due to hardware failures.

Check for any event id’s in the application and system logs and view the

Description of the event. If the event description indicates and hardware

Failures then we need to fix the issues with the hardware.

If the issue is not with the hardware then restore the databases from the backup media. If the backup is not available then perform the hard repair on the databases followed by offline defragmentation and isinteg to fix the physical and logical corruptions in the database.

Understanding and analyzing -1018, -1019, and -1022 Exchange database errors:
