Capture verbose tracing for an Agent or Management Server

This section describes how to capture the trace on the Agent or Management Server.

Prepare for tracing

• Logon to the Agent computer or Management Server

• Open a command prompt and navigate to the Operations Manager “tools” directory.
  • Usually located in “%ProgramFiles%\System Center Operations Manager 2007\Tools”

• Enter the following command to stop the current trace.
  • StopTracing.cmd

· In Windows Explorer, navigate to “%windir%\Temp\OpsMgrTrace“, and delete all files in this directory. This is the current and previous trace data, which can be discarded now.

Start tracing

• Switch back to the command prompt and enter the following command to start a new verbose trace.
  • StartTracing.cmd VER
    Note: “VER” must be capitalized as shown above.

· Keep this command prompt open, as you’ll need to switch back to it in order to stop and format the trace data later.

Wait for issue to reproduce

It is important to wait for the issue to reproduce, and stop tracing immediately after it does. Monitor the problem closely, so that tracing can be stopped immediately after problem occurs.

Stop tracing

• Switch back to the Agent or Management Server and enter the following commands in the command prompt we had left open.
  • StopTracing.cmd
  • FormatTracing.cmd

· After formatting completes, Windows Explorer should open to the %WINDIR%\Temp\OpsMgrTrace directory, displaying the files containing the trace data.

• Select all files and send to compressed folder, as shown below.

· Send the compressed file to your Microsoft support engineer for analysis.
Note: if the compressed file is more than 10MB, as your support engineer to create a FTP workspace to upload the files. Otherwise, it is usually okay to send via email.

Capture verbose tracing for an Operations Manager Console session

This section describes how to capture a verbose trace for a particular Operations Manager Console session.

Prepare for tracing

• Logon to the computer in which you’d like to capture a trace of the Console session.

• Open a command prompt and navigate to the Operations Manager “tools” directory.
  • Usually located in “%ProgramFiles%\System Center Operations Manager 2007\Tools”

Note: Administrator may be required to run tracing on client computers, because the trace writes to the Windows directory, which is usually a protected file system. If there are issues with capturing the trace, ensure that the command prompt is launched under the Administrator account, as shown below.

• Enter the following command to stop the current trace.
  • StopTracing.cmd

· In Windows Explorer, navigate to “%windir%\Temp\OpsMgrTrace“, and delete all files in this directory. This is the current and previous trace data, which can be discarded now.

Start tracing

• Switch back to the command prompt and enter the following command to start a new verbose trace.
  • StartTracing.cmd VER
    Note: “VER” must be capitalized as shown above.

· Keep this command prompt open, as you’ll need to switch back to it in order to stop and format the trace data later.

Wait for issue to reproduce

It is important to wait for the issue to reproduce, and stop tracing immediately after it does. Monitor the problem closely, so that tracing can be stopped immediately after problem occurs.

Stop tracing

• Switch back to the computer hosting the console session and enter the following commands in the command prompt we had left open.
  • StopTracing.cmd
  • FormatTracing.cmd

· After formatting completes, Windows Explorer should open to the %WINDIR%\Temp\OpsMgrTrace directory, displaying the files containing the trace data.

• Select all files and send to compressed folder, as shown below.

· Send the compressed file to your Microsoft support engineer for analysis.
Note: if the compressed file is more than 10MB, ask your support engineer to create a FTP workspace to upload the files. Otherwise, it is usually okay to send via email.

Disable computer group health rollup across the board
Override Target	Context	Parameter	Default Value	Override Value	Scope	Management Pack Object Type	Enforced	Target	Override Management Pack	Target Management Pack	Override Target Management Pack
Computer Security Health Rollup	Computer Group	Enabled	TRUE	FALSE	Class	Monitor	*FALSE	Computer Group	System Center Core Monitoring (overrides)	System Center Core Library	System Center Core Library
Computer Performance Health Rollup	Computer Group	Enabled	TRUE	FALSE	Class	Monitor	*FALSE	Computer Group	System Center Core Monitoring (overrides)	System Center Core Library	System Center Core Library
Computer Configuration Health Rollup	Computer Group	Enabled	TRUE	FALSE	Class	Monitor	*FALSE	Computer Group	System Center Core Monitoring (overrides)	System Center Core Library	System Center Core Library
Computer Availability Health Rollup	Computer Group	Enabled	TRUE	FALSE	Class	Monitor	*FALSE	Computer Group	System Center Core Monitoring (overrides)	System Center Core Library	System Center Core Library
*You may set the enforced parameter to TRUE to force this setting in case of override conflict (not recommended).
If you disabled across the board, but want to enable only for a specific group
Override Target	`	Parameter	Default Value	Override Value	Scope	Management Pack Object Type	Enforced	Target	Override Management Pack	Target Management Pack	Override Target Management Pack
Computer Security Health Rollup	Windows Server 2008 Computer Group	Enabled	TRUE	TRUE	Class	Monitor	*FALSE	Windows Server 2008 Computer Group	System Center Core Monitoring (overrides)	Windows Server 2008 Operating System (Discovery)	System Center Core Library
Computer Performance Health Rollup	Windows Server 2008 Computer Group	Enabled	TRUE	TRUE	Class	Monitor	*FALSE	Windows Server 2008 Computer Group	System Center Core Monitoring (overrides)	Windows Server 2008 Operating System (Discovery)	System Center Core Library
Computer Configuration Health Rollup	Windows Server 2008 Computer Group	Enabled	TRUE	TRUE	Class	Monitor	*FALSE	Windows Server 2008 Computer Group	System Center Core Monitoring (overrides)	Windows Server 2008 Operating System (Discovery)	System Center Core Library
Computer Availability Health Rollup	Windows Server 2008 Computer Group	Enabled	TRUE	TRUE	Class	Monitor	*FALSE	Windows Server 2008 Computer Group	System Center Core Monitoring (overrides)	Windows Server 2008 Operating System (Discovery)	System Center Core Library
*Enforced parameter is not necessary to resolve override conflicts in this case.
If you do not want to disable across the board, but only for specific group
Override Target	Context	Parameter	Default Value	Override Value	Scope	Management Pack Object Type	Enforced	Target	Override Management Pack	Target Management Pack	Override Target Management Pack
Computer Security Health Rollup	Windows Server 2008 Computer Group	Enabled	TRUE	FALSE	Class	Monitor	FALSE	Windows Server 2008 Computer Group	System Center Core Monitoring (overrides)	Windows Server 2008 Operating System (Discovery)	System Center Core Library
Computer Performance Health Rollup	Windows Server 2008 Computer Group	Enabled	TRUE	FALSE	Class	Monitor	FALSE	Windows Server 2008 Computer Group	System Center Core Monitoring (overrides)	Windows Server 2008 Operating System (Discovery)	System Center Core Library
Computer Configuration Health Rollup	Windows Server 2008 Computer Group	Enabled	TRUE	FALSE	Class	Monitor	FALSE	Windows Server 2008 Computer Group	System Center Core Monitoring (overrides)	Windows Server 2008 Operating System (Discovery)	System Center Core Library
Computer Availability Health Rollup	Windows Server 2008 Computer Group	Enabled	TRUE	FALSE	Class	Monitor	FALSE	Windows Server 2008 Computer Group	System Center Core Monitoring (overrides)	Windows Server 2008 Operating System (Discovery)	System Center Core Library

 

· All discoveries run every 12 hours (DFS-R discoveries run every 4 hours)

· Agent Proxy must be enabled on servers hosting Namespaces or Replication to obtain full functionality.

o Suggest just enabling Agent Proxy across the board these days.

· Profiles

o DFS Namespace Discovery Account

§ Scripts requiring account

· DFS Namespace Discovery

· Namespace Server Discovery

· DFS Folders and Folder Targets Discovery

o DFS Paths Access Account

§ For DFS Namespace Folder and Folder Target monitoring.

§ Must have READ permissions on target folders.

§ A Windows Run As Account MUST be created and associated to this Profile.

o File Services Role Monitoring Account

§ No mention of this account in the guide

o DFS Replication Monitoring Account

§ We must create a Run As Account that has administrator privileges on every DFS-R server, because when we need to get a backlog between A and B the script needs to get information from A DFS-R WMI provider and B DFS-R WMI provider. LocalSystem access is not enough.

Client Monitoring

· Enable DFS Client Computer Discovery (Target: Windows Computer, Class: DFS Client Computer)

o For a small set of computers (create a group of Windows Computer objects)

o This enables the DFS-N: Client-Side DFS Folder Availability monitor.

· Configure DFS-N: Client-Side DFS Folder Availability monitor by overriding the UNCPaths parameter. This is a colon delimited list of paths, like \\path1:\\path2:\\etc

Namespace Folder and Folder Target Monitoring

· Enable DFS Namespaces Components Discovery (Target: DFS Namespace Server, Class: DFS Folders)

o There is a parameter named “Path to DFSUtil.exe”, but this is only for the console task. Does not affect discovery. Just set the Enabled param to True.

o Caution: Do not enable this discovery if there are more than 1000 DFS folders in customer deployment! This will adversely affect performance of the agents, and cause excessive network traffic.

· Enabled DFS-N: Folder Target Availability monitor (Target:

o Enable only for a small set of folders (create a group of DFS Folder Targets objects)

Replication Monitoring

· DFS Replication Monitoring Account

o We must create a Run As Account that is a member of the Administrators local group on all monitored DFS-R computers.

· Enable DFS Replication Backlog Discovery (Target: DFS Replication Service, Class: Replication Connection)

o Enable for all objects of Type: DFS Replication Service

 

There are two things to keep in mind while tracing.

1. Tracing uses additional resources on the Root Management Server.

2. Stop tracing as soon as the issue has been reproduced.

This method has only been testing in SCOM 2007 SP1 and R2.

1. Logon to the Root Management Server

2. Open a Command Prompt and navigate to %ProgramFiles%\System Center Operations Manager\Tools. (keep this command prompt open to this locations until you have stopped tracing)

3. Issue the following command

StopTracing.cmd

4. Open explorer and navigate to C:\WINDOWS\temp\OpsMgrTrace.

5. You should see 10-20 *etl, *log and *sum files.

6. While still in explorer, navigate to %ProgramFiles%\System Center Operations Manager\Tools.

7. Make a copy of TracingGuidsBID.txt.

8. Then open TracingGuidsBID.txt. Comment all lines by typing # before each line, except for:

947883FC-801B-8F53-FB37-39730F608894 Microsoft.EnterpriseManagement.HealthService.Modules.Notification.3.0

455329E9-3C18-BB5B-D9B8-B5EA0DF8FCDC Microsoft.Mom.AlertSubscriptionDataSourceModule.1

9. Save and close TracingGuidsBID.txt.

10. Go back to the Command Prompt and issue the following commands:

StartTracing.cmd VER

TraceLogSM.exe -stop TracingGuidsNative

TraceLogSM.exe -stop TracingGuidsUI

11. You are now tracing the Notification Channel components. Continue tracing until the issue has been reproduced.

12. When the issue has reproduced, immediately issue the following command to stop tracing. If tracing isn’t stopped immediately after reproduction, you may risk rolling the log file.

StopTracing.cmd

13. While still in the command prompt, issue the following to format the output

FormatTracing.cmd

14. After formatting has completed, explorer should open at the location where the formatted files were copied to.

Gather Data and Send to Microsoft

Now we’ll package this data and send it to Microsoft. If you have a good understanding of ETL logs and OpsMgr error conditions, you may be able to decipher the log yourself.

1. Gather the *etl, *log and *sum files that were formatted and copied to the trace output directory from step 14 above.

2. Gather the Operations Manager event log from the Root Management Server.

3. Zip these files and send to Microsoft.

4. Make note of the Alert Id for which the alert notification was expected, and send that along with the zipped files or in separate email.

5. Include any other data you feel is relevant. Screenshots of Alert History or duplicate tickets in ticketing system, any logging you’re performing beyond Operations Manager, etc.

Note: Upload the files to the secure FTP site that was provided by Microsoft.

 

Introduction

Periodically, the Health Service Store may become badly fragmented. There is a rule in SCOM that performs a daily defrag of this database file, but it’s an online defrag and doesn’t always do the trick if there are frequent instance space changes occurring.

Performing the Defrag

On the Root Management Server, perform the following steps.

1. Open command propmpt and navigate to “%Program Files%\System Center Operations Manager 2007\Health Service state\health service store“.

2. Type net stop HealthService

a. If service does not shutdown clean, see recoverybelow before continuing.

3. Type esentutl /d HealthServiceStore.edb

4. Type net start HealthService

Recovery

If Health Service incurs a dirty shutdown, replay transaction log before continuing with defragmentation.

1. Type esentutl /r <HealthServiceStoreFile>.edb

a. If recovery doesn’t work, try esentutl /p <HealthServiceStoreFile>.edb

2. Continue with step 3 above