One thing to think about when authoring rules and monitors is performance, and Windows event monitoring is no exception. If you need to search for a string in an event description, and if said event description is not parameterized, this post is for you.
You could add this logic while creating a rule in the Operations console.
The reason you should not do this is because it’s going to require more compute time than is necessary. If the monitored event log typically has a steady stream of events written, a single event detection rule like this could create a significant processing bottleneck on the monitored computer.
This is because all filtering is happening at the data source level, which means every element in your expression filter is checked against every event that is written to the specified log on that computer. Searching for a string in the event description takes time, and this is why it’s a bad practice to implement event description processing in the data source module.
To implement event monitoring with minimal performance impact, I suggest including only these parameters in the data source criteria:
- Event Source
- Event Id
- Event Level
- Event Category
- Event Parameter [0-..]
Furthermore, I suggest including only these operators in the data source criteria:
- Equal
- NotEqual
- Greater
- GreaterEqual
- Less
- LessEqual
So, how is it possible to implement pattern matching while still providing minimal processing impact?
The answer is to process pattern matching criteria in a condition detection module.
Wow, what a concept! Let the data source filter the simple event criteria, then move to the next module to process more complex criteria. That’s the cool thing about using System.ExpressionFilter. The workflow will exit if the expression <> true.
To understand this logic a little better, here is the processing sequence…
Without Condition Detection:
1. DataSource\EventDetection\Provider – match event log and logging computer (if true, move next)
2. DataSource\EventDetection\Filter – process all criteria (if true, move next)
3. WriteAction\Alert – generate an alert
With Condition Detection:
1. DataSource\EventDetection\Provider – match event log and logging computer (if true, move next)
2. DataSource\BasicEventDetection\Filter – process simple criteria (if true, move next)
3. ConditionDetection\FilterDescription – process complex criteria (if true, move next)
4. WriteAction\Alert – generate an alert
Unfortunately, it is not possible to implement the condition detection module in the Operations console. You will need to author this using the R2 Authoring Console, Visual Studio Authoring Extension, or directly in XML. I am providing a rule fragment you can add to your project using VSAE.
Similarly, you could implement this logic in an event detection monitor.
The below fragment detects event 101, with source TEST and event level ERROR in the Application log. If simple criteria matches, move the next module to process complex criteria (match SomePattern in the event description). Generate an alert at the end.
This rule fragment needs to be updated with your criteria and alert details. If you build it, alerts will come – and your manager will thank you.
<ManagementPackFragment SchemaVersion="2.0" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <Monitoring> <Rules> <Rule ID="MyMp.Rule.EventDescriptionMatches" ConfirmDelivery="true" DiscardLevel="100" Enabled="true" Priority="Normal" Remotable="false" Target="Windows!Microsoft.Windows.Computer"> <Category>Alert</Category> <DataSources> <DataSource ID="BasicEventDetection" TypeID="Windows!Microsoft.Windows.EventProvider"> <ComputerName>$Target/Property[Type="Windows!Microsoft.Windows.Computer"]/NetworkName$</ComputerName> <LogName>Application</LogName> <Expression> <And> <Expression> <SimpleExpression> <ValueExpression> <XPathQuery>EventSourceName</XPathQuery> </ValueExpression> <Operator>Equal</Operator> <ValueExpression> <Value>TEST</Value> </ValueExpression> </SimpleExpression> </Expression> <Expression> <SimpleExpression> <ValueExpression> <XPathQuery>EventDisplayNumber</XPathQuery> </ValueExpression> <Operator>Equal</Operator> <ValueExpression> <Value>101</Value> </ValueExpression> </SimpleExpression> </Expression> <Expression> <SimpleExpression> <ValueExpression> <XPathQuery>EventLevel</XPathQuery> </ValueExpression> <Operator>Equal</Operator> <ValueExpression> <Value>1</Value> </ValueExpression> </SimpleExpression> </Expression> </And> </Expression> </DataSource> </DataSources> <ConditionDetection ID="FilterDescription" TypeID="System!System.ExpressionFilter"> <Expression> <RegExExpression> <ValueExpression> <XPathQuery>EventDescription</XPathQuery> </ValueExpression> <Operator>MatchesRegularExpression</Operator> <Pattern>SomePattern</Pattern> </RegExExpression> </Expression> </ConditionDetection> <WriteActions> <WriteAction ID="Alert" TypeID="Health!System.Health.GenerateAlert"> <Priority>1</Priority> <Severity>2</Severity> <AlertMessageId>$MPElement[Name="MyMp.Rule.EventDescriptionMatches.AlertMessage"]$</AlertMessageId> <AlertParameters /> <Suppression /> </WriteAction> </WriteActions> </Rule> </Rules> </Monitoring> <Presentation> <StringResources> <StringResource ID="MyMp.Rule.EventDescriptionMatches.AlertMessage"/> </StringResources> </Presentation> <LanguagePacks> <LanguagePack ID="ENU" IsDefault="true"> <DisplayStrings> <DisplayString ElementID="MyMp.Rule.EventDescriptionMatches"> <Name>Event Description Match Alert Rule</Name> </DisplayString> <DisplayString ElementID="MyMp.Rule.EventDescriptionMatches.AlertMessage"> <Name>Event Description Match</Name> <Description>Event description match pattern specified in rule configuration.</Description> </DisplayString> </DisplayStrings> </LanguagePack> </LanguagePacks> </ManagementPackFragment>
__
Jonathan,
Nice post! I have one question though. Can this be done with a Windows event monitor instead of a rule?
I have an application that will fire off 10 or 15 alerts in a short amount of time when it looses connectivity to the DB. I don’t want to spam the users with the alerts for each one so I’m thinking about writing a timed reset monitor. There is no real great way to detect a good state in the logs for this application to reset on. I see that the code structure of a monitor is a little different in that it doesn’t use datasources like the rule does.
I appreciate any thoughts that you may have.
Brett
Hi Brett. Absolutely, this can be done with a unit monitor. It’s all in the data source and condition detection modules that lead up to and through the unit monitor type. The big difference is, the data source in this example would be moved from the monitoring part of the XML to the type definitions part, and then it would be implemented in a new unit monitor type and finally used in a unit monitor.
Thanks for the reply. I’ll see if I can make that work. I’m pretty new at the coding part of OM.
As we all know that scom throws hundreds of alerts when servers are rebooted due to windows update. To stop these alerts, I am trying to configure a rule to run a script in case of event 22 which will put server in maintenance mode for 15 minutes. Can you please suggest a best way of doing this?
Jonathan,
I have created the monitor but I’ve apparently done something wrong. I’m not getting alerts or health state change when I do an event create to test. Below is what I came up with. Would you mind giving it a look to see if I missed something?
System!System.ExpressionEvaluatorSchema
$Target/Property[Type=”MicrosoftWindowsLibrary7585010!Microsoft.Windows.Computer”]/NetworkName$
$Config/LogName$
EventSourceName
Equal
$Config/EventSourceName$
EventDisplayNumber
Equal
$Config/EventDisplayNumber$
EventLevel
Equal
$Config/EventLevel$
EventDescription
MatchesRegularExpression
$Config/EventDescriptionContains$
$Config/TimerWaitInSeconds$
Hi Jonathan, Does “ConfirmDelivery=”true” has any negative influence here? The explanation by Microsoft is very poor: “Defines whether or not the module following the data source module will notify the data source that it has received the data. The default value is false”.
Does this require entering additional info on the Parameter 1 line such as “equal” or a entry in the “Value” portion of the line?