Install SCOM Agent on Red Hat Enterprise Linux 6 (linux agent installation)

This is a step-by-step article on installing the SCOM agent on a RHEL6 system, both from an SCOM and Linux administrator perspective. Everything in this demonstration assumes a new installation of RHEL6 with no additional configuration. It is also assumed there are no firewalls between the SCOM management servers and the RHEL system. Windows Firewall is not turned on, and the RHEL system is running with default IPTABLES configuration.

Disclaimer: I do not claim to be an experienced Linux admin. This is intended to familiarize SCOM administrators with the entire process. There may be better practices or methods for configuring Linux – comments are  welcome.

If you do not have an SCOM environment or a RHEL system in your lab, you can download evaluation versions from the links below (these links may be outdated at the time you are reading this).

SCOM 2012 SP1 Evaluation

RHEL 6 Evaluation (download the rhel-server-6.3-x86_64-dvd ISO)

Installing RHEL6 on Hyper-V step-by-step

Here are the tasks that will be addressed in this walk-through:

   1: Create a Linux user account for the OpsMgr service

   2: Sudoers file configuration

   3: Configure iptables (Linux firewall)

   4: Verify DNS – forward and reverse lookup

   5: Create a cross-platform monitoring resource pool

   6: Install certificates on resource pool members

   7: Install required management packs

   8: Runas accounts and profiles

   9: Discovery wizard

 

Configuring Linux

Before attempting to discover and install the SCOM agent, an account needs to be created on the Linux system and the sudoers file must be configured to allow the account to execute certain commands. Most out-of-box monitoring activities are performed using low privileges on the Linux system, but some activities do require elevation using sudo.

In this example, SCOM will leverage a single Linux account to perform all activities: opsmgrsvc.

 

Create the opsmgrsvc account

There are several free tools you can use to establish a SSH session to a Linux system. The most common is Putty, which can be downloaded at http://www.putty.org/.

Launch Putty and connect to the Linux host by entering the host name or IP address of  the RHEL system and clicking Open.

image

 

Login with an account that has privileges to add user accounts and modify the sudoers file, or login with root. Create the opsmgrsvc account using the following command:

useradd opsmgrsvc

Next, set the password for the account using the following command:

passwd opsmgrsvc

You will be prompted to enter the password after issuing the command.

Configure sudoers file

Next you will configure the sudoers file, giving the opsmgrsvc account the ability to elevate and execute privileged commands without requiring a password (this is a SCOM requirement).

While still logged into the Linux SSH session as root, open the sudoers file editor by typing visudo and then:

1. Move down the file until you find the line Defaults requiretty, and enter Defaults:opsmgrsvc !requiretty just after it. This setting allows the service account to login without requiring a TTY session.

image

 

2. Move down the file until you find the line root ALL=(ALL) ALL, and enter opsmgrsvc ALL=(root) NOPASSWD: ALL just after it. This setting allows the service account to elevate without requiring a password.

image

 

3. Hit the ESC key to finish editing. To save and close the sudoers file, enter :wq. If you made a mistake in editing the file, you can close without saving your work by entering :q!.

 

NOTE!!! Your Linux administrator will likely want more granular control over which commands the SCOM service account can execute. Below are example sudoers file entries which would enable the opsmgrsvc account to only execute the command necessary for agent installation, maintenance tasks, and baseline monitoring.

More information about sudoers file requirements and examples for other cross-platform systems can be found at http://social.technet.microsoft.com/wiki/contents/articles/7375.configuring-sudo-elevation-for-unix-and-linux-monitoring-with-system-center-2012-operations-manager.aspx.

NOTE: Copy/Paste the following – there is no word wrap, so some of it may float off the screen!

#----------------------------------------------------------------------------------- 
#User configuration for Operations Manager agent: opsmgrsvc

#General requirements for OpsMgr service account
Defaults:opsmgrsvc !requiretty

#Agent discovery, installation, certificate signing, and maintenance tasks
opsmgrsvc ALL=(root) NOPASSWD: /bin/sh -c cp /tmp/scx-opsmgrsvc/scx.pem /etc/opt/microsoft/scx/ssl/scx.pem; rm -rf /tmp/scx-opsmgrsvc; /opt/microsoft/scx/bin/tools/scxadmin -restart
opsmgrsvc ALL=(root) NOPASSWD: /bin/sh -c sh /tmp/scx-opsmgrsvc/GetOSVersion.sh; EC=$?; rm -rf /tmp/scx-opsmgrsvc; exit $EC
opsmgrsvc ALL=(root) NOPASSWD: /bin/sh -c cat /etc/opt/microsoft/scx/ssl/scx.pem
opsmgrsvc ALL=(root) NOPASSWD: /bin/sh -c rpm -e scx
opsmgrsvc ALL=(root) NOPASSWD: /bin/sh -c /bin/rpm -F --force /tmp/scx-opsmgrsvc/scx-1.[0-9].[0-9]-[0-9][0-9][0-9].rhel.[0-9].x[6-8][4-6].rpm; EC=$?; cd /tmp; rpm -rf /tmp/scx-opsmgrsvc; exit $EC
opsmgrsvc ALL=(root) NOPASSWD: /bin/sh -c /bin/rpm -U --force /tmp/scx-opsmgrsvc/scx-1.[0-9].[0-9]-[0-9][0-9][0-9].rhel.[0-9].x[6-8][4-6].rpm; EC=$?; cd /tmp; rpm -rf /tmp/scx-opsmgrsvc; exit $EC

#Log file monitoring
opsmgrsvc ALL=(root) NOPASSWD: /opt/microsoft/scx/bin/scxlogfilereader -p

#Custom shell command monitoring example. Replace <shell command> with the correct command string
#opsmgrsvc ALL=(root) NOPASSWD: /bin/bash -c <shell command>

#Daemon diagnostic and restart recovery tasks (example using cron)
#opsmgrsvc ALL=(root) NOPASSWD: /bin/sh -c ps -ef | grep cron | grep -v grep
#opsmgrsvc ALL=(root) NOPASSWD: /usr/sbin/cron &

#Add sudo logging
Defaults logfile=/var/log/sudolog

#End user configuration for Operations Manager agent
#-----------------------------------------------------------------------------------
 
Configure Linux Firewall (iptables)

The firewall is enabled in a default installation of RHEL6. It comes preconfigured to allow SSH connections, ICMP on local subnet, and very basic network service functionality (these protocol ports may need to be manually configured on other distributions). To see the firewall configuration, run the following command in your SSH session:

iptables --list

image

SCOM discovery and installation uses SSH to copy installation files to the target computer and execute shell scripts, such as GetOSVersion.sh, but before this can happen SCOM needs to connect to the SCX CIM provider on the remote host using WinRM. This occurs over port 1270, which is not open in the default iptables file.

To allow WinRM to do it’s job, add the iptables rule with the following command (this is one line):

iptables -I INPUT 1 -p tcp -s 192.168.2.0/24 --dport 1270 -m state --state NEW,ESTABLISHED -j ACCEPT

Save iptables configuration with the following command:

/sbin/service iptables save

Take a look at the ruleset to make sure it saved correctly.

iptables --list

image

Important: You will need to save your iptables configuration and configure the network interface to restore iptables rules at startup. Otherwise, those rules will disappear the next time the computer restarts. Check your distribution for saving/restoring iptables.

 

iptables command explained

-I INPUT 1 : insert new rule at the beginning of the ruleset.

-p tcp : protocol to match.

-s 192.168.2.0/24 : accept requests from specified host (or hosts). Note: this must be changed for your environment or can be removed from the command to accept from anywhere.

--dport : destination port to match on the host.

-m : match specific connection states.

--state NEW, ESTABLISHED : connection states to match.

-j ACCEPT : accept connection and stop processing the ruleset.

 

DNS Configuration

If you host is not resolvable in DNS – forward and reverse – then read this section. Otherwise, you may skip to the next section about creating a resource pool.

Forward lookup zones are required for SCOM in general, and reverse DNS lookup zones are minimally required for a management server to establish communication with UNIX/Linux agents. connections with the management group. Some DNS systems do not automatically create DNS records, so in this example we will walk thru creating these records and zones in Windows DNS Manager.

 

Create forward lookup record

To register the Linux host in DNS, open DNS Manager and create a new A record in the forward lookup zone of your domain. If you have a reverse DNS lookup zone configured in your domain, select Create associated point (PTR) record. This should automatically create the reverse DNS record, but you may want to verify this on your first attempt.

image

 

Verify your configuration with nslookup.

image

 

In this example, forward lookup is working with nslookup rhel6-01, but reverse lookup isn’t working with nslookup 192.168.2.70. This is because either there was no reverse lookup zone configured in the domain, or the Create associated pointer (PTR) record option was not selected. In this case, a reverse DNS lookup zone did not exist.

 

Create reverse lookup record

If you don’t have a reverse lookup zone in your DNS, you’ll need to create one in DNS Manager. Select the defaults in the wizard, entering your network Id – in this case, 192.168.2.x.

image

 

Now a PTR record can be created for the Linux host. One way to do this is to open the A record created in the previous step (in the forward lookup zone) and select Update associated pointer (PTR) record.

image

 

Testing reverse DNS lookup now returns the Linux host name, and now you’re ready to move onto the next step.

image

 

Create a Resource Pool

In a typical production environment, there will be a resource pool dedicated to monitoring cross-platform systems. This resource pool should contain at least two management servers for high-availability. This example adds management servers OM12-MS02 and OM12-MS03 to the new Cross-Platform Monitoring Resource Pool.

Navigate to Administration > Resource Pools in the Operations console to create a new resource pool.

image

 

Add the management servers.

image

 

Install Certificates on Resource Pool Members

Each management server in the cross-platform monitoring resource pool needs to have it’s certificate installed on each other management server. This is task is performed by running the scxcertconfig tool on each management server; first exporting the certificate from each management server, then importing each other management server’s certificate.

Login to each management server in your cross-platform resource pool, open a command prompt as administrator, navigate to %Program Files%\System Center\Operations Manager 2012\Server folder, and execute the following command:

scxcertconfig.exe -export <hostname.cert>

In this example, <hostname.cert>is replaced with OM12-MS02 on the first management server.

After exporting the certificate on each management server, import the certificate(s) from each other management server in the same manner. in this example, if I’m logged into OM12-MS02, I would execute the following command:

scxcertconfig.exe -import OM12-MS03.cert

More information about this procedure can be read on Technet at http://technet.microsoft.com/en-us/library/hh287152.aspx.

 

Install Management Packs

By default, there are several Unix/Linux management packs installed in SCOM.

image

 

However, this is not enough to perform agent installation using the Discovery Wizard. You will need to import the version specific management packs before even attempting to install a cross-platform agent, because they contain definitions specifically designed to detect which version of the the installation package to install on the target host.

In this example, we minimally need to install the Linux and Red Hat Libraries, and the RHEL6 bundle. These management packs can be found on your installation media, in the Management Packs folder. You can also download most recent versions from the online management pack catalog.

Navigate to Administration > Management Packs in the Operations console to install the required management packs: Microsoft.Linux.RHEL.6.mpb, Microsoft.Linux.Library.mp, and Microsoft.Linux.RedHat.Library.mp.

image

 

At this point, all the prerequisites are satisfied for installing the agent on a Red Hat Enterprise Linux 6 operating system by using the Discovery Wizard. Unless you don’t want to actually monitor the Linux system, you can skip to the Discovery Wizard section.

 

Configuring Runas Accounts and Profiles

One last thing we need to do in order for object discovery and monitoring workflows can run on our Linux system. This isn’t necessarily required to install cross-platform agents, but obviously there wouldn’t be much value in it if this task isn’t completed.

Navigate to Administration > Run As Configuration > UNIX/Linux Accounts in the Operations console and create a new account.

image

 

Create monitoring account

1. On the Account Type screen, select Monitoring account.

2. On the General Properties screen, enter a display name.

image

 

3. Enter your Linux monitoring account credentials on the Account Credentials screen – do not use elevation with this account. If you use elevation with this account, monitoring workflows implementing Linux commands that do not support sudo elevation will fail.

image

 

4. Select More secure on the Distribution Security screen. Then click Create and Close.

 

Create elevated monitoring account

Create another account that will be used for elevated monitoring tasks that require sudo.

1. On the Account Type screen, select Monitoring account.

2. On the General Properties screen, enter a display name.

image

 

3. Enter your Linux monitoring account credentials on the Account Credentials screen – use elevation with this account.

image

 

4. Select More secure on the Distribution Security screen. Then click Create and Close.

 

Associate accounts with profiles

Now the accounts need to be associated with the appropriate Runas profiles.

1. Navigate to Administration > Run As Configuration > Profiles and open the properties of the UNIX/Linux Action Account.

2. Click Add on the Run As Accounts screen, and select the regular monitoring account (not the privileged account). Keep the default – All targeted objects – and click Ok.

3. After clicking Save, the wizard gives an opportunity to securely distribute the account. Click on your account to step into the distribution wizard.

image

 

4. Clicking Add on the Distribution tab launches the Computer Search screen. Select Search by resource pool name option, then click Search. Select your cross-platform resource pool in the Available items box. Click Add, then Ok.

image

 

5. Click Ok, then Close the Run As Profile Wizard.

 

Perform the same steps for the UNIX/Linux Privileged Account profile, using the privileged account you setup earlier.

 

Discovery Wizard

Navigate to Administration > Device Management in the Operations console to start the discovery wizard, and UNIX/Linux computers.

image

 

On the Discovery Criteria screen, Add the Linux host information and click Set credentials. Keep Discovery type set to All computers.

image

 

On the Credentials Settings screen, select User name and password and enter the account information that was created previously. Select This account does not have privileged access on the drop down list. Then click Ok.

image

 

The Credentials section on the Discovery Criteria screen should look like this.

image

 

Click Save, select the Cross-Platform Monitoring Resource Pool, and click Discover.

image

 

The host should be discovered and you can continue by ticking the checkbox and clicking Manage. If you scroll to the right in this window, you can also see that operating system, version, and architecture information was discovered.

image

 

After clicking manage, the wizard goes through an installation, validation, and certificate signing process. Given that all goes well, a typical installation should take 30-60 seconds for a single host. Within minutes you should see the new Linux agent appear in the monitor space, with objects being discovered and monitored.

image

 

 

Here are some useful resources to check in run into problems.

Unable to add the domain to the subject

Troubleshooting UNIX/Linux Agent Discovery in System Center 2012 – Operations Manager

Scenario #1 – “There were no computers that met the specified discovery criteria”

Scenario #2 – “SSH discovery failed.” with “unspecified problem”

Scenario #3 – “Did not find a matching supported agent”

Scenario #4 – “New computer shows as Platform: Unknown and Version: Unknown”

Sudoers Manual

12 thoughts on “Install SCOM Agent on Red Hat Enterprise Linux 6 (linux agent installation)”

  1. Hi. Really nice post! It’s exactly what I was looking for.
    Additionally I’d like to ask you regarding the manual install. Where can I obtain the rpm file?
    Browsing scom agent folder I just found a shell script.

    Any help would be appreciated!

    Regards

    Davi

  2. Hi,

    I’ve followed the steps mentioned and created the (Server Name.cert) certificate kept it in D:\Program Files\Microsoft System Center 2012 R2\Operations Manager\Server path, Linux machine is discovered, once I click on Manage ending with error message” Task invocation failed with error code -2130771918. Error message was: Certificate signing module called with an empty certificate. Check that the certificate on the remote host is valid, and that the remote host can be accessed over SSH”.

  3. for those who are still wondering. the agents can be found here on the management server:

    C:\Program Files\Microsoft System Center 2012 R2\Operations Manager\Server\AgentManagement\UnixAgents\DownloadedKits

  4. This is a great guide.

    I’d like to mention that the firewall rule here is missing the -j ACCEPT parameter. Without it iptables will simply drop the packet. This was driving me nuts for a bit.

    1. Thanks for mentioning that, Brandon. This is actually in the command, but is cut off because there is no word-wrap in the post… Sorry about that! I must have done a copy/paste, and did catch it.

      iptables -I INPUT 1 -p tcp -s 192.168.2.0/24 --dport 1270 -m state --state NEW,ESTABLISHED -j ACCEPT

      There it is with word-wrap.

  5. Hi Jonathan,

    Nice post it worked as expected.

    I have only one issue here where the command to save the firewall config “/sbin/service iptables save” does not seem to work in red hat 7 and above.

    I get this error:
    The service command supports only basic LSB actions (start, stop, restart, try-restart, reload, force-reload, status). For other actions, please try to use systemctl.

    So after every reboot the config erases.

    I also tried using the below command let which ideally says to save the ruining config as a permanent config:

    firewall-cmd –runtime-to-permanent

    output says success as below:

    [root@redhatunix ~]# iptables -I INPUT 1 -p tcp -s 172.0.0.0/16 –dport 1270 -m state –state NEW,ESTABLISHED -j ACCEPT
    [root@redhatunix ~]# firewall-cmd –runtime-to-permanent
    success
    [root@redhatunix ~]#

    But after reboot the config erases causing the Agent to go to a grayed state and works again if i add the rule.

    Any idea on this ?

      1. Ok so i got the correct commandlet to add and save the config in the red hat 7.x firewall.

        Although it is not ip range specific but will just open the port for any source and stays after the box is rebooted as well:

        To add the rule for SCOM Agent:
        firewall-cmd –permanent –add-port=1270/tcp

        To validate:
        firewall-cmd –list-port

  6. Hello,

    Have you had any success with agents on RedHat7? Finding some difficulties with and trying to find out if the issue is related to SELinux, tmpfs (putting files under /tmp) or perhaps sudoers file.

    Any feedback is helpful

    Thanks!

Leave a Reply to Jonathan Cancel reply