2.4 Managing Secure Communication for Agents

Although you normally set the security level for a site during installation, you can change the security level after installation using the NQKeyGenWindows.exe program for Windows agents, or the NQKeyGenUNIX.exe program for UNIX agents. You can also use the programs any time you need to create and manage key file information for one of the agent security options.

2.4.1 Changing the Security Level for Management Servers

After installation, you can use the NQKeyGenWindows.exe program to change the security level for communication between the management server and Windows-based agents. For UNIX agents, use the NQKeyGenUNIX.exe program.

If you change the security setting for the management server, update the security setting for all agents in the site. Also, if you are changing from no security to security level 1 or 2, generate or identify a repository key to use before changing the security level. For more information, see Generating a Repository Key.

To change the security level for the management server:

  1. On a management server computer, open a Command Prompt window and change to the NetIQ\AppManager\bin directory.

  2. Run the appropriate program to set the security level for the management server using the following format:

    NQKeyGenWindows|NQKeyGenUNIX -db database_name:user_name:sql_server\instance ‑seclev level

    For example, to use your current Windows account name and password and set the security level to use encryption only (security level 1) with a QDB installed on the server NYC2006, you would type a command-line similar to this:

    NQKeyGenWindows -db qdb::nyc2006 -seclev 1

    NOTE:

    • All management servers that connect to the same repository must use the same security level.

    • For encryption or management server authentication and encryption, use the same key file.

  3. Change the security level for all of your agents to match the new security level setting. For more information, see Changing the Security Level for Agents.

  4. Stop and restart the NetIQ AppManager Management Service (NetIQms). This allows the management server to receive the new security level information.

2.4.2 Changing the Security Level for Agents

If you change the security level for the management server, you must also update the security setting for every agent.

For UNIX agents, you can change the security setting from the UNIX Agent Manager console. For information about UNIX Agent Manager, see the NetIQ UNIX Agent documentation, which is included in t he AppManager UNIX download package.

To change the security level for a Windows agent:

  1. Start the Control Center console.

  2. In the Enterprise Layout view of the Navigation pane, select the Knowledge Scripts view of a management group that includes the agent computer where you want to change the agent security level.

  3. In the view pane, select the AgentConfigSecurityLevel Knowledge Script.

  4. In the Tasks pane, click Create New Job.

  5. Click the AMAdmin tab in the Knowledge Script pane.

  6. In the Select Servers dialog box, select the agent computer where you want to run the Knowledge Script and then click OK.

  7. Click the Values tab, and:

    • Select the new security level from the Security level list.

    • Set the event notification and event severity parameters as desired.

    NOTE:If you change the security level from security level 1 or 2 to unencrypted communications, the management server ignores the event raised because it is not encrypted. Therefore, no event indicator is displayed in the Control Center console if you change the security level to unencrypted communications. If you are changing from unencrypted communications to security level 1 or 2, you must generate or identify the agent key to use before changing the security level. For more information, see Generating a Repository Key.

  8. Click OK to start the job.

  9. After updating all of your Windows agents, manually stop and restart each management server in the management site by stopping and then restarting the NetIQ AppManager Management Service (NetIQms).

As an alternative, you can run NQKeyGenWindows.exe directly on an agent to set the security level for the agent or to set the security level for a remote agent. For example, to change the security level on an agent to use encryption without authentication, type a command similar to this:

NQKeyGenWindows -agentseclev 1

For more information about using NQKeyGenWindows options, see Key File Utility for Windows Agents.

2.4.3 Generating a Repository Key

If you are using security level 1 or 2 (encryption or authentication and encryption) to secure communications between the management server(s) and agent computers, generate a new encryption key and store it in the repository.

To generate a new repository key for agents:

  1. On a management server computer, open a Command Prompt window and change to the NetIQ\AppManager\bin directory.

  2. Run the appropriate program to generate a new key and store the key information in the repository:

    NQKeyGenWindows|NQKeyGenUNIX -db database_name:user_name:sql_server/instance -new

    Variable

    Description

    database_name

    The name of the AppManager repository.

    user_name

    A valid SQL Server login with permission to access the AppManager repository.

    NOTE:If you are using Windows authentication to connect to the repository, leave the user name blank. If you are using SQL Server authentication, type a SQL Server user name for connecting to the repository. The program prompts for the password to use for the SQL Server account.

    sql_server\instance

    The name of the SQL Server computer and SQL Server instance name (if applicable) where the AppManager repository is installed.

    For example, to run NQKeyGenWindows.exe on a computer named NYC2003 with Windows authentication, type a command similar to this:

    NQKeyGenWindows -db qdb::nyc2003 -new
  3. Type a password for the repository key. If you want to extract the key into a file or use this key in another repository, you need to know this password. For information about sharing a key across multiple repositories, see Extracting and Sharing Key Information from the Repository.

  4. Run the appropriate program with the following command-line format to extract the portion of the key for the agents to use:

    NQKeyGenWindows|NQKeyGenUNIX -db database_name:user_name:sql_server\instance ‑ckey filelocation

    NOTE:If you are using Windows authentication to connect to the repository, leave the user name blank. For SQL Server authentication, type a SQL Server user name for connecting to the repository. The program prompts for the password to use for the SQL Server account.

    In specifying a path for the file, use a directory that you can access from the computers to be managed.

  5. Stop and restart the NetIQ AppManager Management Service (NetIQms) to register the new key with the management server.

2.4.4 Extracting and Sharing Key Information from the Repository

The NQKeyGenWindows.exe and NQKeyGenUNIX.exe programs can extract repository encryption key information and save it in a password-protected file. Saving this information in a file allows you to share a single key across multiple repositories.

If you want to create this password-protected file, run the appropriate program using the following command:

NQKeyGenWindows|NQKeyGenUNIX -db database_name:user_name:sql_server/instance‑skey filelocation

NOTE:If you are using Windows authentication to connect to the repository, leave the user name blank. To use SQL Server authentication, type a SQL Server user name for connecting to the repository. The program prompts for the password to use for the SQL Server account.

To check the key into another repository from the file location specified:

  1. On a management server computer, open a Command Prompt window and change to the NetIQ\AppManager\bin directory.

  2. Run the appropriate program to import the key pair into the repository.

    For example, if you created the key using the password ^myPass and extracted the encrypted key to the file location C:\Security\AMkey.txt, type the following command to import the key pair into the repository QDB01 on the computer SFO2003:

    NQKeyGenWindows|NQKeyGenUNIX -db QDB01:smithj:SFO2003 -change C:\Security\AMkey.txt
  3. Use the password you used to create the key in the repository. Provide the key file password ^myPass.

  4. After you import the key, stop and restart the AppManager Management Service (NetIQms) to register the new key with the management server.

2.4.5 Extracting the Key File

The NQKeyGenWindows.exe and NQKeyGenUNIX.exe programs can extract a portion of the key information stored in the repository and save it in a file. You can then make this agent key file available to all of your agents.

To extract the portion of the key for the agents to use:

  1. On a management server, run the appropriate program with the following command-line format:

    NQKeyGenWindows|NQKeyGenUNIX -db database_name:user_name:SQL_Server/instance‑ckey filelocation

    NOTE:If you are using Windows authentication to connect to the repository, leave the user name blank. For SQL Server authentication, type a SQL Server user name for connecting to the repository. The program prompts for the password to use for the SQL Server account

  2. Specify a path for the file that you can access from the agent computers.

  3. (Conditional) For Windows agents, use the AMAdmin_AgentConfigSecurityKey Knowledge Script to distribute the agent key file to all of your Windows agents. For more information, see Updating the Key File.

  4. (Conditional) For UNIX agents, run the AMAdminUNIX_AgentUpdateSecurityLevel Knowledge Script, replace the old public key file in the UNIX agent data directory with the new public key file, and restart the UNIX agent. For more information, see the AppManager for UNIX Servers Management Guide, available on the AppManager Modules Documentation page.

2.4.6 Updating the Key File

For maximum security and to prevent keys from being compromised over time, periodically create new keys and distribute new key files to all agent computers. This process, called “re‑keying,” applies when you are using security level 1 or 2 (encryption or management server authentication and encryption).

Updating the Key File for Windows Agents

You can choose to update the agent key file manually, which is more secure, or you can add a registry key that allows the agent to automatically detect changes to the key file on the management server and request the new key file. Both methods are described below, as well as a security risk that exists with the automatic update detection feature.

To replace the agent key file manually (most secure method):

When changing the agent key file, update all of the agent computers before updating the management servers. All management servers and agent computers within a management site must use the same security level and the same key file.

Because manual re‑keying requires you to restart all of your management servers, plan carefully. If you cannot update the key file for some agents, you will experience communication failures between the management server and those agents. In addition, any time you update the key file, you might experience a temporary loss of communication between the management server and the agents. Therefore, consider disabling communication with some agents before updating key files or security.

  1. Use the NQKeyGenWindows.exe utility to generate a new key and store the key information in the repository.

  2. Use the NQKeyGenWindows.exe utility to extract the agent portion of the key and save it to a file location.

  3. Use the Control Center console to run the AMAdmin_AgentConfigSecurityKey Knowledge Script on the agent computers you want to update. When creating the job, click the Values tab and:

    • Type the path to the new agent key file in the Location of key file field.

    • Type the password for the new agent key file in the Encryption password field.

    • Set the event notification and severity parameters.

  4. Click OK to start the job.

  5. Verify that all jobs complete successfully.

  6. After updating all of your Windows agents, manually stop and restart each management server in the management site by stopping and then restarting the AppManager Management Service (NetIQms).

To enable the automatic update detection feature (least secure method):

With this method, the agent automatically detects changes to the key file on the management server and requests the new key file using a checksum of the previous key to identify itself. If the management server successfully identifies the agent using the checksum, the management server uses the checksum to encrypt the new key file and sends the new key file to the agent. The agent uses the checksum to decrypt the new key file.

The automatic update detection feature is disabled by default.

WARNING:With this method, any time the agent or the management servers sends the checksum, it sends the checksum as clear text. It is possible for rogue software to capture the checksum and decrypt the new key file.

Be careful when editing your Windows registry. If there is an error in your registry, your computer might become nonfunctional. If an error occurs, you can restore the registry to its state when you last successfully started your computer. For more information, see the Help for the Windows Registry Editor.

  1. On the management server, open the Registry Editor and navigate to HKEY_LOCAL_MACHINE\SOFTWARE\NetIQ\AppManager\4.0\NetIQms\Config (on 32-bit operating systems) or HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\NetIQ\AppManager\4.0\NetIQms\Config (on 64-bit operating systems).

  2. Add a registry key with the following values:

    • Name: RPC_KeyMat_AutoUpdate_Enabled

    • Type: DWORD

    • Data: 1

    If you want to disable the feature later, change the Data value to 0.

Updating the Key File for UNIX Agents

If you have more than one management server in your management site, the management server acting as the current primary management server for the agent must complete the re-keying process. If communication with the acting primary management server is interrupted before re-keying is complete, failover to the inactive management server will not take place and communication with the UNIX agent will be lost.

To prevent this problem, check the status of all management servers and ensure that the agent can communicate with the management server before you start re-key operations. Never stop the UNIX agent management server during re-key operations.

To replace the public/private key pair:

  1. Use the NQKeyGenUNIX.exe utility to create a new public/private key pair and store it in the repository.

  2. Stop and restart the management server so it picks up the new key pair.

After you restart the management server, the next communication from the UNIX agent fails when the agent attempts to authenticate the management server using the old public key. The UNIX agent then uses an encrypted message to request the new public key from the management server by sending a message that includes a checksum for its current key.

The management server uses the checksum to retrieve the key pair from the repository and to encrypt the new public key with the previous private key, and then it sends the signature and the new encrypted public key back to the UNIX agent. The UNIX agent decrypts the new public key using its old public key, which verifies that the new key is from the appropriate management server and begins using the new public key.

You can use the NQKeyGenUNIX.exe utility to remove historical keys from the repository at any time. If you remove the historical keys, however, you must manually replace the public key file on each UNIX agent when you change the public/private key pair. In this case, the automated re‑keying process fails.