AppNote: Configuring an IPsec Tunnel between NBM and NSM



By: gvaidya

June 2, 2005 4:17 am

Reads: 205

Comments:0

Rating:0

This appnote provides detailed configuration & implementation steps to set up a site-to-site (Net to Net) connection between Novell Security Manager (NSM) 5.1 and Novell Border Manager 3.8 servers.

Introduction

Novell Security Manager, powered by Astaro, is the latest offering from Novell in perimeter security. NSM offers packet filtering, intrusion detection, virus and spam filtering, and content filtering, along with VPN services. In addition to PPTP- and L2TP- over-IPsec-based VPN, Novell Security Manager 5.1 also offers IPsec-based VPN service. If NSM 5.1 is deployed in any environment where it needs to securely communicate with an NBM 3.8 server, then a site-to-site (S2S) VPN connection can be established between the two.

This AppNote describes steps for configuring an S2S IPsec tunnel between NSM 5.1 and NBM 3.8. The network setup and IPsec configuration are explained first, followed by the configuration of the IPsec tunnel, using both Certificate and PSK mode. See also the Glossary of Terms at the end of this AppNote.

Setup

The overall setup for the IPsec tunnel is pictured below.

Figure 1 – x

This Appnote assumes certain conditions as described below.

Installed Software

  • NBM 3.8 with latest support pack is installed on the Netware Server (Netware 6.5).
  • NSM 5.1 is installed.
  • Basic net
  • work configuration done for configuring NSM using a web-based interface, such as WebAdmin.

Configuration

  • NBM 3.8 Server Name: F4E
  • WAN Interface IP Address: 164.99.160.98 / 255.255.252.0
  • LAN Interface IP Address: 192.168.10.1 / 255.255.255.0
  • Default Gateway : 164.99.160.88

The NBM 3.8 server connects the internal LAN 192.168.10.0/24 to the Internet. The NBM 3.8 server’s WAN (Internet) interface has the address 164.99.160.98.

  • NSM 5.1 Server Name: NSM
  • WAN Interface IP Address: 172.32.22.12 / 255.255.0.0
  • LAN Interface IP Address: 192.168.99.1 / 255.255.255.0
  • Default Gateway: 172.32.22.88

The NSM 5.1 server connects the internal LAN 192.168.99.0/24 to the Internet. The NSM 5.1 server’s WAN (Internet) interface has the address 172.32.22.12.

Router

For this test scenario we use a router to connect the WAN networks of the NBM 3.8 server (164.99.160.98) and the NSM 5.1 server (172.32.22.12). Addresses of the two interfaces of the router connecting the NBM 3.8 and NSM 5.1 server are: 164.99.160.10 and 172.32.22.10.

IPsec Parameters

For smooth interoperability between NSM 5.1 and NBM 3.8, it is recommended you use the following IPsec parameters for IKE Phase-1 and IKE Phase-2 SA negotiations. If you want to configure another set of parameters, ensure that the configuration at both ends (NBM 3.8 and NSM 5.1) is similar.

Here are the IKE Phase-1 parameters:

  • IKE mode: Main mode
  • IKE Lifetime: 14400 Secs / 240 Mins / 4 hours
  • IKE Encryption: Triple DES (3DES)
  • IKE integrity: MD5
  • IKE Group: (MODP 1024 group 2)
  • IKE Authentication: Certificate OR Pre-Shared Key

Here are the IKE Phase-2 parameters

  • IPsec mode: tunnel (for all VPN connections)
  • IPsec lifetime: 3600 Secs / 60 mins / 1 hour
  • IPsec encryption: Triple DES (3DES)
  • IPsec integrity: MD5
  • PFS: enabled (MODP 1024, group 2)
  • IP compression: disabled
  • Selectors: For all protocols between 192.168.99.0/24 and 192.168.10.0/24 IPv4 Subnets

Configuring the VPN

There are two types of authentication methods available for configuring Site-to-Site VPN between NBM 3.8 & NSM 5.1: X.509 Certificates and Pre-Shared Key (PSK).

Using the Pre-Shared Key is a relatively easy way to configure, but it is less secure and more difficult to maintain, compared to the X.509 certificate mode. Because configuring VPN is largely a one-time effort, it is recommended to use Certificate method. The following sections gives step-by-step procedures to configure VPN using both Certificate and PSK.

For the benefit of new users of NSM 5.1, it is worth introducing the configuration part. NSM 5.1 provides a web-based interface for configuration management. When you connect to NSM 5.1 Server by accessing “https://Your-NSM-Server-Address/” (in our case 192.168.99.1) and providing the username/password, you get the interface shown in Figure 2. The rest of this AppNote refers to menu items shown in the left panel of the browser/interface as shown in Figure-1.

Figure 2 – WebAdmin for Novell Security Manager powered by Astaro

Configuring Definitions and Rules for NSM 5.1

Novell Security Manager 5.1 uses definitions for configuring services. So before configuring IPsec VPN service on NSM 5.1, you should create the definitions needed for VPN service configuration. These are:

  • Network / Interface Definition
  • Packet Filter Rules
  • IPsec Policy Definition

Creating Network Definitions

To create network definition objects,

1. Go to Definitions on the left menu panel
2. Select Networks.

The list of default network definitions appears. Assuming that names given during configuring network interfaces are internal (default) and external respectively, this would create Internal (Network) and External (Interface) definitions by default.

Now you need to configure definitions for NBM server as given in the table below.

Name Type Address Comment
NBM_PUB_Interface Host 164.99.160.98 Public Interface of NBM
NBM_PVT_Network Network 192.168.10.0 / 24 Protected N/W of NBM
External (Interface) Host 172.32.22.12 Public Iface of NSM
Internal (Network) Network 192.168.99.0 / 24 Protected N/W of NSM

Table: Network Definitions

3. Select the drop down-box New Definition from the top panel.

4. Add the network definitions for the NBM server as shown in Figure 3.

Figure 3 – Adding Network Definition

Creating Packet Filter Rule

By default, NSM 5.1 only allows traffic to the management interface. To allow traffic across a protected network after the VPN tunnel is up, you need to create traffic rules (Packet Filter Rules).

  1. Go to Packet Filter in left menu panel.
  2. Select “Rules”.
  3. Select the New Rule drop-down list from the top panel.
  4. Add the following packet filter rules as described in the table below and Figure 4.

Note: There is an option in the VPN configuration for Auto Packet Filtering, but if you want to have specific traffic rules they must be created manually.

Source Destination Service Action
NSM_PVT_Network NBM_PVT_Network ANY Allow
NBM_PVT_Network NSM_PVT_Network ANY Allow

Table: Packet Filter Rules

Figure 4 – Configuring Packet Filter Rules

Creating the IPsec Policy

An IPsec policy object defines parameters for the IKE and IPsec negotiations. To create a new IPsec policy for NBM 3.8 server,

  1. Go to IPSec VPN in the left menu panel.
  2. Select Policies.
  3. Click New and then configure the values in the table below (as shown in Figure 5).
Parameter Value
Name NBM-Policy
ISAKMP (IKE) Settings
IKE Mode Main Mode
Authentication Algorithm 3DES 168bit
Encryption Algorithm MD5 160bit
IKE DH Group DH Group 5 (MODP1536)
SA Lifetime (secs) 14400
IPSec Settings
IPSec Mode Tunnel
IPSec Protocol ESP
Encryption Algorithm 3DES-CBC 168bit
Enforce Algorithm off
Authentication Algorithm MD5 160bit
SA Lifetime (secs) 3600
PFS PFS Group 2 (MODP1024)
Compression off

Table: New Ipsec Policy for NBM

Figure 5 – Adding an IPsec policy on NSM

Configuring IPsec Connection using Certificates

Setting Up Certificates for NSM 5.1

There are three basic steps involved in setting up a Certificate for NSM 5.1:

  1. Generating a new Signing CA
  2. Generating a new CSR for the NSM5.1 Server
  3. Issuing and exporting the Certificate

In this scenario we use the dedicated Certificate Authority (CA) for each VPN server. To configure CA and certificates for the NSM 5.1 server,

  1. Go to IPSec VPN in the left menu panel.
  2. Select CA Management as shown in Figure 6.

Figure 6 – CA Management Interface

To add a new Signing CA (as shown in Figure-6),

  1. Click New in the Certificate Authorities section.
  2. Select Generate on the next screen.
  3. Fill in the corresponding values for the certificate (name, passphrase, key size, country, organization, e-mail ID, etc.).
  4. Click Start to create the CA.

Figure 7 – Creating a new Certificate Authority

Once the CA is created, you can create a Certificate Signing Request (CSR). To add a new CSR (shown in Figure 8),

  1. Click New in the Host CSRs and Certificates section.
  2. Select Generate CSR on the next screen.
  3. Select VPN ID as the IPv4 address
  4. Fill in corresponding values for certificate (i.e. IP Address, Name, passphrase, key size, country, organization, email id etc)
  5. Click Start to create the CSR.

Figure 8 – Generating a Certificate Request for the Server Certificate

After generating the CSR, the next step is to issue the certificate for NSM 5.1 server. To issue the certificate (as shown in Figure 9),

  1. Check the box for the newly created CSR.
  2. From the drop-down list, select Issue CERT for CSR.
  3. Once the certificate is issued, check the box for the newly created Host Certificate.
  4. From the drop-down list, select Download as DER and provide passphrases.
  5. Save the file on the local system (such as NSM-Host-Cert.der)
  6. .

Figure 9 – Issuing and Exporing Certificate from CA Management

Configuring the VPN Service on NBM 3.8

The procedure for configuring the VPN service on the NBM 3.8 Server is shown in Figure 10.

  1. Configure Server Address and Tunnel Address as 164.99.160.98 / 255.255.252.0 and 112.1.1.1 / 255.0.0.0 respectively.
  2. Specify Key life time in minutes (Default value is 480; you may change it to 240 to match configuration at NSM 5.1 server)
  3. Check the Perfect Forward Secrecy check box.
  4. Check “Site to Site” check box, and select the master radio button.
  5. Click Details for Site to Site. It will show the Issuer Certificate (created automatically).
  6. Check the Subject Name and the browse for the server certificate, click it. The Certificate subject Name will automatically appear in the text box
  7. Provide the Protected network of the NBM 3.8 server in the Protected Networks list (in this case it would be 192.168.10.0 / 255.255.255.0)
  8. Click OK after completing the server configuration.

Figure 10 – NBM 3.8 VPN Server Configurationn

For more information, refer to the online documentation at:
http://www.novell.com/documentation/nbm38/index.html

Configuring the Verification CA on Both Servers

For the X509 certificate mode of authentication, both sides verify the authenticity of the certificates. This requires configuring verification CA on both the NBM 3.8 and the NSM 5.1 servers.

*To configure the NBM 3.8 Verification CA on the NSM 5.1 Server, follow the steps below.

First, from iManager, export the Trusted Root certificate (without private key) as “F4E-TrustedRoot.der”:

  1. Go to eDir Maintenance > Modify Object.
  2. Select Trusted Root certificate and export it.

Then, also in iManager, export the Public Key certificate as “F4E-PubKey-Cert.der”:

  1. Go to eDir Maintenance > Modify Object.
  2. Select Public Key Certificate and Export.

Next, upload the Trusted Root certificate as the Verification CA on NSM 5.1:

  1. Click New in the Certificate Authorities section.
  2. Select Upload in the next screen.
  3. Fill in the certificate name.
  4. Select the exported file “F4E-TrustedRoot.der”.
  5. Click “Start” button to upload Verification CA.

Finally, upload the Public Key certificate as the Host certificate on NSM 5.1:

  1. Click New in the Host CSRs and Certificates section.
  2. Select Upload CERT or CSR on the next screen.
  3. Select Certificate for the Type and DER (*.der) for the Format.
  4. Select the exported file “F4E-PubKey-Cert.der”.
  5. Click Start to upload the certificate (NBM-Host-Cert).

*To configure the NSM 5.1 Verification CA on the NBM 3.8 Server, you will need the NSM-Host-Cert.der certificate file you exported earlier.

From iManager, use the exported file NSM-Host-Cert.der to create Trusted Root Object (TRO) in the defalt TRC of NBM 3.8 server:

1. Add and activate the Local Ipsec Key on NSM 5.1.

NSM 5.1 requires setting local IPsec host key. To set and activate local IPSec key for NSM 5.1 server,

2. Go to “IPSec VPN” in the left menu panel.

3. Select “Local Keys” and go to the Local IPSec X.509 Key section.

4. Select NSM-Server from Local Certificates.

5. Enter the passphrase for host certificate and click Save.

This sets the local X.509 IPsec key as the NSM host certificate. The Activated Local key is shown in Figure 11.

Figure 11 – Activating Local Key at NSM Server

Creating a New VPN connection on NSM 5.1

As the final step in configuration on NSM 5.1 server, you need to define a VPN connection to the NBM 3.8 Server. To create a new IPSec connection between the NSM 5.1 and NBM 3.8 servers,

  1. Go to IPSec VPN in the left menu panel.
  2. Select Connections.
  3. Go to the New IPSec Connection section.
  4. Select Standard from the Type drop-down list.

This expands the new connection dialog box as shown in Figure 12.

Figure 12 – Creating new Ipsec connection on NSM Server

5. Enter the parameters shown in the table below to define a new connection, then click Add.

Parameter Value
Name NSM-NBM-Cert
Type Standard (Already Selected)
IPSec Policy NBM-Policy (created previously)
Auto Packet Filter Off
Strict Routing Off
End Point Definition
Local Endpoint External
Remote Endpoint NBM_PUB_Interface
Subnet Definition (optional)
Local Subnet Internal (Network)
Remote Subnet NBM_PVT_Network
Authentication of Remote Subnet(s)
Key X509: NBM-Host-Cert

Table: Parameters for new Ipsec definition on NSM Server

Adding NSM 5.1 as a Slave Server to NBM 3.8

From iManager, go to the Member Lists tab and then click Add. Configure the following parameters as shown in Figure 13.

Figure 13 – Adding NSM 5.1 as 3rd Party VPN server to NBM 3.8

  1. Configure the IP Address and the subnet mask of the NSM 5.1 server.
  2. Configure the tunnel IP Address in the same network as the NBM 3.8 server Tunnel IP. Here we have 112.2.2.2 / 255.0.0.0).
  3. Check the Non-Border Manager check box.
  4. Check Authentication Method.
  5. Configure the NSM 5.1 certificate as Issuer.
  6. Configure the corresponding subject name of certificate.
  7. In the Protected IP Network list, add 192.168.99.0 / 255.255.255.0.
  8. Click Apply and then OK.

Adding Third-Party Traffic Rules for NSM 5.1 on NBM 3.8

The steps below should produce the results shown in Figure 14.

  1. Select Third Party Traffic Rules tab. You will find a DENY rule for newly added NSM 5.1 Server.
  2. To add a new Traffic Rule, click New.
  3. Provide a name for the rule.
  4. Expand the 3rd-Party Server Configuration.
  5. Select the IP Address for NSM 5.1 Server (172.32.22.12) in the 3rd Party Server Gateway Address drop-down list.
  6. Click the Only Use IP List radio button, under Rule Applies To:
  7. Click Add and then provide the network IP Address for the NSM 5.1 Server. Here, it is 192.168.99.0 / 255.255.255.0
  8. Expand the NBM Server Protected Network list.
  9. Click the Only Use IP List radio button, under Rule Applies To:
  10. Click Add and then provide the network IP Address for the NBM 3.8 Server. Here it is 192.168.10.0 / 255.255.255.0
  11. Expand Define Action.
  12. Set the default for traffic to be encrypted.
  13. In the Encryption “key lifetime by time” enter the IKE Phase-2 SAs lifetime value.
  14. Change the default value to 60 as per the configuration on the NSM 5.1 side.
  15. As per the setting for NSM 5.1, make sure the Encryption Algorithm is 3DES and the Authentication Algorithm is HMAC- MD5.
  16. Click Apply and then OK.

Figure 14 – Adding a 3rd Party Traffic rule for NSM 5.1

Verifying the Tunnel

  1. Enable the IPsec connection named “NSM-NBM-Cert” from the NSM 5.1 server.
  2. Initiate the connection /traffic from any client in the protected (LAN) network of NBM 3.8 server to any client in the protected (LAN) network of NSM 5.1 Server, or vice versa. They should be able to ping, with the packets going encrypted between the servers.

Configuring the IPSec Connection using PSK

Setting up the S2S IPsec tunnel between NBM 3.8 and PSK is more or less same as described above, but the certificate configuration step can be omitted. Configuring the VPN service on NBM 3.8 is the same as described above.

Adding a Pre-Shared Key on NSM 5.1

NSM 5.1 requires you to create a remote Pre-Shared Key object for the PSK mode of authentication. To create a new Remote Key,

  1. Go to IPSec VPN from the left menu panel.
  2. Select Remote Keys.
  3. From the New Remote IPSec Key section, select the Key Type as PSK.
  4. Enter the Name as “NBM-PSK” and provide the value of Pre-shared key you intend to use in the Preshared Key field.
  5. Click Add to add the key.

Creating a New VPN Connection on NSM 5.1

Creating a new VPN connection for Pre-Shared key is similar to creating connection with X.509 – the difference is in the NAME and KEY fields. To create new IPSec connection between NSM 5.1 – NBM 3.8 server,

  1. Go to IPSec VPN in left menu panel.
  2. Select Connections.
  3. Go to the New IPSec Connection section.
  4. Select Standard from the Type drop-down list. This expands the new connection dialog.
  5. Enter the parameters to define a new connection. All parameters are the same as described in “Creating a New VPN Connection on NSM 5.1″ above, except that Name is now NSM-NBM-PSK, and Key (under Authentication of Remote Subnets) is PSK: NBM-PSK.
  6. Click Add to create the connection.

Adding NSM 5.1 as a Slave Server to NBM 3.8, for PSK

This process is similar to the one described above in “Adding an NSM 5.1 Server as Slave Server to NBM 3.8,” with minor differences in the authentication mode configuration.

  1. Configure the IP Address and the subnet mask of the NSM 5.1 server.
  2. Configure the tunnel IP Address in the same network as the NBM 3.8 server Tunnel IP. For this AppNote we have used 112.2.2.2 / 255.0.0.0.
  3. Check the Non-Border Manager checkbox.
  4. Check PSS as the Authentication Method.
  5. Configure the Pre-Shared Key the same way as you did for the NSM 5.1 server. For this AppNote, it is “novell”.
  6. In the Protected IP Network list, add 192.168.99.0 / 255.255.255.0.
  7. Click Apply and then OK.

The 3rd Party Traffic rule configuration is the same as described above.

Verifying the Tunnel

  1. Enable the IPsec connection named “NSM-NBM-PSK” from the NSM 5.1 server.
  2. Initiate the connection / traffic from any client in the protected (LAN) network of NBM 3.8 server to any client in the protected (LAN) network of NSM 5.1 Server, or vice versa. They should be able to ping with the packets going encrypted between the servers.

Tips and Tricks

If the tunnel negotiation is failing, you can view the logs in the NSM 5.1 server to determine the cause. To do this,

  1. Go to Local Logs in the left menu panel.
  2. Select Browse.
  3. On the NBM 3.8 machine, view /ETC/IKE/IKE.LOG for LOG messages. You can also look at VPN monitoring on NBM 3.8 for specific log messages and the status of the VPN connection.

Make sure all the IPsec parameters configured on both sides are the same for both Phase-1 and Phase-2 tunnel negotiations.

If the IPsec tunnel is coming up but protected network machines cannot communicate, then verify that firewalls on both NBM 3.8 and NSM 5.1 servers are configured to allow corresponding traffic.

Verify that protected networks are added for both NBM 3.8 and NSM 5.1 servers in the NBM 3.8 VPN configuration.

NBM 3.8 can interoperate with NSM 5.1 VPN on Linux in both Certificate Mode and PSK mode. Moreover, there is no need for both servers to be issued certificates from same certficates authority. Both the CAs can be different.

Glossary of Terms

  • SA (Security Association) – Unidirectional association between two VPN entities
  • S2S (Site to Site): VPN connection between two VPN server which secures two local sites or LANs. Also known as Net to Net VPN
  • IKE – Internet Keying Protocol
  • IPsec – Internet Protocol Security
  • NBM3.8 – Novell Border Manager, Version 3.8. Latest version of Novell Border Manager which has ICSA certification for both Firewall (4.0 Corporate) and IPsec (1.0D)
  • CA – Certificate Authority
  • TRC – Trusted Root Container
  • TRO – Trusted Root Object
VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)

Tags: ,
Categories: Uncategorized

Disclaimer: As with everything else at NetIQ Cool Solutions, this content is definitely not supported by NetIQ, so Customer Support will not be able to help you if it has any adverse effect on your environment.  It just worked for at least one person, and perhaps it will be useful for you too.  Be sure to test in a non-production environment.

Comment