Troubleshooting SSLVPN



By: ncashell

December 26, 2007 7:24 pm

Reads: 1005

Comments:0

Rating:0

Troubleshooting SSLVPN Server Installation Issues
Troubleshooting SSLVPN Server Initialization Issues
Troubleshooting SSLVPN Server Connection Issues
Troubleshooting SSLVPN Server Application Issues
  Troubleshooting Tools
  Troubleshooting Files
  Troubleshooting Steps

When troubleshooting SSLVPN issues, you need to identify the stage where the problem appears. What we will cover in this AppNote are the main areas when problems crop up: installation, initialization, connection, and accessing applications. At each stage, we will outline the key log files and tools, as well as the troubleshooting steps to identify what the issue is.


Troubleshooting SSLVPN Server Installation Issues

Whether you are installing the SSLVPN server on the Linux Access Gateway or on a separate box, the log files from the install are in the /tmp/novell_access_manager/ directory. They start with “sslvpn_inst_*” (where * includes the date and time of the install). Any errors shows at this level need to be addressed; otherwise, everything following may have issues.

The install script is a bash script that runs the install_sslvpn() and configure_sslvpn() methods. It copies over two rpms (see below), sets up the servlet tomcat environment, imports the device, and configures the device with the parameters that were specified during the install. A sample output of the install log file is shown below. Always check to make sure no exceptions were thrown in this file.

Started installation at 2007-05-31_17:15:28
Installing Novell SSLVPN:
Preparing...                ##################################################
novl-sslvpn-servlet         ##################################################
novl-sslvpn                 ##################################################
Configuring Novell SSLVPN:
Old Vcc/VCDNServerAddress: 127.0.0.1
Old Vcc/VCDNServerPort: 2843
Old Vcc/DeviceManagerAddress: 10.1.16.5
Old Vcc/DeviceManagerPort: 8444
Old Vcc/JccAddress: 10.1.16.5

If the device is not imported into the Admin Console after the installation, look at the following:

Messages file in the /var/log directory, to make sure that the VCC and JCC services are up and running successfully
jcc-0.log.0 in the /opt/novell/devman/jcc/logs directory of the SSLVPN server. This file contains the log entries for the server communications module related to interaction of the SSLVPN server with the Administration Console, such as imports, certificates, and configuration.


Troubleshooting SSLVPN Server Initialization Issues

Assuming the SSLVPN server was installed correctly, and a valid configuration has been assigned,

1. Make sure that all the SSLVPN specific services are running correctly.

The SSLVPN server uses the following processes (available with a ‘ps aux’ output, or by just doing ‘pgrep sockd’ and making sure a valid PID is returned).

/opt/novell/sslvpn/bin/connman
sockd -f /etc/opt/novell/sslvpn/sockd.conf
stunnel /etc/opt/novell/sslvpn/stunnel.conf
openvpn --config /etc/opt/novell/sslvpn/openvpn-server.conf

When you are sure these processes are running,

2. Run the ‘sslvpnc –status’ command at the SSLVPN server console.

This gives an indication as to the state of the various processes – ideally, they should all be either “registered” or “running”. The only exception to this rule is with the servlet – it may report that it is not registered. This is actually normal when no request from a browser for the SSLVPN server has been processed by the servlet. Once a request is processed by the SERVLET, the state should go to registered.

linuxlab5:~/novell-access-manager-3.0.1-251 # sslvpnc --status
SOCKD is running
SOCKD is registered
STUNNEL is running
OPENVPN is running
SERVLET is registered

This information is also available on the Health screen for the SSLVPN server in Admin Console.

If a problem exists at this stage, there are a few log files to look at for more details. The log files to analyze depend on the component that appears to be the problem. The main SSLVPN server log files are shown below:

/var/opt/novell/tomcat4/logs/catalina.out : This file contains all the information specific to the servlet component. When the servlet loads, the Tomcat environment must be set up correctly; any errors at this level will the displayed here. The servlet, as we saw above, also interacts with the Connection Manager, and the state of this interaction will be displayed here.

A sample (truncated) output of SSLVPN entries when the servlet is being registered is shown below. Check this file to make sure that the settings match what have been configured and that there are no errors or exceptions reported. If an administrator inputs incorrect information during the SSLVPN install (e.g., an invalid IP address syntax), the install will continue, but the servlet will not register correctly – and that will be visible from this file.

10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN:Initializing all the server
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN : Socket Descriptor = 	Socket[addr=/127.0.0.1,port=2010,localport=19337]
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN : Successfully sent the message 	SERVLET_INIT_REQUEST to connman
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] received message 	SERVLET_INIT_RESPONSE_SUCCESS
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN: received config change message
	
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN : ServerIP Addr -->6d100293
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN : Stunnel Port --> 7777
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN : ProxyIP Addr(Not used currently) --	>11223344
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN : Number of DNS Addresses -->1
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN : DNS Server Address -->100000a
	:
	:
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN : Openvpn Port -->7777
	0/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN : Openvpn Protocol -->udp
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN : Openvpn ServerIP Addr -->6d100293
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN : starting async thread to listen for 	events from connection manager ....
	10/16/07 10:48 PM [thread:SSLVPN_Connman_Async_Thread_0] SSLVPN : Events Socket Descriptor = 	Socket[addr=/127.0.0.1,port=2010,localport=19340]
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN : Successfully sent the message 	Unknown Message : 13065 to connman
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] received message Unknown Message : 12819
	10/16/07 10:48 PM [thread:SSLVPN_Connman_Async_Thread_0] SSLVPN : Successfully sent the message Unknown 	Message : 12548 to connman
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN: Number of servers associated with this 	servlet = 1
	10/16/07 10:48 PM [thread:SSLVPN_Connman_Async_Thread_0] received message Unknown Message : 12824
	10/16/07 10:48 PM [thread:http-147.2.16.109-8080-Processor5] SSLVPN: server ip = 127.0.0.1 port = 2010

/var/log/messages : This file reports details of the SSLVPN server configuration (addresses, ports, policies) at initialization time, as well as the status of all the SSLVPN server daemons. Look at this file to make sure that all the services (VCC, OpenVPN, Dante/server and Policy) report as registered. If any of the services fail to register, an error will be reported here.

Nov 28 12:59:52 linuxlab5 SSLVPN: Server Info :
	Nov 28 12:59:52 linuxlab5 SSLVPN: Server IP Address   :  147.2.16.109
	Nov 28 12:59:52 linuxlab5 SSLVPN: Socks Internal Interface :  eth1
	Nov 28 12:59:52 linuxlab5 SSLVPN: OpenVPN Subnet Address             :  10.8.0.0
	Nov 28 12:59:52 linuxlab5 SSLVPN: OpenVPN Subnet Mask                :  255.255.0.0
	Nov 28 12:59:52 linuxlab5 SSLVPN: OpenVPN IP Address                :  147.2.16.109
	Nov 28 12:59:52 linuxlab5 SSLVPN: Number of DNS entries = 1
	Nov 28 12:59:52 linuxlab5 SSLVPN: DNS Server Address : 10.0.0.1
	:
	Nov 28 12:59:52 linuxlab5 SSLVPN: Rule Name   : Any_Role_TCP_Modify_Network
	Nov 28 12:59:52 linuxlab5 SSLVPN: Role    : Any
	Nov 28 12:59:52 linuxlab5 SSLVPN: priority:   : 1
	Nov 28 12:59:52 linuxlab5 SSLVPN: Destination Network    : 10.0.0.0
	Nov 28 12:59:52 linuxlab5 SSLVPN: Destination Mask    : 255.0.0.0
	Nov 28 12:59:52 linuxlab5 SSLVPN: Protocol    : TCP
	Nov 28 12:59:52 linuxlab5 SSLVPN: Service Name     : AnyTCP
	Nov 28 12:59:52 linuxlab5 SSLVPN: Port     : 0
	:
	Nov 28 12:59:52 linuxlab5 SSLVPN: Starting SSLVPN server ...
	Nov 28 12:59:52 linuxlab5 SSLVPN: Created the VCC socket
	Nov 28 12:59:52 linuxlab5 SSLVPN: Bound VCC socket.
	Nov 28 12:59:52 linuxlab5 SSLVPN: VCC is ready to accept connections DeviceManager
	Nov 28 12:59:52 linuxlab5 sockd[2603]: created new negotiatorchild
	Nov 28 12:59:52 linuxlab5 sockd[2603]: Policy store initialized for negotiator child
	Nov 28 12:59:52 linuxlab5 SSLVPN: OpenVPN is registered with Connman
	Nov 28 12:59:52 linuxlab5 sockd[2604]: created new requestchild
	Nov 28 12:59:52 linuxlab5 sockd[2604]: Policy store initialized for request child
	:
	Nov 28 12:59:52 linuxlab5 sockd[2596]: Policy store initialized
	Nov 28 12:59:52 linuxlab5 sockd[2596]: dante/server v1.1.19 running
	Nov 28 12:59:52 linuxlab5 sockd[2608]: created new iochild
	Nov 28 12	:59:52 linuxlab5 sockd[2608]: Policy store initialized for io child

/var/log/stunnel.log : This file is specific to the Kiosk mode and doesn’t log much during initialisation. The following is what should be displayed when the service is successfully loaded, and we register with the Connection Manager.

2007.11.28 13:06:39 LOG5[2740:1076402400]: stunnel 4.20 on i686-suse-linux with OpenSSL 0.9.7d 17  	
	2007.11.28 13:06:39 LOG5[2740:1076402400]: Threading:PTHREAD SSL:ENGINE Sockets:POLL,IPv4
	2007.11.28 13:06:39 LOG6[2740:1076402400]: file ulimit = 100000 (can be changed with 'ulimit -n')
	2007.11.28 13:06:39 LOG6[2740:1076402400]: poll() used - no FD_SETSIZE limit for file descriptors
	2007.11.28 13:06:39 LOG3[2740:1076402400]: Registering to connman .

/var/log/novell-openvpn.log : This file is specific to the Enterprise mode and shows the OpenVPN server initializing. As with stunnel in Kiosk mode, it needs to register with the Connection Manager but it also needs to initialize the Tun driver required to route traffic through the tunnel encrypted. Use this file to make sure the tun0 device opens and has the correct IP addresses assigned for routing, and that the port has the correct transport (UDP/TCP) and number.

Wed Nov 28 13:10:40 2007 OpenVPN 2.0.9 i686-suse-linux [SSL] [EPOLL] built on Aug 27 2007
	Wed Nov 28 13:10:40 2007 MANAGEMENT: TCP Socket listening on 127.0.0.1:17170
	Wed Nov 28 13:10:40 2007 Successfully registered to ConnMan
	Wed Nov 28 13:10:40 2007 WARNING: --keepalive option is missing from server config
	Wed Nov 28 13:10:40 2007 Diffie-Hellman initialized with 1024 bit key
	Wed Nov 28 13:10:40 2007 WARNING: file '/etc/opt/novell/sslvpn/certs/stunnel_dm.pem' is group or others 	accessible
	Wed Nov 28 13:10:40 2007 WARNING: This configuration may accept clients which do not present a certificate
	Wed Nov 28 13:10:40 2007 TLS-Auth MTU parms [ L:1557 D:138 EF:38 EB:0 ET:0 EL:0 ]
	Wed Nov 28 13:10:40 2007 TUN/TAP device tun0 opened
	Wed Nov 28 13:10:40 2007 /sbin/ifconfig tun0 10.8.0.1 pointopoint 10.8.0.2 mtu 1500
	Wed Nov 28 13:10:40 2007 /sbin/route add -net 10.8.0.0 netmask 255.255.0.0 gw 10.8.0.2
	Wed Nov 28 13:10:40 2007 Data Channel MTU parms [ L:1557 D:1450 EF:57 EB:4 ET:0 EL:0 ]
	Wed Nov 28 13:10:40 2007 UDPv4 link local (bound): 147.2.16.109:7777
	Wed Nov 28 13:10:40 2007 UDPv4 link remote: [undef]
	Wed Nov 28 13:10:40 2007 MULTI: multi_init called, r=256 v=256
	Wed Nov 28 13:10:40 2007 IFCONFIG POOL: base=10.8.0.4 size=16382
	Wed Nov 28 13:10:40 2007 Initialization Sequence Completed

The example below looks at a snippet of the log file where the OpenVPN server was not running when queried for the health check with sslvpnc.

Mon Jun  4 22:37:36 2007 us=380709 OpenVPN 2.0.9 i686-suse-linux [SSL] [EPOLL] built on May 30 2007
	Mon Jun  4 22:37:36 2007 us=380824 MANAGEMENT: TCP Socket listening on 127.0.0.1:41656
	Mon Jun  4 22:37:36 2007 us=381875 Successfully registered to ConnMan
	Mon Jun  4 22:37:36 2007 us=381989 WARNING: --keepalive option is missing from server config
	Mon Jun  4 22:37:36 2007 us=384109 Diffie-Hellman initialized with 1024 bit key
	Mon Jun  4 22:37:36 2007 us=384408 WARNING: file '/etc/opt/novell/sslvpn/certs/stunnel_dm.pem' is group or 	others accessible
	Mon Jun  4 22:37:36 2007 us=384846 WARNING: This configuration may accept clients which do not present a 	certificate
	Mon Jun  4 22:37:36 2007 us=384888 TLS-Auth MTU parms [ L:1557 D:138 EF:38 EB:0 ET:0 EL:0 ]
	Mon Jun  4 22:37:36 2007 us=384938 TCP/UDP: Socket bind failed on local address 10.1.16.5:7777: Cannot assign 	requested address
	Mon Jun  4 22:37:36 2007 us=384950 Exiting
	

It turns out that the IP address we are trying to bind to (10.1.16.5) is an old IP address, and not the IP address we had configured in the Admin Console. The SSLVPN server configuration, stored in /etc/opt/novell/sslvpn/config.xml on the SSLVPN server itself and read in by the various daemons, was not updated with the latest configuration from the Admin Console. Simply running the ‘sslvpnc –update’ on the server forces a re-read of the data from the Admin Console and fixes the issue.


Troubleshooting SSLVPN Server Connection Issues

When the SSLVPN server is configured and a user can connect successfully, the following green light will be displayed in the SSLVPN portal page:

When no green light is observed, the state of the connection status remains in ‘connecting’ or an error message will be displayed instead, as shown below.

The next stage is to try to determine what is causing this error. Based on the authentication flows covered above for the Kiosk and Enterprise mode, you need to find out at what stage the process breaks down. The following is a list of checks to be performed at each stage:

1. Make sure that the Web server defined in the Access Gateway configuration is at the IP address and TCP port of the servlet engine. A common configuration error is to leave the Web server TCP port at the default 80, resulting in a ’504 Gateway Timeout’ error being reported on the browser when trying to connect to the SSLVPN servlet.

2. Make sure that the SSLVPN policy is setup for the SSLVPN protected resource. This policy needs to send the following things:

- Proxy cookie in the X-SSLVPN-PROXY-SESSION-COOKIE header
- Username and password in the Authentication Header
- User roles in the X-SSLVPN-ROLE header

If the servlet fails to receive any of this data, the connection will fail. By looking at the log entries in the ‘Log Entries’ section of the portal, the error details will indicate what the problem was. In this case, I had enough information to take a LAN trace of traffic to the SSLVPN servlet and confirm that the X-SSLVPN-PROXY-SESSION-COOKIE was missing from the request.

3. Make sure that the client downloads and initializes correctly.

After the SSLVPN client binaries have been sent by the SSLVPN servlet to the SSLVPN client, they need to be installed and the components initialized. This may be a common source of errors for customers. The following screenshot shows a typical error:

Again, looking at the logs from the ‘Log entries’ section of the portal, or clicking Save Logs on the same page to create the applet.log file, the following information will be visible:

SSL VPN Applet Logs:

Wed Dec 05 19:32:53 GMT 2007  SSL VPN Applet: Novell SSL VPN ConnectApplet version 3.0.1.170
Wed Dec 05 19:32:53 GMT 2007  SSL VPN Applet: OS Language : English (United Kingdom)
Wed Dec 05 19:32:53 GMT 2007  SSL VPN Applet: Checking Client Integrity. Please wait ...
Wed Dec 05 19:34:06 GMT 2007  SSL VPN Applet: IOException. 2 java.io.FileNotFoundException:
C:\Documents and Settings\ncashell\novl-sslvpn-service-install.exe (The process cannot access the file because it is being used by another process)
Wed Dec 05 19:34:13 GMT 2007  SSL VPN Applet: Error: Failed to download SSL VPN files

In the above example, a download from a previous Enterprise mode connection had failed and certain resources were not released correctly. The log file indicated that the binaries being sent down were not fully downloaded and that the novl-sslvpn-service-install.exe was in use. By simply running the C:\Program Files\Novell SSLVPN Service\uninstall.exe program, the environment was cleaned up so that a subsequent connection from this SSLVPN client to the SSLVPN server allowed all files to download successfully.

4. Make sure that the CIC policies activated on the SSLVPN server can be executed successfully.

Before the connection is made to the SSLVPN server, a number of things need to be validated on the client side. Part of these checks involve detecting the version of the client if already installed, the mode of operation (and offering the Enterprise mode to users even if they do not have root privaledges), gathering the policy info for the user, the Client Integrity Checks (CIC) to be performed, and the certificate validation.

When an error occurs at each of these stages, the corresponding error displayed in the ‘Log entries’ section of the portal will point you to where the problem is. For example, I have a CIC policy defined that requires that a specific virus-checking program needs to be running on my host before the connection can take place. During the initial communication and handshaking with the servlet, a check for that virus-checking program will be carried out on the client. If the application is not located, or is not the right version, or runs on the wrong platform, the connection will fail to come up, and the following error will be reported.

5. Make sure that the certificate handshake is successful.

The certificate exchange occurs when the SSLVPN tunnel is brought up. This occurs at different stages, depending on the mode of operation. With Kiosk mode, the tunnel is brought up when data is sent by the application to the remote server via the tunnel. With Enterprise mode, however, the tunnel is brought up after the client has initialized.

The server certificate that the client expects is the default one. If a third-party certificate is installed and added to the SSLVPN certificate device store, the SSL handshake will fail. Again, looking at the Log entries section of the portal, or hitting the ‘Save logs’ button on the same page to create the applet.log file, the following information will be visible:

SSL VPN Applet Logs :

Wed Nov 21 17:18:57 EST 2007  SSL VPN Applet: Installed SSL VPN trust manager
Wed Nov 21 17:18:57 EST 2007  SSL VPN Applet: Novell SSL VPN ConnectApplet version 3.0.1.170
Wed Nov 21 17:18:57 EST 2007  SSL VPN Applet: OS Language : English (United States)
Wed Nov 21 17:18:57 EST 2007  SSL VPN Applet: Checking Client Integrity. Please wait ...
Wed Nov 21 17:19:00 EST 2007  SSL VPN Applet: Installing SSL VPN client, please wait....
Wed Nov 21 17:19:56 EST 2007  SSL VPN Applet: Starting SSL VPN Client in Enterprise mode...
Wed Nov 21 17:19:59 EST 2007  SSL VPN Applet: MANAGEMENT: Received Fatal Error :>FATAL:VERIFY ERROR, depth=0, error=unable to get local issuer certificate: /C=IE/ST=Dublin/L=Dub/O=Novell/OU=accessManager/CN=*.neorsd.org (OpenSSL)
Wed Nov 21 17:19:59 EST 2007  SSL VPN Applet: Sending disconnect message to Server.
Wed Nov 21 17:19:59 EST 2007  SSL VPN Applet: Uninstalling SSL VPN client, please wait....
Wed Nov 21 17:20:05 EST 2007  SSL VPN Applet: Successfully uninstalled client.

Another common example where the certificate validation fails is when the SSLVPN client and server time is out of sync. The SSLVPN client tries to validate the timestamps on the certificates. If it cannot, an error is thrown in this applet file.

6. Make sure that connectivity to the SSLVPN server is available.

When the binaries are downloaded and initialized, and the versions, policies and certificates are validated, the final stages of the connection includes exchanging data with the SSLVPN server. The SSLVPN server listens out by default on TCP 7777 (Kiosk) or UDP 7777 (Enterprise mode). Data is sent to these ports during the connection phase in Enterprise mode. In Kiosk mode, the data goes to the port only when a user tries to access a remote application through the SSLVPN tunnel. If there are any intermediate devices such as firewalls or Network Address Translators that can intercept, redirect and/or drop requests, the outgoing requests from the SSLVPN client may not reach the server.

The trace below shows the final stages of a connection request to an Enterprise mode server where the UDP requests to port 7777 are being filtered by an intermediate firewall. The end result is that the SSLVPN client never gets any responses to its request, and the connection will fail.

By using ‘tcpdump -i any -s 0 -w ssslvpncon.cap’ you can take a trace of the traffic coming into the SSLVPN server. You need to make sure that all SSLVPN client requests to the SSLVPN server are visible and getting a response. If no requests are visible, take an equivalent trace on the client and make sure they are being sent. If they are, check whether firewalls or NATs are dropping the request or redirecting it to the wrong destination.


Troubleshooting SSLVPN server Application issues

Before looking at troubleshooting application level issues, let’s look at the tools and log files that you can use to troubleshoot the application layer:

Troubleshooting Tools

a) TCPDUMP, Wireshark: These tools (available on Windows and Linux) allow an administrator to decode the traffic in and out on the SSLVPN server. With these tools, you can confirm whether the encrypted data arrived at the SSLVPN server and whether this server proxied or forwarded the request to the back end application. Note that TCPDUMP can trace packets on the loopback interface so SOCKS data is also available for checking. Once the request goes to the application server, there may be enough information to decode the request and response to make sure it is valid – understanding the application data is outside the scope of this document. TCPDUMP is the main tool required for troubleshooting application layer problems.

b) netcat : When the SSLVPN server processes an incoming client request and must send the application data on to the server, it does so to a specific address and port combination. Using netcat on the SSLVPN server, you can make sure there is a listener on the back-end TCP- or UDP-based application. Using ‘netcat <$app_srv_ip_addr> <$port>‘ for remote TCP applications or ‘netcat <$app_srv_ip_addr> <$port> -u’ for UDP based applications, you can confirm that the back-end application is listening for a request by checking for an ‘open’ response. Examples include:

linuxlab5:/var/log # netcat -v -v 147.2.16.159 80
www.mylag.com [147.2.16.159] 80 (http) open
 
linuxlab5:/var/log # netcat -v -v -u 147.2.16.159 123
www.mylag.com [147.2.16.159] 123 (ntp) open

c) SSLVPN portal page: The Policies tab on the SSLVPN portal page defines the applications and subnets that are accessible for this user. When a user cannot access a remote application through the SSLVPN tunnel, check the Policies tab to make sure that the user has a policy allowing access to the application required.

The Log Events tab on the same portal shows any errors that may have occurred accessing the session. Always check this tab for details. There may not always be useful information here – at which point you would analyze the files mentioned in the next section – but be aware of it. More importantly, it confirms the mode of operation being used for this session (Enterprise versus Kiosk) and allow you to look at subsequent log files to gather more useful data.

The Statistics screen shows the amount of data sent and received through the tunnel. It’s a simple way of checking whether the application data is actually going out the tunneled interface. If it is not incrementing as the application is being accessed, the policies may not be correct.

d) SSLVPN configuration ‘Debug Level’ setting: This sets a more verbose logging level for both the OpenVPN and stunnel daemons. With this setting enabled, more verbose information will be available in both the stunnel.log file and novell-openvpn.log files in SSLVPN server/var/messages directory.

Troubleshooting Files

The key troubleshooting files to look at depends on the mode of operation and the OS platform the SSLVPN client is running on. The following list of client-specific files has been broken down into Host OS and SSLVPN modes, and a small explanation is given with both.

Linux Platform

Kiosk mode – ~/.sslvpn/log directory

applet.log: This contains the output of the ‘Log entries’ screen on the SSLVPN portal page. It reports the status on each phase of connection. It will often report errors when the SSLVPN client cannot connect to the server, but will report little after the connection has been made. An example of the SSL VPN Applet Logs might include the following:

Wed Jul 25 14:33:16 BST 2007  SSL VPN Applet: Installed SSL VPN trust manager
Wed Jul 25 14:33:17 BST 2007  SSL VPN Applet: Novell SSL VPN ConnectApplet version 3.0.1.6
Wed Jul 25 14:33:17 BST 2007  SSL VPN Applet: OS Language : English (United Kingdom)
Wed Jul 25 14:33:17 BST 2007  SSL VPN Applet: Checking Client Integrity. Please wait ...
Wed Jul 25 14:33:21 BST 2007  SSL VPN Applet: Installing SSL VPN client, please wait....
Wed Jul 25 14:33:23 BST 2007  SSL VPN Applet: Starting SSL VPN Client in Enterprise mode...
Wed Jul 25 14:33:23 BST 2007  SSL VPN Applet: Unable to run specified command 
Wed Jul 25 14:33:30 BST 2007  SSL VPN Applet: Sending disconnect message to Server.
Wed Jul 25 14:33:31 BST 2007  SSL VPN Applet: ERROR : User closed the browser
Wed Jul 25 14:33:31 BST 2007  SSL VPN Applet: Successfully uninstalled client.

polresolver.log: This is a very useful file in determining DNS details and whether the request has been sent through the tunnel. This file reports details regarding the destination host (IP address and port) that the SSLVPN client is trying to access, as well as the allowed/denied response from the policy engine. A sample entry from this log files might be:

27.11.2007 18:20:14 LOG6: stats_initialize:  bindingto /home/neil/.sslvpn/polResolver/stunipc
27.11.2007 18:20:15 LOG3: Sending Cookie Ack STatus 0
27.11.2007 18:20:15 LOG3: DNS message received from the applet
27.11.2007 18:20:15 LOG6: DNS IP: 10.0.0.1
27.11.2007 18:20:15 LOG6: Domain Name:internal.novell.com
27.11.2007 18:20:15 LOG3: Cannot modify DNS information.Needs root privilages
27.11.2007 18:20:15 LOG3: Unable to update DNS entries
27.11.2007 18:20:18 LOG3: Applet Stats Request Received 
27.11.2007 18:23:17 LOG6: Policy Resolution for toip 11.0.0.11,port 22, proto 2 is resolved to 2
27.11.2007 18:23:17 LOG6: Access to Network\Host (11.0.0.11) is "Allow(Encrypt)" 

SSLVPNINstall.log: This includes details regarding the installation of the SSLVPN binaries sent down by the servlet, and invoking the environment for the SSLVPN client applications to be SSL- enabled. It also includes priority of message in the file so you can gauge the severity of the problem. An example set of entries from this file might include:

DEBUG:    Tue Nov 27 18:20:14 GMT 2007 Client package extracted
INFO:     Tue Nov 27 18:20:14 GMT 2007 /home/neil/.sslvpn_firefox/libdsocks.so already present
DEBUG:    Tue Nov 27 18:20:14 GMT 2007 Libraries setup
DEBUG:    Tue Nov 27 18:20:14 GMT 2007 Updated Stunnel.conf
DEBUG:    Tue Nov 27 18:20:14 GMT 2007 Updated sslize_bashrc
DEBUG:    Tue Nov 27 18:20:14 GMT 2007 Updated sslize_tcshrc
DEBUG:    Tue Nov 27 18:20:14 GMT 2007 Updated sslize script
DEBUG:    Tue Nov 27 18:20:14 GMT 2007 Updated ICA script
WARNING:  Tue Nov 27 18:20:14 GMT 2007 /home/neil/policy.txt - Policy file is not present
DEBUG:    Tue Nov 27 18:20:14 GMT 2007 Execed the binaries
DEBUG:    Tue Nov 27 18:20:14 GMT 2007 Desktop Shortcuts and Toolbar Launchers updated for sslizing
DEBUG:    Tue Nov 27 18:20:14 GMT 2007 SSLVPN Client shell not invoked
INFO:     Tue Nov 27 18:20:14 GMT 2007 SSLVPN Successfully Installed
DEBUG:    Tue Nov 27 18:20:15 GMT 2007 Installsc invoked to create a Secondary SSLVPN Client shell
INFO:     Tue Nov 27 18:20:15 GMT 2007 Secondary SSLVPN Client shell invoked

stunnel.log: This file simply returns the status of the handshake between the stunnel client and server components, including certificate and cipher details. An example entry might include:

LOG5[19864:3083188416]: stunnel 4.20 on i686-suse-linux with OpenSSL 0.9.7g 11 Apr 2005
LOG5[19864:3083188416]: Threading:PTHREAD SSL:ENGINE Sockets:POLL,IPv4
LOG6[19864:3083188416]: file ulimit = 8192 (can be changed with 'ulimit -n')
LOG6[19864:3083188416]: poll() used - no FD_SETSIZE limit for file descriptors
LOG5[19865:3085859744]: sslvpn accepted connection from 127.0.0.1:35454
LOG5[19865:3085859744]: sslvpn connected remote server from 147.2.36.147:56693
LOG5[19865:3085859744]: VERIFY OK: depth=1, /OU=Organizational CA/O=linuxlab5_tree
LOG5[19865:3085859744]: VERIFY OK: depth=0, /CN=test-stunnel/OU=accessManager/O=novell
LOG6[19865:3085859744]: SSL connected: new session negotiated
LOG6[19865:3085859744]: Negotiated ciphers: AES256-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(256) Mac=SHA1

// Enterprise mode

applet.log: Contains the same info as above in Kiosk mode.

OpenVPNInstall.log: Contains the same info as the SSLVPNINstall.log Kiosk mode file above. You can see details about the installation of the OpenVPN client and the subsequent environment setup.

openvpn.log: Contains all the details regarding the Enterprise mode setup. This includes details about the SSLVPN server you need to communicate with, the certificate exchanged, and ciphers selected for the SSL session. It also includes very useful information about the Tun interface setup, including DNS details, MTU of interface, and the hosts and subnets that the user can access through the SSLVPN server. An example of the entries found in such a file might include:

Tue Nov 27 18:43:58 2007 OpenVPN 2.0.9 i686-suse-linux [SSL] [EPOLL] built on Aug 27 2007
Tue Nov 27 18:43:58 2007 MANAGEMENT: TCP Socket listening on 127.0.0.1:47927
Tue Nov 27 18:43:58 2007 Need password(s) from management interface, waiting...
Tue Nov 27 18:43:58 2007 MANAGEMENT: Client connected from 127.0.0.1:47927
Tue Nov 27 18:43:58 2007 IMPORTANT: OpenVPN's default port number is now 1194, based on an official port number assignment by IANA.  OpenVPN 2.0-beta16 and earlier used 5000 as the default port.
Tue Nov 27 18:43:58 2007 Control Channel MTU parms [ L:1557 D:138 EF:38 EB:0 ET:0 EL:0 ]
Tue Nov 27 18:43:58 2007 Data Channel MTU parms [ L:1557 D:1450 EF:57 EB:4 ET:0 EL:0 ]
Tue Nov 27 18:43:58 2007 Local Options hash (VER=V4): '2dd3fcaf'
Tue Nov 27 18:43:58 2007 Expected Remote Options hash (VER=V4): '8114d01c'
Tue Nov 27 18:43:58 2007 UDPv4 link local: [undef]
Tue Nov 27 18:43:58 2007 UDPv4 link remote: 147.2.16.109:7777
Tue Nov 27 18:43:58 2007 TLS: Initial packet from 147.2.16.109:7777, sid=b88c4203 abf34d91
Tue Nov 27 18:43:59 2007 VERIFY OK: depth=1, /OU=Organizational CA/O=linuxlab5_tree
Tue Nov 27 18:43:59 2007 VERIFY X509NAME OK: /CN=test-stunnel/OU=accessManager/O=novell
Tue Nov 27 18:43:59 2007 VERIFY OK: depth=0, /CN=test-stunnel/OU=accessManager/O=novell
Tue Nov 27 18:43:59 2007 Data Channel Encrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
Tue Nov 27 18:43:59 2007 Data Channel Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Tue Nov 27 18:43:59 2007 Data Channel Decrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
Tue Nov 27 18:43:59 2007 Data Channel Decrypt: Using 160 bit message hash'SHA1' for HMAC authentication
Tue Nov 27 18:43:59 2007 Control Channel: TLSv1, cipher TLSv1/SSLv3 DHE-RSA-AES256-SHA, 2048 bit RSA
Tue Nov 27 18:43:59 2007 [test-stunnel] Peer Connection Initiated with 147.2.16.109:7777
Tue Nov 27 18:44:00 2007 SENT CONTROL [test-stunnel]: 'PUSH_REQUEST' (status=1)
Tue Nov 27 18:44:00 2007 PUSH: Received control message: 'PUSH_REPLY,ping 10,ping-exit 120,route 10.8.0.1,dhcp-option DNS 10.0.0.1,dhcp-option DOMAIN internal.novell.com,route 10.0.0.0 255.0.0.0,route 11.0.0.11 255.255.255.255,ifconfig 10.8.0.18 10.8.0.17'
Tue Nov 27 18:44:00 2007 OPTIONS IMPORT: timers and/or timeouts modified
Tue Nov 27 18:44:00 2007 OPTIONS IMPORT: --ifconfig/up options modified
Tue Nov 27 18:44:00 2007 OPTIONS IMPORT: route options modified
Tue Nov 27 18:44:00 2007 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
Tue Nov 27 18:44:00 2007 TUN/TAP device tun0 opened
Tue Nov 27 18:44:00 2007 /sbin/ifconfig tun0 10.8.0.18 pointopoint 10.8.0.17 mtu 1500
Tue Nov 27 18:44:00 2007 /sbin/route add -net 10.8.0.1 netmask 255.255.255.255 gw 10.8.0.17
Tue Nov 27 18:44:00 2007 /sbin/route add -net 10.0.0.0 netmask 255.0.0.0 gw 10.8.0.17
Tue Nov 27 18:44:00 2007 /sbin/route add -net 11.0.0.11 netmask 255.255.255.255 gw 10.8.0.17
Tue Nov 27 18:44:00 2007 Initialization Sequence Completed

Windows Platform

The Kiosk mode files for this platform are identical to that of the corresponding Linux files above. The exceptions are that the polresolver.log, socksclient.log and stunnel.log files are located under the C:\Documents and Settings\<$USER>\Novell\SSLVPN\log directory. The equivalent of the above SSLVPNINstall.log is simply the install.log file and is located under the C:\Documents and Settings\<$USER>\Novell\SSLVPN\install directory.

For the Enterprise mode files, the same files exist as those for the Linux platform above, but again, the directories are different. The applet.log, and openvpn.log files are located in C:\Documents and Settings\<$USER>\Novell\SSLVPN\log.

There is one new file – the SSLVPN service instalation log is located under C:\Program Files\Novell SSLVPN Service. This file displays the details about the installation of the OpenVPN service and the subsequent environment setup.

Troubleshooting Steps

Now that you understand the tools and files available for troubleshooting issues, let’s look at steps to carry out when troubleshooting SSLVPN application-level issues. Note that up to now, no data has been sent across the SSLVPN tunnel at all, even though the connection is successful. Only when you open an application on the client that requires access to an SSLVPN protected application server, will data be sent across the SSLVPN tunnel.

It’s assumed you’ve followed the steps described above to rule out issues with the SSLVPN installation, initialization, or connection layer. Then, the first step is to identify the mode of operation and whether the client has picked up any errors. This data is available by looking at the Log Events tab of the SSLVPN portal page and checking to see whether the problem is with the Kiosk or Enterprise mode. This page may also give some useful data regarding any errors experienced on the SSLVPN tunnel.

Kiosk Mode Issues

If the SSLVPN connection is in Kiosk mode, this rules out a lot of routing issues. The first thing to do is to identify the back-end resource that the user is trying to go to. Then:

1. Make sure no errors exist in the applet.log for the user.
2. Make sure that the command failing is not a ping command. Ping uses ICMP raw sockets, and since we only support TCP and UDP sockets, ping requests will never succeed.
3. Check the polresolver.log on the client to make sure the connected user is allowed access to the host or network that appears to be problematic.

The log file will report an ‘allow’ or ‘deny’ state for that user. For example, here’s what you would see in the log file if the user was not allowed access to the application (ICMP in this case):

Tue Dec  4 12:38:33 2007 Protocol Name:ICMP, Destination Address:11.0.0.11, Destination Port:0, Action : Deny

At this stage, you can assume that the request has been passed down to the stunnel client, to be transmitted out the LAN. One way to confirm this is to do the following:

1. Look at the ‘Statistics’ screen on the SSLVPN portal page and make sure that the packets ‘Sent’ and ‘Received’ counters are incrementing. Or, you can use TCPDUMP or Wireshark on the client, look at the TCP connections to the SSLVPN server port, and make sure they are going out the right LAN interface.

2. Check the routing table on the SSLVPN client and make sure that the next hop for the SSLVPN server IP address is the correct interface.

3. Check the stunnel.log file on the SSLVPN client and make sure that no errors occur there. First you need to launch the client application that needs to talk to the remote server behind the SSLVPN. Then the tunnel will be activated, and any errors here will be displayed in this file. The following shows an issue where the SSL handshake fails because of an invalid server certificate:

2007.11.27 18:20:14 LOG5[19864:3083188416]: stunnel 4.20 on i686-suse-linux with OpenSSL 0.9.7g 11 Apr 2005
2007.11.27 18:20:14 LOG5[19864:3083188416]: Threading:PTHREAD SSL:ENGINE Sockets:POLL,IPv4
2007.11.27 18:20:14 LOG6[19864:3083188416]: file ulimit = 8192 (can be changed with 'ulimit -n')
2007.11.27 18:20:14 LOG6[19864:3083188416]: poll() used - no FD_SETSIZE limit for file descriptors
2007.11.27 18:23:17 LOG5[19865:3085859744]: sslvpn accepted connection from 127.0.0.1:35454
2007.11.27 18:23:17 LOG5[19865:3085859744]: sslvpn connected remote server from 147.2.36.147:56693
2007.11.27 18:23:17 LOG5[19865:3085859744]: MANAGEMENT: Received Fatal Error :>FATAL:VERIFY ERROR, depth=0, error=unable to get local issuer certificate: /C=IE/ST=Dublin/L=Dub/O=Novell/OU=accessManager/CN=*.neorsd.org (OpenSSL

If the packets ‘sent’ are incrementing but the packets ‘received’ are not, you need to look at the server side in more detail.

1. Run ‘tcpdump -i any -s 0 -n -w sslvpn.cap’ on the SSLVPN server. Check the resulting sslvpn.cap file with Wireshark and make sure the requests from the SSLVPN client are arriving at the LAN interface on the server. If this is failing, check a network diagram for any intermediate device that could be processing this request, such as a firewall. Make sure this firewall is capable of forwarding the SSLVPN destined TCP segments to the SSLVPN server.

2. Look at the above sslvpn.cap trace and check to see if requests are being sent on the loopback interface to the SOCKS server (TCP port 1080). If this is failing, make sure that iptables or the SuSEFirewall2 service is not running on the SSLVPN server and blocking the requests. Run ‘rcSuSEFirewall2 stop’ to disable this service temporarily for checking.

3. Assuming you are still in a ‘connected’ state from the SSLVPN client, run the ‘tail -f /var/log/stunnel.conf’ command on the SSLVPN server console and try launching the client application again. Ideally, you should see some data displayed on the SSLVPN server console indicating that the stunnel daemon received data:

// connecting to stunnel server
2007.12.04 14:50:19 LOG7[4978:1076402400]: sslvpn accepted FD=18 from 147.2.36.148:45448
2007.12.04 14:50:19 LOG7[4978:1076472752]: sslvpn started
2007.12.04 14:50:19 LOG7[4978:1076472752]: FD 18 in non-blocking mode
2007.12.04 14:50:19 LOG5[4978:1076472752]: sslvpn accepted connection from 147.2.36.148:45448
// SSL handshake
2007.12.04 14:50:19 LOG7[4978:1076472752]: SSL state (accept): before/accept initialization
2007.12.04 14:50:19 LOG7[4978:1076472752]: SSL state (accept): SSLv3 read client hello A
2007.12.04 14:50:19 LOG7[4978:1076472752]: SSL state (accept): SSLv3 write server hello A
2007.12.04 14:50:19 LOG7[4978:1076472752]: SSL state (accept): SSLv3 write certificate A
2007.12.04 14:50:19 LOG7[4978:1076472752]: SSL state (accept): SSLv3 write server done A
2007.12.04 14:50:19 LOG7[4978:1076472752]: SSL state (accept): SSLv3 flush data
2007.12.04 14:50:19 LOG7[4978:1076472752]: SSL state (accept): SSLv3 read client key exchange A
2007.12.04 14:50:19 LOG7[4978:1076472752]: SSL state (accept): SSLv3 read finished A
2007.12.04 14:50:19 LOG7[4978:1076472752]: SSL state (accept): SSLv3 write change cipher spec A
2007.12.04 14:50:19 LOG7[4978:1076472752]: SSL state (accept): SSLv3 write finished A
2007.12.04 14:50:19 LOG7[4978:1076472752]: SSL state (accept): SSLv3 flush data
2007.12.04 14:50:19 LOG7[4978:1076472752]:    1 items in the session cache
2007.12.04 14:50:19 LOG7[4978:1076472752]:    0 client connects (SSL_connect())
2007.12.04 14:50:19 LOG7[4978:1076472752]:    0 client connects that finished
2007.12.04 14:50:19 LOG7[4978:1076472752]:    0 client renegotiations requested
2007.12.04 14:50:19 LOG7[4978:1076472752]:    1 server connects (SSL_accept())
2007.12.04 14:50:19 LOG7[4978:1076472752]:    1 server connects that finished
2007.12.04 14:50:19 LOG7[4978:1076472752]:    0 server renegotiations requested
2007.12.04 14:50:19 LOG7[4978:1076472752]:    0 session cache hits
2007.12.04 14:50:19 LOG7[4978:1076472752]:    0 session cache misses
2007.12.04 14:50:19 LOG7[4978:1076472752]:    0 session cache timeouts
2007.12.04 14:50:19 LOG6[4978:1076472752]: SSL accepted: new session negotiated
// negotiated parameters
2007.12.04 14:50:19 LOG6[4978:1076472752]: Negotiated ciphers: AES256-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(256) Mac=SHA1
// Connecting to the socks server
2007.12.04 14:50:19 LOG7[4978:1076472752]: FD 19 in non-blocking mode
2007.12.04 14:50:19 LOG7[4978:1076472752]: sslvpn connecting 127.0.0.1:1080
2007.12.04 14:50:19 LOG7[4978:1076472752]: connect_wait: waiting 10 seconds
2007.12.04 14:50:19 LOG7[4978:1076472752]: connect_wait: connected
2007.12.04 14:50:19 LOG5[4978:1076472752]: sslvpn connected remote server from 127.0.0.1:25169
2007.12.04 14:50:19 LOG7[4978:1076472752]: Remote FD=19 initialized
// Application data sent
2007.12.04 14:50:19 LOG7[4978:1076472752]: Received 4 bytes from tunnel
2007.12.04 14:50:19 LOG7[4978:1076472752]: Sent 4 bytes of data

4. Compare the output of your file to determine at what stage the SSLVPN tunnel connection fails. If no errors are reported here, then the data going through the tunnel appears to be fine. You will either have an application-level issue at this stage or a possible routing issue.

To check for routing issues, try and simulate the request to the remote application server from the SSLVPN server itself. The easiest way of doing this is to use the netcat tool described in the tools section above to talk to the remote TCP or UDP port. If a netcat to the remote application from the SSLVPN server fails, then there is a routing or possible firewall issue on the internal network. Sort out this routing issue so that requests from the SSLVPN server reach their destination, and that the responses from this destination arrive back at the SSLVPN server

To check application level issues, one needs to be familiar with the application itself. If the application we’re tunneling is not one of the applications known to have issues with the Kiosk mode described in the Introduction section of this document (e.g. NETBIOS based, Applications opening TCP connections on both directions), then take a LAN trace using tcpdump and try and compare the request/response combination going through the SSLVPN server, with that when the user goes direct to the back end application.

A simple test to narrow the issue down to a specific application and not the SSLVPN tunnel is to create a simple traffic policy allowing a user ssh into a back end server. This is a basic application and assuming all works fine, we can rule out issues with the SSLVPN tunnel and focus on the problem application.

Enterprise mode issues:

If the SSLVPN connection is in Enterprise mode, routing issues become much more prevalent. The initial troubleshooting steps will be similar to that for the Kiosk mode above.

1. Make sure no errors exist in the applet.log for the user.
2. Check the polresolver.log on the client to make sure that the connected user is allowed access to the host or network that appears to be problematic (as in the Kiosk mode above). The log file will report an ‘allow’ or ‘deny’ state for that user. For example, here’s what you would see in the log file if the user was not allowed access to the application (ICMP in this case).

Tue Dec  4 13:38:33 2007 Protocol Name:ICMP, Destination Address:11.0.0.11, Destination Port:0, Action : Allow/Encrypt

At this stage, we can assume that the request has been passed down to the OpenVPN client, to be transmitted out the LAN. One way to confirm this is to do the following:

1. Look at the ‘Statistics’ screen on the SSLVPN portal page and make sure that the packets ‘Sent’ and ‘Received’ counters are incrementing. Or, using tcpdump or wireshark on the client, look at the TCP connections to the SSLVPN server port and make sure they are going out the right LAN interface.

2. Check the routing table on the SSLVPN client and make sure that the next hop for the SSLVPN server IP address is the correct interface.

3. Check the openvpn.log file for errors. The following example below shows a ‘route: netmask doesn’t match route address’ error that is easily missed. This error prevented any requests from going out the tunnel, because the route was invalid. This occurred because the traffic rule subnet address was incorrectly configured in the Admin Console, and this invalid subnet was sent down to the SSLVPN client.

Tue Nov 27 12:02:23 2007 OpenVPN 2.0.9 i686-suse-linux [SSL] [EPOLL] built on Aug 27 2007
	Tue Nov 27 12:02:23 2007 MANAGEMENT: TCP Socket listening on 127.0.0.1:34351
	Tue Nov 27 12:02:23 2007 Need password(s) from management interface, waiting...
	Tue Nov 27 12:02:24 2007 MANAGEMENT: Client connected from 127.0.0.1:34351
	Tue Nov 27 12:02:24 2007 IMPORTANT: OpenVPN's default port number is now 1194, based on an official port 	number assignment by IANA.  OpenVPN 2.0-beta16 and earlier used 5000 as the default port.
	Tue Nov 27 12:02:24 2007 Control Channel MTU parms [ L:1559 D:140 EF:40 EB:0 ET:0 EL:0 ]
	Tue Nov 27 12:02:24 2007 Data Channel MTU parms [ L:1559 D:1450 EF:59 EB:4 ET:0 EL:0 ]
	Tue Nov 27 12:02:24 2007 Local Options hash (VER=V4): '30065675'
	Tue Nov 27 12:02:24 2007 Expected Remote Options hash (VER=V4): '37840390'
	Tue Nov 27 12:02:24 2007 Attempting to establish TCP connection with 195.114.90.136:7777
	Tue Nov 27 12:02:24 2007 TCP connection established with 195.114.90.136:7777
	Tue Nov 27 12:02:24 2007 TCPv4_CLIENT link local: [undef]
	Tue Nov 27 12:02:24 2007 TCPv4_CLIENT link remote: 195.114.90.136:7777
	Tue Nov 27 12:02:24 2007 TLS: Initial packet from 195.114.90.136:7777, sid=e4d37d19 63695d3c
	Tue Nov 27 12:02:24 2007 VERIFY OK: depth=1, /OU=Organizational CA/O=rulx07_tree
	Tue Nov 27 12:02:24 2007 VERIFY X509NAME OK: /CN=run10.axens.local
	Tue Nov 27 12:02:24 2007 VERIFY OK: depth=0, /CN=run10.axens.local
	Tue Nov 27 12:02:25 2007 Data Channel Encrypt: Cipher 'AES-128-CBC' initialized with 128 bit key
	Tue Nov 27 12:02:25 2007 Data Channel Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
	Tue Nov 27 12:02:25 2007 Data Channel Decrypt: Cipher 'AES-128-CBC' initialized with 128 bit key
	Tue Nov 27 12:02:25 2007 Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
	Tue Nov 27 12:02:25 2007 Control Channel: TLSv1, cipher TLSv1/SSLv3 DHE-RSA-AES256-SHA, 1024 bit RSA
	Tue Nov 27 12:02:25 2007 [run10.axens.local] Peer Connection Initiated with 195.114.90.136:7777
	Tue Nov 27 12:02:26 2007 SENT CONTROL [run10.axens.local]: 'PUSH_REQUEST' (status=1)
	Tue Nov 27 12:02:26 2007 PUSH: Received control message: 'PUSH_REPLY,ping 10,ping-exit 120,route 	10.8.0.1,dhcp-option DNS 10.129.224.28,dhcp-option DNS 10.129.224.27,dhcp-option DNS 10.129.228.2,dhcp-	option DOMAIN axens.local,route 10.129.224.40 255.255.252.0,ifconfig 10.8.0.54 10.8.0.53'
	Tue Nov 27 12:02:26 2007 OPTIONS IMPORT: timers and/or timeouts modified
	Tue Nov 27 12:02:26 2007 OPTIONS IMPORT: --ifconfig/up options modified
	Tue Nov 27 12:02:26 2007 OPTIONS IMPORT: route options modified
	Tue Nov 27 12:02:26 2007 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
	Tue Nov 27 12:02:26 2007 TUN/TAP device tun0 opened
	Tue Nov 27 12:02:26 2007 /sbin/ifconfig tun0 10.8.0.54 pointopoint 10.8.0.53 mtu 1500
	Tue Nov 27 12:02:26 2007 /sbin/route add -net 10.8.0.1 netmask 255.255.255.255 gw 10.8.0.53
	Tue Nov 27 12:02:26 2007 /sbin/route add -net 10.129.224.40 netmask 255.255.252.0 gw 10.8.0.53
	route: netmask doesn't match route address
	Usage: route [-nNvee] [-FC] []           List kernel routing tables
	       route [-v] [-FC] {add|del|flush} ...  Modify routing table for AF.
	
	       route {-h|--help} []              Detailed usage syntax for specified AF.
	       route {-V|--version}                  Display version/author and exit.
	
	        -v, --verbose            be verbose
	        -n, --numeric            don't resolve names
	        -e, --extend             display other/more information
	        -F, --fib                display Forwarding Information Base (default)
	        -C, --cache              display routing cache instead of FIB

	  =Use '-A ' or '--'; default: inet
	  List of possible address families (which support routing):
	    inet (DARPA Internet) inet6 (IPv6) ax25 (AMPR AX.25) 
	    netrom (AMPR NET/ROM) ipx (Novell IPX) ddp (Appletalk DDP) 
	    x25 (CCITT X.25) 
	Tue Nov 27 12:02:26 2007 ERROR: Linux route add command failed: shell command exited with error status: 4
	Tue Nov 27 12:02:26 2007 Initialization Sequence Completed

If the packets ‘sent’ are incrementing but the packets ‘received’ are not (very common!), then one must look at the server side in more detail.

1. Run ‘tcpdump -i any -s 0 -n -w sslvpn.cap’ on the SSLVPN server. Check the resulting sslvpn.cap file with Wireshark and make sure the requests from the SSLVPN client are arriving at the LAN interface on the server. If this is failing, check a network diagram for any intermediate device that could be processing this request, such as a firewall. Make sure this firewall is capable of forwarding the SSLVPN destined TCP segments to the SSLVPN server.

2. Look at the /proc/net/dev file and check to see if the tun0 interface ‘Receive’ data is incrementing. This will indicate whether the tun0 OpenVPN interface is receiving any data.

3. If the LAN (eth0 interface) receives data, but the corresponding tun0 counters do not increment, check whether routing is enabled on the SSLVPN server. Run ‘cat /proc/sys/net/ipv4/ip_forward’ and make sure setting is at 1 (enabled).

4. Make sure that the route back to the SSLVPN server is available. By default, the source IP of the incoming request will be in the 10.8.0.0/16 range. When the SSLVPN server routes the request to the destination application server, it will need to have a route back to this 10.8.0.0/16 subnet through the SSLVPN server. Here’s a typical request that reaches the destination application server but has no route back. It shows the following:

linuxlab5:/var/log # tcpdump -i any host 11.0.0.11
tcpdump: WARNING: Promiscuous mode not supported on the "any" device
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
13:22:26.851210 IP 10.8.0.6.40778 > 11.0.0.11.ssh: S 4270098905:4270098905(0) win 5840 
13:22:27.851210 IP 10.8.0.6.40778 > 11.0.0.11.ssh: S 4270098905:4270098905(0) win 5840 

The request is being forwarded to 11.0.0.11 with the source IP address of 10.8.0.6. The host 11.0.0.11 has no route back to the SSLVPN server for 10.8.0.6, so it sends it to the default router, and the datagram eventually gets dropped. Failure to route the response back through the SSLVPN server will result in a communication problem and the application will fail.

There are two options around this:

a) Add a route entry for the 10.8.0.0/16 subnet via the SSLVPN server on all remote application servers (a lot of work), or

b) Use iptables to rewrite the source IP address from the 10.8.0.0/16 subnet address to that of the private interface of the SSLVPN server. A sample command to do this would be

iptables -t nat -A POSTROUTING -s 10.8.0.0/16 -j SNAT –to 10.0.0.1

where 10.0.0.1 is the IP address of the private interface in the SSLVPN server. Looking at the same tcpdump output as above, with iptables remapped source IP addresses, you would see this:

linuxlab5:~ # tcpdump -i any -n icmp
tcpdump: WARNING: Promiscuous mode not supported on the "any" device
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
16:01:46.991061 IP 10.8.0.18 > 11.0.0.11: icmp 64: echo request seq 1
16:01:46.991094 IP 10.0.0.1 > 11.0.0.11: icmp 64: echo request seq 1
16:01:46.991725 IP 11.0.0.11 > 10.0.0.1: icmp 64: echo reply seq 1
16:01:46.991737 IP 11.0.0.11 > 10.8.0.18: icmp 64: echo reply seq 1

You can see that the incoming SSLVPN assigned IP address of 10.8.0.18 is mapped to 10.0.0.1 before sending the request out to the destination host 11.0.0.11.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)

Tags:
Categories: Technical Solutions

Disclaimer: As with everything else at NetIQ Cool Solutions, this content is definitely not supported by NetIQ, so Customer Support will not be able to help you if it has any adverse effect on your environment.  It just worked for at least one person, and perhaps it will be useful for you too.  Be sure to test in a non-production environment.

Comment