18.2 Troubleshooting 100101043 and 100101044 Liberty Metadata Load Errors

The Identity Server is the identity provider for other Access Manager components. The Access Gateways, ESP-enabled SSL VPN servers, and J2EE Agents have Embedded Service Providers. When a device is imported into the Administration Console and an Identity Server configuration is selected for them, a trusted relationship is established with the Identity Server by using test certificates. When you change these certificates or change from using HTTP to HTTPS, you need to ensure that the trusted relationship is reestablished. Metadata is used for establishing trusted relationships.

The metadata exchanged between service providers and identity providers contains public key certificates, key descriptors for message signing, a URL for the SSO service, a URL for the SLO (single logout) service, and so on. With Access Manager, this metadata is accessible on both the Identity Server and the Embedded Service Provider of the device. Errors are generated when either the identity provider could not load the service provider’s metadata (100101043), or the service provider could not load the metadata of the identity provider (100101044).

If users are receiving either of these errors when they attempt to log in, verify the following:

If these steps do not solve your problem, try the following:

18.2.1 The Metadata

If you change the base URL of the Identity Provider, all service providers, including Embedded Service Providers, need to be updated so that they use the new metadata:

Embedded Service Provider Metadata

If you change the base URL of the Identity Provider, all Access Manager devices that have an Embedded Service Provider need to be updated so that new metadata is imported. To force a re-import of the metadata, you need to configure the device so it doesn’t have a trusted relationship with the Identity Server, update the device, reconfigure the device for a trusted relationship, then update the device. The following steps explain how to force the Access Gateway to re-import the metadata of the Identity Server.

  1. In the Administration Console, click Devices > Access Gateways > Edit > Reverse Proxies/Authentication.

  2. Select None for the Identity Server Cluster option, click OK twice, then update the Access Gateway.

  3. Click Edit > Reverse Proxies/Authentication.

  4. Select an Identity Server configuration for the Identity Server Cluster option, click OK twice, then update the Access Gateway.

Service Provider Metadata

If you have set up federation with another provider over the Liberty, SAML 1.1, SAML 2.0, CardSpace, or WS Federation protocol and you change the base URL of the Identity Server, you need to update the provider with the new metadata to reestablish the trusted relationship. If the provider is another Identity Server, follow the procedure below to update the metadata; otherwise, follow the provider’s procedures.

  1. In the Administration Console of the provider, click Devices > Identity Servers > Edit > [Protocol] > [Provider] > Metadata.

  2. Click Reimport.

  3. Follow the steps in the wizard.

    For more information, see Section 7.7, Managing Metadata.

18.2.2 DNS Name Resolution

When the service provider tries to access the metadata on the identity provider, it sends the request to the hostname defined in the base URL configuration of the Identity Server. The base URL in the Identity Server configuration is used to build all the metadata end points.

To view the metadata of the Identity Server with a DNS name of idpcluster.lab.novell.com, enter the following URL:

https://idpcluster.lab.novell.com:8443/nidp/idff/metadata

Scan through the document and notice the multiple references to https://idpcluster.lab.novell.com/... You should see lines similar to the following:

<md:SoapEndpoint>
   https://idpcluster.lab.novell.com:8443/nidp/idff/soap
</md:SoapEndpoint>

<md:SingleLogoutServiceURL>
   https://idpcluster.lab.novell.com:8443/nidp/idff/slo
</md:SingleLogoutServiceURL>

<md:SingleLogoutServiceReturnURL>
   https://idpcluster.lab.novell.com:8443/nidp/idff/slo_return
</md:SingleLogoutServiceReturnURL>

The Embedded Service Provider of the Access Gateway must be able to resolve the idpcluster.lab.novell.com hostname of the Identity Server. To test that it is resolvable, send a ping command with the hostname of the Identity Server. For example, from the Access Gateway:

ping idpcluster.lab.novell.com

The same is true for the Identity Server. It must be able to resolve the hostname of the Access Gateway. To discover the URL for the Access Gateway metadata:

  1. In the Administration Console, click Devices > Access Gateways > Edit > Reverse Proxy/Authentication.

  2. View the Embedded Service Provider section.

    The URL of the metadata is displayed in this section.

To view the metadata, enter the displayed URL. Scan through the document and notice the multiple references to the hostname of the Access Gateway. You should see lines similar to the following. In these lines, the hostname is ag1.provo.novell.com.

<md:SoapEndpoint>
   http://ag1.provo.novell.com:80/nesp/idff/spsoap
</md:SoapEndpoint>

<md:SingleLogoutServiceURL>
   http://ag1.provo.novell.com:80/nesp/idff/spslo
</md:SingleLogoutServiceURL>

<md:SingleLogoutServiceReturnURL>
   http://ag1.provo.novell.com:80/nesp/idff/spslo_return
</md:SingleLogoutServiceReturnURL>

To test that the Identity Server can resolve the hostname of the Access Gateway, send a ping command with the hostname of the Access Gateway. For example, from the Identity Server:

ping ag1.provo.novell.com

To view sample log entries that are logged when a DNS name cannot be resolved, see The Embedded Service Provider Cannot Resolve the Base URL of the Identity Server.

18.2.3 Certificate Names

Ensure that the certificates for the Identity Server and the Embedded Service Provider match the hostnames defined in the metadata URL (see Section 18.2.2, DNS Name Resolution).

When the Identity Server and the Access Gateway are enabled for HTTPS, all communication to these devices requires that the devices send back a server certificate. Not only must the certificate be assigned to the appropriate device, but the subject name of the device certificate must match the hostname of the device it is assigned to.

To verify the certificate name of the Identity Server certificate:

  1. In the Administration Console, click Devices > Identity Servers > Edit.

  2. Click the SSL Certificate icon.

    The SSL Connector keystore is displayed

  3. Verify that the subject name of the certificate matches the DNS name of the Identity Server.

    • If the names match, a certificate name mismatch is not causing your problem.

    • If the names do not match, you need to either create a certificate that matches or import one that matches. For information about how to create a certificate for the Identity Server, see Enabling SSL Communication in the NetIQ Access Manager 3.2 SP3 Setup Guide.

To verify the certificate name of the Access Gateway certificate:

  1. In the Administration Console, click Devices > Access Gateways > Edit > [Name of Reverse Proxy].

  2. Read the alias name of the server certificate, then click the Server Certificate icon.

  3. Verify that the Subject name of the server certificate matches the published DNS name of the proxy service of the Access Gateway.

To view sample log entries that are logged to the catalina.out file when the certificate has an invalid name, see The Server Certificate Has an Invalid Subject Name.

18.2.4 Certificates in the Required Trust Stores

Ensure that the issuers of the Identity Server and Embedded Service Provider certificates are added to the appropriate trusted root containers.

When the server certificates are sent from the identity provider to the service provider client, and from the service provider to the identity provider client, the client needs to be able to validate the certificates. Part of the validation process is to confirm that the server certificate has been signed by a trusted source. By default, well known external trusted certificates are bundled with Access Manager. You can view this list here: Administration Console > Security > Certificates > External Trusted Roots. If the issuer of server certificate is not present in the External Trusted Root list, the import the issuers of the server certificate (intermediate and trusted roots) into the correct trusted root stores:

  • The intermediate and trusted roots of the Embedded Service Provider certificate must be imported into the NIDP Trust Store.

  • The intermediate and trusted roots of the Identity Server certificate must be imported into the ESP Trust Store.

For more information, see Importing a Signed Certificate in the NetIQ Access Manager 3.2 SP3 Administration Console Guide.

If you use certificates generated by the Administration Console CA, the trusted root certificate is the same for the Identity Server and the Embedded Service Provider. If you are using external certificates, the trusted root certificate might not be the same, and there might be intermediate certificates that need to be imported.

To verify the trusted root certificates:

  1. In the Administration Console, click Security > Certificates.

  2. Determine the issuer of the Identity Server certificate and the Embedded Service Provider certificate:

    1. Click the name of the Identity Server certificate, note the name of the Issuer, then click Close.

    2. Click the name of the Embedded Service Provider certificate of the Access Gateway, note the name of the Issuer, then click Close.

    3. (Conditional) If you do not know the names of these certificates, see Section 18.2.3, Certificate Names.

  3. To verify the trusted root for the Identity Server, click Devices> Identity Servers > Edit > Security > NIDP Trust Store.

  4. In the Trusted Roots section, scan for a certificate subject that matches the issuer of the Embedded Service Provider certificate, then click its name.

    • If the Issuer has the same name as the Subject name, then this certificate is the root certificate.

    • If the Issuer has a different name than the Subject name, the certificate is an intermediate certificate in the chain. Click Close, and ensure that another certificate in the trust store is the root certificate. If it isn’t there, you need to import it and any other intermediate certificates between the one you have and the root certificate.

  5. To verify the trusted root for the Embedded Service Provider, click Devices > Access Gateways > Edit >Service Provider Certificates> Trusted Roots.

  6. In the Trusted Roots section, scan for a certificate subject that matches the issuer of the Identity Server certificate, then click its name.

    • If the Issuer has the same name as the Subject name, then this certificate is the root certificate.

    • If the Issuer has a different name than the Subject name, the certificate is an intermediate certificate in the chain. Click Close, and ensure that another certificate in the trust store is the root certificate. If it isn’t there, you need to import it and any other intermediate certificates between the one you have and the root certificate.

  7. (Optional) If you have clustered your Identity Servers and Access Gateways and you are concerned that not all members of the cluster are using the correct trusted root certificates, you can re-push the certificates to the cluster members.

    1. Click Auditing > Troubleshooting > Certificates.

    2. Select the Trust Store of your Identity Servers and Access Gateways, then click Re-push certificates.

    3. Update the Identity Severs and Access Gateways.

    4. Check the command status of each device to ensure that the certificate was pushed to the device. From the Identity Servers page or the Access Gateways page, click the Commands link.

To view sample log entries that are logged to the catalina.out file when a trusted root certificate is missing, see Trusted Roots Are Not Imported into the Appropriate Trusted Root Containers.

18.2.5 Enabling Debug Logging

You can enable Identity Server logging to dump more verbose Liberty information to the catalina.out file on both the Identity Server and the Embedded Service Provider of the Access Gateway.

  1. In the Administration Console, click Devices > Identity Servers > Edit > Logging.

  2. Select Enabled for File Logging and Echo to Console.

  3. In the Component File Logger Levels section, set Application and Liberty to a debug level.

  4. Click OK, update the Identity Server, then update the Access Gateway.

  5. After enabling and applying the changes, duplicate the issue once more to add specific details to the log file for the issue.

    If the error is the 100101044 error, look at the log file on the Embedded Service Provider for the error code

    If the error is the 100101043 error, look at the log file on the Identity Server for the error code.

    On Linux, look at the catalina.out file, and on Windows, look at the stdout.log file.

  6. (Conditional) To view the log files from the Administration Console, click Auditing > General Logging, then select the file and download it.

  7. (Conditional) To view the log files on the device, change to the log directory.

    • On Linux, change to the /var/opt/novell/nam/logs/idp directory.

    • On Windows Server 2008, change to the /Program Files (x86)/Novell/Tomcat/logs directory.

Below are a few typical entries illustrating the most common problems. They are from the catalina.out file of the Embedded Service Provider:

The Embedded Service Provider Cannot Resolve the Base URL of the Identity Server

When the Embedded Service Provider cannot resolve the DNS name of the Identity Server, the metadata cannot be loaded and a hostname error is logged. In the following entries, the Embedded Service Provider cannot resolve the idpcluster.lab.novell.com name of the Identity Server.

<amLogEntry> 2009-08-06T16:24:56Z INFO NIDS Application: AM#500105024:
AMDEVICEID#esp-09C720981EEE4EB4: AMAUTHID#2CA1168DF7343A42C7879
E707C51A03C: ESP is requesting metadata from IDP https://
idpcluster.lab.novell.com/nidp/idff/metadata </amLogEntry>

<amLogEntry> 2009-08-06T16:24:56Z SEVERE NIDS IDFF: AM#100106001:
AMDEVICEID#esp-09C720981EEE4EB4: Unable to load metadata for Embedded
Service Provider: https://idpcluster.lab.novell.com/nidp/idff/
metadata, error: AM#300101046: AMDEVICEID#esp-09C720981EEE4EB4::   Attempted to connect to a url with an unresolvable host name 
</amLogEntry>

<amLogEntry> 2009-08-06T16:24:56Z INFO NIDS Application: AM#500105039:
AMDEVICEID#esp-09C720981EEE4EB4: AMAUTHID#2CA1168DF7343A42C7879
E707C51A03C: Error on session id 2CA1168DF7343A42C7879E707C51A03C,
error 100101044-esp-09C720981EEE4EB4, Unable to authenticate.
AM#100101044: AMDEVICEID#esp-09C720981EEE4EB4:: Embedded Provider
failed to load Identity Provider metadata </amLogEntry>

Trusted Roots Are Not Imported into the Appropriate Trusted Root Containers

When the trusted roots are not imported into the appropriate trusted root containers, a certificate exception is thrown and an untrusted certificate message is logged. In the following log entries, the Embedded Service Provider is requesting metadata from the Identity Server, but the Embedded Service Provider does not trust the Identity Server certificate because the trusted root of the issuer of the Identity Server certificate is not in the Embedded Service Provider’s trusted root container.

<amLogEntry> 2009-08-05T16:07:53Z INFO NIDS Application: AM#500105024:
AMDEVICEID#esp-09C720981EEE4EB4: AMAUTHID#D983B08C28D35221D13 9D33E5324F98F: ESP is requesting metadata from IDP https://idpcluster.lab.novell.com/nidp/idff/metadata </amLogEntry>

<amLogEntry> 2009-08-05T16:07:53Z SEVERE NIDS IDFF: AM#100106001: AMDEVICEID#esp-09C720981EEE4EB4: Unable to load metadata for Embedded ServiceProvider: https://idpcluster.lab.novell.com/nidp/idff/metadata, error: java.security.cert.CertificateException: Untrusted Certificate-  chain </amLogEntry>

<amLogEntry> 2009-08-05T16:07:53Z INFO NIDS Application: AM#500105039:
AMDEVICEID#esp-09C720981EEE4EB4: AMAUTHID#D983 B08C28D35221D139 D33E5324F98F: Error on session id D983B08C28D35221D139D33E5324F98F, error 100101044-esp-09C720981EEE4EB4, Unable to authenticate. AM#100101044: AMDEVICEID#esp-09C720981EEE4EB4:: Embedded Provider failed to load Identity Provider metadata </amLogEntry>

The Server Certificate Has an Invalid Subject Name

When the certificate has an invalid subject name, the handshake fails. In the log entries below, the Embedded Service Provider is requesting metadata from the Identity Server. The server certificate name does not match, so the Embedded Service Provider is unable to authenticate and get the metadata necessary to establish the trusted relationship.

<amLogEntry> 2009-07-05T16:07:53Z INFO NIDS Application: AM#500105024:
AMDEVICEID#esp-09C720981EEE4EB4: AMAUTHID#D983B08C28D35221D139D33 E5324F98F: ESP is requesting metadata from IDP 
https://idpcluster.lab.novell.com/nidp/idff/metadata </amLogEntry>

<amLogEntry> 2009-07-05T16:07:53Z SEVERE NIDS IDFF: AM#100106001: AMDEVICEID#esp-09C720981EEE4EB4: Unable to load metadata for Embedded Service Provider: https://idpcluster.lab.novell.com/nidp/idff/metadata, error: Received fatal alert: handshake_failure </amLogEntry>

<amLogEntry> 2009-07-05T16:07:53Z INFO NIDS Application: AM#500105039:
AMDEVICEID#esp-09C720981EEE4EB4: AMAUTHID#D983B08C28D35221D139D33 E5324F98F: Error on session id D983B08C28D35221D139D33E5324F98F, error 100101044-esp-09C720981EEE 4EB4, Unable to authenticate. AM#100101044: AMDEVICEID#esp-09C720981EEE4EB4: : Embedded Provider failed to load Identity Provider
metadata </amLogEntry>

18.2.6 Testing Whether the Provider Can Access the Metadata

To test whether the metadata is available for download, enter the metadata URL of the identity provider and service provider. If the DNS name of the identity provider is idpcluster.lab.novell.com, open a browser at the Identity Server and enter the following URL:

https://idpcluster.lab.novell.com:8443/nidp/idff/metadata

Open a browser on the Access Gateway Service, then enter the same URL.

Because the Access Gateway Appliance does not have a graphical interface, you need to use the curl command to test whether the Access Gateway Appliance can access the metadata of the Identity Server. If the DNS name of the identity provider is idpcluster.lab.novell.com, enter the following command from the Access Gateway machine:

curl -k https://idpcluster.lab.novell.com:8443/nidp/idff/metadata

To test whether the Identity Server can access the metadata URL of the Access Gateway, open a browser on the Identity Server machine. If the published DNS name of service provider is www.aleris.net, enter the following URL:

https://www.aleris.net/nesp/idff/metadata

18.2.7 Manually Creating Any Auto-Generated Certificates

Occasionally, there are issues where the subject name was auto-generated and the entire configuration appears to be correct, but the 100101044/100101043 error is still reported. Delete the auto-generated certificate and manually re-create the server certificate, making sure that it is added to the relevant devices and stores.