One of the complicating features of troubleshooting Novell Identity Manager is the large number of moving parts. There are all the driver shims, running on remote or local systems, interfacing with the remote system. This takes on many different and wildly diverging forms.
For example the Active Directory driver runs as a DLL that uses Microsoft’s Password API to catch password changes, and gets other events. The Lotus Notes driver (I suppose we should start calling it Lotus Domino really) runs a Notes application, that looks for changes, copies them into a driver created Notes database and synchronizes those changes. The Bi-Directional driver for Midrange servers (AS400, i5os, whatever you call it) has a custom written application that uses the AS400 exits to catch changes in the system.
As you can see each system requires a unique approach to catching changes in the system, sending them on, and then the converse, receiving changes and applying them. This means you need to be able to troubleshoot each unique system in totally different manners. The good news is, you can often rely on the local people who know their systems to do much of the work, but there is always that crossover space where the person doing the Identity Management work has to step. (Often because the application folk are less than thrilled or resistant to the idea of using software they did not select on their system). Then you have errors that happen related to eDirectory, perhaps stuck obituaries, or time being out of sync. In addition to all that, Identity Manager runs on top of an operating system like Netware, Windows, Linux, or a Unix flavour. Errors in those systems (running out of space on the volume hosting the DIBset for example) can trickle up into Identity Manager.
There are all sorts of resources for troubleshooting Identity Manager problems at Cool Solutions, like the following articles.
- A great article about reading DSTrace output: Capturing and Reading Novell Identity Manager Traces
- Generic Identity Manager error messages: Somewhat Obscure IDM Error Codes
- More Identity Manager error messages: Troubleshooting Synchronization in Identity Manager Installations
- A series of articles listing common error messages in the JDBC driver: Error Codes of the Novell Identity Manager Driver for JDBC: Part 1 of 4
Some errors are just plain hard to track down. One that personally I find a bit of a pain, is usually in the Publisher channel, as a connected system tries to write an object to eDirectory, usually on an add event, and it returns a -613 error. 613 is defined as: -613 FFFFFD9B SYNTAX VIOLATION
That seems simple enough, something in the add document is violating the schema rules. This usually occurs on an add event, and often there are many to dozens of attributes in the document to consider.
What makes it frustrating is that the write to eDirectory is an atomic event, which means the entire thing completes or fails in one operation. If even one attribute has an issue, then the entire operation fails. What would be really nice if eDirectory gave better feedback and hinted at what attribute actually had an issue. Alas, this is not to be. Troubleshooting this error is a lot easier the better you understand eDirectory schema. In fact, the more attributes in schema that you know the definition for, off by heart, the easier this is to troubleshoot. Some attributes will surprise you with odd schema syntax requirements.
Here are a couple of quick tips that can help you get started on troubleshooting some of these errors.
DN Syntax attributes: Anything that is Distinguished Name (DN) syntax has two requirements. First it cannot be null, and second it has to reference a valid distinguished name in the directory, in the expected syntax. In fact, in my experience, the fastest way to troubleshoot this is quickly identify all the DN syntax attributes and make sure they have values, and that values are actually valid.
Sized attributes: The next common one I see is for sized attributes. One of the things Identity Manager can often do, is send a blank attribute, for a variety of reasons. (Usually on a merge from a database, where the empty value in a column is not quite NULL, rather it is an empty value, which is subtly different). Regardless, consider any sized attributes you may have.
For example, did you know that the eDirectory attribute Description is sized? From 1 to 1024 I believe. Well that is good to know that there is an upper limit, and I am sure there are cases where it is exceeded to watch out for. But it is the lower limit that is the problem. If you try to send an empty value for Description, that is a syntax violation, since it must be at least 1 character long, and nothing is zero characters long.
A strange one I recently ran into was really hard to troubleshoot. It happens we were using the Work Order driver, and if you are not aware, that driver watched Work Order objects, and if the rules allow it, process them through the subscriber channel, and into the driver shim. They come out the publisher channel as a create event for a Work Order To Do object. Seems like the default naming is to take the Work Order object name, and append a To Do followed by a date stamp.
Well we were naming our Work Order objects with a class (Usually user) then a service name, then the GUID of the Workflow from the User Application. (Yes this was a convoluted, but real world example). We had about 50 character or longer CN’s. This is of course ok, until the Work Order To Do is created with an extra 10-15 characters in the name, and suddenly we broke the size limit on the CN attribute. CN can only go up to 64 characters. Normally not an issue, but in our contrived case, it became one.
I happen to believe this is a bug in the Work Order driver, and it should be watching out for silly things like that, when it is creating a new object. Fixing it is of course easy with a quick test in Placement to be sure the object name is not too long and then truncating it to make sure it does not violate schema. If it does not look like any of these types of attributes are accounting for the error, keep looking, and attribute by attribute look at the schema definition, and then look at the data provided and see where the mismatch occurs.
Some schema types might be a little tricky, things like Path or Backlink syntax, which are structured attributes with three or two (respectively) components. It is important that all components match the schema types. Path syntax is used for defining Home Directory attributes in eDirectory, but it is such a useful compound attribute that many other attributes will leverage it. DirXML-Associations uses this syntax, which has an integer (nameSpace for file system paths, but used to indicate the association state here), then a DN attribute (volume, used for the Driver DN here), followed by a case ignore string (path, used for the association value here).
Backlink syntax is another structured attribute that can be really useful. It is a DN followed by a 32 bit integer. It is meant to hold the remote server DN, and the object ID value in the second part of the attribute. If you need to store a time stamp per driver, this can come in really handy, as CTIME, the native time format inside eDirectory and Identity Manager is a 32 bit integer, representing seconds since 1970. This means you could store the driver DN, and a timestamp value in one attribute, and have it multi valued for all the drivers in use. For example last login time could be really useful to know per system, and this would be an easy way to store it in a consistent fashion.
608 error: The next error that often comes across on the Publisher channel is 608. This error is defined as -608 FFFFFDA0 ILLEGAL ATTRIBUTE and is usually pretty easy to troubleshoot. The most often cause is either you made an error in trying to add an attribute from base schema that is not part of that object. For example trying to add the Login Disabled to a Group object. The attribute is not part of schema. The second most common reason for this error is trying to add an attribute that is part of an auxiliary object class, but the object class is not part of the object yet. If the attribute is added before the Command Transform, the engine will automatically notice that it is part of an auxiliary class that is needed, and add the auxiliary class to user as part of the operation. If your rule does the work too late, then you just need to add a destination attribute of Object Class with the value of the auxiliary class you need.
DirXML Log Event ------------------- Driver: \ACME-LAB\LAB\SERVICES\IDVAULT\MAGIC-JDBC Channel: Publisher Status: Error Message: Code(-9010) An exception occurred: novell.jclient.JCException: modifyEntry -608 ERR_ILLEGAL_ATTRIBUTE
606 error: This error is defined as: -606 FFFFFDA2 ENTRY ALREADY EXISTS When your matching rule is not working perfectly, or if your placement rule is not quite right, you may try to create a user in a location without a unique name. If that happens you are likely to get a 606 error.
606 Entry (object) already exists DirXML Log Event ------------------- Driver: \ACME-LAB\LAB\SERVICES\IDVAULT\MAGIC-JDBC Channel: Publisher Status: Error Message: Code(-9010) An exception occurred: novell.jclient.JCException: createEntry -606 ERR_ENTRY_ALREADY_EXISTS
This is a almost always due to incorrect placement and/or matching rules in the Publisher channel. Look at why the Matching rule did not find the existing user, or why the placement rule did not pick a name for the object that was unique in its container. If you are running into issues with finding a unique naming pattern, look at the Unique Name token that is new in Identity Manager 3.5, you can look at this article: Unique Name Token Functionality in IDM 3.5
630 Error: This error is defined as: -630 FFFFFD8A DIFFERENT TREE You really should never see one of these, but it is pretty easy to force one if you want to, most often by accident. When a DN is specified, it can include the tree name as the first node, or it can start at the second node (usually an Organization (O) or perhaps a Country (C) object). If you include the wrong tree name, the engine is smart enough to notice and return this quite amusing error.
DirXML Log Event ------------------- Driver: \ACME-LAB\LAB\SERVICES\IDVAULT\eDirectory Driver Channel: Publisher Object: \NWIDMTEST\acme_nyc\21ST\TechOps\ThirdTime (LAB\EMPLOYEES\NEW\ThirdTime) Status: Error Message: Code(-9010) An exception occurred: novell.jclient.JCException: createEntry -630 ERR_DIFFERENT_TREE
This particular example was because of a mistake in a Global Configuration Value (GCV) that was copied from the wrong tree, so it had the wrong tree name in the DN. The GCV was used for specifying the base DN in the placement rule. These are some of the more common and annoying eDirectory errors that you may run across in an Identity Manager implementation. These pretty much all only show up on the Publisher channel. The Subscriber channel (with the exception of the eDirectory driver) is usually NOT eDirectory, and so will send its own native error codes in return.