In the first part of this article, I started listing error codes I found when implementing a new Active Directory driver.
I have done this before for other drivers, like the JDBC driver. You can read that series at:
I have enough error codes for part 5 collected, I just have to get around to writing it up!
I would love to see more people write articles like this, because we all run into error codes as we develop new drivers, and it would be awesome if you could just copy and paste the error code into Google and find out that someone else has already run into the error, and written an article explaining what went wrong and how they fixed it.
Personally I think the documentation from Novell should have more of these in the Troubleshooting section, but I also understand how hard that would be to maintain as a document. Nonetheless I think there is a better balance that should be struck, than is currently in use. Some drivers are better than others, but I think the documentation for all of them would be much more useful if more of these types of errors were included.
It seems that no one else is going to do it, so I decided I would do my best to fill this gap. I am always looking for help from others, so if you have any good errors you have saved from trace on any driver, I urge you to submit it to Cool Solutions as an article. You can get points, that you can use to earn all sorts of interesting rewards. They entry level seems high, (after all top prize needs 25.000 points to get the Hawaii cruise) but you can take your points out as Amazon, ThinkGeek, LL Bean, or iTunes store gift cards, with which you can buy almost anything. (I tried, you cannot make car nor mortgage payments with Amazon gift certificates, so almost, but not all!)
The errors below came as I was working on a project with an Active Directory tree, that was really four domains linked. An empty root, and three child domains. This is one of those annoying things in Active Directory. It is called a tree, but really it is just a set of linked domains. Traversing the domain boundaries is always a strange event, either in LDAP, where you get a referral that needs to be processed, or in the tools, where it really feels disjointed as you have to change the focus to the new domain. I miss being able to see an entire tree and not care where the partition boundaries are, like in eDirectory.
Our plan was to build one driver for a single child domain, get everything running, then move all the policies into a Library object in the root of the driver set object. Once we were finished, export that driver to a configuration file, then import it again several times, one per child domain. We planned to handle differences in domain names through the use of Global Configuration Values (GCV’s). This way one rule would work in three different domains with no changes needed.
On to the error codes then:
Duplicate attributes listed in the filter error:
19:00:00 DC0A7BA0 Drvrs: eDirectory ST: DirXML Log Event ------------------- Driver: \ACME-LAB-LDAP\acme\Drivers\IDM\eDirectory Status: Error Message: Code(-9019) Error in vnd.nds.stream://ACME-LAB-LDAP/acme/Drivers/IDM/eDirectory#DirXML-DriverFilter:130 : Duplicate <filter-attr> for value 'Internet EMail Address'.
Now this error is actually from an eDirectory driver, but the same thing will happen in any driver, and I just happened to make the mistake on an eDirectory driver. The filter can be edited two different ways. Via the GUI interface in either Designer or iManager, or you can switch to the XML view.
I happen to find the GUI for filter editing great. But if you have to add a bunch of attributes to the filter I find it painful and slow. What I usually do is switch to the XML view of the filter, and copy an attribute that has the settings I want, then paste it at the bottom of the list for that object class as many times as needed. Then I go back and edit the attribute names to the values I want.
If you have to add say 10 attributes to a bunch of filters, then you can see how copy and pasting can save a LOT of time. The example for that case would be when you realize you have to add say posixAccount attributes to all users and you have an eDirectory driver, which has two filters, one on each half. That means doing it twice. Maybe you have a third system you want to synchronize those attributes too as well, then that is a third time.
What I did was quickly paste four attributes in to each filter, without really checking to see if it was already specified. Alas, that is an error on driver startup, so I had to go back and clean up my mess.
Multiple chained errors:
<nds dtdversion="1.1" ndsversion="8.7"> <source> <product asn1id="" build="20080229_143300" instance="\ACME-LAB-IDV\Acme\Drivers\IDM\Active Directory" version="3.5.3">AD</product> <contact>Novell, Inc.</contact> </source> <output> <status event-id="Xidv1#20090310211846#1#1" level="error" text1="schema violation" type="app-general"> <message>Attribute 'DirXML-PasswordSyncStatus' is not in the application schema</message> <xds-path>/nds/input/add[@event-id='Xidv1#20090310211846#1#1'][@class-name='user'][@src-dn='\ACME-LAB-IDV\Acme\Users\Testing'][@dest-dn='cn=Testing,ou=Users,OU=AMES,Ou=West,dc=americas,dc=acme,dc=corp'][@class-name='user']</xds-path> </status> <status event-id="Xidv1#20090310211846#1#1" level="warning" type="driver-general"> <ldap-err ldap-rc="32" ldap-rc-name="LDAP_NO_SUCH_OBJECT"> <client-err ldap-rc="32" ldap-rc-name="LDAP_NO_SUCH_OBJECT">No Such Object</client-err> <server-err>00000525: NameErr: DSID-031A11A5, problem 2001 (NO_OBJECT), data 0, best match of: '' </server-err> <server-err-ex win32-rc="1317"/> </ldap-err> </status> </output> </nds>
This class of error is very common. There is a series of errors that occur one after another. Usually the document that got sent to the driver, was an <add> event and for some reason it failed, which is the root cause. Then there were a number of <modify> event following it. Perhaps a token of set destination attribute, or add destination attribute, but the token was set to use the when=”after” setting. There are several settings available, “before” the current event, “after” the current event, add to current operation or “direct”. If you select “after”, then you get an new <modify> event in the same XML document, but as a separate operation from the initial <add> event. Of course that <modify> event references the object that was just made in the immediately preceding <add> event. Thus you get an error that the object being modified does not exist, since the <add> just failed.
Probably the most common example of this is adding a user to a group, just after the user is created.
If you were to do this in the other direction, on the Publisher channel, in eDirectory you would probably see some kind of failure on the <add> event, maybe a bad attribute syntax (613), object already exists (606, because your matching rule failed to match correctly), or some other error.
Then the next modify event is to write the group membership on to the user object. That fails with a 601, since the object being referenced, the user does not exist. Then you will see a modify event for the Group, adding a Member value, the DN of the nonexistent user, which will fail with a 613 error (Invalid syntax, since the Member attribute is a DN syntax (Distinguished Name) attribute which cannot be null, and the DN must be valid. The <add> event failed, the user does not exist, but you try to add an attribute pointing at the user, thus the error.
This particular error was the Active Directory equivalent, on the Subscriber channel, thus the error codes are Active Directory error codes. I suppose I should collect all the values and write something about them, all in one location. For using an LDAP browser to login to Active Directory, say to test passwords or other things in Active Directory I wrote an article about the error 49 class of errors: Sub-Error Codes for LDAP Error 49
Trying to move a container back via Move Source, but it failed:
<nds dtdversion="3.5" ndsversion="8.x"> <source> <product version="184.108.40.20694">DirXML</product> <contact>Novell, Inc.</contact> </source> <input> <move class-name="organizationalUnit" dest-dn="OU=Users,OU=Groups,OU=SMITH,OU=East,DC=americas,DC=acme,DC=corp" event-id="Active Directory##1201ff04465##0"> <association>f282355c1b927f4b9b7f768623e26921</association> <parent dest-dn="corp\acme\americas\South\MARSHALL"/> </move> </input> </nds>
I was trying to catch move of structural objects in Active Directory, basically subtree moves, and reset them in Active Directory, (aka move them back) since a subtree move in Active Directory is not a big deal, but needs serious consideration in eDirectory.
This is one of those times where we are comparing apples to oranges. Each Active Directory domain is basically a flat space, with some pretty names added to make it look structured. So moving a subtree of OU’s is really little more than just renaming some attributes so that the structure looks changed. In eDirectory it is a real structural move. The true comparison would be to have the Active Directory move a domain or tree around.
I wrote about this issue (Active Directory Driver, Catch Move Subtree’s Block, and Reset) and offered a solution there. As I was developing this solution I was working through what was needed to get the original context of the container in Active Directory, so it could be used to move the container back from a command.
Above is the <move> event I generated in one of my first attempts. Of course I sent the destination DN in format Identity Manager uses for internal notation, backslash notation.
This generated the following error:
<nds dtdversion="1.1" ndsversion="8.7"> <source> <product asn1id="" build="20080229_143300" instance="\ACME-LAB-IDV\Acme\Drivers\IDM\Active Directory" version="3.5.3">AD</product> <contact>Novell, Inc.</contact> </source> <output> <status event-id="Active Directory##1201ff04465##0" level="error" type="driver-general"> <message>rename failed.</message> <ldap-err ldap-rc="34" ldap-rc-name="LDAP_INVALID_DN_SYNTAX"> <client-err ldap-rc="34" ldap-rc-name="LDAP_INVALID_DN_SYNTAX">Invalid DN Syntax</client-err> <server-err>00000057: LdapErr: DSID-0C090AA4, comment: Error processing name, data 0, v1771</server-err> <server-err-ex win32-rc="87"/> </ldap-err> </status> </output> </nds>
Error is pretty clear. Invalid DN syntax. To fix this, I needed to use ParseDN to change it to LDAP syntax, as you can see in the article I wrote about blocking moves to do this.
Bad DN syntax of another type:
Bad LDAP syntax on DestDN
<nds dtdversion="3.5" ndsversion="8.x"> <source> <product version="220.127.116.1170315 ">DirXML</product> <contact>Novell, Inc.</contact> </source> <input> <add class-name="group" dest-dn="CN=Testing AD Group,,ou=Groups,dc=acme,dc=com" event-id="NJNOVCL1#20090327192359#3#1" qualified-src-dn="C=US\O=NYC\OU=ADGroups\CN=Testing AD Group" src-dn="\IDM\US\NYC\ADGroups\Testing AD Group" src-entry-id="34913" timestamp="1238181839#4"> <add-attr attr-name="description"> <value timestamp="1238181839#4" type="string">Desc</value> </add-attr> </add> </input> </nds>
Here is a group being created in Active Directory on the Subscriber channel, with a bad DN. Can you see the typo? How good are your eyes? Sometimes when troubleshooting bad syntax errors, it feels like playing Wheres Waldo? As you try to see the tree (error) hidden in the thousands of lines of forest (trace).
What is both annoying and helpful at times like this is to just get another set of eyes on the problem. Invariably when I get stuck on this sort of issue and show it to someone the error sticks out like a sore thumb for them. It is annoying because you might have been stuck staring at the trace for hours and still not be able to see it, whereas you show it to someone else and they find it within the first few seconds of looking at it.
This <add> document generated almost the same error as the previous example, LDAP_INVALID_DN_SYNTAX. There is a subtly in the error codes returned that I do not fully understand, so I include it for completeness.
<nds dtdversion="1.1" ndsversion="8.7"> <source> <product asn1id="" build="20080229_143300" instance="\ACME-LAB-IDV\Acme\Drivers\IDM\EMEA-AD" version="3.5.3">AD</product> <contact>Novell, Inc.</contact> </source> <output> <status event-id="mta-gwlab#20090403211313#1#1" level="error" type="driver-general"> <ldap-err ldap-rc="34" ldap-rc-name="LDAP_INVALID_DN_SYNTAX"> <client-err ldap-rc="34" ldap-rc-name="LDAP_INVALID_DN_SYNTAX">Invalid DN Syntax</client-err> <server-err>00002081: NameErr: DSID-03050BF0, problem 2003 (BAD_ATT_SYNTAX), data 0, best match of: 'CN=Testing AD Group,,ou=Groups,dc=acme,dc=com' </server-err> <server-err-ex win32-rc="8321"/> </ldap-err> </status> </output> </nds>
If you could not see the error, it was that the dest-dn is being set as:
CN=Testing AD Group,,ou=Groups,dc=acme,dc=com
There is an extra comma between Testing AD Group and the ou=Groups. Should only be the one. My placement rule had a typo that stuck in the extra comma.
Object is not a leaf:
<nds dtdversion="1.1" ndsversion="8.7"> <source> <product asn1id="" build="20080229_143300" instance="\ACME-LAB-IDV\Acme\Drivers\IDM\Active Directory" version="3.5.3">AD</product> <contact>Novell, Inc.</contact> </source> <input> <delete class-name="Organizational Unit" dest-dn="Acme\Structure\corp\acme\Americas\AnotherSouth" dest-entry-id="33167" e vent-id="Active Directory##120209d615a##0" src-dn="OU=AnotherSouth\0ADEL:7e01cace-bb76-4b78-a933-c5d3fc91a13b,CN=Deleted Obje cts,DC=americas,DC=acme,DC=corp"> <association>ceca017e76bb784ba933c5d3fc91a13b</association> </delete> </input> </nds> [03/19/09 17:23:38.029]:Active Directory PT:Filtering out notification-only attributes. [03/19/09 17:23:38.029]:Active Directory PT:Pumping XDS to eDirectory. [03/19/09 17:23:38.030]:Active Directory PT:Performing operation delete for Acme\Structure\corp\acme\Americas\AnotherSouth. [03/19/09 17:23:38.030]:Active Directory PT:Removing entry Acme\Structure\corp\acme\Americas\AnotherSouth. [03/19/09 17:23:38.083]:Active Directory PT: DirXML Log Event ------------------- Driver: \ACME-LAB-IDV\Acme\Drivers\IDM\Active Directory Channel: Publisher Object: OU=AnotherSouth\0ADEL:7e01cace-bb76-4b78-a933-c5d3fc91a13b,CN=Deleted Objects,DC=americas,DC=acme,DC=corp (Watt s\Structure\corp\acme\Americas\AnotherSouth) Status: Error Message: Code(-9010) An exception occurred: novell.jclient.JCException: removeEntry -629 ERR_ENTRY_IS_NOT_LEAF
Continuing on a similar thread as the previous error, eDirectory does not allow recursive deletes of non empty container nodes. That is, to delete a container object (OU, Organization, Container, Domain (DC)) they need to be empty. Not so in Active Directory, where you are allowed, via the Active Directory Users and Computer MMC snapin to do this sort of event.
Now it turns out, that some LDAP browsing tools allow you to do deletes of non-empty containers. However if you were to watch carefully in DStrace, with the LDAP trace option enabled, you would probably notice that what the LDAP Browser tool is doing, is deleting all the children, then once a container is empty, deleting the container, until it is done.
This approach DOES work with eDirectory and most other LDAP directories, because really it is many deletes, in the correct sequence, whereas the Active Directory MMC seems to actually be sending a single API call to do the subtree delete.
Letting this flow from Active Directory to eDirectory will generate this 629 error, which is a little bit misleading. At least from the perspective that a container is usually not really a leaf object, but if you look at an empty container, it is basically a leaf object at that point. Thus the error definition.
To resolve this (which I still have to figure out how to map), we think we will have to catch this event, query for all objects underneath this container, then for each one, determine if it is a leaf node (either a leaf object (like User, Group, or Computer) and then delete anything that is. If it is not a leaf node, and is a container, keep working down till it is empty and deletable.
This would be a great place to use recursion, but I am not sure how I would do that in DirXML Script. We discussed how this might be possible in XSLT but decided the amount of effort to develop and maintain it would not be useful. Probably we can start by querying for container nodes, and repeat until we get zero containers left in the query. Then query for, and start deleting all children. That would end the loop, and the next iteration would pick the next container, and so on, until we are done. Instead of recursion, probably a while loop inside a for-each loop. This one should be a lot of fun to write and debug. I am looking forward too it when I have some spare time.
Well that is enough error codes for me right now. Stay tuned for part 3, and probably more articles as I have plenty more error codes collected that I need to work through and discuss. I have been collecting eDirectory and GroupWise driver error codes as well, to write similar articles.
Disclaimer: As with everything else at NetIQ Cool Solutions, this content is definitely not supported by NetIQ, so Customer Support will not be able to help you if it has any adverse effect on your environment. It just worked for at least one person, and perhaps it will be useful for you too. Be sure to test in a non-production environment.