When the Active Directory driver will not synchronize attributes

geoffc

By: geoffc

April 19, 2011 12:15 pm

Reads: 329

Comments:0

Rating:0

Recently I was working on an Active Directory driver, and the task we had was to add regular users from the Identity Vault into a new Active Directory domain as Contact object. The notional use case here was to get all the current users into the Exchange address book.

Well it turns out there were a couple of really interesting issues we encountered along the way that are worth discussing, and hopefully will be useful to others.

While we were working in our development lab we did not yet have an actual Exchange server added, we just had an Active Directory domain with some users in it. Since they also wanted the existing users to match and synchronize changes after the match, we needed to handle users properly as well.

None of this is very profound, and the direction we took was to handle users normally, and if we noticed the object was missing an entitlement flag, we would then make a Contact object in Active Directory for it. This was easy to do with a simple use of the Set Operation Class token in Policy Builder, which is actually a pretty cool trick, if you are not aware of it.

This token will change the current class of the operation. You could probably do this in XPATH with a set XML attribute of class-name with the new value, but it is much easier to read using the token meant for the task. I am actually not sure if there are any other consequences or differences between the two approaches, but it seems like built in function is the more appropriate way to handle it.

This was all going great, with the usual minor quirks that come with morphing an objects class (User in eDirectory to Contact in Active Directory for unflagged users) until we got an actual Exchange system installed.

We had migrated in a few dozen Users from the Identity Vault, that became Contacts in Active Directory and they were not showing up in Exchange. Well at least not in the Outlook address book, as we desired.

Digging around, since this was Exchange 2003, in the Exchange Server manager MMC (Microsoft Management Console. Which by the way, if you run mmc.exe on Windows, look at the title bar of the window, it says Console1, you will laugh…) we found that the Address book listings are actually defined by an LDAP query. This is true for “All Contacts” and the other types of Address Book options that show up in Outlook and Outlook Web Access.

Now I like to think I can read LDAP filters quite well, but this one was a mouthful. Here it is, as shown in the MMC console, lets see if you can make heads or tails of this one:

(&(mailnickname=*)(|(&(objectCategory=person)(objectClass=user)(!(homeMDB=*))(!(msExchHomeServerName=*)))(&(objectCategory=person)(objectClass=user)(|(homeMDB=*)(msExchHomeServerName=*)))(&(objectCategory=person)(objectClass=contact))(objectCategory=group)(objectCategory=publicFolder)(objectCategory=msExchDynamicDistributionList)))

However, I knew that a good tool that knows how to handle LDAP filters could parse this into a more human readable format. Now for LDAP operations, I happen to prefer LBE, LDAP Browser and Editor by a guy with a crazy name of Gawor. He used to work at one of the US National Labs (Lawrence Livermore I think) but finally grew up and got a real job, at which point they stopped hosting his client. The good news is you can get it at this link: Jarek Gawor’s excellent LDAP Browser/Editor v2.8.2

This is a 600K file, that is small enough to keep in your email for when you need it, and it is Java based, and the same JAR file runs on a Mac, Windows, or Linux box no problems. This is a great tool, since it is tiny, fast, and completely cross platform. However, it has not been updated in something like eight or nine years, and has some minor quirks.

The other major LDAP tools out there seem to be Softerra’s LDAP Browser which I am ambivalent about, as they went commercial and to my mind it is not great enough to be worth paying for. The other major player seems to be Apache Directory Studio tool (ApacheDS). You can get this at: http://directory.apache.org/

ApacheDS is Eclipse based, like Designer and Analyzer, so it too is cross platform as long as you have Java, however the foot print for Eclipse versus LBE is night and day in terms of memory and CPU.

Nonetheless ApachDS as at least two unique features for which I will load it up and use it. The first is instead of exporting LDAP objects from a directory in LDIF format files, it can also export into CSV (Comma Separated Values) text files, which is awesome if you want to manipulate it in Excel to clean up or rename some of the data. It also claims to export straight to Excel format, but I have never had good luck with that function. Since CSV can be imported directly into Excel in a moments effort, I rarely care.

The second key feature that is less obvious that is very powerful is that the LDAP filter builder has an awesome parser to show you the filter in a much more readable fashion.

So I took that three hundred and thirty three character long LDAP filter from Exchanges Address Book configuration and looked at it in ApacheDS’s LDAP filter builder, and it was displayed in a much more easily understood format:

(&
    (mailnickname=*)
    (|
        (&
            (objectCategory=person)
            (objectClass=user)
            (!(homeMDB=*))
            (!(msExchHomeServerName=*))
        )
        (&
            (objectCategory=person)
            (objectClass=user)
            (|
                (homeMDB=*)
                (msExchHomeServerName=*)
            )
        )
        (&
            (objectCategory=person)
            (objectClass=contact)
        )
        (objectCategory=group)
        (objectCategory=publicFolder)
        (objectCategory=msExchDynamicDistributionList)
    )
)

Now that I see it this way it is pretty easy to see that the filter is looking for objects with a value in the mailNickname attribute that are either a User with some of the Exchange attributes populated, a contact object, a Group, a Public folder, or an msExchDynamicDistributionList object.

That is, there first & sign, is ANDing together the mailNickname = * (i.e. has a value for mailNickname) with the next predicate, which is a bunch of ORed (| sign indicates OR in LDAP filters) conditions. Within the OR group, the first three have some ANDed (&) values. That is, it should be a person, a User without a homeMDB value, and without a msExchHomeServerName value since I guess users with Exchange accounts are already in the list via some other fashion.

You can see there are six different objects (or objects with conditions) that are OR’ed together. As you can see, much much easier to read when broken down this way. I tried doing it by hand, and it did not work anywhere near as well!

But the key was that we were not populating mailNickname on the Contact objects, thus they never would show up in the Exchange address book.

Of course it turns out there was more too it than that. So now that we had Exchange installed, we went into the Exchange Server Manager MMC and made a Contact object and compared the Contact object we made via IDM with the Contact object made via the Exchange MMC, and we found a couple of extra attributes we missed. Specifically we found we should add a proxyAddresses attribute (which I actually should have remembered from another client, hey Eric P!) that is multivalued, and strangely has the formatting of SMTP:user@domain.com.

The other is targetAddress that is single valued and has the same odd formatting. This is how Exchange knows which protocol to send email via, since in theory it looks like it still support X.400. Anyone remember X.400 for email? Like X.500 for directory services, it seemed like a good idea at the time, but in the end, no one really ever used it with all the stuff it needed, thus the L in LDAP for lightweight, since the full X.500 DAP was really really a heavy weight (Watch out Evander Holyfield, X.500 DAP could take you out in a second!). Seems like only IBM ever implemented anything that used all of X.500 (DCE anyone?) or X.400. Heck or even the full seven layer OSI model. SNA was one of the few protocols to actually use all seven layers the way they were meant too.

Well fixing this in DirXML Script is a piece of cake, since adding new attributes, and managing them when things change is exactly what it is designed to do.

The funny part is of course that the Exchange server got installed on the Thursday afternoon right before one of the holiday weekends (Thanks Jeff for the timing!) so of course everyone wanted to go home, and I really wanted to just demonstrate this worked before the holiday. So I was in a rush to implement this, and it was interesting how fast it really happened.

Anyway, so off we go, and tested with a modify, and awesome, the users are showing up in the Address book! Yay!

To be complete, went and did a quick migrate of some other user from the Identity Vault. To make it more challenging, I was on a call with someone discussing the project, while I tried to get this done, and I could not focus well enough to remember what the Create rule policies were, in terms of required attributes. So it took me four tries, each time Operation Vetoed by the Create rule. Fun trying to split your attention.

When I finally got a valid user, I was happy, did a quick look at it in LDAP and everything looked good, except I noticed that mailNickname was missed. What? I know it was sent. Why was it not in Active Directory? And it worked on a modify. What was going on? Well of course I was looking in Dstrace all along to see what was going on. When I looked on the engine side of the trace, I saw an <add> event go through with the correct set of attributes including mailNickname. But Active Directory was not showing it. Next place to look is on the Remote loader trace to see what the shim was doing with the attributes, so lets look at the Remote Loader trace:

DirXML: [12/30/10 17:32:09.76]: 
<nds dtdversion="3.5" ndsversion="8.x">
	<source>
		<product version="3.6.10.4747">DirXML</product>
		<contact>Novell, Inc.</contact>
	</source>
	<input>
		<add class-name="contact" dest-dn="cn=RSMITH7,ou=Contacts,dc=acme,dc=local" event-id="acmedidv3#20101230223222#3#1" qualified-src-dn="O=idv\OU=Users\CN=RSMITH7" src-dn="\acmeIDV\idv\Users\RSMITH7" src-entry-id="214293">
			<add-attr attr-name="displayName">
				<value timestamp="1256696225#32" type="string">Ryan Smith</value>
			</add-attr>
			<add-attr attr-name="givenName">
				<value timestamp="1256696220#26" type="string">Ryan</value>
			</add-attr>
			<add-attr attr-name="dirxml-uACAccountDisable">
				<value timestamp="1293747133#12" type="state">false</value>
			</add-attr>
			<add-attr attr-name="sn">
				<value timestamp="1256696220#27" type="string">Smith</value>
			</add-attr>
			<add-attr attr-name="telephoneNumber">
				<value timestamp="1256696220#16" type="teleNumber">NA</value>
			</add-attr>
			<add-attr attr-name="title">
				<value timestamp="1256696220#24" type="string">User Administrator</value>
			</add-attr>
			<add-attr attr-name="cn">
				<value>RSMITH7</value>
			</add-attr>
			<add-attr attr-name="mail">
				<value type="string">RSMITH7@acmemail.com</value>
			</add-attr>
			<add-attr attr-name="mailNickname">
				<value type="string">RSMITH7</value>
			</add-attr>
			<add-attr attr-name="proxyAddresses">
				<value type="string">SMTP:RSMITH7@acmemail.com</value>
			</add-attr>
			<add-attr attr-name="targetAddress">
				<value type="string">SMTP:RSMITH7@acmemail.com</value>
			</add-attr>
		</add>
	</input>
</nds>

Hmm, there it is, mailNickname in the <add> event in the snippet:

<add-attr attr-name="mailNickname">
	<value type="string">RSMITH7</value>
</add-attr>

Looking good so far. Then the ADdriver.dll shim parses the XDS it received, as the trace shows below:

DirXML: [12/30/10 17:32:09.76]: ADDriver: parse command

  className    contact
  destDN       cn=RSMITH7,ou=Contacts,dc=acme,dc=local
  eventId      acmedidv3#20101230223222#3#1
  association  
DirXML: [12/30/10 17:32:09.76]: ADDriver: MadCommandAdd::onCommand
DirXML: [12/30/10 17:32:09.76]: ADDriver: Imported attribute notes
   MAD syntax     DirectoryString (2.5.5.12,64)
   XDS syntax     string
   Single valued  true
   Case sensitive false
DirXML: [12/30/10 17:32:09.76]: ADDriver: Imported attribute cn
   MAD syntax     DirectoryString (2.5.5.12,64)
   XDS syntax     string
   Single valued  true
   Case sensitive false
DirXML: [12/30/10 17:32:09.76]: ADDriver: Imported class contact
DirXML: [12/30/10 17:32:09.76]: ADDriver: MadCommandAdd::insertXdsAttributes()
DirXML: [12/30/10 17:32:09.76]: ADDriver: displayName
DirXML: [12/30/10 17:32:09.76]: ADDriver: givenName
DirXML: [12/30/10 17:32:09.76]: ADDriver: dirxml-uACAccountDisable
DirXML: [12/30/10 17:32:09.76]: ADDriver: sn
DirXML: [12/30/10 17:32:09.76]: ADDriver: telephoneNumber
DirXML: [12/30/10 17:32:09.76]: ADDriver: title
DirXML: [12/30/10 17:32:09.76]: ADDriver: cn
DirXML: [12/30/10 17:32:09.76]: ADDriver: mail
DirXML: [12/30/10 17:32:09.76]: ADDriver: mailNickname
DirXML: [12/30/10 17:32:09.76]: ADDriver: Imported attribute mailNickname
   MAD syntax     DirectoryString (2.5.5.12,64)
   XDS syntax     string
   Single valued  true
   Case sensitive false
DirXML: [12/30/10 17:32:09.76]: ADDriver: proxyAddresses
DirXML: [12/30/10 17:32:09.76]: ADDriver: Imported attribute proxyAddresses
   MAD syntax     DirectoryString (2.5.5.12,64)
   XDS syntax     string
   Single valued  false
   Case sensitive false
DirXML: [12/30/10 17:32:09.76]: ADDriver: targetAddress
DirXML: [12/30/10 17:32:09.76]: ADDriver: Imported attribute targetAddress
   MAD syntax     DirectoryString (2.5.5.12,64)
   XDS syntax     string
   Single valued  true
   Case sensitive false
   
   

Looking through that, we see for a couple of attributes it had to import attributes (I am not sure exactly what that means, and why for some and not for others, maybe since they are not directly in the filter, the shim does not load them out of AD schema at driver start time) and it shows that it found mailNickname, and correctly read back its schema syntax from Active Directory with this trace snippet:

DirXML: [12/30/10 17:32:09.76]: ADDriver: mailNickname
DirXML: [12/30/10 17:32:09.76]: ADDriver: Imported attribute mailNickname
   MAD syntax     DirectoryString (2.5.5.12,64)
   XDS syntax     string
   Single valued  true
   Case sensitive false
   
   

Ok, it is not case sensitive, it is a DirectoryString, which means pretty much Case Ignore String in eDirectory, which is good, and it is single valued. This all looks great so far.

Next the Remote Loader converts the XDS event to a shim event (basically an LDAP modify) as the trace shows:

DirXML: [12/30/10 17:32:09.76]: ADDriver: Add contact cn=RSMITH7,ou=Contacts,dc=acme,dc=local
LDAPMod operations:
add attribute objectClass
>> contact
add attribute objectCategory
>> CN=Person,CN=Schema,CN=Configuration,DC=acme,DC=local
add attribute displayName
>> Ryan Smith
add attribute givenName
>> Ryan
add attribute sn
>> Smith
add attribute telephoneNumber
>> NA
add attribute title
>> User Administrator
add attribute cn
>> RSMITH7
add attribute mail
>> RSMITH7@acmemail.com
add attribute proxyAddresses
>> SMTP:RSMITH7@acmemail.com
add attribute targetAddress
>> SMTP:RSMITH7@acmemail.com
DirXML: [12/30/10 17:32:09.78]: ADDriver: set userAccountControl returns 0x0041
DirXML: [12/30/10 17:32:09.78]: Loader: subscriptionShim->execute() returned:
DirXML: [12/30/10 17:32:09.78]: Loader: XML Document:
DirXML: [12/30/10 17:32:09.78]: <nds ndsversion="8.7" dtdversion="1.1">
	<source>
		<product version="3.5.10" asn1id="" build="20100709_120000" instance="\acmeIDV\idv\servers\IDMDriverSet\BresnanAD">AD</product>
		<contact>Novell, Inc.</contact>
	</source>
	<output>
		<add-association dest-dn="\acmeIDV\idv\Users\RSMITH7" dest-entry-id="214293" event-id="acmedidv3#20101230223222#3#1">b1c74114f57e9c4aa2d102343ac74cea</add-association>
		<status level="success" event-id="acmedidv3#20101230223222#3#1"/>
	</output>
</nds>
DirXML: [12/30/10 17:32:09.78]: 
DirXML Log Event -------------------
    Driver  = \acmeIDV\idv\servers\IDMDriverSet\BresnanAD
    Thread  = Subscriber Channel
    Object  = \acmeIDV\idv\Users\RSMITH7 (cn=RSMITH7,ou=Contacts,dc=acme,dc=local)
    Level   = success
	
	

Ok, so we had a successful add of a contact object, the shim sent back an <add-association> event, but wow nellie! Where did mailNickname go? It was there a second ago. It let proxyAddresses and targetAddress through. Why did it filter out mailNickname?

It turns out I have a lousy memory for fine detail. But I have a pretty good memory for high level stuff, and while I read a lot, I do not remember a lot of details, but I do know I can use the magic of Google to augment my fine detail memory. A quick search turned up a thread in the forums, that I had even commented on a couple of times,. that I vaguely remembered at a high level, but could not remember any of the fine details. Well my Google memory crutch worked again! (Actually, this is why I started writing for Cool Solutions. I would fix something at work, and then hit it again a month or three later and totally forget how to fix it).

http://forums.novell.com/novell-product-support-forums/identity-manager/im-engine-drivers/404302-ad-driver-stripping-my-attribute-post1948543.html

Anyway, it turns out that on an <add> event, the Active Directory driver knows that a couple of attributes are really Exchange attributes, and for Exchange 2007 and 2010, are managed via PowerShell instead of via direct mapping. Therefore it strips them out. The latest (well 3.5.10 version used to be latest, as I write this, 3.5.13 was out already) version of the driver (Which I was actually running) will not strip them out, if you are set to use the default Exchange provisioning model.

Well it turns out, I had Exchange provisioning disabled, but nonetheless the Global Configuration Variable (GCV) that controlled this setting was still set to Exchange 2007 in the configuration I was using. So I had to go enable Exchange provisioning via the GCV, set the type of Exchange server to Default and save it. Then go back, and Disable Exchange provisioning.

Additionally what worked was changing the set destination attribute for mailNickname to be set after the current operation.

That meant that the <add> event would go through, setting almost all the attributes I needed, but it would then be followed by a <modify> event of just a single attribute, mailNickname, which we had already shown worked.

Anyway, this certainly was one of those interesting issues, that is quite complicated at the end, but really just a series of simple steps, if you can break it down. Hope this helps someone down the road when they run into a similar problem, like the forum post helped me out!

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)

Categories: Uncategorized

Disclaimer: As with everything else at NetIQ Cool Solutions, this content is definitely not supported by NetIQ, so Customer Support will not be able to help you if it has any adverse effect on your environment.  It just worked for at least one person, and perhaps it will be useful for you too.  Be sure to test in a non-production environment.

Comment