Starting a SOAP Driver for IDM – Part 5

geoffc

By: geoffc

October 12, 2010 12:13 pm

Reads:389

Comments:0

Score:Unrated

This entry is part 5 of 9 in the series Getting Started Building a SOAP Driver for IDM

Novell Identity Manager comes with a bunch of prebuilt and out of the box drivers that mostly do what is needed for most cases. However some drivers allow for so much flexibility that no out of the box configuration will ever be complete. The JDBC driver, which can connect to many different databases comes with some of the big ones (Oracle, Microsoft SQL Server) configured, but the rest are sort of up to you. Mostly because almost everyone uses databases differently. Novell has talked about a fan out configuration for the JDBC driver that will come out after the formal release of Identity Manager 4.0. That will probably manage the case of out of the box database models, like using Oracle with ‘Oracle Users’ in which case, you can imagine a model fairly easily where the driver could be set up to push users into many dozens of such Oracle databases that all use the same basic model of users, really differing only in host information (DB server, port, database name, etc) and using entitlements to specify which database the user gets access too.

But the SOAP driver is even harder to provide useful default configurations. Well it ships with a DSML and SPML 1.0 configuration, since those are the only really mature standards for SOAP operations involving users. But SOAP is basically as open ended as you want it to be, and everyone does whatever they want.

For example, the User Application is more about Provisioning than user events directly, but if you really wanted too, you could use the SOAP driver to talk to the User Application. (Actually, it is almost as much fun to use a Workflow, to use a SOAP integration activity, to talk to User Application to do stuff.)

With that said, there are some big targets for SOAP that you could develop configurations for. I started this series to try and provide notions on how you might do that, using Salesforce.com as the target, since it is a pretty big target to aim at. However the concepts involved would be much the same that would be used to try and target any other SOAP system. Heck I used the ideas I developed here in my SOAP integration activity in a Workflow to call User Application web services.

In part 1 of this series, Getting Started Building a SOAP Driver for IDM – Part 1 I discussed some of the things you need to get started building a SOAP driver. I was using the example of Salesforce.com (henceforth known as SFDC, since typing the full name is too much of a pain each time). In Part 1 I focused on how you might start connecting via SOAP to get a session ID,

In Part 2 of this series, Getting Started Building a SOAP Driver for IDM – Part 2 I discussed how you might process the results from SFDC after you submit a login request, and converting it into an <instance> document.

In Part 3 of this series, Getting Started Building a SOAP Driver for IDM – Part 3 I discussed how you might handle query events and their responses.

In Part 4 of this series Getting Started Building a SOAP Driver for IDM – Part 4 I talked about some of the background stuff you need to manage, like attribute syntaxes, and left hanging two more concepts. Subscriber channel write events, like <add> or <modify> events, that need to be sent to SFDC, and the ability to get events onto the Publisher channel.

The good news is that if you peruse the API that SFDC provides for SOAP interface, you will find two useful functions to handle those two cases. There is a getUpdated() function that you call for each object class and requires a start and end time, for which it will return the database Ids for all such objects that were modified in that time window.

For sending events back to SFDC you actually have three choices, and they map somewhat nicely to Identity Manager’s XDS event types. There is a function that nicely maps to an XDS <modify> event called update(), one to an XDS <add> event called insert(), and one much more interesting one, called upsert() which is a combination of the two functions.

The great part about using upsert() is that really you do not need to handle <add> events any different from <modify> events. This makes it much easier since you only have to handle one case, at least in the SFDC direction. Of course an <add> document looks different than a <modify> document in XDS, but that is not as hard to deal with as it might seem at first.

Like the other functionality discussed so far in this series, basically you need to convert the <add> or <modify> XDS event into a SOAP XML document.

As before, I think it is easiest if you can see the before and after documents and then build a policy that gets from here to there. Lets start with <modify> since it turns out to be a little easier than <add>, and once you have <modify> working, <add> is just a few more details, (like Profile to reference, and maybe a few other things like how you get the database ID back, and set associations).

So an XDS <modify> looks something like:

<?xml version="1.0" encoding="UTF-8"?><nds dtdversion="3.5" ndsversion="8.x">
	<source>
		<product version="?.?.?.?">DirXML</product>
		<contact>Novell, Inc.</contact>
	</source>
	<input>
		<modify cached-time="20100426155034.125Z" class-name="Organizational Unit" event-id="nycaidv01m-ebsd#20100426155034#1#1" qualified-src-dn="dc=com\O=ACME\OU=Service" src-dn="\IDVAULT-LAB\com\ACME\Service" src-entry-id="32946" timestamp="1272297034#1">
			<association>IDstringGoesHere</association>
			<modify-attr attr-name="L">
				<add-value>
					<value timestamp="1272297034#1" type="string">46L</value>
				</add-value>
			</modify-attr>
			<modify-attr attr-name="M">
				<remove-value>
					<value timestamp="1272297034#1" type="string">46M</value>
				</remove-value>
			</modify-attr>
			<modify-attr attr-name="N">
				<remove-all-values/>
				<add-value>
					<value timestamp="1272297034#1" type="string">46N</value>
				</add-value>
			</modify-attr>

		</modify>
	</input>
</nds>

This is a silly example with attributes named L, M, and N since I was being lazy. But note that three different type of modify events are mentioned here.

	Attribute L is a simple <add-value> case.
	Attribute M is a common <remove-value> case.
	Attribute N is a common <remove-all-values> then <add-value> case.
	
	

Basically, L is just adding a value. M is just removing one, and N is setting a value by clearing the current values and adding a new one.

You would more commonly see a <remove-value> then an <add-value> if you were changing an existing attribute.

Those are at least three of the basic cases we would need our rule to handle. There are more subtleties that will come up as you test and develop, so handle them each as they come up.

From that XDS event document, we need to generate a SOAP event doc that looks something like this:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:enterprise.soap.sforce.com" xmlns:urn1="urn:sobject.enterprise.soap.sforce.com" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <soapenv:Header>
     <urn:SessionHeader>
        <urn:sessionId>QwWsHJyTPW.1pd0_jXlNKOSU</urn:sessionId>
     </urn:SessionHeader>
  </soapenv:Header>
  <soapenv:Body>
     <urn:upsert>
        <urn:externalIDFieldName>Id</urn:externalIDFieldName>
        <urn:sObjects xsi:type="Account">
           <Id>IDStringGoesHere</Id>
           <L>46L</L>
	   <N>46N</N>
           <fieldsToNull>M</fieldsToNull>
        </urn:sObjects>
     </urn:upsert>
  </soapenv:Body>
</soapenv:Envelope>

Now that is a basic example from the API docs and it gives you a good idea of how some of this should look.

Some things to notice is that you can specify which attribute will be provided as a unique identifier to reference the object you are trying to modify. I am not sure exactly how far you go in terms of using attributes, so I stuck with using the Id attribute in SFDC. For example I do not know what happens if you used say email address, and you had two accounts with the same emails. I never bothered experimenting to figure out what such an upsert() event would generate. I imagine an error document reporting the fact. But you could use the username in SFDC, which in theory ought to be unique as well.

You specify the SFDC object class, in the <urn:sObjects xsi:type=”Account”> node. Of course if you have your own custom object classes in SFDC, they will all end in __c (under bar under bar lower case c) which is more than simply a naming convention, as they seem to enforce it, and often you will get an error message reminding you that if the attribute you tried does not exist, they will recommend you add an __c if it is a custom attribute.

Finally what is a little tricky in XPATH is the fact that the attribute nodes under the sObjects node are the names of the attributes. In XDS land, we usually have identical nodes (<modify-attr> or <add-attr>) that are distinguished by the XML attribute attr-name inside the node. Like <modify-attr attr-name=”L”> or <add-attr attr-name=”M”>. But in SFDC land, they name the node <L> and <M> based on the attribute name. Nothing wrong, just different and slightly different approach to handling it is required.

So how do we get from here to there. Well same approach as you would use to convert from a SOAP XML to XDS <instance> doc as we talked about in previous segments of this series.

This is a little different than before, since now we have all sorts of namespace declarations needed, as you can see in the first node of the SOAP document, the <soapenv:Envelope> node, has four XML namespace declarations:

xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:urn="urn:enterprise.soap.sforce.com"
xmlns:urn1="urn:sobject.enterprise.soap.sforce.com"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

You can paste these into the <policy> node of your rule to add the namespaces to the rule processing flow, or do it using the Namespace button on the upper right hand of the toolbar in Designer. It is after the Deploy, Compare, then Simulator buttons, called Namespace Editor when you mouse over it. To do it ‘by hand’ you would change it in the <policy> node of your Designer rule (flip over to the XML view, and find the <policy> node at the very top, end of the first line) paste the text into it that looks like this:

<policy xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:enterprise.soap.sforce.com" xmlns:urn1="urn:sobject.enterprise.soap.sforce.com" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

You have the input doc, you have the targeted output doc, now build an elegant rule that gets from here to there.

My approach started off nice and simple, and with each edge case I ran into, I updated it to be more and more subtle and sneaky. Not really all THAT subtle, but some cute touches…

Lets look at the simple approach first. (I removed some of the cool tricks, well really just one, and some of my inline comments.).

Before this rule is a rule that limits the events to add or modify events. I could do this with a test for if operation not equal add and another for modify, but I like cheating and instead of using a Case insensitive compare in the if operation equals test, I like to use a regular expression compare, and then test against the value add|modify. In other words, if operation match (equals in a regular expression compare) add|modify.

Nice and simple approach.

<if-operation mode="regex" op="not-equal">add|modify</if-operation>

The actions here would be break(). This way we only process the right kind of events.

Then I have some rules that basically load some variables into driver scoped variables. I used these in a couple of places in the driver, so each time they are used, I need a rule that checks if they are loaded, since it is hard to entirely control the moment they are used.

In this case, I just need the variable SCHEMA-MAP. This ended up being needed, since I found that even with the filter screening out events, sometimes I had events leak through with non mapped attributes, and therefore they were still in the eDirectory namespace, and this meant that the upsert() event would contain invalid data, causing the upsert to fail with a SFDC error message.

I think I tracked down all such instances, but decided to make the rule more robust by double checking since that is a fatal event from SFDC’s point of view. While I was working on that, I came up with a great single line XPATH that basically implements the schema mapping rule. You can read more about it in this article:

Anyway, to get the SCHEMA-MAP variable made available, I load it with this action. I additionally learned a trick from the SAP HR CMP driver, which you can read about in the following series of articles:

SAP Business Logic:

That is a lot of reading, but the relevant lesson I took out is that your rule can possibly event, before the driver is fully instantiated, and in fact will get stuck in the query. Thus you first do a query for the Object Class of the driver DN (via the GCV dirxml.auto.driverdn which always has the Driver DN available, and lets you reference the current driver in common code). The GCV does not work until the driver is properly started, so if you make sure you get a value back, then you know it is safe to proceed.

Anyway, this rule loads up the schema map into a driver scoped local variable. I would change the actual DN to point at your actual Schema Map object, since the name is somewhat up to you to define.

<do-if>
	<arg-conditions>
		<and>
			<if-local-variable mode="regex" name="SCHEMA-MAP" op="not-equal">.+</if-local-variable>
		</and>
	</arg-conditions>
	<arg-actions>
		<do-set-local-variable name="SCHEMA-MAP" scope="driver">
			<arg-node-set>
				<token-xml-parse>
					<token-base64-decode>
						<token-src-attr name="XMLData" notrace="true">
							<arg-dn>
								<token-global-variable name="dirxml.auto.driverdn"/>
								<token-text xml:space="preserve">\[ACME] Schema Map</token-text>
							</arg-dn>
						</token-src-attr>
					</token-base64-decode>
				</token-xml-parse>
			</arg-node-set>
		</do-set-local-variable>
	</arg-actions>
	<arg-actions/>
</do-if>

As you can see this is a pretty complex rule, and I think I will tackle explaining this in part 6 of this series, since this has gone on longer than I expected, just to get to this point.

<rule>
	<description>[acme] Convert Modify to upsert() SOAP doc</description>
	<comment name="author" xml:space="preserve">Geoffrey Carman</comment>
	<comment name="version" xml:space="preserve">5</comment>
	<comment name="lastchanged" xml:space="preserve">Aug 6, 2010</comment>
	<conditions>
		<and>
			<if-operation mode="case" op="equal">modify</if-operation>
		</and>
	</conditions>
	<actions>
		<do-append-xml-element expression=".." name="soapenv:Envelope"/>
		<do-append-xml-element expression="../soapenv:Envelope" name="soapenv:Header"/>
		<do-append-xml-element expression="../soapenv:Envelope/soapenv:Header" name="urn:SessionHeader"/>
		<do-append-xml-element expression="../soapenv:Envelope/soapenv:Header/urn:SessionHeader" name="urn:sessionId"/>
		<do-append-xml-text expression="../soapenv:Envelope/soapenv:Header/urn:SessionHeader/urn:sessionId">
			<arg-string>
				<token-local-variable name="SessionID"/>
			</arg-string>
		</do-append-xml-text>
		<do-append-xml-element expression="../soapenv:Envelope" name="soapenv:Body"/>
		<do-append-xml-element expression="../soapenv:Envelope/soapenv:Body" name="urn:upsert"/>
		<do-append-xml-element expression="../soapenv:Envelope/soapenv:Body/urn:upsert" name="urn:externalIDFieldName"/>
		<do-append-xml-text expression="../soapenv:Envelope/soapenv:Body/urn:upsert/urn:externalIDFieldName">
			<arg-string>
				<token-text xml:space="preserve">Id</token-text>
			</arg-string>
		</do-append-xml-text>
		<do-append-xml-element expression="../soapenv:Envelope/soapenv:Body/urn:upsert" name="urn:sObjects"/>
		<do-if>
			<arg-conditions>
				<and>
					<if-xpath op="true">@class-name</if-xpath>
				</and>
			</arg-conditions>
			<arg-actions>
				<do-set-local-variable name="CLASS-NAME" scope="policy">
					<arg-string>
						<token-xpath expression="@class-name"/>
					</arg-string>
				</do-set-local-variable>
			</arg-actions>
			<arg-actions/>
		</do-if>
		<do-set-local-variable name="EDIR-CLASS" scope="policy">
			<arg-string>
				<token-map dest="edir-class-name" src="class-name" table="\[root]\com\ACME\Drivers\IDM\[ACME]-Library\[acme] SFDC Object mapping table">
					<token-local-variable name="CLASS-NAME"/>
				</token-map>
			</arg-string>
		</do-set-local-variable>
		<do-set-xml-attr expression="../soapenv:Envelope/soapenv:Body/urn:upsert/urn:sObjects" name="xsi:type">
			<arg-string>
				<token-local-variable name="CLASS-NAME"/>
			</arg-string>
		</do-set-xml-attr>
		<do-if>
			<arg-conditions>
				<and>
					<if-xpath op="true">@dest-dn</if-xpath>
				</and>
			</arg-conditions>
			<arg-actions>
				<do-set-local-variable name="DEST-ID" scope="policy">
					<arg-string>
						<token-xpath expression="@dest-dn"/>
					</arg-string>
				</do-set-local-variable>
			</arg-actions>
			<arg-actions>
				<do-if>
					<arg-conditions>
						<and>
							<if-association op="available"/>
						</and>
					</arg-conditions>
					<arg-actions>
						<do-set-local-variable name="DEST-ID" scope="policy">
							<arg-string>
								<token-association/>
							</arg-string>
						</do-set-local-variable>
					</arg-actions>
					<arg-actions>
						<do-if>
							<arg-conditions>
								<and>
									<if-src-attr name="acmeGAId" op="available"/>
								</and>
							</arg-conditions>
							<arg-actions>
								<do-set-local-variable name="DEST-ID" scope="policy">
									<arg-string>
										<token-src-attr name="acmeGAId"/>
									</arg-string>
								</do-set-local-variable>
							</arg-actions>
							<arg-actions>
								<do-status level="warning">
									<arg-string>
										<token-text xml:space="preserve">No acmeGAId available, and cannot process the change.</token-text>
									</arg-string>
								</do-status>
								<do-veto/>
							</arg-actions>
						</do-if>
					</arg-actions>
				</do-if>
			</arg-actions>
		</do-if>
		<do-append-xml-element expression="../soapenv:Envelope/soapenv:Body/urn:upsert/urn:sObjects" name="Id"/>
		<do-append-xml-text expression="../soapenv:Envelope/soapenv:Body/urn:upsert/urn:sObjects/Id">
			<arg-string>
				<token-local-variable name="DEST-ID"/>
			</arg-string>
		</do-append-xml-text>
		<do-for-each>
			<arg-node-set>
				<token-xpath expression="modify-attr"/>
			</arg-node-set>
			<arg-actions>
				<do-set-local-variable name="ATTR-NAME" scope="policy">
					<arg-string>
						<token-xpath expression="$current-node/@attr-name"/>
					</arg-string>
				</do-set-local-variable>
				<do-if>
					<arg-conditions>
						<or>
							<if-xpath op="true">$current-node/@attr-name=$SCHEMA-MAP/attr-name-map/attr-name[@class-name=$EDIR-CLASS]/app-name</if-xpath>
							<if-xpath op="true">$current-node/@attr-name=$SCHEMA-MAP/attr-name-map/attr-name[not(@class-name)]/app-name</if-xpath>
						</or>
					</arg-conditions>
					<arg-actions>
						<do-if>
							<arg-conditions>
								<or>
									<if-xpath op="true">$current-node/remove-value</if-xpath>
									<if-xpath op="true">$current-node/remove-all-values</if-xpath>
								</or>
							</arg-conditions>
							<arg-actions>
								<do-if>
									<arg-conditions>
										<and>
											<if-xpath op="true">$current-node/add-value</if-xpath>
										</and>
									</arg-conditions>
									<arg-actions>
										<do-set-local-variable name="VALUE" scope="policy">
											<arg-string>
												<token-xpath expression="$current-node/add-value"/>
											</arg-string>
										</do-set-local-variable>
										<do-if>
											<arg-conditions>
												<and>
													<if-local-variable mode="regex" name="VALUE" op="equal">.+</if-local-variable>
												</and>
											</arg-conditions>
											<arg-actions>
												<do-append-xml-element expression="../soapenv:Envelope/soapenv:Body/urn:upsert/urn:sObjects" name="$ATTR-NAME$"/>
												<do-append-xml-text expression="../soapenv:Envelope/soapenv:Body/urn:upsert/urn:sObjects/*[last()]">
													<arg-string>
														<token-xpath expression="$current-node/add-value/value"/>
													</arg-string>
												</do-append-xml-text>
											</arg-actions>
											<arg-actions>
											</arg-actions>
										</do-if>
									</arg-actions>
									<arg-actions>
										<do-append-xml-element expression="../soapenv:Envelope/soapenv:Body/urn:upsert/urn:sObjects" name="urn1:fieldsToNull"/>
										<do-append-xml-text expression="../soapenv:Envelope/soapenv:Body/urn:upsert/urn:sObjects/urn1:fieldsToNull">
											<arg-string>
												<token-local-variable name="ATTR-NAME"/>
											</arg-string>
										</do-append-xml-text>
									</arg-actions>
								</do-if>
							</arg-actions>
							<arg-actions>
								<do-set-local-variable name="VALUE" scope="policy">
									<arg-string>
										<token-xpath expression="$current-node/add-value"/>
									</arg-string>
								</do-set-local-variable>
								<do-if>
									<arg-conditions>
										<and>
											<if-local-variable mode="regex" name="VALUE" op="equal">.+</if-local-variable>
										</and>
									</arg-conditions>
									<arg-actions>
										<do-append-xml-element expression="../soapenv:Envelope/soapenv:Body/urn:upsert/urn:sObjects" name="$ATTR-NAME$"/>
										<do-append-xml-text expression="../soapenv:Envelope/soapenv:Body/urn:upsert/urn:sObjects/*[last()]">
											<arg-string>
												<token-xpath expression="$current-node/add-value/value"/>
											</arg-string>
										</do-append-xml-text>
									</arg-actions>
									<arg-actions>
									</arg-actions>
								</do-if>
							</arg-actions>
						</do-if>
					</arg-actions>
					<arg-actions/>
				</do-if>
			</arg-actions>
		</do-for-each>
		<do-if>
			<arg-conditions>
				<and>
					<if-xpath op="not-true">count(../soapenv:Envelope/soapenv:Body/urn:upsert/urn:sObjects/*) > 1</if-xpath>
				</and>
			</arg-conditions>
			<arg-actions>
				<do-strip-xpath expression="self::soapenv:Envelope"/>
				<do-veto/>
			</arg-actions>
			<arg-actions/>
		</do-if>
		<do-strip-xpath expression="self::modify"/>
	</actions>
</rule>

Series Navigation<< Getting Started Building a SOAP Driver for IDM – Part 4Getting Started Building a SOAP Driver for IDM – Part 6 >>
0 votes, average: 0.00 out of 50 votes, average: 0.00 out of 50 votes, average: 0.00 out of 50 votes, average: 0.00 out of 50 votes, average: 0.00 out of 5 (0 votes, average: 0.00 out of 5)
You need to be a registered member to rate this post.
Loading ... Loading ...

Tags: , ,
Categories: Identity Manager, Technical Solutions

Disclaimer: As with everything else at NetIQ Cool Solutions, this content is definitely not supported by NetIQ, so Customer Support will not be able to help you if it has any adverse effect on your environment.  It just worked for at least one person, and perhaps it will be useful for you too.  Be sure to test in a non-production environment.

Comment