A Forum reader recently asked:
“Can someone point me to some documentation that can tell me how to optimize LDAP lookups with eDirectory 188.8.131.52? I am having a problem with our Novell 6.0 SP5 server hitting 100% utilization with LDAP lookups when we get hit hard. I don’t know how to set up an index, but I’m getting the feeling that is what I need to do to improve performace. How do I check to see if the lookups are indexed?
Also, I have seen some mention somewhere about exporting LDAP to a flat file – would that help?”
And here’s the response from Dave Gersic …
An LDAP search has four interesting components from your point of view. They are:
- base – what container to start searching from
- scope – subtree search from the base, or not
- filter – what to search for
- attribute list – what to return for entries matching the filter
Often, you configure the search base to be the tree root, with a scope of subtree, to search the whole tree. That’s obviously the most useful possible search, but the most costly. If you can restrict the search by lowering the base to some other layer of your hierarchical tree, or by using non-subtree searches, then they’ll be faster by simply not looking at as many entries. This is often not especially helpful, though, if you’re really needing to look at all users.
Filters and Attributes
A filter is a compound list of things to search for. You can use AND, OR, and NOT operators to combine multiple search terms. Any part of an object in the directory can be used in a filter. So, if you want all users, a filter of “Objectclass=User” would return them all. If you want all users with a first name of “Steve,” it would be expressed as:
In general, the more complex and complicated the filter, the more horsepower and time required to do the search.
An attribute list can be described as “what do you want to know?”. If you don’t specify an attribute list, you get back all attributes. So, if you wanted the CN, First Name, and Last Name, for all users whos First Name is “Steve”, you could do something like the following, which shows the Filter and the Attributes list:
"(&(objectclass=user)(givenname=steve))" cn givenname surname
Generally, reading a bunch of attributes isn’t slow, but finding the entries to read them from could be, if your filters are overly complicated or using non-indexed search attributes.
So what to index? Here are some steps to follow:
1. Turn on DSTRACE on your server and log it to a file.
2. Turn off everything but +LDAP.
3. Run your application and turn the log off.
4. Find the search in the log, which will show you the filter being used.
5. Look at the attributes used in the filter, then go to your server and make sure that each one of them is Indexed.
To check to see what need to be indexed, first you need to know the attributes. You can see those in the filter (mail and description). Then you go look at the LDAP Group object for your server and see if those are the real attribute names, or if they’re being mapped to something else. (In this case, I think “mail” is really “internet email address” and “description” is “description”, but check your configuration to see if that’s true). Then you go to the Server object, go to the Indexes page or role or whatever, and look to see if those are indexed (mail might be, description probably is not). If not, you add them. Given the filters above, value indexes are probably what you want (ie: looking for “jycgumm” is a value index, but look for “jyc*” would use a substring index).
Depending on the search run, value or substring indexes may be useful. I seem to recall reading somewhere that at least two characters have to be specified for a substring index to be used, so “givenname=s*” is a non-indexed search, but “givenname=ste*” is indexed, assuming that there’s a substring index established for the givenname attribute.
Value indexes are used to find exact values (no wildcards). Everybody who has a surname of “steve” is a value index, because “steve” does not have a wildcard in it.
Substring indexes are used for partial values with wildcards. Everybody who has a surname of “ste*” would use a substring index, and could return values of “steve” an “stevens” and “stevedore” since those all match the pattern “ste*”.
Using a flat file won’t really do you any good. Using another tree or server might, but tree design and hardware come in to play then, plus you’re increasing your system complexity. It might be good, or bad – I’d try the index configuration first, as that’s quick, easy, should help, and doesn’t greatly affect your current system configuration or complexity.
Decoding the filter takes a bit of thinking sometimes, but if you’ve ever seen prefix notation before, it’ll make sense. “|” is the symbol used for “or” operations. If you haven’t seen prefix notation before, it’s just a matter of switching around the operation (what you want to do) and the operands (what you’re doing the operation with) from the infix notation you’re probably more familiar with.
Your filter here is:
(OR (mail = something) (description = something))
Or, rearranged in to the way you’d say it in english (infix notation):
(mail = something) OR (description = something)
If you have control over the search filter being used, then minimizing the filter used and indexing the attributes used in the filter is your best bet. If you do NOT have control over the search filter, then using DSTrace with the +LDAP flag to see what filter is being used will show you what needs to be indexed.