2.1 FLAIM Database

eDirectory uses FLAIM as its database. FLAIM (Flexible Adaptable Information Manager) is used for traditional, volatile, and complex information. It is a very scalable database engine that supports multiple readers and a single-writer concurrency model. Readers do not block writers and writers do not block readers.

Physically, FLAIM organizes data in blocks. Some of the blocks are typically held in memory. They represent the block cache. The entry cache (sometimes called a record cache) caches logical entries from the database. Entries are constructed from the items in the block cache. FLAIM maintains hash tables for both caches. The hash bucket size is periodically adjusted based on the number of items.

By default eDirectory uses a block size of 4 KB. The block cache size for caching the complete DIB is equal to the DIB size, and the size required for the entry cache is about two to four times the DIB size.

While retrieving an entry, FLAIM first checks for the entry in the entry cache. If the entry exists, reading from the block cache isn't necessary. While retrieving a block from the disk, FLAIM first checks for the block in the cache. If the block exists, a disk read operation isn't necessary.

When an entry is added or modified, the corresponding blocks for that entry are not directly committed to the disk, so the disk and memory might not be in sync. However, the updates made to the entry are logged to the roll-forward log (RFL). An RFL is used to recover transactions after a system failure.

Least Recently Used (LRU) is the replacement algorithm used for replacing items in the cache.

2.1.1 Checkpoint

A checkpoint brings the on-disk version of the database to the same coherent state as the in-memory (cached) database. FLAIM can perform a checkpoint during the minimal update activity on the database. It runs every second and writes the dirty blocks (dirty cache) to the disk. Blocks that are modified in the cache but not yet written to the disk are called “dirty blocks”. FLAIM acquires a lock on the database and performs the maximum amount of possible work until either the checkpoint completes or another thread is waiting to update the database. To prevent the on-disk database from becoming too far out of sync, there are conditions under which a checkpoint is forced even if threads are waiting to update the database:

  • If the checkpoint thread cannot complete a checkpoint within a specified time interval (the default is 3 minutes), it is forced and the dirty cache is cleaned.

  • If the size of the dirty cache is larger than the maxdirtycache (if set), a checkpoint is forced to bring down the dirty cache size to mindirtycache (if set) or to zero.

2.1.2 Indexes

An index is a set of keys arranged in a way that significantly speeds up the task of finding any particular key within the index. Index keys are constructed by extracting the contents of one or more fields (attributes) from the entries. Indexes are maintained in the block cache. Any changes to the indexed attributes requires changes in the index blocks.

eDirectory defines a default set of indexes for system attributes (fields). System attributes such as parentID and ancestorID are used for one-level and subtree searches. These indexes cannot be suspended or deleted. The directory internally uses them. Default indexes are defined for attributes such as CN, Surname, Given Name, and so on. Indexes can be of type presence, value, and substring indexes. These indexes can be suspended. On deletion they are automatically re-created.

You can use iManager or the ndsindex Lightweight Directory Access Protocol (LDAP) utility to create indexes. Indexes are server-specific.

By enabling the Storage Manager (StrMan) tag in DSTrace (ndstrace), you can view the index chosen for the search queries.

The following example is for a DSTrace log for a subtree search using "cn=admin", CN.

3019918240 StrMan: Iter #b239c18 query ((Flags&1)==1) && ((CN$217A$.Flags&8=="admin") && (AncestorID==32821))
3019918240 StrMan: Iter #b239c18 index = CN$IX$220

The following example is for an DSTrace log for a subtree search using "Description= This is for testing", AncestorID.

2902035360 StrMan: Iter #83075b0 query ((Flags&1)==1) && ((Description$225A$.Flags&8=="This is for testing") && (AncestorID==32821))
2902035360 StrMan: Iter #83075b0 index = AncestorID_IX

To improve the search performance with the server side sort, use the -a option to prefix the AncestorID attribute to the list of attributes passed while creating a new index.

The below table explains the trend of increasing DIB size while creating simple/compound indexes for a certain number of objects. We recommend you to refer to this test data while creating simple/compound indexes:

Number of Objects

Increase in DIB size with simple index containing attribute size of 20 characters

Increase in DIB size with simple index containing attribute size of 200 characters

Increase in DIB size with compound index containing attribute size of 200 characters

23K

24KB

3MB

7MB

47K

50KB

6MB

15MB

0.5 Million

4MB

55MB

100MB

1 Million

9MB

111MB

210MB

2.1.3 Roll-Forward Log

FLAIM logs operations for each update transaction in a roll-forward log (RFL) file. An RFL is used to recover transactions from a system failure or when restoring from a backup. The RFL file is truncated after every checkpoint is completed unless it is turned on (rflkeepfiles) by using a hot continuous backup.

2.1.4 FLAIM Attribute Containerization

To ensure optimal utilization of the entry cache and enhanced performance of attribute search operations, FLAIM stores attributes with larger values or higher number of values in a separate location namely, Attribute Container. By default the attributes will be moved to the container automatically when the attribute:

  • has greater than 25 values

  • has a value greater than 2048 bytes

To disable the automatic containerization of attributes, add disablemovetoattrcontainer =1 in the _ndsdb.ini file and restart eDirectory.

eDirectory provides you the flexibility of scheduling the attribute movement. You first view the attributes that are ready to be moved and then schedule their movement as per your convenience.

To view the number of attributes ready for movement to attribute containers, run the ndscheck command. To view the details of attributes, use iMonitor dsContainerReadyAttrs attribute on the Pseudo server objects in the Agent Configuration. User can also find those attributes which are marked for indexing in the Agent Health.

You can start the attribute containerization by using the single object repair option of ndsrepair for the Pseudo server object. To containerize an attribute, issue the ndsrepair command with the new advance switch -am followed by the name of the attribute as below:

ndsrepair –J <Pseudo server object ID> –Ad –AM/–am <attribute name>

After moving an attribute to the Attribute Container, eDirectory creates a system index with the name of the attribute. When an attribute is containerized, you cannot move it back to the original container.

NOTE:If an attribute has a value greater than 2048 bytes, containerization still happens but eDirectory does not create any system index.