2.1 FLAIM Database

eDirectory uses FLAIM as its database. FLAIM (Flexible Adaptable Information Manager) is used for traditional, volatile, and complex information. It is a very scalable database engine that supports multiple readers and a single-writer concurrency model. Readers do not block writers and writers do not block readers.

Physically, FLAIM organizes data in blocks. Some of the blocks are typically held in memory. They represent the block cache. The entry cache (sometimes called a record cache) caches logical entries from the database. Entries are constructed from the items in the block cache. FLAIM maintains hash tables for both caches. The hash bucket size is periodically adjusted based on the number of items.

By default eDirectory uses a block size of 4 KB. The block cache size for caching the complete DIB is equal to the DIB size, and the size required for the entry cache is about two to four times the DIB size.

While retrieving an entry, FLAIM first checks for the entry in the entry cache. If the entry exists, reading from the block cache isn't necessary. While retrieving a block from the disk, FLAIM first checks for the block in the cache. If the block exists, a disk read operation isn't necessary.

When an entry is added or modified, the corresponding blocks for that entry are not directly committed to the disk, so the disk and memory might not be in sync. However, the updates made to the entry are logged to the roll-forward log (RFL). An RFL is used to recover transactions after a system failure.

Least Recently Used (LRU) is the replacement algorithm used for replacing items in the cache.

2.1.1 Checkpoint

A checkpoint brings the on-disk version of the database to the same coherent state as the in-memory (cached) database. FLAIM can perform a checkpoint during the minimal update activity on the database. It runs every second and writes the dirty blocks (dirty cache) to the disk. Blocks that are modified in the cache but not yet written to the disk are called “dirty blocks”. FLAIM acquires a lock on the database and performs the maximum amount of possible work until either the checkpoint completes or another thread is waiting to update the database. To prevent the on-disk database from becoming too far out of sync, there are conditions under which a checkpoint is forced even if threads are waiting to update the database:

  • If the checkpoint thread cannot complete a checkpoint within a specified time interval (the default is 3 minutes), it is forced and the dirty cache is cleaned.

  • If the size of the dirty cache is larger than the maxdirtycache (if set), a checkpoint is forced to bring down the dirty cache size to mindirtycache (if set) or to zero.

2.1.2 Indexes

An index is a set of keys arranged in a way that significantly speeds up the task of finding any particular key within the index. Index keys are constructed by extracting the contents of one or more fields (attributes) from the entries. Indexes are maintained in the block cache. Any changes to the indexed attributes requires changes in the index blocks.

eDirectory defines a default set of indexes for system attributes (fields). System attributes such as parentID and ancestorID are used for one-level and subtree searches. These indexes cannot be suspended or deleted. The directory internally uses them. Default indexes are defined for attributes such as CN, Surname, Given Name, and so on. Indexes can be of type presence, value, and substring indexes. These indexes can be suspended. On deletion they are automatically re-created.

You can use iManager or the ndsindex Lightweight Directory Access Protocol (LDAP) utility to create indexes. Indexes are server-specific.

By enabling the Storage Manager (StrMan) tag in DSTrace (ndstrace), you can view the index chosen for the search queries.

The following example is for a DSTrace log for a subtree search using “cn=admin”, CN.

3019918240 StrMan: Iter #b239c18 query ((Flags&1)==1) && ((CN$217A$.Flags&8=="admin") && (AncestorID==32821))
3019918240 StrMan: Iter #b239c18 index = CN$IX$220

The following example is for an DSTrace log for a subtree search using “Description= This is for testing”, AncestorID.

2902035360 StrMan: Iter #83075b0 query ((Flags&1)==1) && ((Description$225A$.Flags&8=="This is for testing") && (AncestorID==32821))
2902035360 StrMan: Iter #83075b0 index = AncestorID_IX

2.1.3 Roll-Forward Log

FLAIM logs operations for each update transaction in a roll-forward log (RFL) file. An RFL is used to recover transactions from a system failure or when restoring from a backup. The RFL file is truncated after every checkpoint is completed unless it is turned on (rflkeepfiles) by using a hot continuous backup.