Enabling eDirectory's event caching: Journal Event Caching vs. XDAS/CEF caching

eDirectory provides an event system whereby an event consumer can register for an event and be notified when it occurs.  A number of utilities and applications can take advantage of event reporting such as ndstrace, audit, etc.

There are two challenges to maintaining a journal event cache in which order must be maintained:
1. Holding all local events that have been created but not yet reported to the local consumer.
2. When a connection is lost, holding specific event notifications not yet sent to a remote reporting server.

eDirectory has two event caching mechanisms.  Each is separate and helps solve one of the challenges outlined above.

---

Journal Event Cache

Description: this cache applies to ALL journal events: ndstrace, XDAS, NAudit, IDM events, etc.  A location on disk can be set to store cached events.  This location will be used rather than memory when the number being created becomes greater than the number that can be processed.  This cache resides in the NDSD event system and is at a lower layer than the XDAS cache.

Pros: this helps reduce NDSD's memory footprint by storing those events not yet processed to disk instead.

Cons: consumes additional disk space but compression lowers this requirement.  It can also be slower to process the events since they must first be written to then retrieved from disk rather than memory.

Configuration: this cache's settings are controlled via environment variables set in the ndsd script. 

- NDSD_EVENT_DISK_CACHE
Enables the Journal Event Cache.

- NDSD_EVENT_DISK_CACHE_DIR
- There is no Journal Event cache setting for specifying the size.  The Journal cache will use file sizes of 4MB or less while implementing its own compression upon them.

---

XDAS Cache

Description: The cache is implemented in the xadauditds layer and is ONLY used when:
1. XDAS specific events are ready to be sent to a remote auditing server.
AND
2. The remote server cannot be reached. 

Pros: Prevents the loss of audit event information when a remote audit server cannot be contacted.  This cache is only used when required.  The events are released once the remote server's connection is reestablished.

Cons: other than some additional disk space used, none since it is only used if there is a problem.

Configuration: this cache's settings are controlled via variables set in the xdasconfig.properties file.

- log4j.appender.S.CacheEnabled
Enables the XDAS Cache for storing XDAS events locally.

- log4j.appender.S.CacheDir
Optionally specifies the directory to use (/var/opt/novell/eDirectory)

- log4j.appender.S.CacheMaxFileSize
Specifies the maximum file size.  Values can be from 50MB to 4GB.  The default is 512MB.

---

These cache methods can be used together.  Consider the following scenario. 
a. An XDAS audit event for a login is thrown but its reporting to the consumer is delayed behind other earlier events.  The event gets written to the Journal Event Cache. 
b. The Journal thread comes along and releases this event from the Journal Cache. 
c. The configured remote audit server cannot be contacted.  The event goes into the XDAS Cache. 
d. The remote server is brought online again.  The event is released from the XDAS cache and sent to a remote syslog appender.