6.5 Configuring Data Retention Policies

The data retention policies control when data should be deleted from the system. A retention policy contains a filter that is used to identify the events for which the retention policy applies and the minimum and maximum number of days these events should be kept in the system.

You can configure one or more data retention policies to control the duration for which specific types of events are retained in Sentinel. Except for the Raw Data Retention policy, all of the configured policies apply to the event data.

The configured retention policies are displayed in the data retention policy table. By default, the data retention policy table is refreshed every 30 seconds to reflect the changes made by multiple administrators.

6.5.1 Raw Data Retention Policy

The raw data retention policy controls the duration for which the raw data is kept in the system before it is deleted.The raw data retention policy cannot be deleted or disabled. However, you can modify the Keep at most and Keep at Least values, which determine the maximum and minimum number of days to keep the raw data file.

The process to delete raw data files runs every time the server is started, every hour because that is when the raw data files are closed, and whenever the Keep at most value is changed. All the files exceeding the retention time are removed permanently from the primary and secondary storage locations.

6.5.2 Event Data Retention Policies

The event data retention policies control the duration for which different types of event data are kept in the system before being deleted.

To create a data retention policy:

  1. Log in to the Sentinel Web interface as a user in the administrator role.

  2. Click Storage > Configuration.

  3. In the Data Retention section, click Create.

  4. Use the following information to create the data retention policy.

    Policy name: Specify a name for the retention policy.

    The policy name must be unique and must contain alphanumeric characters.

    Filter: Click the Filter icon to select a saved or shared filter.

    or

    Specify a filter value. The filter value uses the same syntax as searches.

    For example, assume that the Filter field contains a filter such as sev:[3 TO 5] AND (evt:"SyslogNICListener"). This filter value matches all the events with a severity of 3, 4 or 5 and event name SyslogNICListener.

    For more information, see Configuring Filters in the NetIQ Sentinel User Guide.

    For example, use sev:[0 TO 1] to define a retention policy that applies to all events with a severity of 0 or 1.

    Keep at least: Specify the minimum number of days to retain the events in the system. The value must be a valid positive integer.

    Keep at most: (Optional) Specify the maximum number of days for which the events should be retained in the system.

    The value must be a valid positive integer and must be greater than or equal to the Keep at least value. If no value is specified, the system retains the events in the system until the space is available.

  5. Click Save. The newly created policy is displayed in the data retention table.

    The table also contains the following additional columns:

    Size: Displays the amount of space used to store the events for each retention policy.

    Events: Displays the number of events for the selected retention policy.

    The policies are sorted in alphabetical order by policy name. The default retention policy is always shown as the last policy in the list.

    For more information, see Section 6.9, Data Deletion Policy.

6.5.3 Rules for Applying a Retention Policy

An event could match the filter criteria of multiple data retention policies.

To determine which data retention policy will apply to an event and, therefore, how long an event will be retained before deleting it from the primary and secondary data stores, apply the following rules:

  1. If an event meets the criteria of only one data retention policy filter, that data retention policy is applied to the event.

  2. If an event does not meet the criteria for any of the data retention policies, the default data retention policy is applied to that event.

  3. If an event meets the criteria for more than one of the data retention policies, the following guidelines are used to determine which data retention policy should be applied:

    • If the maximum retention period of a policy is shorter than the others, that policy is applied. (If the maximum retention period is not specified for a policy, the policy is considered to have a long maximum retention period.)

    • If multiple matching policies have the same shortest maximum retention period, the policy with the longest minimum retention period is applied.

    • If multiple matching policies have the same shortest maximum retention period and the same longest minimum retention period, the system arbitrarily applies one of the policies.