A.3 Architecture Overview

The Sentinel system is responsible for receiving events from the Collector Manager. The events are then displayed in real-time in an Active View and logged into a database for historical analysis.

At a high level, the Sentinel system uses a relational database and is comprised of Sentinel processes and a reporting engine. The system accepts events from the Collector Manager as its input. The Collector Manager interfaces with third-party products and normalizes the data from these products. The normalized data is then sent to the Sentinel processes and database.

Historical analysis and reporting can be done using the Sentinel integrated reporting engine. The reporting engine extracts data from the database and integrates the report displays into the Sentinel Control Center using HTML documents over an HTTP connection.

Figure A-1 Sentinel Architecture

A.3.1 iSCALE Platform

The Sentinel iSCALE architecture is built using a standards-based, service-oriented architecture (SOA) that combines the advantages of in-memory processing and distributed computing. iSCALE is a specialized message bus capable of handling high data volumes.

Message Bus

The iSCALE message bus allows for independent scaling of individual components while also allowing for standards-based integration with external applications. The key to scalability is that, unlike other distributed software, no two peer components communicate with each other directly. All components communicate through the message bus, which is capable of moving thousands of message packets per second.

Leveraging the unique features of the message bus, the high-throughput communication channel can maximize and sustain a high data throughput rate across the independent components of the system. Events are compressed and encrypted on the wire for secure and efficient delivery from the edge of the network or collection points to the hub of the system, where real-time analytics are performed.

The iSCALE message bus employs a variety of queuing services that improve the reliability of the communication beyond the security and performance aspects of the platform. Using a variety of transient and durable queues, the system offers unparalleled reliability and fault tolerance. For instance, important messages in transit are saved (by being queued) in case of a failure in the communication path. The queued message is delivered to the destination after the system recovers from the failure state.

Figure A-2 iSCALE Message Bus

Channels

The iSCALE platform employs a data-driven or event-driven model that allows independent scaling of components for the entire system based on the workload. This provides a flexible deployment model because each customer’s environment varies: one site can have a large number of devices with low event volumes; another site can have fewer devices with very high event volumes. The event densities (that is, the event aggregation and event multiplexing pattern on the wire from the collection points) are different in these cases and the message bus allows for consistent scaling of disparate workloads.

iSCALE takes advantage of an independent, multi-channel environment, which virtually eliminates contention and promotes parallel processing of events. These channels and sub-channels work not only for event data transport but also offer fine-grain process control for scaling and load balancing the system under varying load conditions. Using independent service channels such as control channels and status channels, in addition to the main event channel, allows sophisticated and cost-effective scaling of event-driven architecture.

A.3.2 Sentinel Event

Sentinel receives information from devices, normalizes this information into a structure called a Sentinel Event, or Event for short and sends the event for processing. Events are processed by the real time display, correlation engine and the backend server.

An event comprises of more than 200 tags. Tags are of different types and of different purposes. There are some predefined tags such as severity, criticality, destination IP and destination port. There are two sets of configurable tags: Reserved Tags are for Novell internal use to allow future expansion and Customer Tags are for customer extensions.

Tags can be repurposed by renaming them. The source for a tag can either be external, which means that it is set explicitly by the device or the corresponding Collector or referential. The value of a referential tag is computed as a function of one or more other tags using the mapping service. For example, a tag can be defined to be the building code for the building containing the asset mentioned as the destination IP of an event. For example, a tag can be computed by the mapping service using a customer defined map using the destination IP from the event.

Mapping Service

Map Service allows a sophisticated mechanism to propagate business relevance data throughout the system. This facility aids scalability and provides an extensibility advantage by enabling intelligent data transfer between different nodes of the distributed system.

Map Service is a data propagation facility that gives the ability to cross-reference Vulnerability Scanner data with Intrusion Detection System signatures and more (for example, asset data, business-relevant data). This allows immediate notification when an attack is attempting to exploit a vulnerable system. Three separate components provide this functionality:

  • Collection of real time events from an intrusion detection source;

  • Comparing those signatures to the latest vulnerability scans; and

  • Cross referencing an attack feed through Sentinel Advisor (an optional product module, which cross-references between real-time IDS attack signatures and the user’s vulnerability scanner data).

Map Service dynamically propagates information throughput the system without impacting system load on the system. When important data sets (that is, “maps” such as asset information or patch update information) are updated in the system, the Map Service propagates the updates across the system, which can often get to be hundreds of megabytes in size.

iSCALE’s Map Service algorithms handle large referential data sets across a production system processing large real-time data volumes. These algorithms are “update-aware” and selectively push only the changes or “delta data sets” from the repository to the edge or system perimeter.

Streaming Maps

Map Service employs a dynamic update model and streams the maps from one point to another, avoiding the build up of large static maps in dynamic memory. The value of this streaming capability is particularly relevant in a mission-critical real-time system such as Sentinel where there needs to be a steady, predictive and agile movement of data independent of any transient load on the system.

Exploit Detection (Mapping Service)

Sentinel provides the ability to cross-reference event data signatures with Vulnerability Scanner data. Users are notified automatically and immediately when an attack is attempting to exploit a vulnerable system. This is accomplished through:

  • Advisor Feed

  • Intrusion detection

  • Vulnerability scanning

  • Firewalls

Advisor provides a cross-reference between event data signatures and vulnerability scanner data. Advisor feed has an alert and attack feed. The alert feed contains information about vulnerabilities and threats. The attack feed is a normalization of event signatures and vulnerability plug-ins. For more information on Advisor, see Section 8.0, Advisor Usage and Maintenance.

You require at least one vulnerability scanner and either an IDS, IPS, or firewall. The IDS and Firewall DeviceName should appear in the RV31 field of the event. Also, the IDS and Firewall must properly populate the DeviceAttackName (rt1) field (for example, WEB-PHP Mambo uploadimage.php access).

The Advisor feed is sent to the database and then to the Exploit Detection Service. The Exploit Detection Service generates one or two files depending on the kind of data that has been updated.

Figure A-3 Exploit Detection

The Exploit Detection Map Files are used by the Mapping Service to map attacks to exploits of vulnerabilities.

Vulnerability Scanners scan for system (asset) vulnerable areas. IDS detects attacks (if any) against these vulnerable areas. Firewalls detect if any traffic is against any of these vulnerable areas. If an attack is associated with any vulnerability, the asset has been exploited.

The Exploit Detection Service generates the exploitdetection.csv file at:

$ESEC_HOME/bin/map_data

The exploitDetection.csv is generated after one of the following:

  • Advisor feed

  • Vulnerability scan

  • Sentinel Server Startup (if enabled in das_query.xml, disabled by default)

By default, there are two configured event columns used for exploit detection and they are referenced from a map (all mapped tags have the Scroll icon).

  • Vulnerability

  • AttackId

Figure A-4 Event Columns

When the Vulnerability field (vul) equals 1, the asset or destination device is exploited. If the Vulnerability field equals 0, the asset or destination device is not exploited.

The map name for the exploitdetection.csv file is IsExploitWatchlist.

There are two types of data sources:

  • External: Retrieves information from the Collector

  • Referenced from Map: Retrieves information from a map file to populate the tag.

The Vulnerability tag has a column entry “_EXIST_”, which means that map result value will be 1 if the key is in IsExploitWatchlist (exploitDetection.csv file) or 0 if it is not. The key columns for the vulnerability tag are IP and NormalizedAttackId. When an incoming event with a DestinationIP event tag that matches the IP column entry and an AttackId event tag that matches the NormalizedAttackId column entry in the same row, the result is one (1). If no match is found in a common row, the result is zero (0).

Figure A-5 Vulnerability and Data Source

A.3.3 Event Source Management

Sentinel 6 delivers a centralized event source management framework to facilitate data source integration. This framework enables all aspects of configuring, deploying, managing and monitoring data Collectors for a broad set of systems, which include databases, operating systems, directories, firewalls, intrusion detection/prevention systems, antivirus applications, mainframes, Web and application servers, and many more.

Using adaptable and flexible technology is central to Sentinel’s event source management strategy, which is achieved through interpretive Collectors that parse, normalize, filter and enrich the events in the data stream.

These Collectors can be modified as needed and are not tied to a specific environment. An integrated development environment allows for interactive creation of Collectors using a “drag and drop” paradigm from a graphical user interface. Non-programmers can create Collectors, ensuring both current and future requirements are met in an ever-changing IT environment. The command and control operation of Collectors (for example, start, stop and so on) is performed centrally from the Sentinel Control Center. The event source management framework takes the data from the source system, performs the transformations and presents the events for later analysis, visualization and reporting purposes. The framework delivers the following components and benefits:

  • Collectors: Parse and normalize events from various systems

  • Connectors: Connect to the data source to get raw data

  • Taxonomy: Allows data from disparate sources to be categorized consistently

  • Filtering: Eliminates irrelevant data at the point of collection, saving bandwidth and disk space.

  • Business relevance: Offers a way to enrich event data with valuable information

  • Collector builder: An Integrated Development Environment (IDE) for building custom Collectors to collect from unique or proprietary systems

  • Live view: User interface for managing live event sources.

  • Scratch pad: User interface for offline design of event source configuration.

A.3.4 Application Integration

External application integration through standard APIs is central to Sentinel. For example, when dealing with a third party trouble-ticketing system, Sentinel 6 can open an initial ticket in its own iTRAC workflow remediation system. Sentinel then uses bi-directional API to communicate with the other trouble ticketing systems—for example, Remedy allowing straightforward integration with external systems.

The API is Web Services-based and therefore allows any external systems that are SOAP-aware to take advantage of pervasive integration with the Sentinel system.

A.3.5 Time

The time of an event is very critical to its processing. It is important for reporting and auditing purposes as well as for real time processing. The correlation engine processes time-ordered streams of events and detects patterns within events as well as temporal patterns in the stream. However, the device generating the event might not know the real time when the event is generated. In order to accommodate this, Sentinel allows two options in processing alerts from security devices: trust the time the device reports and use that as the time of the event, or do not trust the device time and instead stamp the event at the time it is first processed by Sentinel (by the Collector).

Sentinel is a distributed system and is made up of several processes that can be in different parts of the network. In addition, there can be some delay introduced by the device. In order to accommodate this, the Sentinel processes reorder the events into a time-ordered stream before processing.

The following illustration explains the concept of Sentinel Time.

Figure A-6 Sentinel Time

  1. By default, the event time is set to Collector Manager time. The ideal time is the device time. Therefore it is best to set the event time to the device time if the device time is available, accurate, and properly parsed by the Collector.

  2. Events are sorted into 30 second buckets so that they can be viewed in Active Views. By default, the events that have a timestamp within a 5 minute range from the DAS_RT server time (in the past or future) are processed normally. Events that have timestamps more than 5 minutes in the future do not show in the Active Views, but are inserted into the database. Events that have timestamps more than 5 minutes and less than 24 hours in the past are still shown in the charts, but are not shown in the event data for that chart. A drill down operation is necessary to retrieve those events from the database.

  3. Correlation reorder buffer. If the event time is more than 30 seconds older than the server time, the correlation engine does not process the events.

  4. If the event time is older than 5 minutes than the Collector Manager time (correct time), events are directly routed to the database.

A.3.6 System Events

System Events is a means to report on the status and status change of the system. There are three types of events generated by the internal system, they are:

  • Internal Events

  • Performance Events

  • Audit Events

Internal Events

Internal Events are informational and describe a single state or change of state in the system. They report when a user logs in or fails to authenticate, when a process is started or a correlation rule is activated.

Performance Events

Performance Events are generated on a periodic basis and describe average resources used by different parts of the system.

Audit Events

Audit Events are generated internally. Each time an audited method is called or an audited data object is modified, audit framework generates audit events. There are two types of Audit Events. One which monitors user actions for example, user login/out, add/delete user and another which monitors system actions/health, for example, process start/stop.

Some of these events used to be called Internal Events (mainly for system actions/health monitoring). So the functionality of Audit Events is similar to Internal Events. Audit Events can be logged into log files, saved into database, and sent out as Audit Event at the same time. (Internal Events are only sent out as events.).

All System Events populate the following attributes:

  • ST (Sensor Type) field: For internal events it is set to “I” and for performance events it is set to “P”

  • Event ID: A unique UUID for the event

  • Event Time: The time the event was generated

  • Source: The UUID of the process that generated the event

  • Sensor Name: The name of the process that generated the event (for example, DAS_Binary)

  • RV32 (Device Category): Set to “ESEC”

  • Collector: “Performance” for performance events and “Internal” for internal events

In addition to the common attributes, every system event also sets the resource, sub resource, the severity, the event name and the message tags. For internal events, the event name specific enough to identify the exact meaning of the event (for example, UserAuthenticationFailed). The message tags add some specific detail; in the above example the message tag will contain the name of the user, the OS name if available and the machine name). For performance events the event name is generic describing the type of statistical data and the data itself is in the message tag.

Performance events are sent directly to the database. To view them, do a quick query.

For more information, see Section B.0, System Events for Sentinel.

A.3.7 Processes

The following processes and Windows service communicate with each other through iSCALE - the message-oriented middleware (MOM).

The following is the architecture for Sentinel Server.

Figure A-7 Sentinel Server Architecture

Sentinel Service (Watchdog)

Watchdog is a Sentinel Process that manages other Sentinel Processes. If a process other than Watchdog stops, Watchdog will report this and will then restart that process.

If this service is stopped, it will stop all Sentinel processes on that machine. It executes and reports health of other Sentinel processes. This process is launched by the “Sentinel” Windows Service or the “sentinel” UNIX service.

Data Access Service (DAS) Process

The Data Access Service (DAS) process is Sentinel Server's persistence service and provides an interface to the database. It provides data driven access to the database backend.

DAS is a container, composed of five different processes. Each process is responsible for different types of database operations. These processes are controlled by the following configuration files:

  • das_binary.xml: Used for event and correlated event insertion operations

  • das_query.xml: All other database operations

  • activity_container.xml: Used for executing and configuring activity service

  • workflow_container.xml: Used for configuring the workflow (iTRAC) service

  • das_rt.xml: Used for configuring the Active Views function within the Sentinel Control Console

DAS receives requests from the different Sentinel processes, converts them to a query against the database, processes the result from the database and converts it that back to a reply. It supports requests to retrieve events for Quick Query and Event Drill Down, to retrieve vulnerability information and advisor information and to manipulate configuration information. DAS also handles logging of all events being received from the Collector Manager and requests to retrieve and store configuration information.

Correlation Engine Process (correlation_engine)

The Correlation Engine (correlation_engine) process receives events from the Collector Manager and publishes correlated events based on user-defined correlation rules.

Collector Manager

Collector Manager services, processes and sends events.

iSCALE

It is a message-oriented middleware (MOM) that provides the communication platform for all other Sentinel processes.