Need for ActiveMQ Clustering:

It is observed that a JDBC Fan-Out Driver connects to only a single Fan-Out agent via ActiveMQ.

In case of Driver failure, all the events triggered from Identity vault are stored in driver cache and once the driver comes up, subsequently all the events will get processed without any event loss. Similarly in the case of failure in JDBC Driver instances, the events will be placed in individual instance queues of ActiveMQ and these events will be processed as delayed events ensuring no event loss.

What if the single ActiveMQ connecting JDBC Fan-Out Driver and Fan-Out Agent Fails??. Identity vault events will get processed by JDBC Fan-Out Driver and will be placed in ActiveMQ for further processing. If ActiveMQ running on a server goes down for some reason at this time, Fan-Out Driver instances will not be able to get any events for further processing and hence there will be a event loss.

The solution for above problem is ActiveMQ Clustering.

JDBC Fan-Out Driver:

The JDBC Fan-Out driver relies on the following independent components.

Fan-Out Driver Shim: The Fan-Out driver shim is a Java-based interface driver. The driver shim virtually connects to the Fan-Out agent through ActiveMQ.

ActiveMQ: The Fan-Out driver and the Fan-Out agent use ActiveMQ for transferring the Subscriber events, configuration data, and the queries. The Fan-Out agent creates a separate queue for each JDBC driver instance. The JDBC driver instances wait for the events in their respective queues.

Fan-Out Agent: The Fan-Out agent is a standalone Java process that works independently of the Identity Vault. The Fan-Out agent loads the JDBC driver instances based on the configuration of the connection objects in the Fan-Out driver.

Figure 1-1 Fan-Out Driver Configuration

Figure 1-1 Fan-Out Driver Configuration

The Fan-Out process works as follows:

  1. The Identity Vault stores the connection object configuration. The configuration includes connection, authentication, and trace information.
  2. The Fan-Out driver receives the initialization document from the engine.
  3. The Fan-Out driver and the Fan-Out agent perform a handshake to establish a connection. This signals the start of the communication between the agent and the driver. The handshake is done through challenge-response sets.
  4. The Fan-Out driver queries the Identity Vault for the connection objects associated with this driver and creates multiple initialization documents based on the content in the connection objects.
  5. The Fan-Out driver sends the initialization documents to the Fan-Out agent through ActiveMQ.
  6. The Fan-Out agent loads the JDBC driver instances.
  7. The Fan-Out driver sends the events to the Fan-Out agent through ActiveMQ.
  8. The Fan-Out agent determines which JDBC driver instances this event should be sent to.
  9. The JDBC driver processes the event and sends the status of the event to the Fan-Out driver through ActiveMQ.
  10. The Fan-Out driver sends this status to the Identity Manager engine.

For more information see: https://www.netiq.com/documentation/idm45drivers/jdbc_fanout/data/b1imcw6p.html

ActiveMQ:

ActiveMQ is a open source message oriented middleware (MOM) from the Apache Software Foundation. ActiveMQ is written in Java and allows communication between systems using theJMS(Java Message Service) specification.

The most important advantage of using a MOM is to loosen the coupling between applications. It mediates communication amongst applications, minimizing the mutual awareness that applications should have of each other in order to be able to exchange messages

Need for ActiveMQ Clustering:

From Figure 1-1, it is observed that a JDBC Fan-Out Driver connects to only a single Fan-Out agent via ActiveMQ.

In case of Driver failure, all the events triggered from Identity vault are store in driver cache and once the driver comes up, subsequently all the events will get processed without any event loss. Similarly in the case of failure in JDBC Driver instances, the events will be placed in individual instance queues of ActiveMQ and these events will be processed as delayed events ensuring no event loss.

What if the single ActiveMQ connecting JDBC Fan-Out Driver and Fan-Out Agent Fails??. Identity vault events will get processed by JDBC Fan-Out Driver and will be placed in ActiveMQ for further processing. If ActiveMQ running on a server goes down for some reason at this time, Fan-Out Driver instances will not be able to get any events for further processing and hence there will be a event loss.

The solution for above problem is ActiveMQ Clustering.

ActiveMQ Clustering:
Apache ActiveMQ supports a number of clustered modes of operation, of which the master-slave failover mode is probably the most widely used. In this mode of operation, two or more message brokers compete for the lock on a shared filestore; the broker that obtains the lock becomes the master, and any others remain slaves. Only the master broker will accept network connections from clients, so this mode of operation works in conjuction with the ActiveMQ client runtime, which detects the master and connects to it. The roles in the cluster will only change if the current master fails, in which case a slave will promote itself to master by taking control of the lock which the previous master released.

Figure 1-2  ActiveMQ Clustering

Figure 1-2 ActiveMQ Clustering

From the figure 1-2, there will be ActiveMQ Master and ActiveMQ Slave. JDBC Fan-Out Driver and Fan-Out Agent are connected to ActiveMQ Master. There can be any number of ActiveMQ Slaves present. ActiveMQ uses a flat file database called ‘kahadb’ while will be mounted on a nfs server. This shared database will be accessed by both ActiveMQ Master and Slave. At any point of time, only one ActiveMQ will have lock to the shared database and other will be waiting for the lock till the failure happens in its peer. Once when there is a failure in the master ActiveMQ, the slave DB will get acquire lock to the shared database and event processing will continue.

Steps for ActiveMQ Clustering:

  1. Configure nfs server pointing to the shared database directory.
  2. Configure nfs clientsin both Primary and Secondary servers and mount shared database in nfs server to a local activemq database directory.
  3. In both Primary and Secondary servers , edit <ActiveMQ directory>/conf/activemq.xml and change database directory to local database directory as below.
    <broker xmlns="http://activemq.apache.org/schema/core" brokerName="amq02"
    dataDirectory="/mnt/amq_data">
    <!-- ...blablablabla...-->
    <persistenceAdapter>
     <kahaDB directory="/mnt/amq_data/kahadb"/>
    </persistenceAdapter>
  4. In Primary ActiveMQ Server, edit <ActiveMQ directory>/conf/activemq.xml and append below lines.
    <networkConnectors>
             <networkConnector
               uri="static:(failover:(tcp://secondary IP:61616)?randomize=false)"
               dynamicOnly="true"
               networkTTL="3"
               duplex="true"/>
       </networkConnectors>
  5. Start ActiveMQ in primary server and then start ActiveMQ in secondary server.
  6. In Linux, Edit <FanoutAgent Installation Location>/config/fanoutagent-config.properties, change “netiq.fanoutagent.connection.url” to “failover: (tcp://primary:61616,tcp://secodary:61616)?randomize=false” and restart fanout agent.
  7. Edit Fanout driver properties and change Authentication contex to “failover: (tcp://primary:61616,tcp://secodary:61616)?randomize=false” and restart Fan-Out Driver.
3 votes, average: 5.00 out of 53 votes, average: 5.00 out of 53 votes, average: 5.00 out of 53 votes, average: 5.00 out of 53 votes, average: 5.00 out of 5 (3 votes, average: 5.00 out of 5)
You need to be a registered member to rate this post.
Loading...

Disclaimer: As with everything else at NetIQ Cool Solutions, this content is definitely not supported by NetIQ, so Customer Support will not be able to help you if it has any adverse effect on your environment.  It just worked for at least one person, and perhaps it will be useful for you too.  Be sure to test in a non-production environment.

Leave a Reply

One Comment

By: rneelam
Jul 11, 2016
1:38 pm
Reads:
1,213
Score:
5
Active Directory Authentication Automation Cloud Computing Cloud Security Configuration Customizing Data Breach DirXML Drivers End User Management Identity Manager Importing-Exporting / ICE/ LDIF Intelligent Workload Management Knowledge Depot LDAP Monitoring Open Enterprise Server Passwords Reporting Secure Access Sentinel Supported Troubleshooting Workflow