Our IDM product line is based on data synchronization technology. From time to time I come across requests for virtualization in identity management projects for various reasons. Some of them hold up, others don’t and fall after only a short investigation. Read on to learn about some of the misconceptions that exist out there regarding synchronization versus virtualization.I went out and did some research on what the general understanding of a virtual and a meta directory is. I found an article on Wikipedia very interesting, actually interesting enough to make changes to it. The article originally stated:

When compared against most metadirectory technologies, virtual directory implementations typically offer several advantages:

  • a simpler administration model,
  • better reaction times against changes as the data is read directly from the source,
  • better adoption in the Corporate IT politics as the ownership of data is not changed,
  • better match for environments where the bulk transfer of changes are inappropriate

When I read that I thought this is seriously wrong. I made the following changes:

When compared against metadirectory technologies, virtual directory implementations offer potential advantages and suffer from certain disadvatages:

potential advantages:

  • In certain political climates it may be preferrable to not synchronize data to a central identity vault. In all the other cases, however, synchronization offers unique advantages (some of which are listed under disadvantages below)
  • Better match for environments where the bulk transfer of changes are inappropriate. An example might be transactional systems which hold information about a lot of transactions but only summaries or only the last couple of transactions should actually be retrieved through the directory service.
  • Potentially better reaction times against changes in low load/request environments as the data is read directly from the source. This advantage may turn quickly into a huge disadvantage in heavy load/request scenarios when all the backend systems are put under heavy load.


  • All data is always available as long as the central identity vault is available. In a virtual directory implementation, some of the delegated data source may not be available and requests may return no or only incomplete data.
  • A central identity vault is usually easier made high-available and fault-tolerant than a conglomeration of separate data stores.
  • In heavy load/request environments the identity vault absorbs all client requests thus protecting the backend systems from having to handle the whole load.
  • Using close-to-realtime synchronization technologies offer comparable performance even in a load/request environment

Now what I really want to understand from anyone who has to share some insights is: Have you been using our products and have you come across situations where virtualization would have come in handy or even saved your project? Have you not used our products because they do synchronization and no virtualization of identity data?

0 votes, average: 0.00 out of 50 votes, average: 0.00 out of 50 votes, average: 0.00 out of 50 votes, average: 0.00 out of 50 votes, average: 0.00 out of 5 (0 votes, average: 0.00 out of 5)
You need to be a registered member to rate this post.
Categories: Uncategorized

Disclaimer: As with everything else at NetIQ Cool Solutions, this content is definitely not supported by NetIQ, so Customer Support will not be able to help you if it has any adverse effect on your environment.  It just worked for at least one person, and perhaps it will be useful for you too.  Be sure to test in a non-production environment.

Leave a Reply

Leave a Comment

  • Luke says:

    Historical Data. We want our Enterprise Directory to be up to date and accurate about current data. IDM does this very well. However, we also have a need for some applications to have access to historical data as well as the current data. We already have a data warehouse that keeps all the historical data, and we don’t want or plan to duplicate it. That means that with your solution, the apps that need both will have to know how to look in the Enterprise Directory and the data warehouse. This is not ideal as we are trying to provide a one stop shop.

    Here’s an example. We have an application that provides an online collaboration space (sites) for students (as well as others, but lets focus on students.) We need to know when current students join and leave class lists to update memberships to sites. We also have to create new sites when new classes are added. IDM and eDir serve this need very well. However, after the classes end, these sites remain, for years. Years later, the application needs to be able to verify that an alumnus was in fact enrolled in the class that the site is linked to. This requires a lookup to the data warehouse.

    The alternative is to send all the current data to the application and have it store and sync a copy, and have it maintain it’s own history data. This means two disconnected copies history. Bad, bad, bad. If eDir could provide the current directly, and virtualize the historical, we would be set. We could use another vendors virtual directory, but that would mean adding another layer to our Enterprise Directory design and infrastructure that folks aren’t interested in paying for or supporting.

  • Thank you for this great real-world use-case for a combination of synchronization and virtualization, Luke. Please drop me an email at vscheuber-at-novell-dot-com so that I can stay in touch with you.

    Thanks again, Luke!

  • Matt, I appreciate your thoughts, even though I have not yet been able to read them. Your blog on blogspot seems to be experiencing severe problems at the moment and is only returning server error 500. I’ll try again later to follow your link.

  • Mr Scheuber, I believe Enterprise customers see benefits in using Virtual Directory technology in partnership with synch tools. The “AND” architecture delivers far more interoperability and faster application deployment then an “OR” option wouldn’t you agree ?

  • Mr Tyrrell, the idea of having and AND-option available is intriguing. I’m not that sure about enhanced interoperability and faster application deployment in general but I can see cases where having both makes sense politically and/or technically. From my experience most of the time is spent analyzing the IS situation and figuring out what the SHOULD is supposed to be (or the other way around?) and not the technical implementation. What were your thoughts behind your statement of “more interoperability and faster application deployment”?

  • Chris Van Den Abbeele says:

    another “pro” for a real virtual directory comes from the side of compliance. Several European Privacy Protection Acts state that some privacy data cannot be stored in a combination with other privacy data in the same database.

    An example (that I made up myself) : some health-insurance company might be entitled to :
    1. store information on your income and financial health in one database
    2. and store information on your physical health (but in another database)

    A person who is entitled to see the full picture is allowed to see the combined information in one screen, but the data cannot be stored in the same database.

    ps :
    to be perfectly honnest : This information was brought to me as being reliable but I do not have it first hand from some compliance handbook.

  • Chris, thank you very much for your insights. Any more information on these privacy protection acts would be helpful to better understand the real business needs.

  • Its a good conversation. Here are my comments..

    Virtual Directory actually complements a Meta Directory Solution.

    You would use Meta Directory Solution to Syncrhonize Enterprise Identity Data
    across all the data stores to keep the informatiion Consistent.

    If you take a look at any large enterprise, there are atleast 10 different Systems
    each having your Identity Information such as First name, Last Name, Phone, Address, etc.

    In this scenario, Meta Directory solution helps to keep the data in Sync
    with all the data stores.

    However, as Organizations build new systems, buy other COTS products
    two basic requirements they have to address:
    Keep the User Identity data in sync with enterprise data

    Enforce Strict Access Control (it will be tough to move all the
    old and new Products or Systems authorization information
    into Enterprise Directory)

    Now if you were to use Enterprise Directory (syncrhnozied
    using your Meta Directory product) there is a significant
    effort involved in creating those sync rules with new data store.

    However if you use Virtual Directory,
    you can join any data store on the back end and can control
    access to applications without having to deploy
    a meta directory solution (especially for access control).

    Another Example:

    Authentication/Aurhoziation often times is not a straightforward
    LDAP Bind/validation alogn with Group or RBAC.
    Especially applications in Insurance, HealthCare, Financial, etc requires more than
    just validating uid/pwd to authenticate the user.

    If you deploy just a Meta Directory Solution,
    a) you have to syncrhonize data with Enterprise Directory
    b) Think about the time it takes to syncrhonize 10 data sources into
    one directory and the time the user has to wait if they need access
    to all the 10 applications.


  • Hi Matt, I read your post a day later and have yet to respond back to you. You bring up a couple of good points and valid examples that I can understand and follow. Thank you for your participation in this thread!

  • Hi Ram, thank you for jumping in, here! Some thoughts from my side:
    Agreed with your points where a Meta Directory would come in handy. I do not necessarily agree (or understand?) that access control issues of any kind should be handled through virtualization. The two most important reasons not to do that are performance and system availability.
    Having a virtualization layer on top of your application farm creates 2 bottle necks: the virtualization engine plus each of the connected systems. I take it that bottle neck #1 (the virtualization engine) is solved by its vendor which leaves us with the other applications. Now if these were all applictions optimized for read access and all local to the virtualization engine, we would probably get only a marginal performance hit compared to a Meta Directory. But looking at the IT landscapes I have seen in the field, this ideal situation can hardly ever be found. Many enterprise applications are not optimized for read performance but are optimized for their primary purpose which is usually not authentication and authorization. This makes each request to the virtualization engine take as long as the slowest application to respond to the redirected request. In addition to that, all data transformations have to be done on-the-fly for every request. It is a fact that virtual directories respond slower to requests than Meta Directories. Now this may or may not affect you but when it comes to authentication and authorization, it usually does. I have also heard comments on attaching a cache to the Virtual Directory but that almost puts away the difference between a Meta and a Virtual Directory, doesn’t it?
    You are not the only one making the point that the integration of a virtualization solution is less intrusive and easier than the integration of a Meta Directory. I don’t see this to be true or I can’t follow your thoughts on this. Here is how I see the implementation story: You need a good understanding of the data model and flow in both cases. You also have to understand the format of the data and how it has to be transformed to serve the requester. A Meta Directory usually allows you to implement data transformation policies or rules and also deals with problems like creation and placement of objects. It is true that all these policies need to be implemented but if we turn around and look at the Virtual Directory, we will find the same set of policies and rules required over there: Data needs to be transformed and objects need to be virtually created and placed in the virtual environment at request time. I don’t see any gain in implementation time.
    So far the only reasons that really held up in my mind are:
    – Regulations and laws that require things to be in a certain way which exclude Meta Directories as an option.
    – Political reasons where companies or individuals just don’t want data to be synchronized
    – A lot of fast changing data in surrounding applications
    – Loads of stale data which needs to be accessed sporadically only
    Just some thoughts from my side.

  • Thanks for taking time to reply.
    I really like this kind of conversation so please feel free to post your thoughts.

    I understand you need to have complete understanding of data formats for both
    Synchronization vs Virtualization. Yes caching defeats the purpose of virtualization to some extent.
    I am totally on board to do Syncrhonization when it comes to Identity Data (First name, Last name, email, etc).
    However when it comes to access control the synchrnoization usually slows down the process.
    (I am not saying VD is the perfect solution for all the customer scenarios but however
    its worth considering as part of Enterprise Architecture)

    Organization XYZ have Three or more COTS products where the access control is maintained by them.
    You have a Portal which determines who can see what links in the Portal (and many of them links to the COTS product pages)
    You have multiple Custom home grown application which uses AD, any other LDAP, or database for RBAC.
    you have Web Access Management product that should validate all the access requests.
    Company is fast growing where it buys and sells business units, creates many apps, buys other COTS products.

    Now if you were to tie these together via Syncrhonization, you have to bring each system
    in place to syncrhonize roles and members from their system into Enterprise Directory (and configure your
    access management product against it).

    (I am not debating about the Architecture of Syncrhonization Solution here. It can very well be designed with flexibility
    if data sources are known upfront..however that usually isnt the case )

    Now if you were to use Virtualization In place, you configure your Web Access Management and Portal to go against
    the Virtual Directory.
    VD in turn talks to other Data sources for Roles and Members.

    This addresses the regulatory issues, political issues, and obviously the data that is left to the Application
    or Product to manage (Roles and members).


  • Tim Paul says:

    most of the story is right on, but misses features of a virtual directory focusing on a virtual directory as only a proxy engine, which it is not. virtual directories can offer real-time synchronization AND persistent data, negating most of his “disadvantages”. Meta is old, Virtual is new and more adaptive

By: vscheuber
Jan 18, 2007
4:08 pm
Active Directory Authentication Automation Cloud Computing Cloud Security Configuration Customizing Data Breach DirXML Drivers End User Management Identity Manager Importing-Exporting / ICE/ LDIF Intelligent Workload Management IT Security Knowledge Depot LDAP Monitoring Open Enterprise Server Passwords Reporting Secure Access Supported Troubleshooting Workflow