It is important that you collect only the data you require for an adequate understanding of your systems or for charts, graphs, and reports. If you collect too much data, it can be hard to manage and require constant database maintenance.
Typically, you collect data when you want to:
Identify normal operating environment performance values or baseline performance values
Diagnose problems (for example, to view the top CPU consumers after finding that processes are failing)
Create data streams for real-time charts or graphs to identify short‑term trends or compare performance data
Store historical information for trend analysis, capacity planning, or service-level reporting
Although most organizations collect data for a combination of these reasons, it’s important to consider which jobs you use to collect data, how frequently the jobs run, how much data is returned, and how you will use the data in specific charts, graphs, and reports. Don’t collect data for all jobs; only collect it from the jobs from which you need charts, graphs, and reports.
As an example, consider a Knowledge Script that checks server connectivity. You might want to run this monitoring job every 5 minutes to ensure prompt notification if the server connection goes down. However, with data collection enabled, you could run the script less frequently, or only during business hours. By adjusting the schedule for data collection, you avoid cluttering the database with unnecessary information.
In some cases, getting a data point every five minutes is useful. Often, however, frequent data collection doesn’t really provide you with any more useful information than if you were collecting the data point at a longer interval. For example, CPU and memory usage can change significantly in a five-minute period, but logical disk space is not likely to change as frequently and can be effectively monitored at 12- or 24-hour intervals.
As a general guideline, when you monitor values that can change frequently, collect data more frequently so that a statistical analysis of the data can provide a realistic picture of activity on the monitored system. For example, if you collect data on CPU utilization once an hour, or even every 15 minutes, you can capture data spikes or lulls that do not accurately reflect performance, resulting in reports that provide an inaccurate view. When monitoring values that change more slowly, like logical disk space or database growth, you can collect data at longer intervals and still achieve an accurate assessment.
The best practice is to have a clearly defined purpose for collecting the data, such as producing a weekly report of server availability or the top ten e-mail users. It is also important to keep in mind that every data point stored in the repository requires more data space and more database management. Having a clear purpose and understanding the impact of the data you are collecting helps you make the most intelligent decisions about when and how to collect data.
To illustrate the importance of this point, consider a Knowledge Script such as NT_TopCpuProcs, which reports the CPU utilization of all processes running on a managed computer. In most cases, you should only run this job when investigating a problem on the computer. If you enable data collection, you should not run this job on multiple computers or on a regular schedule because of the potential overhead in database space and performance involved in returning so much data.
Therefore, the first step in defining your data-handling policies is to make a list of the specific charts, graphs, and reports you need and the period of time for which you want to view information. For example, determine whether you are interested in real-time charts and graphs only on a daily basis or want to keep charts and graphs active for one or two weeks. Similarly, consider whether you are interested in hourly data or weekly summaries. Although your list is likely to change over time, identifying a few specific charts, graphs, and reports early on can help you collect only the data you need.