With the Sample assessment, you can generate an Executive Summary report and a Complete report. For more information, see Section 8.0, Generating Reports.
Expand the Report tab, click the Generate view tab, and then review the following topics.
The Executive Summary report provides a high-level overview of the results of each component of the VoIP Readiness Assessment.
To generate the Executive Summary report:
In the Generate view, select Executive Summary.
Click Generate Report. The report is generated in Microsoft Word and opens in its own window.
The Executive Summary begins by summarizing the VoIP Readiness Assessment, including which assessments were run and whether they ran to completion:
The Network Inventory section of the summary indicates the number of routers, switches, and links that were discovered.
Executive Summary reports do not include information about Configuration assessments and Bandwidth modeling.
Scroll down through the Executive Summary into the section of results from the Utilization Assessment. After the summary of relevant information about when utilization monitoring was performed and what result ranges were configured, you see the following chart:
The Utilization Readiness Summary is the highest-level summary of the results from utilization monitoring. At a glance, you can see that which of the monitored links had enough “Good” utilization measurements to receive a “Good”, “Acceptable, or “Poor” rating.
The Executive Summary also includes charts that illustrate router, switch, and link utilization measurements broken out by day and by hour. Again, a quick look makes it apparent that the quality of the utilization statistics might vary slightly by the day of the week.
In fact, the Link Measurements by Hour chart, farther into the report, might reveal that utilization measurements were significantly worse between the hours of 8:00 AM and 6:00 PM, the hours when, more than likely, most users were accessing the network.
The “VoIP Quality Assessment” section of the Executive Summary includes information about the Potential Lost Revenue calculation. Vivinet Assessor found that 17.47% of the simulated VoIP calls made during the assessment were either “Poor” or “Unavailable.”
Based on a projected loss of $42.67 per hour, per employee, in the Financial industry, Vivinet Assessor has calculated that the Poor and Unavailable calls in this assessment could lead to a revenue loss of as much as $745.44 per hour.
Farther down, the Call Quality Summary chart shows the percentage of all calls that fell into each call quality category.
This chart provides a first indication that further investigation of the results is required: although 45% of the simulated calls were Good quality and 37% were Acceptable quality, 13% were Poor quality and 5% were Unavailable.
Just below the Call Quality Summary chart, the Factors Affecting Call Quality chart indicates that the greatest impairment factors in the Sample assessment were codec at 82%, delay at 14%, and lost data at 4%. You can analyze these results with the information provided by the Complete report. For more information, see Section 12.6.2, Sample Complete Report.
Scroll down to the Call Quality Summary by Call Group chart. At a glance, you can tell which call groups had the lowest call quality. With a MOS of 3.50, the worst-performing call group was Raleigh_Bldg 1-SanJose:G.723.1-ACELP. The endpoints in this call group are located in the Raleigh and San Jose offices. Two call groups include these: Raleigh_Bldg 1-SanJose: G.723.1-ACELP and Raleigh_Bldg 1-SanJose: G.711u.
Although the Raleigh_Bldg 1-San Jose link experienced much better performance using the G.711u codec, it was still the worst-performing call group using that codec. According to the result ranges defined for this assessment (shown on the previous page of the report), the G.711u codec produced Good results (MOS of 4.18) between these, while the G.723.1-ACELP codec produced Poor results (MOS of 3.50).
The Complete report expands on the information provided by the Executive Summary, allowing you to drill down into results for a more detailed analysis.
To generate the Complete report:
Click the Generate view tab and select Complete Report.
In the Filter Content tree, expand VoIP Quality Assessment and then click to check Call Group Details. Selecting this option provides a breakout of results per call group.
Click Generate Report and then click Yes to save your changes. The report is generated in Microsoft Word and opens in its own window. This process may take several minutes. The Complete report is much longer than the Executive Summary report.
In the report, find Network Inventory in the Table of Contents. Click the corresponding page number to go straight to that section. Discovered routers, switches, and links are identified in tables. The names of devices and links are hyperlinks that take you to an appendix with some very detailed information. Click a hyperlink to see the information that is available for each device and link type.
HINT:It is easier to navigate within the Complete report if you use the Back and Forward arrow buttons in Word. Click Toolbars on the View menu, and then select the Web toolbar.
Click Back to return to the Table of Contents. To see how the routers fared during the Utilization Assessment, find the Router Utilization Details subsection and click on the corresponding page number to see a table.
The router names correspond to their locations. One router in the table might be the best-performing one, while another router might perform the worst, with CPU and output queue drops as the worst measurements. Both measurements suggest that this router is congested.
Return to the Table of Contents and navigate to the Router Average CPU Utilization by Day and Router Average CPU Utilization by Hour sections.
This section of the Complete report provides information about the poorly performing call group. This codec appears to be the major source of the poor-quality calls made by this call group. However, it is best to make sure no other factors were involved before purchasing the phones that will use this codec.
Navigate to Factors Affecting Call Quality by Call Group, located in the Calls by Group section of the VoIP quality assessment to view results. For more information, see Section 7.11.2, Reviewing Codec Types.
To determine if was delay a major problem, navigate to the Delay Evaluation by Call Group chart, which. Notice that 89.66% of delay values recorded for this call group fell into the Acceptable range.
The delay for this call group is not alarming. However, the average delay for this call group hovered around 176 ms, as seen in the Delay Summary by Call Group chart, and most vendors believe that delay of more than 150 ms impairs call quality. Even though delay stayed within the Acceptable range, the delay impairment combined with the G.723.1 codec impairment was sufficient to drive call quality to a Poor level. Because the G.723.1 codecs have a theoretical maximum MOS of only 3.69, even a minor impairment can make their call quality Poor.
The G.723.1 call group may not be the only low performer. In the Factors Affecting Call Quality by Call Group chart, notice that the Raleigh_Bldg 1-San Jose: G.711u call group shows data loss of .18 MOS point.
First, find out whether a network performance problem occurred during the assessment to cause the excessive data loss. Perhaps a problem existed on the link between Raleigh and San Jose. Navigate to the Call Quality Evaluation by Day chart.
Notice that call quality held pretty steady on each day of the assessment except for Monday, May 21, when 19.11% of all MOS scores fell into the Poor range.
The Factors Affecting Call Quality by Call Group chart indicates that lost data was the largest MOS impairment factor for the G.711u call group, so look more closely at the Lost Data charts for this call group. Navigate to the Lost Data Evaluation by Day chart.
The chart indicates that Monday, May 21, experienced the highest amount of data loss: 0.07%. On that day, 4.53% of all lost data values fell into the Acceptable range. For an even more granular view of the data, navigate to the Lost Data Evaluation by Hour chart.
Notice that at its worst, data loss at 2 PM was low enough to be considered Good at 89%. Most VoIP vendors recommend data loss of no more than 0.50% to maintain good call quality, so our loss of .07% (and less for the rest of the week) is not considered significant. Data loss for the G.711u call group is not a major contributor to the overall low MOS.
The network delay results for the Raleigh_Bldg 1-San Jose: G.723.1-ACELP call group raise the question: Are the cross-country links adequate to carry high-quality VoIP in their present state?
The delay on these links was enough, when combined with the low-performing G.723.1-ACELP codec, to push call quality to an Acceptable or Poor level. And, according to the Link Utilization Details section of the Complete report, even the VoIP readiness of all the links was rated only Acceptable after the Utilization assessment.
Even using the high-performing G.711u codec, some calls between Raleigh and San Jose still have either Poor or Acceptable quality. Upgrading the fractional T3 line may work, but is expensive.
However, perhaps adding or changing QoS, which is recommended by most VoIP vendors, will improve call quality. You may want to deploy some call groups that use QoS and others that do not. You can compare the results to see which method provides better call quality. For more information, see Section 7.11.7, Reviewing Quality of Service.