What is sampling mode?

Sampling mode is a method by which the AMD handles traffic volume that exceeds its capabilities. When faced with an overwhelming traffic volume, instead of degrading the quality of measurements of the monitoring software, the AMD artificially limits the volume by not analyzing all client-server communications (all TCP/IP sessions between given pair).

How does sampling mode work?

Sampling mode is activated as soon as the monitoring software detects that the AMD is dropping network packets. Once activated, sampling mode decreases the traffic volume to be analyzed by 5% until the AMD stops dropping packets. The reduction of traffic volume is achieved by ignoring the observed traffic (ignoring whole client-server sessions). This method generally does not affect metrics based on averages, but statistics for volume are necessarily skewed as the amount of traffic analyzed is reduced.

Sampling mode remains active as long as the packet drops occur on the AMD. Persistent packet drops downgrade (5% at a time) the analysis volume until it reaches a minimum of 5% of the total observed volume. This means that if your AMD is continuously dropping packets, it will eventually analyze only 5% of observed traffic and 95% will be analyzed based on sampled data.

Figure 1. Single traffic spike with default AMD sampling setting

While a single traffic spike will not cause persistent AMD sampling, in the extreme case where your analyzed volume drops to 5%, the return to the full volume analysis may take a long time using the default 5% per hour recovery setting.

AMD sampling single spike

 

The AMD remembers the volume of traffic it had to activate for each level of sampling. Once the traffic volume reduces, the AMD will automatically lower the sampling level and eventually analyze all observed traffic again. If no changes in traffic volume and no packet drops occur for a certain time (1 hour by default), the AMD will lower sampling by one step (5%). You can set the time for the volume analysis recovery by modifying the interval at which the sampling mode increases the analysis volume. This setting is located on the AMD in the rtm.config file.

Change the default value of 3600 seconds (which is 1 hour) to the number of seconds you want between sampling mode increases.

 sampling.config.noDropTimePeriod=3600

Note:

We recommend that you do not set this value lower than 900 seconds (which is 15 minutes).
Figure 2. Single traffic spike with custom AMD sampling setting

An example of a single traffic spike and AMD sampling set to 15 minutes. Once the traffic load returns to normal, and the AMD stops dropping packets, AMD sampling decreases while the analyzed traffic increases, both by 5% every 15 minutes until analyzed traffic reaches 100% and AMD sampling becomes inactive.

AMD sampling 15 minutes

When should I disable sampling?

The sampling mode is enabled by default however, it is not active. There is no need to disable AMD sampling in most deployments monitoring and analyzing an acceptable traffic volume. It is beneficial to have sampling mode enabled in the event an unnoticed increase in traffic volume downgrades AMD performance.

You might consider disabling sampling mode if your deployment experiences expected spikes in traffic volume. Such spikes (for example, a regularly scheduled backup tasks) can cause the AMD to briefly drop packets, which will activate sampling mode, which in turn will bring down the analyzed traffic volume down by 5% for each second the AMD is dropping packets. The recovery of analyzed traffic to 100% is, by default, scheduled to increase by 5% every 1 hour. So, for example, a scheduled backup occurring every hour and generating a traffic spike that causes sampling mode to possibly reduce the analyzed volume to 70% can have an ongoing effect because the AMD never returns to full volume analysis before the next scheduled backup.

Note:

Keep in mind that AMD sampling, when active, ignores whole client-server sessions, which has a minor effect on volume-related statistics and an insignificant effect on averages. However, if you disable AMD sampling, any network packets that are dropped are done so in an uncontrolled manner (without regard to sessions). This may have a significant effect on metrics related to errors, volume, and averages.
Figure 3. Scheduled traffic spikes with default AMD sampling setting

If your scheduled traffic spikes occur on a regular basis and sampling mode does not recover in time, your AMD will not operate at its full capability. If your monitoring profile does not require you to analyze the spike traffic, it might be beneficial to allow the AMD to randomly drop packets during such spikes but maintain the proper analysis volume the rest of the time.

Here is an example of scheduled traffic spikes that cause a persistent AMD sampling. Such cases warrant either a custom AMD sampling setting, or disabling AMD sampling.

Scheduled_Spikes

How do I read the AMD performance data on CAS reports?

The AMD performance data is presented on the AMD global packet stats report on CAS. The metrics presented on this report are described in the Wire-level packet distribution section. To examine your AMD sampling to received packets ratio, hide all the metrics but leave the RX packets and Dropped (sampling) visible. The RX packets is the volume of packets received by the driver and Dropped (sampling) is the volume of packets dropped that were sampled. The higher Dropped (sampling) volume, the more your RX packets is sampled. You can review your AMD sampling for a specific date by adjusting the Resolution and Time range.

  • No labels