Using Retention Policies - Part 1

 

Learn how to create retention policies to control how much of your data is stored, and for how long.

Transcript

Use retention policies to manage collection and storage of your Intermapper server data.

Intermapper stores collected data locally for use in charts. It can also export to a database for long-term storage and reporting. In larger installations, you might even have multiple Intermapper servers exporting to one central database. Even a modest network monitoring 150 devices and with no charts or special probes can generate as much as 100 megabytes of data per day.

Use retention policies to specify how long you want to retain data both locally and in the database at what resolution. You may want to store certain data longer than other data. You can create retention policies that specify how long you want to retain data in each category. Original data is the entire set of raw data values collected by the Intermapper server.

Five minute samples are statistics computed over five minute sets of raw data. They include the minimum, maximum, mean, and standard deviation across all the values in each five minute period. These are computed as the data comes in so you can see up-to-date statistics at any time. Hourly and daily samples are the same statistics but computed over each hour and each day's worth of data for each data set. The categories I've just mentioned apply to data that's been exported to the database. Chart data is the local storage used to present Intermapper strip charts and as a staging area for the database.

Six retention policies are available by default. Three of these are not editable. The 24 Hour policy keeps local data forever on the server and raw data, 5-minute, hourly, and daily samples for one day in the database. The Forever policy keeps all data forever, both locally and in the database. When you choose the None policy, no data is collected, stored locally, or exported to the database. The 24 Hour policy is selected by default.

Three additional, editable retention policies are available. The Autorecord policy is automatically assigned to certain important data sets. By default, it acts exactly like the 24 Hour policy but can be edited to keep data for longer periods. The Chart Only policy stores data only locally and does not export to the database. Edit the IM46Charts policy to specify retention for charts created in versions prior to 5.0. By default, this earlier data is saved forever so until you edit the policy, that data is retained.

You probably want to create, at least, one retention policy that retains data for longer than 24 hours. You create and edit retention policies from the server settings Windows retention policies pane. Start by creating a retention policy you want to apply to all maps by default. If you have certain devices or maps for which you want to retain more or less data, create additional policies.

I'm going to create three policies. By default, I want original data retained for 24 hours, 5-minute data for two weeks, hourly data for six months, and daily data for five years. This allows me to create relatively detailed reports for up to six months and reports with daily information for up to five years. Remember, daily samples for a data set take only 365 records per year.

I'll also create a policy for use with my map of critical servers. I'll make this one retain more data for a shorter period, original data for 24 hours, 5-minute data for two months, hourly data for one year, and daily data forever. This allows me to create detailed weekly reports going back two months and to be able to see general trends at any point in the past.

And finally, I'll create a policy to use with the SLA Probe on my main Cisco switch to monitor ping and packet loss. Original data for 24 hours, 5-minute data for two months, hourly data for six months, and daily data for one year. This allows me to review detailed spikes in ping or packet loss over a two month period and to spot monthly trends.

You can set a default retention policy at the server level, and you can override this policy at the map, device, and interface level and even assign policies to individual data sets. This is described in part two of this video.