Purging SQL Monitor data
Published 28 October 2013
SQL Monitor stores a large amount of high, medium and low volume data in the Data Repository. To prevent the Data Repository database using all your hard disk space, SQL Monitor checks for old data every hour, and purges data that is older than the time specified for each category in your purging policy.
Configuring data purging
Go to the Configuration tab. Under Data Management, select Data purging:
Data is stored in three separate categories: Machine data, SQL Server data and Alert data. Each category is broken down further to make it easier to identify whether the data could become high, medium or low volume, and where the data is displayed in SQL Monitor. The default purge setting for each type of data is displayed in brackets:
- Machine data:
- Basic machine data (2 months) - Medium volume data displayed on the Host machine, Cluster, and SQL Server instance overview pages, and as metrics listed under Machine on the Analysis page. By default, this data is purged after 2 months.
- Windows process data (1 month) - Medium volume data displayed as the System processes (top 10) on the Host or Cluster machine overview pages.
- SQL Server data:
- Basic SQL Server data (2 months) - Medium volume data displayed on the SQL Server instance overview pages, and as metrics listed under SQL Server on the Analysis page.
- SQL process data (1 month) - High volume data displayed as SQL user processes (top 10 by CPU usage) on the SQL Server instance overview pages, and as alert details when Trace is turned off.
- Performance diagnostics data (1 month) - High volume data displayed as the Top 10 waits and Top 10 queries on the SQL Server instance and Database overview pages, and the performance data on the Alert details page.
- Database performance metric data (2 months) - High volume data displayed on the Database overview pages, and as metrics listed under Database on the Analysis page.
- Custom metric data (2 months) - Low volume data collected for custom metrics displayed on the Analysis page.
- Alert data:
- Basic alert data (1 month) - Low volume data displayed on the Alert pages of raised alerts and including alert details, history of occurrences and performance data.
- Trace data (3 days) - High volume data displayed under SQL Processes/Profiler trace in the Performance data section of the Alert pages when Trace is turned on.
The default purge settings are designed to keep the data you're most interested in for longer without using up too much disk space. To change these settings, select a different time limit from the drop-down list for each category and click Save settings.
How frequently should I purge data?
Your purging frequency depends on:
- how important it is for you to retain historic information about each category. For example, for auditing purposes, you may need to keep data about your machines for longer than alert data.
- the amount of disk space you have available in your Data Repository to store collected data. If space is an issue, you may consider purging high volume data on a more frequent basis.
There is a Do not purge option for each category, which means data will be stored indefinitely in your Data Repository, but you should ensure that you have enough disk space available to accommodate potentially high volumes of data. For more information, see Growth in size of Data Repository database.
What is the effect of purging data?
- For overview data, a dash (--) or (no data) will be displayed on the overview pages next to a value if you rewind time to a point for which data does not exist because it has been purged.
For alert data, purging will result in older alerts no longer being displayed in the application.
For analysis graphs, a blank area will be displayed on the graph for the duration purged.
Reducing data retention
If you decide to reduce the amount of data being retained, for example reducing Basic Machine Data from 2 months to 2 weeks, then the purge process will require time to process the data. Therefore say that there is 5GB of Basic Machine Data, and a change should reduce that to 3GB, the process needs to identify and delete 2GB of data.