MSDN Blogs: SharePoint 2010 Web Analytics for page requests is less than expected

Background:

I had the pleasure of working with a customer dealing with SharePoint 2010 web analytics where the page requests weren't showing the expected values. We verified that the usage logs were in fact logging usage and we could see requests for our web application in question. So we knew that logging was configured correctly.  We however still didn't see the expected results in the web analytic reports.

Cause:

The Cause of this could really be a lot of different things. After Validating that the usage files were being imported and they did in fact contain usage entries for our web application in question I turned to the ULS logs.

A common problem with the Import is that the Usage database could be "full", which means that all additional data is just thrown out.  You would see this in the ULS logs at the time of the import:

"11/11/2015 07:00:02.18 OWSTIMER.EXE (0x276C) 0x4768 SharePoint Foundation Health i0m6 High Table RequestUsage_Partition20 has 443006976 bytes that has exceeded the max bytes 443006976"

Note :  The ULS logs would indicate the partition ID & usage Definition  affected.

In above example ...

RequestUsage_Partition20

Partition ID =20

Usage Definition : Request Usage  ( Page Usage) 

These type of errors show for any type of Usage Definition category.

The Total Size of Usage Partitions can be confirmed by a Query to the Configuration Table of Usage Application DB

Select * from Configuration (nolock)

The property name is "Max Total Bytes-Usage Definition Name" , for eg. "Max Total Bytes-RequestUsage" or "Max Total Bytes-FeatureUsage" etc

The value for this property is Total bytes available across all days in the retention period ( default 14 days , can be changed by Set-SpUsageDefinition).

Resolution:

My preferred resolution:

• is to simply lower the retention period for the page requests.
• https://technet.microsoft.com/en-us/library/ff607542.aspx

Acceptable resolution:

• Increase the partition size using OM code , followed by creating a new Database
• The property MaxTotalSizeInBytes is exposed by OM code for the Usage definition and can be  updated  with  below code snippet. Here is an example of updating the "Page Requests" Usage Definition:

$def=Get-SPUsageDefinition  -Identity "page requests"

$def.MaxTotalSizeInBytes=12400000000

$def.update()

• Create a new Logging database and attach to the Usage Service Application using the following command:

Get-SPUsageapplication |Set-SPUsageApplication -DatabaseServer <> -DatabaseName <>

• Note : Since the Usage Service Application can have only one usage database creating a new Database will cause loss of existing data being retained in the existing Logging database.  The OM code will set the new Size in the Configuration object  "Microsoft.SharePoint.Administration.SPRequestUsageDefinition" but this will not get reflected on the Usage Database until we create a new database.

Scripts:

Select * from Configuration (nolock)

against the usage database will tell us everything we need to know about this subject

SELECT dbo.fn_GetPartitionSize('RequestUsage_Partition20')

will tell us the current size of the partition

Select max(LogTIme) from  RequestUsage_Partition20 (nolock)

will tell us how long we enter usage information into the database

More Information:

http://blogs.msdn.com/b/sharepoint_strategery/archive/2012/04/16/usage-file-and-web-analytics-reports-with-blind-spots.aspx