Filter tracked data

By default, the Sitefinity Insight connector tracks all interactions regardless of whether the visitor who completed the interaction is an internal employee or a customer or prospect who browsed your site. In some cases, such a mix of internal and external data may jeopardize the accuracy of your statistics, scoring models, and reports. 

RECOMMENDATION: We recommend that you consider carefully what kind of data you want included in your actual statistics and reports. Basically, consider who else, apart from your customers, will browse your site and whether this may result in contaminating your data.
We recommend that you set up your traffic filters as early as possible as historical data cannot be retroactively removed from your statistics.

Following are some examples of other parties, apart from your customers, that may affect collected data and statistics:

• Content editors on the site
  Since usually content editors check the results of their work in the site's Preview mode, this interaction is reported and becomes part of the page visits data on the site.
• QAs testing the website after deployment
• Developers testing the site
• Employees of the company
  It is often the case that employees need to use the site in their daily work, for example the Support team reading the documentation. These interactions are also reported and become part of the statistics.
• Test automation
  It is common practice to automate heartbeat test on websites for crucial business processes, such as purchase, user registration, payment methods, and so on. Thus, you are aware of potential threats and errors early on. Such interactions are reported to Insight as well.

  EXAMPLE: You are measuring the number of new visitors on your site. You set automation tests to check your site availability each 30 minutes. Consequently, the resulting statistics include the automation test count if you do not explicitly exclude the testing results from the final statistics.

• Use of third-party systems for monitoring like Pingdom and PagerDuty
  Such systems perform requests from various locations against your site on predefined schedule. The IP addresses of the systems' servers usually are published, so you can easily exclude them from your datasets.
• Web crawlers
  If your company uses crawlers to warn them in cases there are potential problems with their sites being rated well, for example broken links.
• Search Engine Indexers
  Sitefinity Insight automatically filters out interactions coming from the most popular search engines - Google, Bing, and Yahoo.

PREREQUISITES: To configure IP filtering, you need to be either a Sitefinity Insight account admin or a data center admin. For more information, see Manage users in data centers and accounts.

To filter data and define which visitors to exclude from tracking and statistics, you filter the visitors based on IP address or IP range, so that you do not spam your live data with test accounts and interactions. This includes interactions from the parties, described in the previous section, as well as network segment of public IP address. In addition, by default, Insight excludes junk traffic, caused by bots and spiders, such as:

• Google bot
• Baidu spider
• Facebook external hits
• Bing bot
• Slurp bot
• Siteimprove bot
• Jigsaw_W3C_CSS_Validator
• All other user agents, containing 'crawler' in their names
• All other user agents, containing 'spider' in their names
• All user agents containing 'bot' in their names

To exclude visitors by IP address, you need the static IP address range (or specific IP address) for the particular network or organization that you to filter out from your data results. Next:

1. Open Sitefinity Insight
2.  Navigate to the Administration dashboard
3. Click the data center that you want to configure.
4. Click Traffic filters.
5. Click Create a filter.
6. Name the filter, for example, Internal employees
7. In the IP addresses field, enter one of the following:
   • A single IP address.
   • The start and end of the IP address range.
     In case you need to filter out the inbound traffic, caused by your employees internally, configure all of your public IP addresses.
     You may enter multiple IP addresses. In this case, enter them one per line.
     

     NOTE: In general, companies have multiple ISPs in case one of them fail. If so, configure all ISPs and filter out their IP address ranges.

8. Click Save.

On the Traffic filters page, you can see a list of all filters you defined and can delete any filter you no longer use. Once you delete a filter, Insight starts tracking the respective IP or range of IPs again.

NOTE: Keep in mind that the statistics for the respective IP(s) will not include the window, during which the filter was active.