Performance with InMemory vs Distributed output cache
Test results summary
Using distributed cache delivers up to 200 times better startup and warmup performance for a “cold start”, when pages have not yet been compiled, and up to 5 times better startup and warmup when a node is restarted (pages have already been compiled). This results in improved scalability and high-availability, as well as better browsing experience for users who hit a newly started node. In-memory output cache storage certainly excels at providing the best average page response times once the site is warmed up. Distributed output cache still delivers very good average page response times, and this performance can be affected positively (or negatively) based on the distributed cache storage used.
Following is a detailed analysis of the performance improvements comparing in-memory and distributed output cache storages.
Test setup
Testing the Sitefinity CMS performance with distributed cache storage is executed with the following setup:
|
Hosting configuration
|
|
Setup type
|
Sitefinity running on 3 web server nodes in load balanced configuration hosted on AWS.
|
|
Database server
|
c5.xlarge - 4CPU, 8GB RAM
|
|
Web Server node 1
|
c5.xlarge - 4CPU, 8GB RAM
|
|
Web Server node 2
|
c5.xlarge - 4CPU, 8GB RAM
|
|
Web Server node 3
|
c5.xlarge - 4CPU, 8GB RAM
|
|
Redis cache configuration
|
Amazon ElastiCache for Redis cache used for NLB messaging and distributed cache storage.
1 node, cache.t2.medium.
Both Redis node and Sitefinity Web server nodes configured in the same AWS Region.
|
NOTE: When using distributed cache, the performance of your Sitefinity CMS site depends directly on the network latency and distributed cache storage type. For example, running the tests described in this article with ElastiCache for Redis node of type of cache.t2.medium results in 2 times better performance compared to using cache.t2.small. The cache.t2.small and cache.t2.medium are among the most affordable Redis cache storage types and are categorized as Low to Moderate network performance. Using higher-rated distributed cache storage might result in even better performance.
|
Sitefinity CMS configuration
|
|
Domain configuration
|
Website domain set to the load balancer URL Note: this configuration is required for the warmup module normal operation if distributed cache is used
|
|
Pages
|
250 MVC pages
|
|
Content
|
Content block widget on each page
|
|
Page size (in KB)
|
600KB
|
|
Warm-up module configuration
|
maxPagesAfterInitializationPerSite="500"
|
|
Output cache settings:
|
|
|
Cache duration (in seconds)
|
2000
|
|
varyByHost
|
true
|
Test results
The test results list the Sitefinity CMS performance in startup, warmup, and load test to reflect the full scenario of a newly started/restarted site spinning up and warming up, as well as website browsing under load by site visitors.
Startup and warmup
|
Test scenario description
|
Metric
|
In-memory output cache
|
Distributed output cache
|
|
Start any node for the first time (pages have not been compiled yet)
|
Compilation and warmup duration
|
1000 seconds
|
- 1000 seconds for the first node
- 5 seconds for all other nodes (all pages already cached in Redis by the 1st node, no compilation or content processing needed)
|
|
Restart any node
(pages have already been compiled and exist in the ASP.NET Temporary fields)
|
Warmup duration
|
22 seconds
|
5 seconds
|
|
Scale by adding a new node
|
Compilation and warmup duration
|
1000 seconds
|
5 seconds
|
User load
Test setup
Testing the Sitefinity CMS website user load performance with distributed cache is executed with the following setup:
|
Hosting configuration
|
|
Setup type
|
Sitefinity running on 3 web server nodes in load balanced configuration hosted on Microsoft Azure.
|
|
Database server
|
Microsoft Azure SQL (Standard S2: 20-100DTUs)
|
|
Web Server node 1
|
7 GB RAM, 2 CPUs
|
|
Web Server node 2
|
7 GB RAM, 2 CPUs
|
|
Web Server node 3
|
7 GB RAM, 2 CPUs
|
|
Redis cache configuration
|
Microsoft Azure Redis cache used for NLB messaging and distributed cache storage.
1 node, C3; 6144 MB cache
Both Redis node and Sitefinity Web server nodes configured in the same Azure Region.
|
NOTE: When using distributed cache, the performance of your Sitefinity CMS site depends directly on the network latency and distributed cache storage type. Using higher-rated distributed cache storage might result in even better performance.
|
Sitefinity CMS configuration
|
|
Domain configuration
|
Website domain set to the load balancer URL Note: this configuration is required for the warmup module normal operation if distributed cache is used
|
|
Pages and Content
|
The Sitefinity Quantum demo website is used for the test. For more information see Sitefinity Quantum GitHub repository.
|
|
Output cache settings:
|
|
|
Cache duration (in seconds)
|
2000
|
|
varyByHost
|
true
|
The load test has been executed using JMeter v.5.1 load test tool.
|
Load test configuration
|
|
User load
|
20
|
|
Load pattern
|
Constant pattern (user browse the website all the time)
|
|
Test duration
|
10 min
|
The load test is executed against a fully warmed up website (all nodes warmed up and initialized the output cache) to measure the differences in website performance when reading from In-memory vs Distributed cache.
Test results
|
Metric
|
In-memory output cache
|
Distributed output cache
|
|
Average response time
|
0.38 seconds
|
0.56 seconds
|