Performance test on Jetty/HSQLDB with a single wiki between 8.4.6 and 11.8
Environment
- Ubuntu 19.04 64b
- Quad core i7-8750H
- Oracle JDK 1.8.0_222 64b
- Dumbbench 0.111
- Yourkit 2018.04-b87
- standard jetty+hsqldb
- permdir in a SSD
- jetty/hsqldb package
- only main wiki
- configuration
- Admin user:
- hidden documents enabled
- Admin user:
- import
Most of response time statistics are made with Dumbbench, you can install it using cpan.
For example on Debian/Ubuntu:
$ sudo cpanm Module::Install
$ sudo cpanm Dumbbench
Unless specifically indicated otherwise all the tests are executed with a fully initialized instance (core extensions don't run anymore, SOLR index is fully up to date, etc.) and logged in with Admin user. Basically except for a few special tests we are using an instance that already been started and run until the proc is flat for some time.
Login with Admin user:
Tests
Summary
The skin is back on track compared to 8.4.6 mostly thanks to asynchonous elements (which means it's still slower but it now feels similar). 11.8 contains more documents (70 more) than 8.4.6 which make most tasks in charge of loading all documents take longer, it also contains more wiki components which take slighly more time to save (because they are registered at save time). While the memory is more optimized (explaining the lower memory used before initializing the DB) we simply store more stuff in it (mostly more cached wiki components and also a bit of asynchronous elements).
"similar": difference is lower than 10%
"slightly": difference is lower than 20%
Speed
Memory
Actions | Difference |
---|---|
Heap Memory after jetty startup | -49 |
Heap Memory after full SOLR index | +53 |
Jetty startup
Unit: milliseconds
You get this from the log with the line looking like 2016-10-19 16:52:25.309:INFO:oejs.Server:main: Started @8434ms.
- 8.4.6: 6646
- 11.8: 9631
First access
First HTTP request after restarting Jetty.
First access to not existing page without UI
(/xwiki/bin/get/NoSpace/NoPage)
Unit: milliseconds
Firefox network statistics to load the root resource (image/js/css/etc. resources are not taken into account).
- 8.4.6: 4463
- 11.8: 5838
First access to not existing page with UI
(/xwiki/bin/view/NoSpace/NoPage)
Unit: seconds
Done by hand with a chronometer (image/js/css/etc. resources are not taken into account).
- 8.4.6: 9
- 11.8: 10
Reload
XWiki is doing nothing on background and everything is both in server and client cache already. Values are calculated using dumbbench (image/js/css/etc. resources are not taken into account).
Reload of not existing page without UI
Unit: milliseconds
- 8.4.6: 3.8
- 11.8: 4
Reload of not existing page with UI
Unit: milliseconds
- 8.4.6: 67.5
- 11.8: 64.99
Reload of empty page without UI
Unit: milliseconds
- 8.4.6: 3.23
- 11.8: 3.7
Reload of empty page with UI
Unit: milliseconds
- 8.4.6: 64.98
- 11.8: 70.4
Reload of Main.WebHome without UI
Unit: milliseconds
- 8.4.6: 3.55
- 11.8: 6.4
Reload of Main.WebHome with UI
Unit: milliseconds
- 8.4.6: 65.1
- 11.8: 72.2
SOLR
Full SOLR reindex
Unit: seconds
Delete the data/solr folder, restart, load home page and calculate the time of activity of the SOLR index thread using Yourkit (Yourkit is slowing down a bit all that but what is important is the difference anyway).
- 8.4.6: 76
- 11.8: 102
SOLR sync
Only the thread determining what need to be indexed and not the actual indexing.
<perm dir>/data/jobs/status/solr/indexer/status.xml
when index is empty
Unit: nanoseconds
- 8.4.6: 900920830
- 11.8: 1931773802
when there is nothing to do
Unit: nanoseconds
- 8.4.6: 407215988
- 11.8: 553533001
Search
XWiki is doing nothing on background and everything is both in server and client cache already. Values are calculated using dumbbench (image/js/css/etc. resources are not taken into account).
Unit: milliseconds
Result of search finding lots of results
Login with Admin user and make sure hidden document display is enabled.
- 8.4.6: 134
- 11.8: 148
Result of search finding one result
- 8.4.6: 93.2
- 11.8: 102
Rendering
Page with 1000 macros without UI
Unit: milliseconds
- 8.4.6: 61
- 11.8: 57
Page with 1000 html macros without UI
Unit: milliseconds
- 8.4.6: 602.4
- 11.8: 56.4
Wiki creation
Unit: nanoseconds
From flavor
Create the wiki test1 from default flavor and get the time spend from the job (</rootStep> in <permdir>/jobs/status/wikicreation/createandinstall/test1/status.xml).
- 8.4.6: 21127813415
- 11.8: 16015857760
From template
Make test1 a template wiki.
Create the wiki test2 from wiki test1 and get the time spend from the job (</rootStep> in <permdir>/jobs/status/wikicreation/createandinstall/test2/status.xml).
- 8.4.6: 18780437402
- 11.8: 31799311910
Heap Memory
Attach Yourkit to get the information.
Unit: MB
Heap Memory after jetty startup
Start jetty and wait until Java is not doing anything on both (wait for potential initialization of various stuff like core extension cache, etc). Force garbage collector before looking at "used".
- 8.4.6: 121 used, 908 allocated
- 11.8: 72 used, 608 allocated
Heap Memory after full SOLR index
Delete the data/solr folder, restart, load home page then wait until nothing is happening in background. Force garbage collector before looking at "used".
Unit: MB
- 8.4.6: 196 used, 970 allocated
- 11.8: 249 used, 971 allocated