• Environment
• Tests
  • Summary
    • Speed
    • Memory
  • Jetty startup
  • First access
    • First access to not existing page without UI
    • First access to not existing page with UI
  • Reload
    • Reload of not existing page without UI
    • Reload of not existing page with UI
    • Reload of empty page without UI
    • Reload of empty page with UI
    • Reload of Main.WebHome without UI
    • Reload of Main.WebHome with UI
  • SOLR
    • Full SOLR reindex
    • SOLR sync
      • when index is empty
      • when there is nothing to do
    • Search
      • Result of search finding lots of results
      • Result of search finding one result
  • Rendering
    • Page with 1000 macros without UI
    • Page with 1000 html macros without UI
  • Wiki creation
    • From flavor
    • From template
  • Heap Memory
    • Heap Memory after jetty startup
    • Heap Memory after full SOLR index

Environment

• Windows 11 64bit version 24H2
• Quad core i7-9750H
• Oracle JDK 21.0.5 (build 21.0.5+9-LTS-239)
• Dumbbench 0.505
• VisualVM 2.1.10
• permdir in a SSD
• standard Jetty/HSQLDB package
• only main wiki
• configuration:
  • Admin user:
    • hidden documents enabled
• import perf.xar

Most of response time statistics are made with Dumbbench, you can install it using cpan.
For example on Debian/Ubuntu:

$ sudo apt-get install cpanminus
$ sudo cpanm Module::Install
$ sudo cpanm Dumbbench

On Windows 11:

Unless specifically indicated otherwise all the tests are executed with a fully initialized instance (core extensions don't run anymore, SOLR index is fully up to date, etc.) and logged in with Admin user. Basically except for a few special tests we are using an instance that already been started and run until the proc is flat for some time.

Login with Admin user (using port 1115):

.\wget.exe --save-cookies cookies.txt --post-data 'j_username=Admin&j_password=admin&j_rememberme=true' "http://127.0.0.1:1115/xwiki/bin/loginsubmit/XWiki/XWikiLogin?j_username=Admin&j_password=admin&j_rememberme=true"

Tests

Summary

"similar": difference is lower than 10%
"slightly": difference is lower than 20%

Speed

Memory

Jetty startup

Unit: milliseconds

This information is obtained from the log with the line looking like 2022-06-15 14:24:27.163:INFO :oejs.Server:main: Started Server@503d687a{STARTING}[10.0.7,sto=5000] @23806ms.

• 15.10.16: 24861
• 16.10.3: 28234

Results of tests for 16.10.3:

Test 1: 28916
Test 2: 32866
Test 3: 33223
Test 4: 28437
Test 5: 28234

First access

First HTTP request after restarting Jetty.

First access to not existing page without UI

(/xwiki/bin/get/NoSpace/NoPage)

Unit: milliseconds

Firefox network statistics to load the root resource (image/js/css/etc. resources are not taken into account).

• 15.10.16: 36046
• 16.10.3: 35129

First access to not existing page with UI

(/xwiki/bin/view/NoSpace/NoPage)

Unit: seconds

Done by hand with a chronometer (image/js/css/etc. resources are not taken into account).

• 15.10.16: 42
• 16.10.3: 43

Reload

XWiki is doing nothing on background and everything is both in server and client cache already. Values are calculated using dumbbench (image/js/css/etc. resources are not taken into account).

Reload of not existing page without UI

dumbbench --float -p 0.1 -i 100 -- wget -q -o NUL -O NUL --load-cookies cookies.txt "http://127.0.0.1:1115/xwiki/bin/get/NoSpace/NoPage"

Unit: milliseconds

• 15.10.16: 356
• 16.10.3: 777

Results of tests for 16.10.3:

Test 1: 0.783600 (784 ms)
Test 2: 0.795100 (795 ms)
Test 3: 0.784100 (784 ms)
Test 4: 0.777300 (777 ms)
Test 5: 0.783520 (784 ms)

Reload of not existing page with UI

dumbbench --float -p 0.1 -i 100 -- wget -q -o NUL -O NUL --load-cookies cookies.txt "http://127.0.0.1:1115/xwiki/bin/view/NoSpace/NoPage"

Unit: milliseconds

• 15.10.16: 367
• 16.10.3: 901

Results of tests for 16.10.3:

Test 1: 0.900600 (901 ms)
Test 2: 0.908400 (908 ms)
Test 3: 0.905200 (905 ms)
Test 4: 0.903800 (904 ms)
Test 5: 0.902700 (903 ms)

Reload of empty page without UI

dumbbench --float -p 0.1 -i 100 -- wget -q -o NUL -O NUL --load-cookies cookies.txt "http://127.0.0.1:1115/xwiki/bin/get/EmptySpace/EmptyPage"

Unit: milliseconds

• 15.10.16: 37
• 16.10.3: 115

Results of tests for 16.10.3:

Test 1: 0.119110 (119 ms)
Test 2: 0.116600 (117 ms)
Test 3: 0.115060 (115 ms)
Test 4: 0.114660 (115 ms)
Test 5: 0.114830 (115 ms)

Reload of empty page with UI

dumbbench --float -p 0.1 -i 100 -- wget -q -o NUL -O NUL --load-cookies cookies.txt "http://127.0.0.1:1115/xwiki/bin/view/EmptySpace/EmptyPage"

Unit: milliseconds

• 15.10.16: 507
• 16.10.3: 1309

Results of tests for 16.10.3:

Test 1: 1.331400 (1331 ms)
Test 2: 1.317500 (1317 ms)
Test 3: 1.329300 (1329 ms)
Test 4: 1.309100 (1309 ms)
Test 5: 1.331000 (1331 ms)

Reload of Main.WebHome without UI

dumbbench --float -p 0.1 -i 100 -- wget -q -o NUL -O NUL --load-cookies cookies.txt "http://127.0.0.1:1115/xwiki/bin/get/Main/WebHome"

Unit: milliseconds

• 15.10.16: 47
• 16.10.3: 122

Results of tests for 15.10.4:

Test 1: 0.122470 (122 ms)
Test 2: 0.121760 (122 ms)
Test 3: 0.121980 (122 ms)
Test 4: 0.122330 (122 ms)
Test 5: 0.124250 (124 ms)

Reload of Main.WebHome with UI

dumbbench --float -p 0.1 -i 100 -- wget -q -o NUL -O NUL --load-cookies cookies.txt "http://127.0.0.1:1115/xwiki/bin/view/Main/WebHome"

Unit: milliseconds

• 15.10.16: 525
• 16.10.3: 1323

Results of tests for 16.10.3:

Test 1: 1.322700 (1323 ms)
Test 2: 1.346700 (1347 ms)
Test 3: 1.339800 (1340 ms)
Test 4: 1.337300 (1337 ms)
Test 5: 1.343200 (1343 ms)

SOLR

Full SOLR reindex

Unit: seconds

Delete the data/solr folder, restart, load home page and calculate the time of activity of the SOLR index thread using VisualVM (XWiki Solr index thread from Threads Tab > Running column, after it's parked).

• 15.10.16: 250
• 16.10.3: 242

SOLR sync

Only the thread determining what need to be indexed and not the actual indexing.

<perm dir>/data/jobs/status/solr/indexer/status.xml

when index is empty

Unit: nanoseconds

• 15.10.16: 1786012000 (1.79 s)
• 16.10.3: 2780173900 (2.78 s)

Results of tests for 16.10.3:

Test 1: 2881255000
Test 2: 2921206800
Test 3: 2780173900
Test 4: 2805631700
Test 5: 3210457300

when there is nothing to do

Unit: nanoseconds

• 15.10.16: 766288300 (766 ms)
• 16.10.3: 1041906900 (1041 ms)

Results of tests for 16.10.3:

Test 1: 1153749100
Test 2: 1041906900
Test 3: 1108128400
Test 4: 1199181400
Test 5: 1056905300

Search

XWiki is doing nothing on background and everything is both in server and client cache already. Values are calculated using dumbbench (image/js/css/etc. resources are not taken into account).

Unit: milliseconds

Result of search finding lots of results

Login with Admin user and make sure hidden document display is enabled.

dumbbench --float -p 0.1 -i 100 -- wget -q -o NUL -O NUL --load-cookies cookies.txt """http://127.0.0.1:1115/xwiki/bin/view/Main/Search?sort=score&sortOrder=desc&r=1&f_type=DOCUMENT&text=*"""

• 15.10.16: 802
• 16.10.3: 1518

Results of tests for 16.10.3:

Test 1: 1.565000 (1565 ms)
Test 2: 1.541500 (1542 ms)
Test 3: 1.539900 (1540 ms)
Test 4: 1.517800 (1518 ms)
Test 5: 1.536440 (1536 ms)

Result of search finding one result

dumbbench --float -p 0.1 -i 100 -- wget -q -o NUL -O NUL --load-cookies cookies.txt """http://127.0.0.1:1115/xwiki/bin/view/Main/Search?sort=score&sortOrder=desc&r=1&f_type=DOCUMENT&text=name%3A1000Macros"""

• 15.10.16: 619
• 16.10.3: 1361

Results of tests for 16.10.3:

Test 1: 1.356200 (1356 ms)
Test 2: 1.365800 (1366 ms)
Test 3: 1.369400 (1369 ms)
Test 4: 1.360700 (1361 ms)
Test 5: 1.368500 (1369 ms)

Rendering

Page with 1000 macros without UI

dumbbench --float -p 0.1 -i 100 -- wget -q -o NUL -O NUL --load-cookies cookies.txt "http://127.0.0.1:1115/xwiki/bin/get/Test/1000Macros"

Unit: milliseconds

• 15.10.16: 131
• 16.10.3: 221

Results of tests for 16.10.3:

Test 1: 0.222130 = (222 ms)
Test 2: 0.221270 = (221 ms)
Test 3: 0.221460 = (221 ms)
Test 4: 0.221540 = (221 ms)
Test 5: 0.222830 = (223 ms)

Page with 1000 html macros without UI

dumbbench --float -p 0.1 -i 100 -- wget -q -o NUL -O NUL --load-cookies cookies.txt "http://127.0.0.1:1115/xwiki/bin/get/Test/1000HTMLMacros"

Unit: milliseconds

• 15.10.16: 527
• 16.10.3: 644

Results of tests for 16.10.3:

Test 1: 0.644000 (644 ms)
Test 2: 0.651900 (652 ms)
Test 3: 0.649800 (650 ms)
Test 4: 0.660300 (660 ms)
Test 5: 0.669300 (669 ms)

Wiki creation

Unit: nanoseconds

From flavor

Create the wiki test1 from default flavor and get the time spent from the job (</rootStep> in <permdir>/jobs/status/wikicreation/createandinstall/test1/status.xml).

• 15.10.16: 57944759200 (57 s)
• 16.10.3: 73771563100 (73 s)

Results of tests for 16.10.3:

Test 1: 78100649200 (78 s)
Test 2: 73771563100 (73 s)
Test 3: 73845821700 (74 s)
Test 4: 74987512300 (75 s)
Test 5: 75246548700 (75 s)

From template

Make test1 a template wiki.

Create the wiki test2 from wiki test1 and get the time spend from the job (</rootStep> in <permdir>/jobs/status/wikicreation/createandinstall/test2/status.xml).

• 15.10.16: 56911236000 (56 s)
• 16.10.3: 57348456400 (57 s)

Heap Memory

Attach VisualVM to get the information.

Unit: MB

Heap Memory after jetty startup

Start jetty and wait until Java is not doing anything on both (wait for potential initialization of various stuff like core extension cache, etc). Force garbage collector before looking at "used".

• 15.10.16: 106 used, 180 allocated
• 16.10.3: 107 used, 269 allocated

Heap Memory after full SOLR index

Delete the data/solr folder, restart, load home page then wait until nothing is happening in background. Force garbage collector before looking at "used".

Unit: MB

• 15.10.16: 278 used, 947 allocated
• 16.10.3: 321 used, 1053 allocated