Wiki source code of Performance test on Jetty/HSQLDB with a single wiki between 6.4.5 and 7.2

Last modified by Thomas Mortagne on 2015/09/25 09:32
Hide line numbers
  1: {{toc/}}
  2: 
  3: = Environment =
  4: 
  5: {{velocity}}
  6: #if ($xcontext.action == 'edit')
  7:   ; Previous version
  8:   : $doc.next
  9:   ; Next version
 10:   : $doc.previous
 11: #end
 12: {{/velocity}}
 13: 
 14: * Ubuntu 15.04 64b
 15: * Quad core i7-3630QM
 16: * Oracle JDK 1.8.0_60 64b
 17: * standard jetty+hsqldb
 18: * permdir in a SSD
 19: * only main wiki
 20: * configuration
 21: ** ##xwiki.cfg##:
 22: *** ##xwiki.store.cache.capacity=500##
 23: ** ##Admin user:##
 24: *** ##hidden documents enabled##
 25: 
 26: Most response time statistics were made with [[Dumbbench>>https://github.com/tsee/dumbbench]], you can install it using cpan.
 27: For example on Debian/Ubuntu:
 28: 
 29: {{code language="bash"}}
 30: $ sudo apt-get install cpanminus
 31: $ sudo cpanm Dumbbench
 32: {{/code}}
 33: 
 34: Unless specifically indicated otherwise all the test are executed with a fully initialized instance (core extensions don't run anymore, SOLR index is fully up to date, etc.) and logged in with Admin user. Basically except for a few special tests we are using an instance that already been started and run until the proc is flat for some time.
 35: 
 36: Login with Admin uer:
 37: 
 38: {{code language="none"}}
 39: wget --save-cookies cookies.txt --post-data 'j_username=Admin&j_password=admin&j_rememberme=true' -O /dev/null -o /dev/null "http://127.0.0.1:8080/xwiki/bin/loginsubmit/XWiki/XWikiLogin?j_username=Admin&j_password=admin&j_rememberme=true"
 40: {{/code}}
 41: 
 42: = Tests =
 43: 
 44: == Summary ==
 45: 
 46: "(% style="color:orange" %)similar(%%)": difference is lower than 10%
 47: "slightly": difference is lower than 20%
 48: 
 49: Note hat most of the speed related values are an average of very moving results, a lot of things is happening during a HTTP request and it's far from stable duration (that's why 10% may sounds a lot for something called "similar" but the variable can go up and down around 5% sometimes so 10% average is really not that much of a clear win).
 50: 
 51: === Speed ===
 52: 
 53: |=(% colspan="2" %)Actions|=Difference
 54: |(% colspan="2" %){{sl}}Jetty startup{{/sl}}|(% style="color:orange" %)similar\\
 55: |(% rowspan="2" width="150" %){{sl}}First access{{/sl}}|(% width="300" %){{sl id="HFirstaccesstonotexistingpagewithoutUI"}}not existing page without UI{{/sl}}|(% style="color:orange" %)similar\\
 56: |{{sl id="HFirstaccesstonotexistingpagewithUI"}}not existing page with UI{{/sl}}|(% style="color:orange" %)similar\\
 57: |(% rowspan="4" %){{sl}}Reload{{/sl}}|{{sl id="HReloadofnotexistingpagewithoutUI"}}not existing page without UI{{/sl}}|(% style="color:orange" %)similar\\
 58: |{{sl id="HReloadofnotexistingpagewithUI"}}not existing page with UI{{/sl}}|(% style="color:green" %)20% faster\\
 59: |{{sl id="HReloadofMain.WebHomewithUI"}}Main.WebHome with UI{{/sl}}|(% style="color:green" %)20% faster\\
 60: |{{sl id="HReloadofMain.WebHomewithoutUI"}}Main.WebHome without UI{{/sl}}|(% style="color:green" %)30% faster\\
 61: |(% rowspan="5" %){{sl}}SOLR{{/sl}}|{{sl}}Full SOLR reindex{{/sl}}|(% style="color:orange" %)similar\\
 62: |{{sl id="Hwhenindexisempty"}}SOLR sync when index is empty{{/sl}}|(% style="color:orange" %)similar\\
 63: |{{sl id="Hwhenthereisnothingtodo"}}SOLR sync when there is nothing to do{{/sl}}|(% style="color:orange" %)similar\\
 64: |{{sl}}Result of search finding lots of results{{/sl}}|(% style="color:green" %)25% faster\\
 65: |{{sl}}Result of search finding one result{{/sl}}|(% style="color:green" %)20% faster\\
 66: |(% rowspan="2" %){{sl}}Rendering{{/sl}}|{{sl}}Page with 1000 macros without UI{{/sl}}|(% style="color:darkgreen" %)/34\\
 67: |{{sl}}Page with 1000 html macros without UI{{/sl}}|(% style="color:darkgreen" %)/6\\
 68: 
 69: === Memory ===
 70: 
 71: |=(% width="450" %)Actions|=Difference
 72: |{{sl}}Heap Memory after jetty startup{{/sl}}|(% style="color:orange" %)similar\\
 73: |{{sl}}Heap Memory after full SOLR index{{/sl}}|(% style="color:red" %)+50MB\\
 74: 
 75: == Jetty startup ==
 76: 
 77: Unit: seconds
 78: 
 79: Execute {{code language="shell"}}echo `date` && ./start_xwiki.sh{{/code}} and compare "Server started..." log date the one printed before jetty startup.
 80: 
 81: * {{previous/}}: 9
 82: * {{next/}}: 10
 83: 
 84: == First access ==
 85: 
 86: First HTTP request after restarting Jetty.
 87: 
 88: === First access to not existing page without UI ===
 89: 
 90: (/xwiki/bin/get/NoSpace/NoPage)
 91: 
 92: Unit: seconds
 93: 
 94: Firefox network statistics to load the root resource (image/js/css/etc. resources are not taken into account).
 95: 
 96: * {{previous/}}: 6
 97: * {{next/}}: 6
 98: 
 99: === First access to not existing page with UI ===
100: 
101: (/xwiki/bin/view/NoSpace/NoPage)
102: 
103: Unit: seconds
104: 
105: Done by hand with a chronometer (image/js/css/etc. resources are not taken into account).
106: 
107: * {{previous/}}: 14
108: * {{next/}}: 14
109: 
110: == Reload ==
111: 
112: XWiki is doing nothing on background and everything is both in server and client cache already. Values are calculated using dumbbench (image/js/css/etc. resources are not taken into account).
113: 
114: === Reload of not existing page without UI ===
115: 
116: {{code language="none"}}
117: dumbbench -i 100 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/get/NoSpace/NoPage"
118: {{/code}}
119: 
120: Unit: milliseconds
121: 
122: * {{previous/}}: 7
123: * {{next/}}: 6
124: 
125: === Reload of not existing page with UI ===
126: 
127: {{code language="none"}}
128: dumbbench -p 0.1 -i 100 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/view/NoSpace/NoPage"
129: {{/code}}
130: 
131: Unit: milliseconds
132: 
133: * {{previous/}}: 76
134: * {{next/}}: 56
135: 
136: === Reload of Main.WebHome with UI ===
137: 
138: {{code language="none"}}
139: dumbbench -p 0.1 -i 100 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/view/Main/WebHome"
140: {{/code}}
141: 
142: Unit: milliseconds
143: 
144: * {{previous/}}: 153
145: * {{next/}}: 115
146: 
147: === Reload of Main.WebHome without UI ===
148: 
149: {{code language="none"}}
150: dumbbench -p 0.1 -i 100 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/get/Main/WebHome"
151: {{/code}}
152: 
153: Unit: milliseconds
154: 
155: * {{previous/}}: 77
156: * {{next/}}: 50
157: 
158: == SOLR ==
159: 
160: === Full SOLR reindex ===
161: 
162: Unit: seconds
163: 
164: Delete the data/solr folder, restart, load home page and calculate the time of activity of the SOLR index thread using Yourkit (Yourkit is slowing down a bit all that but what is important is the difference anyway).
165: 
166: * {{previous/}}: 62
167: * {{next/}}: 68
168: 
169: === SOLR sync ===
170: 
171: Only the thread determining what need to be indexed and not the actual indexing.
172: 
173: ==== when index is empty ====
174: 
175: Unit: seconds
176: 
177: * {{previous/}}: 2
178: * {{next/}}: 2
179: 
180: ==== when there is nothing to do ====
181: 
182: Unit: seconds
183: 
184: * {{previous/}}: < 1
185: * {{next/}}: < 1
186: 
187: === Search ===
188: 
189: XWiki is doing nothing on background and everything is both in server and client cache already. Values are calculated using dumbbench (image/js/css/etc. resources are not taken into account).
190: 
191: Unit: milliseconds
192: 
193: ==== Result of search finding lots of results ====
194: 
195: Login with Admin user and make sure hidden document display is enabled.
196: 
197: {{code language="none"}}
198: dumbbench -p 0.1 -i 100 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/view/Main/Search?sort=score&sortOrder=desc&r=1&f_type=DOCUMENT&text=*"
199: {{/code}}
200: 
201: * {{previous/}}: 315
202: * {{next/}}: 233
203: 
204: ==== Result of search finding one result ====
205: 
206: {{code language="none"}}
207: dumbbench -p 0.1 -i 100 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/view/Main/Search?sort=score&sortOrder=desc&r=1&f_type=DOCUMENT&text=%22Learn+how+to+use+XWiki%22"
208: {{/code}}
209: 
210: * {{previous/}}: 190
211: * {{next/}}: 140
212: 
213: == Rendering ==
214: 
215: === Page with 1000 macros without UI ===
216: 
217: [[1000 ~~{~~{id}} macros>>attach:Test.1000Macros.xar]].
218: 
219: {{code language="none"}}
220: dumbbench -p 0.1 -i 10 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/get/Test/1000Macros"
221: {{/code}}
222: 
223: Unit: milliseconds
224: 
225: * {{previous/}}: 2874
226: * {{next/}}: 83
227: 
228: === Page with 1000 html macros without UI ===
229: 
230: [[1000 standalone ~~{~~{html}} macros>>attach:Test.1000HTMLMacros.xar]].
231: 
232: {{code language="none"}}
233: dumbbench -p 0.1 -i 10 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/get/Test/1000HTMLMacros"
234: {{/code}}
235: 
236: Unit: milliseconds
237: 
238: * {{previous/}}: 5371
239: * {{next/}}: 870
240: 
241: == Heap Memory ==
242: 
243: Attach Yourkit to get the information.
244: 
245: Unit: MB
246: 
247: === Heap Memory after jetty startup ===
248: 
249: Start jetty and wait until java is not doing anything on both (wait for potential initialization of various stuff like core extension cache, etc). Force garbage collector before looking at the values.
250: 
251: * {{previous/}}: 51 used, 365 allocated
252: * {{next/}}: 45 used, 375 allocated
253: 
254: === Heap Memory after full SOLR index ===
255: 
256: Delete the data/solr folder, restart, load home page then wait until nothing is happening in background. Force garbage collector before looking at the values.
257: 
258: Unit: MB
259: 
260: * {{previous/}}: 97 used, 503 allocated
261: * {{next/}}: 145 used, 505 allocated
Wiki source code of Performance test on Jetty/HSQLDB with a single wiki between 6.4.5 and 7.2

About

About

Support

Platform

User Guide

Admin Guide

Developer Guide

Projects

XWiki

Extensions

Other

Contribute

Status

Practices

Under the Hood

Get Involved

Get Connected
