Wiki source code of Performance test on Jetty/HSQLDB with a single wiki between 8.4.4 and 9.2
Version 223.20 by Thomas Mortagne on 2017/04/05 09:59
Show last authors
author | version | line-number | content |
---|---|---|---|
1 | {{toc/}} | ||
2 | |||
3 | = Environment = | ||
4 | |||
5 | {{velocity}} | ||
6 | #if ($xcontext.action == 'edit') | ||
7 | ; Next version | ||
8 | : $doc.next | ||
9 | ; Previous version | ||
10 | : $doc.previous | ||
11 | #end | ||
12 | {{/velocity}} | ||
13 | |||
14 | * Ubuntu 16.10 64b | ||
15 | * Quad core i7-6700HQ | ||
16 | * Oracle JDK 1.8.0_121 64b | ||
17 | * Dumbbench 0.10 | ||
18 | * standard jetty+hsqldb | ||
19 | * permdir in a SSD | ||
20 | * only main wiki | ||
21 | * configuration | ||
22 | ** ##Admin user:## | ||
23 | *** ##hidden documents enabled## | ||
24 | * import attach:perf.xar | ||
25 | |||
26 | Most of response time statistics are made with [[Dumbbench>>https://github.com/tsee/dumbbench]], you can install it using cpan. | ||
27 | For example on Debian/Ubuntu: | ||
28 | |||
29 | {{code language="bash"}} | ||
30 | $ sudo apt-get install cpanminus | ||
31 | $ sudo cpanm Dumbbench | ||
32 | {{/code}} | ||
33 | |||
34 | Unless specifically indicated otherwise all the tests are executed with a fully initialized instance (core extensions don't run anymore, SOLR index is fully up to date, etc.) and logged in with Admin user. Basically except for a few special tests we are using an instance that already been started and run until the proc is flat for some time. | ||
35 | |||
36 | Login with Admin user: | ||
37 | |||
38 | {{code language="none"}} | ||
39 | wget --save-cookies cookies.txt --post-data 'j_username=Admin&j_password=admin&j_rememberme=true' -O /dev/null -o /dev/null "http://127.0.0.1:8080/xwiki/bin/loginsubmit/XWiki/XWikiLogin?j_username=Admin&j_password=admin&j_rememberme=true" | ||
40 | {{/code}} | ||
41 | |||
42 | = Tests = | ||
43 | |||
44 | == Summary == | ||
45 | |||
46 | There hasn't been much work on performance in 9.0 so it's very similar to 8.4.4. The speed up in the Jetty startup is because Let's Encryp extension has been removed in 9.0. | ||
47 | |||
48 | "(% style="color:orange" %)similar(%%)": difference is lower than 10% | ||
49 | "slightly": difference is lower than 20% | ||
50 | |||
51 | {{info}} | ||
52 | Note that most of the speed related values are an average of very moving results, a lot of things is happening during a HTTP request and it's far from stable duration (that's why 10% may sounds a lot for something called "similar" but the variable can go up and down around 5% sometimes so 10% average is really not that much of a clear win). Dumbbench based tests are executed several times and the lowest result is selected. | ||
53 | {{/info}} | ||
54 | |||
55 | === Speed === | ||
56 | |||
57 | |=(% colspan="2" %)Actions|=Difference | ||
58 | |(% colspan="2" %){{sl}}Jetty startup{{/sl}}|(% style="color:orange" %)similar | ||
59 | |(% rowspan="2" width="150" %){{sl}}First access{{/sl}}|(% width="300" %){{sl id="HFirstaccesstonotexistingpagewithoutUI"}}not existing page without UI{{/sl}}|(% style="color:orange" %)similar | ||
60 | |{{sl id="HFirstaccesstonotexistingpagewithUI"}}not existing page with UI{{/sl}}|(% style="color:orange" %) | ||
61 | |(% rowspan="6" %){{sl}}Reload{{/sl}}|{{sl id="HReloadofnotexistingpagewithoutUI"}}not existing page without UI{{/sl}}|(% style="color:orange" %)similar | ||
62 | |{{sl id="HReloadofnotexistingpagewithUI"}}not existing page with UI{{/sl}}|(% style="color:orange" %)similar | ||
63 | |{{sl id="HReloadofemptypagewithoutUI"}}empty page without UI{{/sl}}|(% style="color:orange" %)similar | ||
64 | |{{sl id="HReloadofemptypagewithUI"}}empty page with UI{{/sl}}|(% style="color:orange" %)similar | ||
65 | |{{sl id="HReloadofMain.WebHomewithoutUI"}}Main.WebHome without UI{{/sl}}|(% style="color:orange" %)similar | ||
66 | |{{sl id="HReloadofMain.WebHomewithUI"}}Main.WebHome with UI{{/sl}}|(% style="color:orange" %)similar | ||
67 | |(% rowspan="5" %){{sl}}SOLR{{/sl}}|{{sl}}Full SOLR reindex{{/sl}}|(% style="color:orange" %)similar | ||
68 | |{{sl id="Hwhenindexisempty"}}SOLR sync when index is empty{{/sl}}|(% style="color:red" %)+70% | ||
69 | |{{sl id="Hwhenthereisnothingtodo"}}SOLR sync when there is nothing to do{{/sl}}|(% style="color:red" %)+70% | ||
70 | |{{sl}}Result of search finding lots of results{{/sl}}|(% style="color:orange" %)similar | ||
71 | |{{sl}}Result of search finding one result{{/sl}}|(% style="color:orange" %)similar | ||
72 | |(% rowspan="2" %){{sl}}Rendering{{/sl}}|{{sl}}Page with 1000 macros without UI{{/sl}}|(% style="color:orange" %)similar | ||
73 | |{{sl}}Page with 1000 html macros without UI{{/sl}}|(% style="color:orange" %)similar | ||
74 | |||
75 | === Memory === | ||
76 | |||
77 | |=(% width="450" %)Actions|=Difference | ||
78 | |{{sl}}Heap Memory after jetty startup{{/sl}}|(% style="color:green" %)-3MB | ||
79 | |{{sl}}Heap Memory after full SOLR index{{/sl}}|(% style="color:orange" %)similar | ||
80 | |||
81 | == Jetty startup == | ||
82 | |||
83 | Unit: seconds | ||
84 | |||
85 | You get this from the log with the line looking like {{code language="none"}}2016-10-19 16:52:25.309:INFO:oejs.Server:main: Started @8434ms{{/code}}. | ||
86 | |||
87 | * {{previous/}}: 10 | ||
88 | * {{next/}}: 10 | ||
89 | |||
90 | == First access == | ||
91 | |||
92 | First HTTP request after restarting Jetty. | ||
93 | |||
94 | === First access to not existing page without UI === | ||
95 | |||
96 | (/xwiki/bin/get/NoSpace/NoPage) | ||
97 | |||
98 | Unit: seconds | ||
99 | |||
100 | Firefox network statistics to load the root resource (image/js/css/etc. resources are not taken into account). | ||
101 | |||
102 | * {{previous/}}: 5.5 | ||
103 | * {{next/}}: 5.7 | ||
104 | |||
105 | === First access to not existing page with UI === | ||
106 | |||
107 | (/xwiki/bin/view/NoSpace/NoPage) | ||
108 | |||
109 | Unit: seconds | ||
110 | |||
111 | Done by hand with a chronometer (image/js/css/etc. resources are not taken into account). | ||
112 | |||
113 | * {{previous/}}: 10 | ||
114 | * {{next/}}: 10 | ||
115 | |||
116 | == Reload == | ||
117 | |||
118 | XWiki is doing nothing on background and everything is both in server and client cache already. Values are calculated using dumbbench (image/js/css/etc. resources are not taken into account). | ||
119 | |||
120 | === Reload of not existing page without UI === | ||
121 | |||
122 | {{code language="none"}} | ||
123 | dumbbench -p 0.1 -i 1000 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/get/NoSpace/NoPage" | ||
124 | {{/code}} | ||
125 | |||
126 | Unit: milliseconds | ||
127 | |||
128 | * {{previous/}}: 6.6 | ||
129 | * {{next/}}: 6.4 | ||
130 | |||
131 | === Reload of not existing page with UI === | ||
132 | |||
133 | {{code language="none"}} | ||
134 | dumbbench -p 0.1 -i 1000 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/view/NoSpace/NoPage" | ||
135 | {{/code}} | ||
136 | |||
137 | Unit: milliseconds | ||
138 | |||
139 | * {{previous/}}: 85 | ||
140 | * {{next/}}: 85 | ||
141 | |||
142 | === Reload of empty page without UI === | ||
143 | |||
144 | {{code language="none"}} | ||
145 | dumbbench -p 0.1 -i 1000 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/get/EmptySpace/EmptyPage" | ||
146 | {{/code}} | ||
147 | |||
148 | Unit: milliseconds | ||
149 | |||
150 | * {{previous/}}: 6.8 | ||
151 | * {{next/}}: 6.5 | ||
152 | |||
153 | === Reload of empty page with UI === | ||
154 | |||
155 | {{code language="none"}} | ||
156 | dumbbench -p 0.1 -i 1000 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/view/EmptySpace/EmptyPage" | ||
157 | {{/code}} | ||
158 | |||
159 | Unit: milliseconds | ||
160 | |||
161 | * {{previous/}}: 82 | ||
162 | * {{next/}}: 80 | ||
163 | |||
164 | === Reload of Main.WebHome without UI === | ||
165 | |||
166 | {{code language="none"}} | ||
167 | dumbbench -p 0.1 -i 1000 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/get/Main/WebHome" | ||
168 | {{/code}} | ||
169 | |||
170 | Unit: milliseconds | ||
171 | |||
172 | * {{previous/}}: 8 | ||
173 | * {{next/}}: 8 | ||
174 | |||
175 | === Reload of Main.WebHome with UI === | ||
176 | |||
177 | {{code language="none"}} | ||
178 | dumbbench -p 0.1 -i 1000 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/view/Main/WebHome" | ||
179 | {{/code}} | ||
180 | |||
181 | Unit: milliseconds | ||
182 | |||
183 | * {{previous/}}: 85 | ||
184 | * {{next/}}: 86 | ||
185 | |||
186 | == SOLR == | ||
187 | |||
188 | === Full SOLR reindex === | ||
189 | |||
190 | Unit: seconds | ||
191 | |||
192 | Delete the data/solr folder, restart, load home page and calculate the time of activity of the SOLR index thread using Yourkit (Yourkit is slowing down a bit all that but what is important is the difference anyway). | ||
193 | |||
194 | * {{previous/}}: 53 | ||
195 | * {{next/}}: 56 | ||
196 | |||
197 | === SOLR sync === | ||
198 | |||
199 | Only the thread determining what need to be indexed and not the actual indexing. | ||
200 | |||
201 | ==== when index is empty ==== | ||
202 | |||
203 | Unit: milliseconds | ||
204 | |||
205 | * {{previous/}}: 906 | ||
206 | * {{next/}}: 1611 | ||
207 | |||
208 | ==== when there is nothing to do ==== | ||
209 | |||
210 | Unit: milliseconds | ||
211 | |||
212 | * {{previous/}}: 406 | ||
213 | * {{next/}}: 694 | ||
214 | |||
215 | === Search === | ||
216 | |||
217 | XWiki is doing nothing on background and everything is both in server and client cache already. Values are calculated using dumbbench (image/js/css/etc. resources are not taken into account). | ||
218 | |||
219 | Unit: milliseconds | ||
220 | |||
221 | ==== Result of search finding lots of results ==== | ||
222 | |||
223 | Login with Admin user and make sure hidden document display is enabled. | ||
224 | |||
225 | {{code language="none"}} | ||
226 | dumbbench -p 0.1 -i 100 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/view/Main/Search?sort=score&sortOrder=desc&r=1&f_type=DOCUMENT&text=*" | ||
227 | {{/code}} | ||
228 | |||
229 | * {{previous/}}: 157 | ||
230 | * {{next/}}: 172 | ||
231 | |||
232 | ==== Result of search finding one result ==== | ||
233 | |||
234 | {{code language="none"}} | ||
235 | dumbbench -p 0.1 -i 100 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/view/Main/Search?sort=score&sortOrder=desc&r=1&f_type=DOCUMENT&text=name%3A1000Macros" | ||
236 | {{/code}} | ||
237 | |||
238 | * {{previous/}}: 116 | ||
239 | * {{next/}}: 118 | ||
240 | |||
241 | == Rendering == | ||
242 | |||
243 | === Page with 1000 macros without UI === | ||
244 | |||
245 | {{code language="none"}} | ||
246 | dumbbench -p 0.1 -i 100 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/get/Test/1000Macros" | ||
247 | {{/code}} | ||
248 | |||
249 | Unit: milliseconds | ||
250 | |||
251 | * {{previous/}}: 61 | ||
252 | * {{next/}}: 62 | ||
253 | |||
254 | === Page with 1000 html macros without UI === | ||
255 | |||
256 | {{code language="none"}} | ||
257 | dumbbench -p 0.1 -i 100 -- wget -q -o /dev/null -O /dev/null --load-cookies cookies.txt "http://127.0.0.1:8080/xwiki/bin/get/Test/1000HTMLMacros" | ||
258 | {{/code}} | ||
259 | |||
260 | Unit: milliseconds | ||
261 | |||
262 | * {{previous/}}: 718 | ||
263 | * {{next/}}: 683 | ||
264 | |||
265 | == Heap Memory == | ||
266 | |||
267 | Attach Yourkit to get the information. | ||
268 | |||
269 | Unit: MB | ||
270 | |||
271 | === Heap Memory after jetty startup === | ||
272 | |||
273 | Start jetty and wait until java is not doing anything on both (wait for potential initialization of various stuff like core extension cache, etc). Force garbage collector before looking at "used". | ||
274 | |||
275 | * {{previous/}}: 62 used, 567 allocated | ||
276 | * {{next/}}: 59 used, 500 allocated | ||
277 | |||
278 | === Heap Memory after full SOLR index === | ||
279 | |||
280 | Delete the data/solr folder, restart, load home page then wait until nothing is happening in background. Force garbage collector before looking at "used". | ||
281 | |||
282 | Unit: MB | ||
283 | |||
284 | * {{previous/}}: 162 used, 732 allocated | ||
285 | * {{next/}}: 162 used, 785 allocated |