A Simple Loading Performance Test on zkdemo

{{Template:Smalltalk_Author|
|author=Henri Chen, Principal Engineer, Potix Corporation
|date=October 05, 2006
|version=Applicable to ZK 2.1.2 and later.
}}


=The Test=

The loading performance test is done by using the ''record and playback'' method. First ''record'' the http requests sent by client into a testing script and then playback with multiple client threads simultaneously to simulate multiple users. We found the zkdemo testing script can perform 400 simultaneous client threads on a Mac mini equipped with 512MB memory with an 163.5 ms average response time.



=The Server=

* Mac mini (Intel) on the 100 Mb local area network.
* 1.66GHz Intel Core Duo Processor
* 512MB DDR2 SDRAM (minus 64MB shared display buffer)
* Mac OS X 1.4.7
* Tomcat 5.5.20 with maxThreads set to 1000
* Java 1.5_06 with max heap set to 384MB (-Xmx384M)


=The Client=

* Desktop PC on the 100 Mb local area network.
* 1.4GHz Pentium 4
* 256MB RAMBUS PC-800 DRAM
* Window 2000 Professional
* [http://grinder.sourceforge.net/ grinder-3.0-beta30]
* Java 1.5_06 with default heap size



=The Sample Application=

* zkdemo-all.war (ZK 2.1.2)
* Rewritten all zscript codes (BeanShell interpreter code) with compiled java classes.
* Increase page definition caching size and some other fine tunes.



=The Methodology=

* Record testing script with Grinder TCPProxy.
* Playback and collect information with Grinder Agent and Grinder Console.
* The testing script is playback for 30 runs for each thread.
* The simultaneous threads number is changed from 50 ~ 450 for each test.
* The first run (run 0) is ignored to let the server reach its stability.



=The Results=

* ART: Average Response Time for a test.
* Max ARRT: Maximum Average Request Response Time for a test.
* Max RT: Maximum Response Time for a test.
{|border="1"
|-
! Threads || ART (ms) || ART Standard Deviation (ms) || Max ARRT (ms) ||  Max RT (ms)
|-
| 50||2.62||7.42||5.9||261
|-
|100||4.6||10.52||10.8||1202
|-
|150||5.43||14.26||10.9||1252
|-
|200||12.2||48.4||25||1292
|-
|250||27.85||94.46||56.4||2504
|-
|300||52.63||140.62||105||1672
|-
|350||118.04||204.72||155||2013
|-
|400||163.5||266.72||248||2113 
|-
|450||Grinder (client side Java) complains "Out Of Memory"
|}

=Step By Step=
<br>
==The server side==

deploy the zkdemo-all.war to Tomcat server.
<br>

==The client side==

:1. Unzip the [http://zkoss.org/smalltalks/zktest/zktest.zip zktest.zip] into a directory, say c:\zktest.
:2. Edit setGrinderEnv.cmd to reflect your testing environment.
::* GRINDERPATH=c:\grinder-3.0-beta30
::* GRINDERPROPERTIES=c:\zktest
::* JAVA_HOME=c:\jdk1.5.0_06
:3. Edit the file grinder.properties. Change the thread number and make sure the url is point to your Tomcat server.
::* grinder.threads=50.
::* url=<nowiki>http://X.X.X.X:8080</nowiki>
:4. Open a command box:
::* Change directory to c:\zktest
::* Execute "startConsole"
::* "The Grinder Console" GUI console should popup.
::* On the grinder GUI console, set "Ignore 0 Samples" to 120 ~ 480 depends on the thread number.
:5. Open another command box (Let's call it "agent" command box)
::* Change directory to c:\zktest
::* Execute "startAgent"
:6. On the grinder GUI console, select menu "Action/Start Processes". The grinder start sending requests to server and collecting information.
:7. The "agent" command box will show up something like:
<code><pre>
2006/10/3 AM 09:34:28 (agent): worker xxx-0 started
2006/10/3 AM 09:34:33 (process xxx-0): starting threads
</pre></code>
:8.When the test is done, the "agent" command box will show messages as follows: 
<code><pre>
2006/10/3 AM 09:54:06 (agent): finished, waiting for console signal
</pre></code>
:9. On the grinder GUI console
::* Select "Results" tab and you will see the statistics for the test.
::* The new collected data would be logged in c:\zktest\log
:10. Change the threads number, restart the "agent" and "console" and start over.

=Summary=

Under our testing environment, the zkdemo testing script can perform 400 simultaneous client threads on a Mac mini equipped with 512MB memory with an 163.5 ms average response time. For those interested in the collected raw data, you can find them [http://www.zkoss.org/smalltalks/zktest/log.zip here]. The ZK team also welcome you to collect your own data, see what happens, and give us feedback. That is why open source software is better, isn't it?


<comment>http://docs.zkoss.org/wiki/A_Simple_Loading_Performance_Test_on_zkdemo</comment>


[[Category:ZK]]
[[Category:Performance]]
{{Template:Smalltalk_Footer|
|name=Potix Corporation
}}