A Simple Loading Performance Test on zkdemo
Henri Chen, Principal Engineer, Potix Corporation
October 05, 2006
Applicable to ZK 2.1.2 and later.
The Test
The loading performance test is done by using the record and playback method. First record the http requests sent by client into a testing script and then playback with multiple client threads simultaneously to simulate multiple users. We found the zkdemo testing script can perform 400 simultaneous client threads on a Mac mini equipped with 512MB memory with an 163.5 ms average response time.
The Server
- Mac mini (Intel) on the 100 Mb local area network.
- 1.66GHz Intel Core Duo Processor
- 512MB DDR2 SDRAM (minus 64MB shared display buffer)
- Mac OS X 1.4.7
- Tomcat 5.5.20 with maxThreads set to 1000
- Java 1.5_06 with max heap set to 384MB (-Xmx384M)
The Client
- Desktop PC on the 100 Mb local area network.
- 1.4GHz Pentium 4
- 256MB RAMBUS PC-800 DRAM
- Window 2000 Professional
- grinder-3.0-beta30
- Java 1.5_06 with default heap size
The Sample Application
- zkdemo-all.war (ZK 2.1.2)
- Rewritten all zscript codes (BeanShell interpreter code) with compiled java classes.
- Increase page definition caching size and some other fine tunes.
The Methodology
- Record testing script with Grinder TCPProxy.
- Playback and collect information with Grinder Agent and Grinder Console.
- The testing script is playback for 30 runs for each thread.
- The simultaneous threads number is changed from 50 ~ 450 for each test.
- The first run (run 0) is ignored to let the server reach its stability.
The Results
- ART: Average Response Time for a test.
- Max ARRT: Maximum Average Request Response Time for a test.
- Max RT: Maximum Response Time for a test.
| Threads | ART (ms) | ART Standard Deviation (ms) | Max ARRT (ms) | Max RT (ms) |
|---|---|---|---|---|
| 50 | 2.62 | 7.42 | 5.9 | 261 |
| 100 | 4.6 | 10.52 | 10.8 | 1202 |
| 150 | 5.43 | 14.26 | 10.9 | 1252 |
| 200 | 12.2 | 48.4 | 25 | 1292 |
| 250 | 27.85 | 94.46 | 56.4 | 2504 |
| 300 | 52.63 | 140.62 | 105 | 1672 |
| 350 | 118.04 | 204.72 | 155 | 2013 |
| 400 | 163.5 | 266.72 | 248 | 2113 |
| 450 | Grinder (client side Java) complains "Out Of Memory" |
Step By Step
The server side
deploy the zkdemo-all.war to Tomcat server.
The client side
- 1. Unzip the zktest.zip into a directory, say c:\zktest.
- 2. Edit setGrinderEnv.cmd to reflect your testing environment.
- GRINDERPATH=c:\grinder-3.0-beta30
- GRINDERPROPERTIES=c:\zktest
- JAVA_HOME=c:\jdk1.5.0_06
- 3. Edit the file grinder.properties. Change the thread number and make sure the url is point to your Tomcat server.
- grinder.threads=50.
- url=http://X.X.X.X:8080
- 4. Open a command box:
- Change directory to c:\zktest
- Execute "startConsole"
- "The Grinder Console" GUI console should popup.
- On the grinder GUI console, set "Ignore 0 Samples" to 120 ~ 480 depends on the thread number.
- 5. Open another command box (Let's call it "agent" command box)
- Change directory to c:\zktest
- Execute "startAgent"
- 6. On the grinder GUI console, select menu "Action/Start Processes". The grinder start sending requests to server and collecting information.
- 7. The "agent" command box will show up something like:
2006/10/3 AM 09:34:28 (agent): worker xxx-0 started
2006/10/3 AM 09:34:33 (process xxx-0): starting threads
- 8.When the test is done, the "agent" command box will show messages as follows:
2006/10/3 AM 09:54:06 (agent): finished, waiting for console signal
- 9. On the grinder GUI console
- Select "Results" tab and you will see the statistics for the test.
- The new collected data would be logged in c:\zktest\log
- 10. Change the threads number, restart the "agent" and "console" and start over.
Summary
Under our testing environment, the zkdemo testing script can perform 400 simultaneous client threads on a Mac mini equipped with 512MB memory with an 163.5 ms average response time. For those interested in the collected raw data, you can find them here. The ZK team also welcome you to collect your own data, see what happens, and give us feedback. That is why open source software is better, isn't it?
| Copyright © Potix Corporation. This article is licensed under GNU Free Documentation License. |