Author
Henri Chen, Principal Engineer, Potix Corporation

Date
October 05, 2006

Version
Applicable to ZK 2.1.2 and later.

The Test

The loading performance test is done by using the record and playback method. First record the http requests sent by client into a testing script and then playback with multiple client threads simultaneously to simulate multiple users. We found the zkdemo testing script can perform 400 simultaneous client threads on a Mac mini equipped with 512MB memory with an 163.5 ms average response time.

The Server

Mac mini (Intel) on the 100 Mb local area network.
1.66GHz Intel Core Duo Processor
512MB DDR2 SDRAM (minus 64MB shared display buffer)
Mac OS X 1.4.7
Tomcat 5.5.20 with maxThreads set to 1000
Java 1.5_06 with max heap set to 384MB (-Xmx384M)

The Client

Desktop PC on the 100 Mb local area network.
1.4GHz Pentium 4
256MB RAMBUS PC-800 DRAM
Window 2000 Professional
grinder-3.0-beta30
Java 1.5_06 with default heap size

The Sample Application

zkdemo-all.war (ZK 2.1.2)
Rewritten all zscript codes (BeanShell interpreter code) with compiled java classes.
Increase page definition caching size and some other fine tunes.

The Methodology

Record testing script with Grinder TCPProxy.
Playback and collect information with Grinder Agent and Grinder Console.
The testing script is playback for 30 runs for each thread.
The simultaneous threads number is changed from 50 ~ 450 for each test.
The first run (run 0) is ignored to let the server reach its stability.

The Results

ART: Average Response Time for a test.
Max ARRT: Maximum Average Request Response Time for a test.
Max RT: Maximum Response Time for a test.

Threads	ART (ms)	ART Standard Deviation (ms)	Max ARRT (ms)	Max RT (ms)
50	2.62	7.42	5.9	261
100	4.6	10.52	10.8	1202
150	5.43	14.26	10.9	1252
200	12.2	48.4	25	1292
250	27.85	94.46	56.4	2504
300	52.63	140.62	105	1672
350	118.04	204.72	155	2013
400	163.5	266.72	248	2113
450	Grinder (client side Java) complains "Out Of Memory"

Step By Step

The server side

deploy the zkdemo-all.war to Tomcat server.

The client side

1. Unzip the zktest.zip into a directory, say c:\zktest.

2. Edit setGrinderEnv.cmd to reflect your testing environment.

GRINDERPATH=c:\grinder-3.0-beta30
GRINDERPROPERTIES=c:\zktest
JAVA_HOME=c:\jdk1.5.0_06

3. Edit the file grinder.properties. Change the thread number and make sure the url is point to your Tomcat server.

grinder.threads=50.
url=http://X.X.X.X:8080

4. Open a command box:

Change directory to c:\zktest
Execute "startConsole"
"The Grinder Console" GUI console should popup.
On the grinder GUI console, set "Ignore 0 Samples" to 120 ~ 480 depends on the thread number.

5. Open another command box (Let's call it "agent" command box)

Change directory to c:\zktest
Execute "startAgent"

6. On the grinder GUI console, select menu "Action/Start Processes". The grinder start sending requests to server and collecting information.

7. The "agent" command box will show up something like:

2006/10/3 AM 09:34:28 (agent): worker xxx-0 started 2006/10/3 AM 09:34:33 (process xxx-0): starting threads

8.When the test is done, the "agent" command box will show messages as follows:

2006/10/3 AM 09:54:06 (agent): finished, waiting for console signal

9. On the grinder GUI console

Select "Results" tab and you will see the statistics for the test.
The new collected data would be logged in c:\zktest\log

10. Change the threads number, restart the "agent" and "console" and start over.

Summary

Under our testing environment, the zkdemo testing script can perform 400 simultaneous client threads on a Mac mini equipped with 512MB memory with an 163.5 ms average response time. For those interested in the collected raw data, you can find them here. The ZK team also welcome you to collect your own data, see what happens, and give us feedback. That is why open source software is better, isn't it?

Anonymous

A Simple Loading Performance Test on zkdemo

Namespaces

More

Page actions

Contents