Step by Step Trouble Shooting
Performance SOP
Identify the Bottleneck
Usually the bottleneck can be found in one of these areas:
- client
- network
- server
To breakdown a slow performing web application is a good idea to start, where the bad performance is perceived... in the browser. Most modern browsers provide very sophisticated tools supporting the search for a bottleneck and draw some conclusions, and eliminate other possible causes easily.
- Dev tools (Network)
- Chrome -> [F12]
- Firefox -> [F12] or Firebug
- IE9+ -> [F12]
- IE8 -> fiddler2
File:Chrome developer tools network.jpg
Investigating the network traffic following the questions below the biggest problem area(s) should become apparent after a few minutes:
1. Are there one or more long running requests?
NO → #Client Side Issue
YES
↓
2. Is it a static resource?
YES (js, css, images...) → check #ZK Server Configuration (debug / cache / compression) → STILL SLOW → #Network Issue
NO (dynamic request into ZK application)
- *.zul = full page request (can be followed by ajax requests)
- zkau/* = ajax request
↓
3. Which PHASE of the request is slowest ? (examples based on Chrome developer tools)
File:Chrome developer tools network example phases.jpg
CONNECTING (or one of Proxy, DNS Lookup, Blocking, SSL)
- 3.a) Is this a network problem (everything between browser and ZK Application)?
- test ping / trace route to different servers
- test dns lookup timing
- YES → #Network Issue
- NO → #Server Side Issue (application takes long time to accept connection, or even times out)
SENDING
- 3.b) Is the request unreasonably big?
- YES → #Client Side Issue
- NO
- ↓
- 3.c) Is the bandwidth low?
- e.g. try upload the same amount of data to the server via ftp/scp to check possible upload speed
- YES → #Network Issue
- NO → #Server Side Issue (application server receiving request data slowly)
WAITING → #Server Side Issue (application server taking long time to prepare response)
RECEIVING
- 3.d) Is the response unreasonably big?
- YES → #ZK Server Configuration (render on demand / compression)
- NO
- ↓
- 3.e) Is the bandwidth low?
- e.g.try to download the same amount of data from the server via ftp/scp to check download speed
- YES → ask your administrator to fix it ;)
- NO → #Server Side Issue (appserver sending response data slowly)
Client Side Issue
client side
profiling client side - debug JS Chrome FF Firebug IE
CPU performance memory leaks
Client Side Profiling
Memory Issue
Server Side Issue
http://www.computerperformance.co.uk/HealthCheck/Disk_Health.htm#Disk%20Bottleneck%202 OS or VM
Perfmon.exe
LogicalDisk(_Total) -> Avg. Disk Queue Length LogicalDisk(_Total) -> Current Disk Queue Length PhysicalDisk(_Total) -> % Disk Time Processor Information(_Total) -> % Processor Time
JVisualVM
- Busy vs. IO wait - Java Debugging - Task manager / Process explorer / top - GC logging - profiling - memory leaks
Appserver config e.g. check number of concurrent requests
ZK Monitor/Statistics...
- Sessions - Desktops
Server Side Profiling
Memory Issue
ZK Server Configuration
- check debug mode → zk config should not be enabled (disabled by default)
- check caching config → should not be disabled (enabled by default)
- http://books.zkoss.org/wiki/ZK_Configuration_Reference/zk.xml/The_Library_Properties/org.zkoss.web.classWebResource.cache
- http://books.zkoss.org/wiki/ZK_Configuration_Reference/zk.xml/The_Library_Properties/org.zkoss.zk.WPD.cache
- http://books.zkoss.org/wiki/ZK_Configuration_Reference/zk.xml/The_Library_Properties/org.zkoss.zk.WCS.cache
- check compression settings → should not be disabled (enabled by default)
- consider/check render on demand settings
- http://books.zkoss.org/wiki/ZK_Developer%27s_Reference/Performance_Tips/Client_Render_on_Demand
- http://books.zkoss.org/wiki/ZK_Developer%27s_Reference/Performance_Tips/Listbox,_Grid_and_Tree_for_Huge_Data/Turn_on_Render_on_Demand
- http://books.zkoss.org/wiki/ZK_Configuration_Reference/zk.xml/The_Library_Properties/org.zkoss.zul.client.rod
- http://books.zkoss.org/wiki/ZK_Configuration_Reference/zk.xml/The_Library_Properties/org.zkoss.zul.grid.initRodSize
- http://books.zkoss.org/wiki/ZK_Configuration_Reference/zk.xml/The_Library_Properties/org.zkoss.zul.listbox.initRodSize
- http://books.zkoss.org/wiki/ZK_Configuration_Reference/zk.xml/The_Library_Properties/org.zkoss.zul.tree.initRodSize (ZK 7)
- TODO ??? maybe more tweaks ???
- check http://books.zkoss.org/wiki/ZK_Developer%27s_Reference/Performance_Tips
Network Issue
If something in your network infrastrucure (routers, proxies, web servers...) is causing the performance issues, there is little you can do as web application developer.
Here some ideas to identify possible bottlenecks trying to reduce network complexity by:
- using ip addresses directly → indicates DNS problem
- avoiding proxies, routers, firewalls (e.g. access the application server from a remote desktop "closer" to the actual server)
- accessing the application server directly, instead of going through a webserver or load balancer
- disabling SSL and check difference
→ "Kindly" inform your network administrator about your observations and ask for help identifying, excluding, fixing these infrastructure problems.