Testing ZK-based Application with QF Test

Author
Hawk Chen, Engineer, Potix Corporation

Date
March, 2015

Version
QF-Test 4.0.3

Overview

QF-Test is a GUI testing tool for Java and Web, a previous small talk, Testing ZK Applications at Business Level with QF-Test, already gives us an overview of the tool. As a team member of ZK I was interested in trying it out myself and understand how it could benefit our users. This is why I started this article. In this article, I will talk about how I tested ZK application with this tool and I will share some useful tips about recording and replaying. Also my testing samples are shared in gitlab for your reference. I did not have the chance to try out advanced features such as remote execution and reading test data from data sources like Excel files, you can explore them by yourself if you are interested.

How QF Test differs from Other Testing Tools

When it comes to test we need to find out what we want to test. Server side logic or client side display? Functional test or stress test? The following tools are popular functional testing tools among the ZK community.

ZATS Mimic is a unit test library that is designed to test an application's server-side logic during agile development. It plays a complementary role to a GUI (client-side) test tool like Selenium or QF-test.

Selenium might be most users' first choice to perform a browser test against a ZK application as it's something most people are familiar with. However according to ZK Testing with Selenium IDE, you might need to deal with ID generator and locator by yourself.

Sahi is also a web automation test tool, it provides special functions in Sahi scripting language to identify ZK widget without ID generator. You can learn more from ZK Testing with Sahi.

QF-Test focuses on creating test cases in a graphical tree, which allows business testers to create and read tests without having much coding knowledge. The special integration for ZK offers an optimized component recognition algorithm, which filters for the most important ZK HTML elements and ZK ids. Due to that special effort for ZK widgets, you can access content of data containers like trees, tables or lists directly by one click without taking care of the internal HTML structure of those widgets. Nevertheless, it's also possible to use XPATH or CSS selectors to determine objects. Find an online video describing QF-Test's approach for component recognition at https://www.youtube.com/watch?v=VE_mJAZpMx0.

Record and Replay a Test Case

The most intuitive way in QF-Test to create a test case is to record your actions on a browser. QF-Test tutorial describes how you can create a test case in details. Here, I just mention a few key points to give a basic idea.

Quick Wizard

Extra \ Quick Wizard is a good place to start a new test case. Only a few things to notice:

• Fill a URL to a zul page like http://localhost:8080/qfsut/form.zul
• In "Ajax Toolkit", choose "ZK"

Then you will see a "Setup" node appears at "Extras" Node:

Record

Run your ZK application. Choose the "Setup" node and click "Start test run" (or press enter), QF-test will execute "Setup" node, open a custom browser window and connect to the specified URL.

Then you now can click "Start recording" button and perform the steps to test the application.

Check

After you perform some test actions, you would like to verify some conditions on the page. Click "Record check" button first and hover your mouse pointer on the ZK component you want to check, and you shall see a blue box highlighted on the ZK component recognized by QF-test. Left-click will add a default check, text check. Right click and you can select different checks, like "Image" or "Visible".

You can visit http://www.qfs.de/qftest/manual/en/checks.html to know more details about all supported checks.

Put Them Together

After clicking "Stop recording" button , a "Sequence" node with recorded actions will be automatically created at "Extras" node. Finally, add a "clean up" node. To make things clearer, I put these 3 nodes into a "Test case" node like:

When creating multiple test cases, you can just copy a test case node and modify without using QuickStart Wizard.

Recognized Components

After stopping the recording, you will also find that there are nodes created under "Window and Component" node. Those nodes are components identified during recording and will be used by the tool. If you remove a component node which is referenced by a test case, it will cause ComponentNotFoundException.

Standard Library

Every Test-suite includes a standard library by default.

It contains lots of procedures for accessing and checking components, file system and database access, logging messages or screenshots, and performs cleanup in a generic way. You can invoke them with "Call procedure" node in a Sequence node to be part of your testing process.

For a complete list of the standard library, please refer to http://www.qfs.de/include/qfs_pkgdoc/qfs_pkgdoc.html.

Simulate User Actions

QF-test can record most of the user actions, but some actions cannot be recorded and replayed directly. I have here shared some tips on how you can change a recorded event's properties manually and make them work.

Drag and Drop

If you just record drag and drop action and replay it, you will find it doesn't work as expected. As a workaround I simulate "drag and drop" with the following steps:

1. When recording, click on the source item and click on the destination, end record
2. Change Event details manually. Change source click to MOUSE_DRAG_FROM and destination click to MOUSE_DROP_TO
3. Adjust X, Y value of drag or drop position in fine grain

Please refer to sample test case Drag and Drop.

Hover

Hovering a mouse pointer is just like moving a mouse, it cannot be recorded directly. The steps to workaround are:

1. When recording, click the place you want to hover on.
2. Change the Mouse Event "Event detail" to MOUSE_DRAG_OVER.

Please refer to sample test case Hover.

Move Scrollbar

This divides into 2 cases: move a browser's scrollbar and a Listbox's scrollbar (or Grid's).

Browser's Scrollbar

Moving a browser's scrollbar is recorded as mouse clicks event and replay them doesn't actually move the browser's scrollbar. We also can achieve this action by using MOUSE_DRAG_FROM and MOUSE_DRAG_TO, just like what we do for "Drag and Drop". But actually you don't need to replay moving a browser's scrollbar in most cases, just replay the actions performed on components. QF-test will scroll to the target component automatically.

Listbox's Scrollbar

We can also achieve this action by MOUSE_DRAG_FROM and MOUSE_DRAG_TO, but the position it moves at replaying is not precise. Hence, each replay will move the scrollbar to different positions and fails a test case easily. I suggest to use PageUp and PageDown to scroll the view instead.

Upload

You have to check Replay as "hard" event property in the node of clicking "Browse..." button.

Call the procedure qfs.web.input.fileUpload can also do the same trick.

Other Tips

Inputting a Datebox

When recording a Datebox's input, you should input with text instead of selecting a day on the small calendar. Because Datebox will show the current month by default. If you select the small calendar, the month you recorded might not be the same as the month you replay. This will cause an unexpected input and fails your test case.

Differentiate Multiple Message Boxes

Multiple message boxes on the same page should have different titles or QF-test cannot differentiate them.

Perform a Check

Image Check

Image Check is very useful when you cannot perform text check or want to check an overall condition instead of checking one widget by another. But it's very common that a screenshot captured at the run-time has a minor difference from the target screenshot. QF-test default image comparison algorithm is pixel by pixel. So any minor difference may fail the image check. Fortunately, we can change comparison algorithm to tolerant image deviation for a "Check image" node by specifying

algorithm=identity;expected=0.95

at "Algorithm for image comparison" field.

For complete algorithm you can use, please refer to http://www.qfs.de/qftest/manual/en/tech_imagealgorithmdetails.html#sec_imagealgorithmdetails.

Downloaded File Check

There is no check option for downloaded files, you have to call a procedure qfs.utils.files.compareFiles to compare two files by specifying file paths.

Note that even if comparison procedure returns false it will not fail a test case automatically. We need to add the following steps to report an error when the procedure returns false:

1. set "Variable for return value" in "Call procedure" node (the screenshot above), e.g. filesAreEquals
2. add If node and specify the condition with not ${filesAreEquals} (the screenshot below)
3. add a node to log an error message or throw an exception (the screenshot below)

Summary and Download

In general I found QF Test easy to setup and useful for performing most of the GUI test cases against ZK. Thanks to its built-in Resolvers that identifies ZK components and handles it automatically without having the testers to code, setup ID and/or locators. The QF Support Test Team was also helpful and responsive in providing me solutions.
I have shared my test cases on gitlab. You can download these test cases and the example application and run them up by yourselves which may help you understand how the tool works with ZK better.
Sample ZK application: http://gitlab.zkoss.org/hawkchen/qfsut
Sample Test Suites: http://gitlab.zkoss.org/hawkchen/qfsut/tree/master/qftest
It contains test cases I created while trying out QF-test. You can feel free to use them as a studying reference:

Note that QF Test is a product of Quality First Software GmbH, please consult Quality First Software in case of any licensing and/or technical support matters.

Comments

Welcome any feedback. Is this article useful to you? Do you want to know more topics about it? Leave your comments.

Comments

Copyright © Potix Corporation. This article is licensed under GNU Free Documentation License.