Testing ZK-based Application with QF Test

Author
Hawk Chen, Engineer, Potix Corporation

Date
March, 2015

Version
QF-Test 4.0.3

Overview

QF-Test is a GUI testing tool for Java and Web, a previous small talk, Testing ZK Applications at Business Level with QF-Test, already gives us an overview of the tool. As a team member of ZK I would like to try it out myself and learn how good this product is and understand how it could benefit our users and that is why I started this article. In this article, I will talk about how I tested ZK application with this tool and I will share some useful tips about recording and replaying. I did not have the chance to try out advanced features such as remote execution and reading test data from data sources like Excel files, you can explore them by yourself if you are interested.

How QF Test differs from Other Testing Tools

ZATS Mimic is a unit test library that is designed to test an application's server-side logic during agile development. It plays a complementary role to a GUI (client-side) test tool like Selenium or QF-test.

Selenium might be most users' first choice to perform a browser test against a ZK application as it's something most people are familiar with. However according to ZK Testing with Selenium IDE, you might need to deal with ID generator and locator by yourself.

Sahi is also a web automation test tool, it provides special functions in Sahi scripting language to identify ZK widget without ID generator. You can learn more from ZK Testing with Sahi.

QF-Test focuses on creating test cases in a graphical tree, which allows business testers to create and read tests without having much coding knowledge. The special integration for ZK offers an optimized component recognition algorithm, which filters for the most important ZK HTML elements and ZK ids. Due to that special effort for ZK widgets, you can access content of data containers like trees, tables or lists directly by one click without taking care of the internal HTML structure of those widgets. Nevertheless, it's also possible to use XPATH or CSS selectors to determine objects. Find an online video describing QF-Test's approach for component recognition at https://www.youtube.com/watch?v=VE_mJAZpMx0.

Record and Replay a Test Case

The most intuitive way in QF-Test to create a test case is to record your actions on a browser. QF-Test tutorial describes how you can create a test case in details. Here, I just mention a few key points to give a basic idea.

Quick Wizard

Extra \ Quick Wizard is a good place to start a new test case. Only a few things to notice:

• Fill a URL to a zul page like http://localhost:8080/qfsut/form.zul
• In "Ajax Toolkit", choose "ZK"

Then you will see a "Setup" node appears at "Extras" Node:

Record

Run your ZK application. Choose the "Setup" node and click "Start test run" (or press enter), QF-test will execute "Setup" node, open a custom browser window and connect to the specified URL.

Then you now can click "Start recording" button and perform the steps to test the application.

Check

After you perform some test actions, you would like to verify some conditions on the page. Click "Record check" button first and hover your mouse pointer on the ZK component you want to check, and you shall see a blue box highlighted on the ZK component recognized by QF-test. Left-click will add a default check, text check. Right click and you can select different checks, like "Image" or "Visible".

You can visit http://www.qfs.de/qftest/manual/en/checks.html to know more details about all supported checks.

Put Them Together

After clicking "Stop recording" button , a "Sequence" node with recorded actions will be automatically created at "Extras" node. Finally, add a "clean up" node. To make things clearer, I put these 3 nodes into a "Test case" node like:

When creating multiple test cases, you can just copy a test case node and modify without using QuickStart Wizard.

Recognized Components

After stopping the recording, you will also find that there are nodes created under "Window and Component" node. Those nodes are components identified during recording and will be used by the tool. If you remove a component node which is referenced by a test case, it will cause ComponentNotFoundException.

Standard Library

Every Test-suite includes a standard library by default.

It contains lots of procedures for accessing and checking components, file system and database access, logging messages or screenshots, and performs cleanup in a generic way. You can invoke them with "Call procedure" node in a Sequence node to be part of your testing process.

For a complete list of the standard library, please refer to http://www.qfs.de/include/qfs_pkgdoc/qfs_pkgdoc.html.

Simulate User Actions

QF-test can record most of the user actions, but some actions cannot be recorded and replayed directly. I have here shared some tips on how you can change a recorded event's properties manually and make them work.

Drag and Drop

If you just record drag and drop action and replay it, you will find it doesn't work as expected. As a workaround I simulate "drag and drop" with the following steps:

1. When recording, click on the source item and click on the destination, end record
2. Change Event details manually. Change source click to MOUSE_DRAG_FROM and destination click to MOUSE_DROP_TO
3. Adjust X, Y value of drag or drop position in fine grain

Please refer to sample test case Drag and Drop.

Hover

Hovering a mouse pointer is just like moving a mouse, it cannot be recorded directly. The steps to workaround are:

1. When recording, click the place you want to hover on.
2. Change the Mouse Event "Event detail" to MOUSE_DRAG_OVER.

Please refer to sample test case Hover.

Move Scrollbar

This divides into 2 cases: move a browser's scrollbar and a Listbox's scrollbar (or Grid's).

Browser's Scrollbar

Moving a browser's scrollbar is recorded as mouse clicks event and replay them doesn't actually move the browser's scrollbar. We also can achieve this action by using MOUSE_DRAG_FROM and MOUSE_DRAG_TO, just like what we do for "Drag and Drop". But actually you don't need to replay moving a browser's scrollbar in most cases, just replay the actions performed on components. QF-test will scroll to the target component automatically.

Listbox's Scrollbar

We can also achieve this action by MOUSE_DRAG_FROM and MOUSE_DRAG_TO, but the position it moves at replaying is not precise. Hence, each replay will move the scrollbar to different positions and fails a test case easily. I suggest to use PageUp and PageDown to scroll the view instead.

Upload

You have to check Replay as "hard" event property in the node of clicking "Browse..." button.

Call the procedure qfs.web.input.fileUpload can also do the same trick.

Other Tips

Input in a Datebox

When recording a Datebox's input, you should input with text instead of selecting a day on the small calendar. Because Datebox will show the current month by default. The month you record might not be the same as the month when you replay. This will cause an unexpected input and fail your test case.

Differentiate Multiple Message Boxes

Multiple message boxes on the same page should have different titles or QF-test cannot differentiate them.

Perform a Check

Image Check

Image Check is very useful when you cannot perform text check or want to check an overall condition instead of checking one widget by one. But it's very common that a screenshot captured at the run-time has a minor difference from the target screenshot. QF-test default image comparison algorithm is pixel by pixel. So any minor difference may fail the image check. Fortunately, we can change comparison algorithm to tolerant image deviation for a "Check image" node by specifying

algorithm=identity;expected=0.95

at "Algorithm for image comparison" field.

For complete algorithm you can use, please refer to http://www.qfs.de/qftest/manual/en/tech_imagealgorithmdetails.html#sec_imagealgorithmdetails.

Downloaded File Check

There is no such check option, you have to call a procedure qfs.utils.files.compareFiles to compare two files by specifying file pathes.

But comparison procedure returns false will not make a test case fail automatically. We can fail a test case upon procedure result as follows:

1. set "Variable for return value" in "Call procedure" node (the screenshot above), e.g. filesAreEquals
2. add If node and specify the condition with not ${filesAreEquals} (the screenshot below)
3. add a node to log an error message or throw an exception (the screenshot below)

Download

You can download test cases and the example application and run it by yourselves. You may understand better. Sample ZK application: http://gitlab.zkoss.org/hawkchen/qfsut

Sample Test Suites: http://gitlab.zkoss.org/hawkchen/qfsut/tree/master/qftest It contains test cases I made for trying QF-test. Readers can use them as a reference:

Comments

Welcome any feedback. Is this article useful to you? Do you want to know more topics about it? Leave your comments.

Comments

Copyright © Potix Corporation. This article is licensed under GNU Free Documentation License.