Testing ZK-based Application with QF Test

Author
Hawk Chen, Engineer, Potix Corporation

Date
March 12, 2015

Version
QF-Test 4.0.3
ZK 7

Overview

QF-Test is a GUI testing tool for Java and Web, a previous small talk, Testing ZK Applications at Business Level with QF-Test, already gives us an overview of the tool. As a team member of ZK, I was interested in trying it out myself and understand how it could benefit our users. This is why I started this article.

QF-Test has a complete tutorial and following their tutorial most of the test cases run well. However for some specific user actions manual tweaking is required. In this article, I will focus on sharing these useful tweaking tips especially around recording and replaying. Also, my testing samples are shared in gitlab for your reference. I did not have the chance to try out advanced features such as remote execution and reading test data from data sources like Excel files, you can explore them by yourself if you are interested.

How It Is Different from Other Testing Tools

When it comes to test we need to think about what we want to test. Server-side logic or client side display? Functional test or stress test? The following tools are popular functional testing tools among the ZK community.

ZATS Mimic is a unit test library that is designed to test an application's server-side logic during agile development. It plays a complementary role to a GUI (client-side) test tool like Selenium and QF-test.

Selenium might be most users' first choice to perform a browser test against a ZK application as it's something most people are familiar with. However according to ZK Testing with Selenium IDE, you might need to deal with ID generator and locator by yourself.

Sahi is also a web automation testing tool, it provides special functions in Sahi scripting language to identify ZK widget without ID generator. You can learn more from ZK Testing with Sahi.

QF-Test focuses on creating test cases in a graphical tree, which allows business testers to create and read tests without having much coding knowledge. The special integration for ZK offers an optimized component recognition algorithm, which filters for the most important ZK HTML elements and ZK ids. Due to that special effort for ZK widgets, you can access content of data containers like trees, tables or lists directly by one click without having to take care of the internal HTML structure of those widgets. Nevertheless, it's also possible to use XPATH or CSS selectors to determine objects. Find an online video describing QF-Test's approach for component recognition at https://www.youtube.com/watch?v=VE_mJAZpMx0.

Record and Replay a Test Case

The most intuitive way in QF-Test to create a test case is to record your actions on a browser. QF-Test tutorial describes how you can create a test case in details. For step-by-step setup, please reference their tutorial. Here, I just mention a few key points to give you a basic idea about the record and replay process.

Quick Wizard

Extra \ Quick Wizard is a good place to start a new test case. Only a few things to notice:

• Fill a URL to a zul page like http://localhost:8080/qfsut/form.zul
• In "Ajax Toolkit", choose "ZK"

Then you will see a "Setup" node appears at "Extras" Node:

Record

Run your ZK application. Choose the "Setup" node and click "Start test run" (or press enter), QF-test will execute "Setup" node, open a custom browser window and connect to the specified URL.

Then you now can click "Start recording" button and perform the steps to test the application.

Check

After you perform some test actions, you would normally like to verify some conditions on the page. Click "Record check" button first and hover your mouse pointer on the ZK component you want to check, and you shall see a blue box highlighted on the ZK component recognized by QF-test. Left-click will add a default check, text check. Right click and you can select different checks, like "Image" or "Visible".

You can visit http://www.qfs.de/qftest/manual/en/checks.html to know more details about all supported checks.

Put Them Together

After clicking "Stop recording" button , a "Sequence" node with recorded actions will be automatically created at "Extras" node. Finally, add a "clean up" node. To make things clearer, I put these 3 nodes into a "Test case" node like:

When creating multiple test cases, you can just copy a test case node and modify without using QuickStart Wizard.

Recognized Components

After recording has stopped, you will also find that there are nodes created under "Window and Component" node. Those nodes are components identified during recording and will be used by the tool. If you remove a component node which is referenced by a test case, it will cause ComponentNotFoundException.

Simulate User Actions

Following the steps illustrated above most of the cases would run out of box. However there are some actions that cannot be recorded and replayed directly and need some tweaking. Here under I will share some tips on how you can change a recorded event's property manually and make it work.

Standard Library

Every Test-suite includes a standard library by default.

It contains lots of procedures for accessing and checking components, file system and database access, logging messages or screenshots and performs cleanup in a generic way. You can invoke them with "Call procedure" node in a Sequence node to be part of your testing process. This library will be used later in this section.

For a complete list of the standard library, please refer to http://www.qfs.de/include/qfs_pkgdoc/qfs_pkgdoc.html.

Drag and Drop

If you just record drag and drop action and replay it, you will find it doesn't work as expected. As a workaround I simulate "drag and drop" with the following steps:

1. When recording, click on the source item and click on the destination, end record
2. Change Event details manually. Change source click to MOUSE_DRAG_FROM and destination click to MOUSE_DROP_TO
3. Adjust X, Y value of drag or drop position in fine grain

Please refer to sample test case Drag and Drop.

Hover

Hovering a mouse pointer is just like moving a mouse, it cannot be recorded directly. The workaround steps are:

1. When recording, click the place you want to hover on.
2. Change the Mouse Event "Event detail" to MOUSE_DRAG_OVER.

Please refer to sample test case Hover.

Move Scrollbar

This divides into 2 cases: move a browser's scrollbar and a Listbox's scrollbar (or Grid's).

Browser's Scrollbar

Moving a browser's scrollbar is recorded as mouse clicks events and replaying them doesn't actually move the browser's scrollbar. We can also achieve this action by using MOUSE_DRAG_FROM and MOUSE_DRAG_TO, just like what we do for "Drag and Drop". But actually you don't need to replay moving a browser's scrollbar in most cases, just replay the actions performed on components. QF-test will scroll to the target component automatically.

Listbox's Scrollbar

We can also achieve this action by MOUSE_DRAG_FROM and MOUSE_DRAG_TO, but the position it moves at replaying is not precise. Hence, each replay will move a scrollbar to different positions and fails a test case easily. I suggest using PageUp and PageDown to scroll the view instead.

Upload

You have to check Replay as "hard" event property in the node of clicking "Browse..." button.

Call the procedure qfs.web.input.fileUpload can also do the same trick.

Perform a Check

As mentioned in previous sections, after recording a test case you can add a check easily by left or right-click on the target component. Most of the checks run out of box as illustrated in their tutorial but not all. Here are some tips that will hopefully make your life easier:

Image Check

Image Check is very useful when you cannot perform text check or want to check an overall condition instead of checking widgets one by one. However, it's very common that a screenshot captured at the run-time has a minor difference from the target screenshot. QF-test default image comparison algorithm is pixel by pixel, so any minor difference may fail the image check. Fortunately, we can change comparison algorithm to tolerant image deviation for a "Check image" node by specifying

algorithm=identity;expected=0.95

at "Algorithm for image comparison" field.

For complete algorithm you can use, please refer to http://www.qfs.de/qftest/manual/en/tech_imagealgorithmdetails.html#sec_imagealgorithmdetails.

Downloaded File Check

There is no check option for downloaded files, you have to call a procedure qfs.utils.files.compareFiles to compare two files by specifying file paths.

However, comparison procedure returns false will not make a test case fail automatically, we can fail a test case by the procedures below:

1. set "Variable for return value" in "Call procedure" node (the screenshot above), e.g. filesAreEquals
2. add If node and specify the condition with not ${filesAreEquals} (the screenshot below)
3. add a node to log an error message or throw an exception (the screenshot below)

Other Tips

Inputting a Datebox

When recording a Datebox's input, you should input with text instead of selecting a day from the small calendar. Because Datebox will show the current month by default. If you select from the small calendar, the month you recorded might not be the same as the month you replay. This will cause an unexpected input and fails your test case.

Differentiate Multiple Message Boxes

Multiple message boxes on the same page should have different titles or QF-test cannot differentiate them.

Summary and Download

In general I found QF-Test is easy to setup and useful for performing most of the GUI test cases against ZK. Thanks to its built-in resolvers that identifies ZK components and handles it automatically without having the testers to code, setup ID and/or locators. Also, their complete tutorial was a good reference.

I hope these tips covers most of the tricky parts and saves your time like how the QF-Test Support Team helped me. I have shared my test cases on gitlab. You can download these test cases and the example application and run them up by yourselves which may help you understand how the tool works with ZK better.

Sample ZK application: http://gitlab.zkoss.org/hawkchen/qfsut
Sample Test Suites: http://gitlab.zkoss.org/hawkchen/qfsut/tree/master/qftest
It contains test cases I created while trying out QF-test. Feel free to use them as a study reference:

Note that QF-Test is a product of Quality First Software GmbH, please consult Quality First Software in case of any licensing and/or technical support matters.

References

Welcome any feedback. Is this article useful to you? Do you want to know more topics about it? Leave your comments.

Comments

Copyright © Potix Corporation. This article is licensed under GNU Free Documentation License.