ZK Testing with Selenium IDE"
Robertwenzel (talk | contribs) |
|||
(159 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
{{Template:Smalltalk_Author| | {{Template:Smalltalk_Author| | ||
|author=Robert Wenzel, Engineer, Potix Corporation | |author=Robert Wenzel, Engineer, Potix Corporation | ||
− | |date=July 2013 | + | |date=July 30, 2013 |
|version=ZK 6.5 (or later) | |version=ZK 6.5 (or later) | ||
}} | }} | ||
− | |||
=Introduction= | =Introduction= | ||
+ | |||
+ | So you have your nice ZK application and are interested in, how to be sure it will work in different browsers (and keep working after changes, which always happen). Of course you have tested everything possible with your unit and integration tests ;-). You have maxed out the possibilities offered by [http://www.zkoss.org/product/zats ZATS] to ensure ZK components play well together in a simulated environment. | ||
+ | |||
+ | Finally you come to a point when it is necessary to ensure your application (or at least the key features) works and renders well in a real environment (with different browsers, real application server, real network traffic/latency etc.) - so eventually Selenium comes into play. | ||
+ | |||
==About this article== | ==About this article== | ||
− | |||
− | |||
− | |||
− | + | There are several approaches in creating Selenium Tests: | |
− | + | ||
− | + | The programmatic way - suitable for developers - is already covered in [[Small_Talks/2009/February/How_to_Test_ZK_Application_with_Selenium | this Small Talk]]. This way offers maximum control over the test execution to write reusable, flexible, maintainable and even "intelligent" test cases. Of course writing such test cases may require some time and resources. | |
+ | |||
+ | Sometimes it is up to a less technical person to create these test cases. This is when one often hears about '''"record and replay"''' tools. | ||
+ | Selenium IDE is not only such a tool, it has record and replay features but also many more options to tweak and extend your test cases after recording. In few cases the recorded tests can be instantly re-playable. However, especially when it comes to AJAX there are several issues, that will prevent your test cases from executing out-of-the-box. | ||
+ | |||
+ | Here are two examples (you can easily find more): | ||
+ | * problems with [http://stackoverflow.com/questions/4203559/some-mouse-clicks-dont-register-in-selenium Missing clicks] | ||
+ | * someone recorded this [http://www.youtube.com/watch?v=k7U2mh65Xys video] explaining several issues | ||
+ | But not to worry, this article will show you how to deal with these! | ||
+ | |||
+ | The major conclusion is: | ||
+ | See Selenium IDE more as a record/tweak/maintain and replay/debug tool to build your tests cases. | ||
+ | |||
+ | This article will show (using an example web application) how to: | ||
+ | * enable useful initial test recording in ZK | ||
+ | * tweak the tests so that they replay | ||
+ | * keep your tests maintainable and robust to changes in your pages | ||
+ | * add missing actions that fail to record at all | ||
+ | |||
+ | Many ideas in the Article are based on solutions found for general problems in several forums and previous project experiences. Some simply reflects my personal preference on how things may be handled. This article does not show the only way to write tests for a ZK application, just highlight problems and solutions, and discuss some considerations taken and why this approach was chosen. | ||
+ | |||
+ | Happy reading and commenting !! | ||
==Environment== | ==Environment== | ||
− | * Firefox | + | * Firefox 22 |
− | + | * Selenium IDE 2.1.0 (Selenium IDE 2.0.0 is not working in FF >=22) | |
* JDK 6/7 | * JDK 6/7 | ||
− | * ZK 6.5.3 | + | * ZK 6.5.3 |
* Maven 3.0 | * Maven 3.0 | ||
=Steps= | =Steps= | ||
− | ==Initialize your | + | ==Initialize your environment for this example== |
− | * | + | * Install Selenium IDE plugin in Firefox via Addons or [http://docs.seleniumhq.org/download/ download] [http://wiki.openqa.org/display/SIDE/Home documentation page] |
− | + | * Maven 3.0 [http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html#Installation download & install] | |
− | * Maven 3.0 | + | * [[#Download|download]] the example project, it contains the following folders: |
− | * | + | ** selenium_example/ (the example application, maven project) |
− | ** | + | ** extensions/ (the selenium extensions used in this article) |
− | + | ** test_suite/ (a Selenium Test Suite illustrating the contepts discussed here) | |
− | * | + | * run on command line |
− | * | + | cd selenium_example_app |
− | * localhost:8080 | + | mvn jetty:run |
− | + | : or execute ''selenium_testing_app/launch_for_PROD.bat / .sh'' | |
+ | * open test app http://localhost:8080/selenium_testing | ||
==Initial Situation== | ==Initial Situation== | ||
+ | # Launch Firefox | ||
# Launch Selenium IDE CTRL+ALT+S | # Launch Selenium IDE CTRL+ALT+S | ||
− | # open example app localhost:8080 | + | # Set the base URL to "http://localhost:8080/" |
− | # Record login-test and replay it | + | # open example app http://localhost:8080/selenium_testing |
+ | # Record login-test and try to replay it | ||
<table cellpadding="1" cellspacing="1" border="1"> | <table cellpadding="1" cellspacing="1" border="1"> | ||
Line 69: | Line 94: | ||
Looks hard to read / maintain and doesn't even work :'( | Looks hard to read / maintain and doesn't even work :'( | ||
− | Looking at the recorded commands and comparing with page source we notice the IDs are generated and changing with every | + | Looking at the recorded commands and comparing the IDs with the page source we notice the IDs are generated and changing with every page refresh. So no chance to record / replay this way. |
A way to make the IDs predictable is shown in the next Section. | A way to make the IDs predictable is shown in the next Section. | ||
Line 75: | Line 100: | ||
==Custom Id Generator== | ==Custom Id Generator== | ||
− | As mentioned in other Small Talks about testing one strategy is to use a custom IdGenerator implementation to create predictable, readable (with a business meaning) and easily selectable (by selenium) component | + | As mentioned in other Small Talks about testing, one strategy is to use a custom IdGenerator implementation to create predictable, readable (with a business meaning) and easily selectable (by selenium) component IDs. |
<source lang="java"> | <source lang="java"> | ||
Line 100: | Line 125: | ||
return uuid.length() == 0 ? "zkcomp_" + i : uuid.append(i).toString(); | return uuid.length() == 0 ? "zkcomp_" + i : uuid.append(i).toString(); | ||
} | } | ||
+ | ... | ||
+ | //omitted code see download section Selenium-IDE-example.zip: | ||
+ | //selenium_testing_app/src/main/java/org/zkoss/selenium_example/TestingIdGenerator.java) | ||
</source> | </source> | ||
Line 125: | Line 153: | ||
mvn jetty:run | mvn jetty:run | ||
− | '' | + | or execute ''selenium_testing_app/launch_for_TEST.bat / .sh'' |
+ | |||
Now we get nicer and deterministic IDs when recording a test case. | Now we get nicer and deterministic IDs when recording a test case. | ||
Line 155: | Line 184: | ||
</table> | </table> | ||
− | This already looks nice, and self explaining - | + | This already looks nice, and self explaining - but unfortunately still fails to replay |
Why this happens and how to fix it? See next section ... | Why this happens and how to fix it? See next section ... | ||
Line 161: | Line 190: | ||
==ZK specific details== | ==ZK specific details== | ||
− | + | Due to the nature of rich web applications, JavaScript is generally heavily used, but Selenium IDE does not record all events by default. In our case here the "blur" events of the input fields have not been recorded, but ZK relies on them to determine updated fields. So, we need to manually add selenium commands "fireEvent target blur".Unfortunately Selenium does not offer a nice equivalent like "focus" action. | |
− | |||
− | |||
− | So we need to manually add selenium commands "fireEvent target blur" | ||
<table cellpadding="1" cellspacing="1" border="1"> | <table cellpadding="1" cellspacing="1" border="1"> | ||
Line 200: | Line 226: | ||
<tr> | <tr> | ||
<td>assertLocation</td> | <td>assertLocation</td> | ||
− | <td> | + | <td>*/selenium_testing/index.zul</td> |
<td></td> | <td></td> | ||
</tr> | </tr> | ||
Line 215: | Line 241: | ||
</table> | </table> | ||
− | Replaying the script | + | Replaying the script now works :D, and we see the overview page (also added a few verifications for that). |
==Selenium Extensions== | ==Selenium Extensions== | ||
− | If you find the syntax of "fireEvent" hard to remember or think it is too inconvenient to add an additional line just to update an input field there's help using a '''Selenium Core extension'''. The extension - file usually called '''user-extensions.js''' | + | If you find the syntax of "fireEvent" hard to remember or think it is too inconvenient to add an additional line just to update an input field there's help using a '''Selenium Core extension'''. The extension - file usually called '''user-extensions.js''' can be used in both Selenium IDE and Selenium RC when running the tests outside of Selenium IDE (please refer to [http://docs.seleniumhq.org/docs/08_user_extensions.jsp selenium documentation]). |
This snippet shows 2 simple custom actions: | This snippet shows 2 simple custom actions: | ||
;blur | ;blur | ||
− | : | + | : convenient action to avoid "fireEvent locator blur" |
;typeAndBlur | ;typeAndBlur | ||
: combines the type with a blur event, to make ZK aware of the input change automatically | : combines the type with a blur event, to make ZK aware of the input change automatically | ||
Line 232: | Line 258: | ||
var element = this.page().findElement(locator); | var element = this.page().findElement(locator); | ||
// Fire the "blur" event | // Fire the "blur" event | ||
− | + | this.doFireEvent(locator, "blur") | |
}; | }; | ||
Line 242: | Line 268: | ||
this.page().replaceText(element, text); | this.page().replaceText(element, text); | ||
// Fire the "blur" event | // Fire the "blur" event | ||
− | + | this.doFireEvent(locator, "blur") | |
}; | }; | ||
</source> | </source> | ||
− | To enable it just | + | To enable it just add the file (''extensions/user-extension.js'' in the example zip) to the Selenium IDE configuration. |
− | (the file contains another extension ... more about | + | (the file contains another extension ... more about when discussing [[#Custom_Locator_and_LocatorBuilder|custom locators]]) |
− | : (Selenium | + | : (Selenium IDE Options > Options... > [General -Tab] > Selenium Core extensions) |
− | Then restart Selenium | + | Then restart Selenium IDE - close the window, and reopen it - e.g. by [CTRL+ALT+S] |
Adapted test using the '''blur''' action: | Adapted test using the '''blur''' action: | ||
Line 313: | Line 339: | ||
It is not required to use either of these extensions, but saving 1 line for each input is my personal preference. | It is not required to use either of these extensions, but saving 1 line for each input is my personal preference. | ||
− | Another thing to keep in mind is robustness and maintenance effort of test-cases when changes in the UI happen ... the generated numbers | + | Another thing to keep in mind is robustness and maintenance effort of test-cases when changes in the UI happen ... the generated numbers at the end of each ID are still an obstacle in this way as they are prone to change every time a component is added or removed above that component. |
Wouldn't it be nice to avoid having to adapt test cases that often? Read on... | Wouldn't it be nice to avoid having to adapt test cases that often? Read on... | ||
− | ==Improve robustness and maintainability | + | ==Improve robustness and maintainability== |
===Locators in Selenium=== | ===Locators in Selenium=== | ||
Line 335: | Line 361: | ||
//div[starts-with(@id, 'billingAddress_')]//input[starts-with(@id, 'street_')] | //div[starts-with(@id, 'billingAddress_')]//input[starts-with(@id, 'street_')] | ||
− | Another example | + | Another example to locate the "delete" button in the currently selected row of a listbox using ID prefixes, CSS-class and text comparison. |
//div[starts-with(@id, 'overviewList_')]//tr[contains(@class, 'z-listitem-seld')]//button[text() = 'delete'] | //div[starts-with(@id, 'overviewList_')]//tr[contains(@class, 'z-listitem-seld')]//button[text() = 'delete'] | ||
− | NOTE: Make sure not too use the "//" operator too extensively in XPath, as it might perform badly | + | NOTE: Make sure not too use the "//" operator too extensively in XPath, as it might perform badly (for me this never was a matter so far, as there are ''much worse'' performance bottle necks, affecting your test execution speed - [[#Waiting for AJAX|discussed later]]). |
− | This [https://www.simple-talk.com/dotnet/.net-framework/xpath,-css,-dom-and-selenium-the-rosetta-stone/ sheet provided by Michael Sorens] is an excellent reference with many helpful examples how to select page elements in various situations. | + | This [https://www.simple-talk.com/dotnet/.net-framework/xpath,-css,-dom-and-selenium-the-rosetta-stone/ sheet provided by Michael Sorens] is an excellent reference with many helpful examples on how to select page elements in various situations. |
So now we can change the test case to use this kind of XPath locator, searching only by ID prefix: | So now we can change the test case to use this kind of XPath locator, searching only by ID prefix: | ||
Line 372: | Line 398: | ||
===Custom Locator and LocatorBuilder=== | ===Custom Locator and LocatorBuilder=== | ||
− | If you don't want to change the locator to XPath manually | + | If you don't want to change the locator to XPath manually every time after you record a test case, Selenium IDE has more to offer. Even improving the readability. |
− | + | ====Custom Locator==== | |
− | : In the user-extensions.js from above you'll find a custom locator that will hide the XPath complexity for simple locate scenarios. It is based on the format of IDs generated by TestingIdGenerator (from above). | + | : In the ''user-extensions.js'' from above you'll find a custom locator that will hide some of the XPath complexity for simple locate scenarios. It is based on the format of IDs generated by TestingIdGenerator (from above). |
− | <source | + | <source lang="javascript"> |
− | // The "inDocument" is | + | // The "inDocument" is the document you are searching. |
PageBot.prototype.locateElementByZkTest = function(text, inDocument) { | PageBot.prototype.locateElementByZkTest = function(text, inDocument) { | ||
// Create the text to search for | // Create the text to search for | ||
Line 398: | Line 424: | ||
</source> | </source> | ||
− | + | This locator will work in 2 forms | |
1. zktest=#login_ | 1. zktest=#login_ | ||
Line 411: | Line 437: | ||
# will also test the elements html tag, and find the <input> element with the ID-prefix "username_..." | # will also test the elements html tag, and find the <input> element with the ID-prefix "username_..." | ||
− | = | + | In addition, it is possible to append any further XPath to narrow down the results. |
+ | |||
+ | e.g. (to locate the second option in a <select> element) | ||
+ | zktest=select#priority_ /option[2] | ||
+ | will become in XPath | ||
+ | //select[starts-with(@id, 'priority_')]/option[2] | ||
+ | |||
+ | Even if this locator is very basic (could be extended if one needs), It already improves readability of many use-cases. | ||
− | + | ====Custom LocatorBuilder==== | |
− | Add file zktest-Selenium-IDE-extension.js to selenium config | + | In order to make this really handy, Selenium IDE offers extensions too ('''don't mix this up with Selenium Core Extension''') |
+ | |||
+ | Add IDE extension file ''extensions/zktest-Selenium-IDE-extension.js'' from example.zip to selenium config | ||
: (Selenium IDE Options > Options... > [General -Tab] > Selenium IDE extensions) | : (Selenium IDE Options > Options... > [General -Tab] > Selenium IDE extensions) | ||
Line 421: | Line 456: | ||
: (Selenium IDE Options > Options... > [Locator Builders - Tab]) | : (Selenium IDE Options > Options... > [Locator Builders - Tab]) | ||
− | It contains the following LocatorBuilder which will generate the locator in form "zktest=elementTag#IdPrefix" mentioned above using only the first part of the ID (the ID prefix - its business name). If an ID does not contain 3 tokens separated by "_" it will use the full ID instead including the sequential number, this will indicate | + | It contains the following LocatorBuilder which will generate the locator in form "zktest=elementTag#IdPrefix" mentioned above using only the first part of the ID (the ID prefix - its business name). If an ID does not contain 3 tokens separated by "_" it will use the full ID instead of including the sequential number, this will indicate that the element has no ID specified and another location approach may be better, or just give it an ID in the zul file. |
− | <source | + | <source lang="javascript"> |
LocatorBuilders.add('zktest', function(e) { | LocatorBuilders.add('zktest', function(e) { | ||
var idTokens = e.id.split("_") | var idTokens = e.id.split("_") | ||
Line 437: | Line 472: | ||
</source> | </source> | ||
− | Restart Selenium | + | Restart Selenium IDE and record the test case again... and you'll get this (after changing the ''type'' to ''typeAndBlur'' events). |
<table cellpadding="1" cellspacing="1" border="1"> | <table cellpadding="1" cellspacing="1" border="1"> | ||
Line 463: | Line 498: | ||
</table> | </table> | ||
− | == | + | =More Tests= |
+ | |||
+ | Let's record a test case for editing and saving an existing "feedback item" in the list, verify the results, and a logout test. | ||
+ | These tests introduce new challenges testing an ajax applications with Selenium - and their solution or workarounds. | ||
+ | |||
+ | Now that we are actually changing data I adapted the login test, to use a random user each time. So that the Mock implementation will serve fresh data for each test run, without having to restart the server, or having to cleanup (there are other strategies possible to do the same - that's just what I use here). | ||
+ | |||
+ | <table cellpadding="1" cellspacing="1" border="1"> | ||
+ | <tr><th rowspan="1" colspan="3">login test</th></tr> | ||
+ | <tr> | ||
+ | <td>store</td> | ||
+ | <td>javascript{'testUser'+Math.floor(Math.random()*10000000)}</td> | ||
+ | <td>username</td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>open</td> | ||
+ | <td>/selenium_testing/login.zul</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>typeAndBlur</td> | ||
+ | <td>zktest=input#username_</td> | ||
+ | <td>${username}</td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>typeAndBlur</td> | ||
+ | <td>zktest=input#password_</td> | ||
+ | <td>${username}</td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>clickAndWait</td> | ||
+ | <td>zktest=button#login_</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>assertLocation</td> | ||
+ | <td>*/selenium_testing/index.zul</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>verifyTextPresent</td> | ||
+ | <td>logout ${username}</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | </table> | ||
+ | |||
+ | ==Edit and Save Test== | ||
+ | |||
+ | After recording and applying the '''typeAndBlur''' actions we get the following. | ||
+ | |||
+ | <table cellpadding="1" cellspacing="1" border="1"> | ||
+ | <tr><th rowspan="1" colspan="3">edit save test</th></tr> | ||
+ | <tr> | ||
+ | <td>open</td> | ||
+ | <td>/selenium_testing/index.zul</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>zktest=#button_27</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>typeAndBlur</td> | ||
+ | <td>zktest=input#subject_</td> | ||
+ | <td>Subject 1 (updated)</td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>typeAndBlur</td> | ||
+ | <td>zktest=input#topic_</td> | ||
+ | <td>consulting</td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>select</td> | ||
+ | <td>zktest=select#priority_</td> | ||
+ | <td>label=low</td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>zktest=#listitem_62</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>typeAndBlur</td> | ||
+ | <td>zktest=textarea#comment_</td> | ||
+ | <td>test comment (content also updated)</td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>zktest=button#submit_</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>assertTextPresent</td> | ||
+ | <td>Feedback successfully updated</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>verifyTextPresent</td> | ||
+ | <td>subject=Subject 1 (updated)</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>verifyTextPresent</td> | ||
+ | <td>[text=test comment (content also updated)]</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>verifyTextPresent</td> | ||
+ | <td>priority=low</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>zktest=button#backToOverview_</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>assertTextPresent</td> | ||
+ | <td>Feedback Overview</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | </table> | ||
+ | |||
+ | We still see some hard-coded numbers. I'll just delete the "click zktest=#listitem_62" as the "select" will command will just do fine (Selenium IDE will sometimes record more than you need, and sometimes not enough - by default). | ||
+ | |||
+ | Interesting for now is the locator "zktest=#button_27" - one of the the "edit" button. If we try to give it an ID in the zul code it will fail to render (duplicate ID), because the button is rendered multiple times - once for each <listitem>. (one workaround could be to surround it by an <idspace> component or include the buttons from a different file). But it is not necessary - using the right locator XPath. | ||
+ | |||
+ | ===Locating an Element inside a Listbox=== | ||
+ | |||
+ | If you just want to locate the first edit button in the list you can use (it will find the button by its text and not by its ID): | ||
+ | |||
+ | pure XPath: | ||
+ | //*[starts-with(@id, 'overviewList_')]//button[text() = 'edit'] | ||
+ | |||
+ | or combine with the "zktest" selector from the selenium extension above | ||
+ | zktest=#overviewList_ //button[text() = 'edit'] | ||
+ | |||
+ | more interesting is it to locate by index, e.g. exactly the second edit button | ||
+ | xpath=(//*[starts-with(@id, 'overviewList_')]//button[text() = 'edit'])[2] | ||
+ | |||
+ | or even more fun to select the "edit" button inside a <listitem> containing a specific text in a cell | ||
+ | xpath=//*[starts-with(@id, 'overviewList_')]//*[starts-with(@id, 'listitem_')][.//*[starts-with(@id, 'listcell_')][text() = 'Subject 2']]//button[text() = 'edit'] | ||
+ | This can get infinitely complex, and reduce the readability of your testcase. That's why it is also a good idea to write '''comments''' (see screenshot below) in your test case (yes, test cases can have comments too). | ||
+ | |||
+ | Even though complex this last locator is still quite stable to changes in the UI. Especially when the <listbox> content changes, you can still use it to find the same <listitem> (and its contained button) several times in a test case. | ||
+ | |||
+ | ===Using Variables=== | ||
+ | |||
+ | Repeating those complex locators in the test scripts will give you a headache maintaining them, when e.g. the label of the button changes from "edit" to "update". It is better to "store" locator strings or parts of them in variables, which can be reused throughout the test. | ||
+ | |||
+ | [[File:variables_and_comments.png|variables and comments example]] | ||
+ | |||
+ | These are just a few general ideas, about what can be done to reduce the work in maintaining test cases, increasing stability or make them more readable - at reasonable investment when setting them up (recording and adjusting) initially. | ||
+ | |||
+ | ==Waiting for AJAX== | ||
+ | |||
+ | However rerunning the test suite at full speed (which is what we should always aim for) will or may sometimes - fail, because we don't wait long enough for ajax responses to render... there is no full page reload so Selenium IDE does not wait automatically. | ||
+ | |||
+ | An idea might be to '''reduce the test speed'''... which would affect every step in all test cases, so we try to avoid that. Also in our feedback example there is a longer operation when clicking the "submit" button (2 seconds). Delaying every Step by 2 seconds would be devastating for our overall replay duration. | ||
+ | |||
+ | Another possibility is using the '''pause''' command and specify the number of milliseconds to wait. Also here the execution speed might vary so we are either waiting too short, so that errors still occur, or we wait too long ("just to be sure") and waste time - both not desirable. | ||
+ | |||
+ | Luckily Selenium comes with a variety of '''waitFor...''' actions, which stops the execution until a condition is met. A useful subsection is: | ||
+ | * waitForTextPresent - wait until a text is present anywhere on the page | ||
+ | * waitForText - wait until an element matching a locator has the text | ||
+ | * waitForElementPresent - wait until an element matched by a locator becomes rendered | ||
+ | * waitForVisible - wait until an element matched by a locator becomes visible | ||
+ | |||
+ | The previously recorded "edit save test" would likely fail in line 3 just after the click on the edit button. | ||
+ | Between Step 2 and 3 ZK is updating parts of the page using AJAX, causing a short delay (but long enough for our test to fail - when running at full speed). | ||
+ | |||
+ | <table cellpadding="1" cellspacing="1" border="1"> | ||
+ | <tr><th rowspan="1" colspan="3">edit save test</th></tr> | ||
+ | <tr> | ||
+ | <td>open</td> | ||
+ | <td>/selenium_testing/index.zul</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>zktest=#overviewList_ //button[text() = 'edit']</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>typeAndBlur</td> | ||
+ | <td>zktest=input#subject_</td> | ||
+ | <td>Subject 1 (updated)</td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>typeAndBlur</td> | ||
+ | <td>zktest=input#topic_</td> | ||
+ | <td>consulting</td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>select</td> | ||
+ | <td>zktest=select#priority_</td> | ||
+ | <td>label=low</td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>typeAndBlur</td> | ||
+ | <td>zktest=textarea#comment_</td> | ||
+ | <td>test comment (content also updated)</td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>zktest=button#submit_</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>assertTextPresent</td> | ||
+ | <td>Feedback successfully updated</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>...</td> | ||
+ | <td>...</td> | ||
+ | <td>...</td> | ||
+ | </tr> | ||
+ | </table> | ||
+ | |||
+ | So we have to wait e.g. until the new Headline "New/Edit Feedback" is present that the AJAX update has finished. | ||
+ | And wait after submitting the feedback article accordingly - replace the "assertTextPresent" with "waitForTextPresent". Also the last assert... can be replaced. | ||
+ | |||
+ | <table cellpadding="1" cellspacing="1" border="1"> | ||
+ | <tr><th rowspan="1" colspan="3">edit save test</th></tr> | ||
+ | <tr> | ||
+ | <td>open</td> | ||
+ | <td>/selenium_testing/index.zul</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>zktest=#overviewList_ //button[text() = 'edit']</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>waitForTextPresent</td> | ||
+ | <td>New/Edit Feedback</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>typeAndBlur</td> | ||
+ | <td>zktest=input#subject_</td> | ||
+ | <td>Subject 1 (updated)</td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>typeAndBlur</td> | ||
+ | <td>zktest=input#topic_</td> | ||
+ | <td>consulting</td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>select</td> | ||
+ | <td>zktest=select#priority_</td> | ||
+ | <td>label=low</td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>typeAndBlur</td> | ||
+ | <td>zktest=textarea#comment_</td> | ||
+ | <td>test comment (content also updated)</td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>zktest=button#submit_</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>waitForTextPresent</td> | ||
+ | <td>Feedback successfully updated</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>verifyTextPresent</td> | ||
+ | <td>subject=Subject 1 (updated)</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>verifyTextPresent</td> | ||
+ | <td>[text=test comment (content also updated)]</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>verifyTextPresent</td> | ||
+ | <td>priority=low</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>zktest=button#backToOverview_</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>waitForTextPresent</td> | ||
+ | <td>Feedback Overview</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | </table> | ||
+ | |||
+ | Now this test can run as fast as possible adapting automatically, to any speedup, or slowdown of the test server. | ||
− | + | It is also possible to '''implicitly wait''' in general for AJAX to finish at a technical level [[http://agilesoftwaretesting.com/selenium-wait-for-ajax-the-right-way/ covered here]]. Personally I think it is tempting at first. After a second thought, I prefer the idea to '''explicitly wait''' (via waitForXXX) for your expected results - only when there is need to wait. | |
− | = | + | Like that you find out directly that something on your page has affected the performance (= user experience) - in case the test suddenly fails after a change affecting the responsiveness of your application. |
− | ==automatically | + | Additionally, there might be another delay after the AJAX response has finished, until the expected component is finally rendered on the page or visible (e.g. due to an animation). So this approach may still fail - just bear that in mind. |
− | ===different browsers | + | |
+ | ==Logout Test== | ||
+ | |||
+ | This test should be very simple, but still there is a catch. When trying to record this test, we notice that Seleniume-IDE won't record clicking on the logout button. Selenium IDE is not very predictable about which events are recorded and which are ignored. In this case the it is a <toolbarbutton> which is rendered as a <div> so maybe that's why. | ||
+ | |||
+ | (There is a [[http://stackoverflow.com/questions/4203559/some-mouse-clicks-dont-register-in-selenium stackoverflow question about this]] and also an [[http://wiki.openqa.org/display/SIDE/Contributed+Extensions+and+Formats#ContributedExtensionsandFormats-Recordingeveryclicksonpage IDE extension]] available to enable recording of ALL clicks in a test. I tried this one - and didn't like the big number of clicks recorded. In the end I leave it up to you to uncomment this in the [[#Custom LocatorBuilder|IDE extension for Locator Builder]] provided above and judge yourself - maybe you want to test exactly this.) | ||
+ | |||
+ | As we know its name (or we can easily find out with tools like firebug) we just add another click event manually and change the locator to "zktest=div#logout_" | ||
+ | |||
+ | <table cellpadding="1" cellspacing="1" border="1"> | ||
+ | <tr><th rowspan="1" colspan="3">logout test</th></tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>zktest=div#logout_</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>waitForTextPresent</td> | ||
+ | <td>Login (with same username as password)</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>assertLocation</td> | ||
+ | <td>*/selenium_testing/login.zul</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | </table> | ||
+ | |||
+ | =Other Components Examples= | ||
+ | Other components require some extra attention too when recording events on them (sometimes you might wonder why the results are varying). | ||
+ | So here are just a few examples. | ||
+ | ==Button (mold="trendy")== | ||
+ | The "new/edit page" of the example application contains 2 submit buttons - rendered using different molds: "default" and "trendy". Both have the same effect (save the feedback item). When you try to record the click event of the second one - labelled "submitTrendy" - Selenium IDE just ignores the click. | ||
+ | |||
+ | Funny thing is, when you first focus another window on your desktop and then click the button directly - without prior focusing the browser window - it gets recorded. But our custom locator builder does not match here. So we get some other generic locator strings. (The "trendy" button is rendered using a layout table and images for the round corners.) | ||
+ | |||
+ | [[File:button-trendy-record.png|button trendy recorded locator]] | ||
+ | |||
+ | The second choice contains the component ID, we can use to build our own locator string. | ||
+ | zktest=#submitTrendy_ | ||
+ | This would usually be sufficient but not in this example. When we test it, clicking on the "Find" button in Selenium IDE we see it is selecting the whole cell around it. ZK has generated some html elements of the layout grid around our button sharing the same ID prefix. | ||
+ | Inspecting the element in the browser we can see the element we really want to click and that it is an <span> element with a style class "z-button" | ||
+ | <source><span id="submitTrendy_button_129" class="z-button"></source> | ||
+ | |||
+ | So a more specific locator would look like this | ||
+ | zktest=span#submitTrendy_ | ||
+ | or that | ||
+ | zktest=#submitTrendy_ [@class = "z-button"] | ||
+ | |||
+ | equivalent XPaths are: | ||
+ | //span[starts-with(@id, 'submitTrendy_')] | ||
+ | //*[starts-with(@id, 'submitTrendy_')][@class="z-button"] | ||
+ | |||
+ | ==Menu== | ||
+ | Recording events on a <menubar> (even with submenus) is generally very simple. Unless you click the "wrong" areas while recording highlighted in red see image below. If you just click the menu(item)-texts (green) Selenium IDE will create nice locators, otherwise it will fallback to some other strategy, using CSS or generated XPath which might surprise you (this is no bug in Selenium IDE, it is just a different html element receiving the click event). | ||
+ | |||
+ | <table cellpadding="1" cellspacing="1" border="1"> | ||
+ | <tr><th rowspan="1" colspan="3">menu test (not working for us)</th></tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>css=td.z-menu-inner-m > div</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>css=span.z-menuitem-img</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | </table> | ||
+ | |||
+ | [[File:menu-areas-to-avoid.png|avoid red areas, and click on the green areas]] | ||
+ | |||
+ | What you get avoiding the '''red''' areas. | ||
+ | |||
+ | <table cellpadding="1" cellspacing="1" border="1"> | ||
+ | <tr><th rowspan="1" colspan="3">menu test (what we want)</th></tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>zktest=button#feedbackMenu_</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>zktest=a#newFeedbackMenuItem_</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | </table> | ||
+ | |||
+ | ==Tabbox== | ||
+ | |||
+ | When all your <tab>s and <tabpanel>s (and nested components) have unique IDs everything is simple, but when it comes to dynamically added tabs without nice Ids things get a bit tricky. | ||
+ | |||
+ | So in our example some manual work is required to record interactions with the "comments" tabbox. | ||
+ | |||
+ | The "new comment"-button is again a toolbarbutton like the "logout"-button, so its events are not recorded automatically to create a new comment-tab just add this "click zktest=div#newComment_" manually will create a new tab. Now how to activate the new tab (Comment 1). | ||
+ | |||
+ | ===Locating a Tab=== | ||
+ | |||
+ | Automatic recording will give us this, which will work for now, and fail again after a change to the page. | ||
+ | click zktest=#tab_261-cnt | ||
+ | |||
+ | To avoid the hard-coded index, and truly being sure the second tab (inside our comments tabbox) is selected I would prefer this | ||
+ | |||
+ | //div[starts-with(@id, 'comments_')]//li[starts-with(@id, 'tab_')][2] | ||
+ | or (locating by label) | ||
+ | //div[starts-with(@id, 'comments_')]//li//span[text() = 'Comment 1'] | ||
+ | |||
+ | It could be simplified increasing the chance of possible future failure (if there is another li inside the "comments" tabbox) using the custom zktest locator | ||
+ | zktest=div#comments_ //li//span[text() = 'Comment 1'] | ||
+ | |||
+ | ===Locating a Tabpanel=== | ||
+ | |||
+ | Locating e.g. the <textarea> to edit the comment of the currently selected tab is somewhat more difficult, as the <tabpanels> component is not in the same branch of the Browser-DOM tree as the <tabs> component. | ||
+ | |||
+ | Easiest is to locate by index (if you know it): | ||
+ | //div[starts-with(@id, 'comments_')]//div[starts-with(@id, 'tabpanel_')][2]//textarea | ||
+ | or | ||
+ | zktest=div#comments_ //div[starts-with(@id, 'tabpanel_')][2]//textarea | ||
+ | |||
+ | Selecting by label of selected Tab is somewhat complex (I am sure there is a simpler solution to it - if you know feel free to share): | ||
+ | xpath=(//div[starts-with(@id, 'comments_')]//textarea[starts-with(@id, 'comment_')])[count(//div[starts-with(@id, 'comments_')]//li[.//span[text() = 'Comment 2']]/preceding-sibling::*)+1] | ||
+ | |||
+ | Once again we see locators are very flexible - just be creative. | ||
+ | |||
+ | ==Combobox== | ||
+ | |||
+ | Comboboxes are also very dynamic components. So if you just want to test your page, it is easiest to just "typeAndBlur" a value into them. | ||
+ | |||
+ | typeAndBlur zktest=input#topic_ theValue | ||
+ | |||
+ | If you really need to test the <comboitems> are populated correctly, my first suggestion is to test your ListSubModel implementation in a unit test in pure java code. But if you really really need to test this in a page test you can select the combobox-dropdown button with: | ||
+ | |||
+ | zktest=i#topic_ /i | ||
+ | or | ||
+ | zktest=i#topic_ [contains(@id, '-btn')] | ||
+ | XPath equivalents | ||
+ | //i[starts-with(@id, 'topic_')]/i | ||
+ | or | ||
+ | //i[starts-with(@id, 'topic_')][contains(@id, '-btn')] | ||
+ | |||
+ | To select the "consulting" <comboitem> - via mouse - you could use the following: | ||
+ | |||
+ | <table cellpadding="1" cellspacing="1" border="1"> | ||
+ | <tr><th rowspan="1" colspan="3">New Test</th></tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>zktest=i#topic_ /i</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>waitForVisible</td> | ||
+ | <td>zktest=div#topic_ /table</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <td>click</td> | ||
+ | <td>zktest=div#topic_ //tr[starts-with(@id, 'comboitem_')]//td[text() = 'consulting']</td> | ||
+ | <td></td> | ||
+ | </tr> | ||
+ | </table> | ||
+ | |||
+ | In many cases there is no general recipe, about what is right. In most cases it helps to inspect the page source and do what you need. | ||
+ | |||
+ | =Putting it all together= | ||
+ | The example package contains a running test suite featuring all the discussed topics from above. | ||
+ | |||
+ | It contains 4 test cases. 3 from above and a longer test case "new re-edit test" showing some more complex component interactions. | ||
+ | * login test | ||
+ | * edit save test | ||
+ | * new re-edit test | ||
+ | * logout test | ||
+ | |||
+ | To see it running you need to: | ||
+ | # Unpack the [[#Download|downloaded]] zip file | ||
+ | # Launch the testing application in test mode (''/selenium_testing_app/launch_for_TEST.bat'' or ''selenium_testing_app/launch_for_TEST.sh'') | ||
+ | # Open the Selenium IDE in Firefox ([CTRL+ALT+S]) | ||
+ | # Make sure the extensions are configured (''/extensions/user-extension.js'') - if not, configure them, close and restart Selenium IDE | ||
+ | # In Selenium IDE open the sample test suite (''/test_suite/suite.html'') | ||
+ | # Check the Base URL - should be ''http://localhost:8080/'' | ||
+ | # Click the button "Play entire Test Suite" | ||
+ | # Lean back and watch... | ||
+ | |||
+ | =Run against different Browsers using Selenium Server= | ||
+ | |||
+ | There are many ways of running your tests against different browsers using Selenium Server. As this is not the focus of this document, here are just a few command line examples to see if your suite will actually run outside of Selenium IDE using [http://docs.seleniumhq.org/download/ Selenium Server 2.33] (while writing this document the latest version 2.33 only supports Firefox up to version 21.0) | ||
+ | |||
+ | Place the ''selenium-server-standalone-2.33.0.jar'' in the folder where you unzipped this example and run a command prompt at the same path: | ||
+ | (of course you might have to adapt the paths of your browser executables) | ||
+ | |||
+ | execute with Firefox '''21.0''': | ||
+ | java -jar selenium-server-standalone-2.33.0.jar -userExtensions extensions/user-extensions.js -htmlSuite "*firefox C:\Program Files (x86)\Mozilla Firefox 21\firefox.exe" "http://localhost:8080" test_suite/suite.html test_suite/results.html | ||
+ | |||
+ | execute with google chrome: | ||
+ | java -jar selenium-server-standalone-2.33.0.jar -userExtensions extensions/user-extensions.js -htmlSuite "*googlechrome C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" "http://localhost:8080" test_suite/suite.html test_suite/results.html | ||
+ | |||
+ | =Appendix= | ||
+ | ==Download== | ||
+ | [http://sourceforge.net/projects/zkforge/files/Small%20Talks/ZK%20Testing%20with%20Selenium%20IDE/Selenium-IDE-example.zip/download selenium-IDE-example.zip] | ||
+ | |||
+ | =Comments= | ||
+ | {{#tag:comment | ||
+ | |http://books.zkoss.org/wiki/{{SUBJECTPAGENAME}}|disqus=1 | ||
+ | |}} | ||
+ | |||
+ | [[category: Unit Test|2]] | ||
+ | [[Category:ZK]] | ||
+ | [[Category:Performance]] | ||
+ | |||
+ | {{Template:Smalltalk_Footer| | ||
+ | |name=Potix Corporation | ||
+ | }} |
Latest revision as of 02:29, 7 February 2023
Robert Wenzel, Engineer, Potix Corporation
July 30, 2013
ZK 6.5 (or later)
Introduction
So you have your nice ZK application and are interested in, how to be sure it will work in different browsers (and keep working after changes, which always happen). Of course you have tested everything possible with your unit and integration tests ;-). You have maxed out the possibilities offered by ZATS to ensure ZK components play well together in a simulated environment.
Finally you come to a point when it is necessary to ensure your application (or at least the key features) works and renders well in a real environment (with different browsers, real application server, real network traffic/latency etc.) - so eventually Selenium comes into play.
About this article
There are several approaches in creating Selenium Tests:
The programmatic way - suitable for developers - is already covered in this Small Talk. This way offers maximum control over the test execution to write reusable, flexible, maintainable and even "intelligent" test cases. Of course writing such test cases may require some time and resources.
Sometimes it is up to a less technical person to create these test cases. This is when one often hears about "record and replay" tools. Selenium IDE is not only such a tool, it has record and replay features but also many more options to tweak and extend your test cases after recording. In few cases the recorded tests can be instantly re-playable. However, especially when it comes to AJAX there are several issues, that will prevent your test cases from executing out-of-the-box.
Here are two examples (you can easily find more):
- problems with Missing clicks
- someone recorded this video explaining several issues
But not to worry, this article will show you how to deal with these!
The major conclusion is: See Selenium IDE more as a record/tweak/maintain and replay/debug tool to build your tests cases.
This article will show (using an example web application) how to:
- enable useful initial test recording in ZK
- tweak the tests so that they replay
- keep your tests maintainable and robust to changes in your pages
- add missing actions that fail to record at all
Many ideas in the Article are based on solutions found for general problems in several forums and previous project experiences. Some simply reflects my personal preference on how things may be handled. This article does not show the only way to write tests for a ZK application, just highlight problems and solutions, and discuss some considerations taken and why this approach was chosen.
Happy reading and commenting !!
Environment
- Firefox 22
- Selenium IDE 2.1.0 (Selenium IDE 2.0.0 is not working in FF >=22)
- JDK 6/7
- ZK 6.5.3
- Maven 3.0
Steps
Initialize your environment for this example
- Install Selenium IDE plugin in Firefox via Addons or download documentation page
- Maven 3.0 download & install
- download the example project, it contains the following folders:
- selenium_example/ (the example application, maven project)
- extensions/ (the selenium extensions used in this article)
- test_suite/ (a Selenium Test Suite illustrating the contepts discussed here)
- run on command line
cd selenium_example_app mvn jetty:run
- or execute selenium_testing_app/launch_for_PROD.bat / .sh
- open test app http://localhost:8080/selenium_testing
Initial Situation
- Launch Firefox
- Launch Selenium IDE CTRL+ALT+S
- Set the base URL to "http://localhost:8080/"
- open example app http://localhost:8080/selenium_testing
- Record login-test and try to replay it
New Test | ||
open | /selenium_testing/login.zul | |
type | id=jXDW5 | test |
type | id=jXDW8 | test |
clickAndWait | id=jXDWb |
Looks hard to read / maintain and doesn't even work :'(
Looking at the recorded commands and comparing the IDs with the page source we notice the IDs are generated and changing with every page refresh. So no chance to record / replay this way.
A way to make the IDs predictable is shown in the next Section.
Custom Id Generator
As mentioned in other Small Talks about testing, one strategy is to use a custom IdGenerator implementation to create predictable, readable (with a business meaning) and easily selectable (by selenium) component IDs.
public class TestingIdGenerator implements IdGenerator {
public String nextComponentUuid(Desktop desktop, Component comp,
ComponentInfo compInfo) {
int i = Integer.parseInt(desktop.getAttribute("Id_Num").toString());
i++;// Start from 1
StringBuilder uuid = new StringBuilder("");
desktop.setAttribute("Id_Num", String.valueOf(i));
if(compInfo != null) {
String id = getId(compInfo);
if(id != null) {
uuid.append(id).append("_");
}
String tag = compInfo.getTag();
if(tag != null) {
uuid.append(tag).append("_");
}
}
return uuid.length() == 0 ? "zkcomp_" + i : uuid.append(i).toString();
}
...
//omitted code see download section Selenium-IDE-example.zip:
//selenium_testing_app/src/main/java/org/zkoss/selenium_example/TestingIdGenerator.java)
The IDs will look like this:
- {id}_{tag}_{##} (for components with a given id)
- {tag}_{##} (for components without a given id - e.g. listitems in a listbox)
- {zkcomp}_{##} (for other cases, just to make them unique)
e.g. a button in zul-file
<button id="login" label="login" />
will become something like this in HTML in browser (number can vary):
<button id="login_button_12" class="z-button-os" type="button">login</button>
In order to separate production from test configuration we can include an additional config file at startup to enable the TestingIdGenerator only for testing. In ZK this is possible by setting the library property org.zkoss.zk.config.path (also refer to our Testing Tips)
- e.g. via VM argument -Dorg.zkoss.zk.config.path=/WEB-INF/zk-test.xml
Windows:
set MAVEN_OPTS=-Dorg.zkoss.zk.config.path=/WEB-INF/zk-test.xml mvn jetty:run
Linux:
export MAVEN_OPTS=-Dorg.zkoss.zk.config.path=/WEB-INF/zk-test.xml mvn jetty:run
or execute selenium_testing_app/launch_for_TEST.bat / .sh
Now we get nicer and deterministic IDs when recording a test case.
After recording login-test again and we get this:
login test | ||
---|---|---|
open | /selenium_testing/login.zul | |
type | id=username_textbox_6 | test |
type | id=password_textbox_9 | test |
clickAndWait | id=login_button_12 |
This already looks nice, and self explaining - but unfortunately still fails to replay
Why this happens and how to fix it? See next section ...
ZK specific details
Due to the nature of rich web applications, JavaScript is generally heavily used, but Selenium IDE does not record all events by default. In our case here the "blur" events of the input fields have not been recorded, but ZK relies on them to determine updated fields. So, we need to manually add selenium commands "fireEvent target blur".Unfortunately Selenium does not offer a nice equivalent like "focus" action.
login test | ||
---|---|---|
open | /selenium_testing/login.zul | |
type | id=username_textbox_6 | test |
fireEvent | id=username_textbox_6 | blur |
type | id=password_textbox_9 | test |
fireEvent | id=password_textbox_9 | blur |
clickAndWait | id=login_button_12 | |
assertLocation | */selenium_testing/index.zul | |
verifyTextPresent | Welcome test | |
verifyTextPresent | Feedback Overview |
Replaying the script now works :D, and we see the overview page (also added a few verifications for that).
Selenium Extensions
If you find the syntax of "fireEvent" hard to remember or think it is too inconvenient to add an additional line just to update an input field there's help using a Selenium Core extension. The extension - file usually called user-extensions.js can be used in both Selenium IDE and Selenium RC when running the tests outside of Selenium IDE (please refer to selenium documentation).
This snippet shows 2 simple custom actions:
- blur
- convenient action to avoid "fireEvent locator blur"
- typeAndBlur
- combines the type with a blur event, to make ZK aware of the input change automatically
Selenium.prototype.doBlur = function(locator) {
// All locator-strategies are automatically handled by "findElement"
var element = this.page().findElement(locator);
// Fire the "blur" event
this.doFireEvent(locator, "blur")
};
Selenium.prototype.doTypeAndBlur = function(locator, text) {
// All locator-strategies are automatically handled by "findElement"
var element = this.page().findElement(locator);
// Replace the element text with the new text
this.page().replaceText(element, text);
// Fire the "blur" event
this.doFireEvent(locator, "blur")
};
To enable it just add the file (extensions/user-extension.js in the example zip) to the Selenium IDE configuration. (the file contains another extension ... more about when discussing custom locators)
- (Selenium IDE Options > Options... > [General -Tab] > Selenium Core extensions)
Then restart Selenium IDE - close the window, and reopen it - e.g. by [CTRL+ALT+S]
Adapted test using the blur action:
login test | ||
---|---|---|
open | /selenium_testing/login.zul | |
type | id=username_textbox_6 | test |
blur | id=username_textbox_6 | |
type | id=password_textbox_9 | test |
blur | id=password_textbox_9 | |
clickAndWait | id=login_button_12 |
Same test using the typeAndBlur:
login test | ||
---|---|---|
open | /selenium_testing/login.zul | |
typeAndBlur | id=username_textbox_6 | test |
typeAndBlur | id=password_textbox_9 | test |
clickAndWait | id=login_button_12 |
It is not required to use either of these extensions, but saving 1 line for each input is my personal preference.
Another thing to keep in mind is robustness and maintenance effort of test-cases when changes in the UI happen ... the generated numbers at the end of each ID are still an obstacle in this way as they are prone to change every time a component is added or removed above that component.
Wouldn't it be nice to avoid having to adapt test cases that often? Read on...
Improve robustness and maintainability
Locators in Selenium
To remove the hard coded running numbers of the component IDs from our test cases Selenium offers e.g. XPath or CSS locators.
Using XPath is it possible to select a node by its ID-prefix:
//input[starts-with(@id, 'username_')]
This will work as long as the prefix "username_" is unique on the page, otherwise will perform the action on the first element found. So it is a good idea for this scenario to give widgets suitable IDs (plus: it will also improve the readability of source code).
In more complex cases one can select nested components to distinguish components with the same ID (e.g. in different ID spaces). e.g. if the "street" component appears several times on the page use this:
//div[starts-with(@id, 'deliveryAddress_')]//input[starts-with(@id, 'street_')] //div[starts-with(@id, 'billingAddress_')]//input[starts-with(@id, 'street_')]
Another example to locate the "delete" button in the currently selected row of a listbox using ID prefixes, CSS-class and text comparison.
//div[starts-with(@id, 'overviewList_')]//tr[contains(@class, 'z-listitem-seld')]//button[text() = 'delete']
NOTE: Make sure not too use the "//" operator too extensively in XPath, as it might perform badly (for me this never was a matter so far, as there are much worse performance bottle necks, affecting your test execution speed - discussed later).
This sheet provided by Michael Sorens is an excellent reference with many helpful examples on how to select page elements in various situations.
So now we can change the test case to use this kind of XPath locator, searching only by ID prefix:
login test | ||
---|---|---|
open | /selenium_testing/login.zul | |
typeAndBlur | //input[starts-with(@id, 'username_')] | test |
typeAndBlur | //input[starts-with(@id, 'password_')] | test |
clickAndWait | //button[starts-with(@id, 'login_')] |
Done that, the order of the components may change without breaking the test, as long as the actual IDs are not changed. Of course this requires some manual work, but the effort invested once will pay off quickly as your project evolves.
Custom Locator and LocatorBuilder
If you don't want to change the locator to XPath manually every time after you record a test case, Selenium IDE has more to offer. Even improving the readability.
Custom Locator
- In the user-extensions.js from above you'll find a custom locator that will hide some of the XPath complexity for simple locate scenarios. It is based on the format of IDs generated by TestingIdGenerator (from above).
// The "inDocument" is the document you are searching.
PageBot.prototype.locateElementByZkTest = function(text, inDocument) {
// Create the text to search for
var text2 = text.trim();
var separatorIndex = text2.indexOf(" ");
var elementNameAndIdPrefix = (separatorIndex != -1 ? text2.substring(0, separatorIndex) : text2).split("#");
var xpathSuffix = separatorIndex != -1 ? text2.substring(separatorIndex + 1) : "";
var elementName = elementNameAndIdPrefix[0] || "*";
var idPrefix = elementNameAndIdPrefix[1];
var xpath = "//" + elementName + "[starts-with(@id, '" + idPrefix + "')]" + xpathSuffix;
return this.xpathEvaluator.selectSingleNode(inDocument, xpath, null, this._namespaceResolver);
};
This locator will work in 2 forms
1. zktest=#login_ 2. zktest=input#username_
Internally generated/queried XPaths will be
1. //*[starts-with(@id, 'login_')] 2. //input[starts-with(@id, 'username_')]
- will look for any element with an ID starting with "login_"
- will also test the elements html tag, and find the <input> element with the ID-prefix "username_..."
In addition, it is possible to append any further XPath to narrow down the results.
e.g. (to locate the second option in a <select> element)
zktest=select#priority_ /option[2]
will become in XPath
//select[starts-with(@id, 'priority_')]/option[2]
Even if this locator is very basic (could be extended if one needs), It already improves readability of many use-cases.
Custom LocatorBuilder
In order to make this really handy, Selenium IDE offers extensions too (don't mix this up with Selenium Core Extension)
Add IDE extension file extensions/zktest-Selenium-IDE-extension.js from example.zip to selenium config
- (Selenium IDE Options > Options... > [General -Tab] > Selenium IDE extensions)
and move it up in the Locator Builder priority list.
- (Selenium IDE Options > Options... > [Locator Builders - Tab])
It contains the following LocatorBuilder which will generate the locator in form "zktest=elementTag#IdPrefix" mentioned above using only the first part of the ID (the ID prefix - its business name). If an ID does not contain 3 tokens separated by "_" it will use the full ID instead of including the sequential number, this will indicate that the element has no ID specified and another location approach may be better, or just give it an ID in the zul file.
LocatorBuilders.add('zktest', function(e) {
var idTokens = e.id.split("_")
if (idTokens.length > 2) {
idTokens.pop();
idTokens.pop();
return "zktest=" + e.nodeName.toLowerCase() + "#" + idTokens.join("_") + "_";
} else {
return "zktest=" + "#" + e.id;
}
return null;
});
Restart Selenium IDE and record the test case again... and you'll get this (after changing the type to typeAndBlur events).
New Test | ||
---|---|---|
open | /selenium_testing/login.zul | |
typeAndBlur | zktest=input#username_ | test |
typeAndBlur | zktest=input#password_ | test |
clickAndWait | zktest=button#login_ |
More Tests
Let's record a test case for editing and saving an existing "feedback item" in the list, verify the results, and a logout test. These tests introduce new challenges testing an ajax applications with Selenium - and their solution or workarounds.
Now that we are actually changing data I adapted the login test, to use a random user each time. So that the Mock implementation will serve fresh data for each test run, without having to restart the server, or having to cleanup (there are other strategies possible to do the same - that's just what I use here).
login test | ||
---|---|---|
store | javascript{'testUser'+Math.floor(Math.random()*10000000)} | username |
open | /selenium_testing/login.zul | |
typeAndBlur | zktest=input#username_ | ${username} |
typeAndBlur | zktest=input#password_ | ${username} |
clickAndWait | zktest=button#login_ | |
assertLocation | */selenium_testing/index.zul | |
verifyTextPresent | logout ${username} |
Edit and Save Test
After recording and applying the typeAndBlur actions we get the following.
edit save test | ||
---|---|---|
open | /selenium_testing/index.zul | |
click | zktest=#button_27 | |
typeAndBlur | zktest=input#subject_ | Subject 1 (updated) |
typeAndBlur | zktest=input#topic_ | consulting |
select | zktest=select#priority_ | label=low |
click | zktest=#listitem_62 | |
typeAndBlur | zktest=textarea#comment_ | test comment (content also updated) |
click | zktest=button#submit_ | |
assertTextPresent | Feedback successfully updated | |
verifyTextPresent | subject=Subject 1 (updated) | |
verifyTextPresent | [text=test comment (content also updated)] | |
verifyTextPresent | priority=low | |
click | zktest=button#backToOverview_ | |
assertTextPresent | Feedback Overview |
We still see some hard-coded numbers. I'll just delete the "click zktest=#listitem_62" as the "select" will command will just do fine (Selenium IDE will sometimes record more than you need, and sometimes not enough - by default).
Interesting for now is the locator "zktest=#button_27" - one of the the "edit" button. If we try to give it an ID in the zul code it will fail to render (duplicate ID), because the button is rendered multiple times - once for each <listitem>. (one workaround could be to surround it by an <idspace> component or include the buttons from a different file). But it is not necessary - using the right locator XPath.
Locating an Element inside a Listbox
If you just want to locate the first edit button in the list you can use (it will find the button by its text and not by its ID):
pure XPath:
//*[starts-with(@id, 'overviewList_')]//button[text() = 'edit']
or combine with the "zktest" selector from the selenium extension above
zktest=#overviewList_ //button[text() = 'edit']
more interesting is it to locate by index, e.g. exactly the second edit button
xpath=(//*[starts-with(@id, 'overviewList_')]//button[text() = 'edit'])[2]
or even more fun to select the "edit" button inside a <listitem> containing a specific text in a cell
xpath=//*[starts-with(@id, 'overviewList_')]//*[starts-with(@id, 'listitem_')][.//*[starts-with(@id, 'listcell_')][text() = 'Subject 2']]//button[text() = 'edit']
This can get infinitely complex, and reduce the readability of your testcase. That's why it is also a good idea to write comments (see screenshot below) in your test case (yes, test cases can have comments too).
Even though complex this last locator is still quite stable to changes in the UI. Especially when the <listbox> content changes, you can still use it to find the same <listitem> (and its contained button) several times in a test case.
Using Variables
Repeating those complex locators in the test scripts will give you a headache maintaining them, when e.g. the label of the button changes from "edit" to "update". It is better to "store" locator strings or parts of them in variables, which can be reused throughout the test.
These are just a few general ideas, about what can be done to reduce the work in maintaining test cases, increasing stability or make them more readable - at reasonable investment when setting them up (recording and adjusting) initially.
Waiting for AJAX
However rerunning the test suite at full speed (which is what we should always aim for) will or may sometimes - fail, because we don't wait long enough for ajax responses to render... there is no full page reload so Selenium IDE does not wait automatically.
An idea might be to reduce the test speed... which would affect every step in all test cases, so we try to avoid that. Also in our feedback example there is a longer operation when clicking the "submit" button (2 seconds). Delaying every Step by 2 seconds would be devastating for our overall replay duration.
Another possibility is using the pause command and specify the number of milliseconds to wait. Also here the execution speed might vary so we are either waiting too short, so that errors still occur, or we wait too long ("just to be sure") and waste time - both not desirable.
Luckily Selenium comes with a variety of waitFor... actions, which stops the execution until a condition is met. A useful subsection is:
- waitForTextPresent - wait until a text is present anywhere on the page
- waitForText - wait until an element matching a locator has the text
- waitForElementPresent - wait until an element matched by a locator becomes rendered
- waitForVisible - wait until an element matched by a locator becomes visible
The previously recorded "edit save test" would likely fail in line 3 just after the click on the edit button. Between Step 2 and 3 ZK is updating parts of the page using AJAX, causing a short delay (but long enough for our test to fail - when running at full speed).
edit save test | ||
---|---|---|
open | /selenium_testing/index.zul | |
click | zktest=#overviewList_ //button[text() = 'edit'] | |
typeAndBlur | zktest=input#subject_ | Subject 1 (updated) |
typeAndBlur | zktest=input#topic_ | consulting |
select | zktest=select#priority_ | label=low |
typeAndBlur | zktest=textarea#comment_ | test comment (content also updated) |
click | zktest=button#submit_ | |
assertTextPresent | Feedback successfully updated | |
... | ... | ... |
So we have to wait e.g. until the new Headline "New/Edit Feedback" is present that the AJAX update has finished. And wait after submitting the feedback article accordingly - replace the "assertTextPresent" with "waitForTextPresent". Also the last assert... can be replaced.
edit save test | ||
---|---|---|
open | /selenium_testing/index.zul | |
click | zktest=#overviewList_ //button[text() = 'edit'] | |
waitForTextPresent | New/Edit Feedback | |
typeAndBlur | zktest=input#subject_ | Subject 1 (updated) |
typeAndBlur | zktest=input#topic_ | consulting |
select | zktest=select#priority_ | label=low |
typeAndBlur | zktest=textarea#comment_ | test comment (content also updated) |
click | zktest=button#submit_ | |
waitForTextPresent | Feedback successfully updated | |
verifyTextPresent | subject=Subject 1 (updated) | |
verifyTextPresent | [text=test comment (content also updated)] | |
verifyTextPresent | priority=low | |
click | zktest=button#backToOverview_ | |
waitForTextPresent | Feedback Overview |
Now this test can run as fast as possible adapting automatically, to any speedup, or slowdown of the test server.
It is also possible to implicitly wait in general for AJAX to finish at a technical level [covered here]. Personally I think it is tempting at first. After a second thought, I prefer the idea to explicitly wait (via waitForXXX) for your expected results - only when there is need to wait.
Like that you find out directly that something on your page has affected the performance (= user experience) - in case the test suddenly fails after a change affecting the responsiveness of your application.
Additionally, there might be another delay after the AJAX response has finished, until the expected component is finally rendered on the page or visible (e.g. due to an animation). So this approach may still fail - just bear that in mind.
Logout Test
This test should be very simple, but still there is a catch. When trying to record this test, we notice that Seleniume-IDE won't record clicking on the logout button. Selenium IDE is not very predictable about which events are recorded and which are ignored. In this case the it is a <toolbarbutton> which is rendered as a <div> so maybe that's why.
(There is a [stackoverflow question about this] and also an [IDE extension] available to enable recording of ALL clicks in a test. I tried this one - and didn't like the big number of clicks recorded. In the end I leave it up to you to uncomment this in the IDE extension for Locator Builder provided above and judge yourself - maybe you want to test exactly this.)
As we know its name (or we can easily find out with tools like firebug) we just add another click event manually and change the locator to "zktest=div#logout_"
logout test | ||
---|---|---|
click | zktest=div#logout_ | |
waitForTextPresent | Login (with same username as password) | |
assertLocation | */selenium_testing/login.zul |
Other Components Examples
Other components require some extra attention too when recording events on them (sometimes you might wonder why the results are varying). So here are just a few examples.
Button (mold="trendy")
The "new/edit page" of the example application contains 2 submit buttons - rendered using different molds: "default" and "trendy". Both have the same effect (save the feedback item). When you try to record the click event of the second one - labelled "submitTrendy" - Selenium IDE just ignores the click.
Funny thing is, when you first focus another window on your desktop and then click the button directly - without prior focusing the browser window - it gets recorded. But our custom locator builder does not match here. So we get some other generic locator strings. (The "trendy" button is rendered using a layout table and images for the round corners.)
The second choice contains the component ID, we can use to build our own locator string.
zktest=#submitTrendy_
This would usually be sufficient but not in this example. When we test it, clicking on the "Find" button in Selenium IDE we see it is selecting the whole cell around it. ZK has generated some html elements of the layout grid around our button sharing the same ID prefix. Inspecting the element in the browser we can see the element we really want to click and that it is an element with a style class "z-button"
<span id="submitTrendy_button_129" class="z-button">
So a more specific locator would look like this
zktest=span#submitTrendy_
or that
zktest=#submitTrendy_ [@class = "z-button"]
equivalent XPaths are:
//span[starts-with(@id, 'submitTrendy_')] //*[starts-with(@id, 'submitTrendy_')][@class="z-button"]
Menu
Recording events on a <menubar> (even with submenus) is generally very simple. Unless you click the "wrong" areas while recording highlighted in red see image below. If you just click the menu(item)-texts (green) Selenium IDE will create nice locators, otherwise it will fallback to some other strategy, using CSS or generated XPath which might surprise you (this is no bug in Selenium IDE, it is just a different html element receiving the click event).
menu test (not working for us) | ||
---|---|---|
click | css=td.z-menu-inner-m > div | |
click | css=span.z-menuitem-img |
What you get avoiding the red areas.
menu test (what we want) | ||
---|---|---|
click | zktest=button#feedbackMenu_ | |
click | zktest=a#newFeedbackMenuItem_ |
Tabbox
When all your <tab>s and <tabpanel>s (and nested components) have unique IDs everything is simple, but when it comes to dynamically added tabs without nice Ids things get a bit tricky.
So in our example some manual work is required to record interactions with the "comments" tabbox.
The "new comment"-button is again a toolbarbutton like the "logout"-button, so its events are not recorded automatically to create a new comment-tab just add this "click zktest=div#newComment_" manually will create a new tab. Now how to activate the new tab (Comment 1).
Locating a Tab
Automatic recording will give us this, which will work for now, and fail again after a change to the page.
click zktest=#tab_261-cnt
To avoid the hard-coded index, and truly being sure the second tab (inside our comments tabbox) is selected I would prefer this
//div[starts-with(@id, 'comments_')]//li[starts-with(@id, 'tab_')][2]
or (locating by label)
//div[starts-with(@id, 'comments_')]//li//span[text() = 'Comment 1']
It could be simplified increasing the chance of possible future failure (if there is another li inside the "comments" tabbox) using the custom zktest locator
zktest=div#comments_ //li//span[text() = 'Comment 1']
Locating a Tabpanel
Locating e.g. the <textarea> to edit the comment of the currently selected tab is somewhat more difficult, as the <tabpanels> component is not in the same branch of the Browser-DOM tree as the <tabs> component.
Easiest is to locate by index (if you know it):
//div[starts-with(@id, 'comments_')]//div[starts-with(@id, 'tabpanel_')][2]//textarea
or
zktest=div#comments_ //div[starts-with(@id, 'tabpanel_')][2]//textarea
Selecting by label of selected Tab is somewhat complex (I am sure there is a simpler solution to it - if you know feel free to share):
xpath=(//div[starts-with(@id, 'comments_')]//textarea[starts-with(@id, 'comment_')])[count(//div[starts-with(@id, 'comments_')]//li[.//span[text() = 'Comment 2']]/preceding-sibling::*)+1]
Once again we see locators are very flexible - just be creative.
Combobox
Comboboxes are also very dynamic components. So if you just want to test your page, it is easiest to just "typeAndBlur" a value into them.
typeAndBlur zktest=input#topic_ theValue
If you really need to test the <comboitems> are populated correctly, my first suggestion is to test your ListSubModel implementation in a unit test in pure java code. But if you really really need to test this in a page test you can select the combobox-dropdown button with:
zktest=i#topic_ /i or zktest=i#topic_ [contains(@id, '-btn')]
XPath equivalents
//i[starts-with(@id, 'topic_')]/i or //i[starts-with(@id, 'topic_')][contains(@id, '-btn')]
To select the "consulting" <comboitem> - via mouse - you could use the following:
New Test | ||
---|---|---|
click | zktest=i#topic_ /i | |
waitForVisible | zktest=div#topic_ /table | |
click | zktest=div#topic_ //tr[starts-with(@id, 'comboitem_')]//td[text() = 'consulting'] |
In many cases there is no general recipe, about what is right. In most cases it helps to inspect the page source and do what you need.
Putting it all together
The example package contains a running test suite featuring all the discussed topics from above.
It contains 4 test cases. 3 from above and a longer test case "new re-edit test" showing some more complex component interactions.
- login test
- edit save test
- new re-edit test
- logout test
To see it running you need to:
- Unpack the downloaded zip file
- Launch the testing application in test mode (/selenium_testing_app/launch_for_TEST.bat or selenium_testing_app/launch_for_TEST.sh)
- Open the Selenium IDE in Firefox ([CTRL+ALT+S])
- Make sure the extensions are configured (/extensions/user-extension.js) - if not, configure them, close and restart Selenium IDE
- In Selenium IDE open the sample test suite (/test_suite/suite.html)
- Check the Base URL - should be http://localhost:8080/
- Click the button "Play entire Test Suite"
- Lean back and watch...
Run against different Browsers using Selenium Server
There are many ways of running your tests against different browsers using Selenium Server. As this is not the focus of this document, here are just a few command line examples to see if your suite will actually run outside of Selenium IDE using Selenium Server 2.33 (while writing this document the latest version 2.33 only supports Firefox up to version 21.0)
Place the selenium-server-standalone-2.33.0.jar in the folder where you unzipped this example and run a command prompt at the same path: (of course you might have to adapt the paths of your browser executables)
execute with Firefox 21.0:
java -jar selenium-server-standalone-2.33.0.jar -userExtensions extensions/user-extensions.js -htmlSuite "*firefox C:\Program Files (x86)\Mozilla Firefox 21\firefox.exe" "http://localhost:8080" test_suite/suite.html test_suite/results.html
execute with google chrome:
java -jar selenium-server-standalone-2.33.0.jar -userExtensions extensions/user-extensions.js -htmlSuite "*googlechrome C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" "http://localhost:8080" test_suite/suite.html test_suite/results.html
Appendix
Download
Comments
Copyright © Potix Corporation. This article is licensed under GNU Free Documentation License. |