Archive for August 2008
We are proud to announce the release of LDTP 1.3.0. This release features number of important breakthroughs in LDTP as well as in the field of Test Automation. This release note covers a brief introduction on LDTP followed by the list of new features and major bug fixes which makes this new version of LDTP the best of the breed. Useful references have been included at the end of this article for those who wish to hack / use LDTP.
Linux Desktop Testing Project is aimed at producing high quality test automation framework (C / Python) and cutting-edge tools that can be used to test Linux Desktop and improve it. It uses the Accessibility libraries to poke through the application’s user interface. The framework also has tools to record test-cases based on user events in the interface of the application which is under testing. We strive to help in building a quality desktop.
Whats new in this release:
* Shreyank Gupta has contributed Object Oriented LDTP based generation of scripts in LDTP editor.
* Thanks to Philipp Wagner for reporting bugs on LDTP.
* Thanks to Ara Pulido for reporting utf-8 related bugs.
* Fixes required for VMware Player automation, reported by Gaurav Sharma
* Fixes required for VMware Workstation globalization testing, reported by Nagappan Alagappan
Congratulations Nagappan and the rest of the LDTP team!
We hope that we will be able to ship this version with Ubuntu 8.10 (Intrepid Ibex).
Next week we will be having an Ubuntu Developer Week. Ubuntu Developer Week is a series of online workshops where you can:
- learn about different packaging techniques
- find out more about different development teams
- check out the efforts of the world-wide Development Community
- participate in open Q&A sessions with Ubuntu developers
- and much more…
Though the schedule is not yet closed, you can still check it out at https://wiki.ubuntu.com/UbuntuDeveloperWeek/Prep
As you can see I will be giving a session about automated testing for Ubuntu. Please, feel free to drop by and learn how to run the already existing tests as well as creating your own tests easily.
In the previous post we talked about the difficulties when trying to recognize widgets through its accessibility information. If normally is a pain, it can certainly be worse when dealing with labels.
Labels text does not only change their text when the language is changed, but they can change their text in different states of the application work flow. Let’s imagine that an application uses the same label to show error messages to the user (very common scenario). If the label does not have correctly set its ‘name’ in the accessibility information (very very common scenario), then the name of the label will be the current text of the label, and our test script won’t be able to gather errors or any other information from these labels.
If the text of the label has some kind of structure, though, some of this information can be retrieved, using regular expressions to search for common patterns, for example, if error messages always start with the text ‘Error:’ we could get the rest of the text of the label to obtain the error message.
For Ubuntu we are using some of these “pattern” labels to get some useful information from the applications. In the update-manager a label with the pattern “You can install n updates.”, is shown every time at least one update is available, where n is the number of available updates. The method number_udpates, part of the Ubuntu testing library, scans the available labels for that pattern and returns the number of available updates. Anyone wanting to write a test for update-manager can use this and any other available method in his or her test script.
One of the main issues when getting information from the AT-SPI layer is that most of the accessibility information is missing. Names are almost never set, therefore, objects take the name of the current text.
This is something generally painful, as tests will need to be localized for each language. Let’s imagine that we have a form called “Update Manager”, then the object would be frmUpdatemanager. If we change the target system language, let’s say, to Spanish, then the window will be named “Gestor de Actualizaciones”, and the accessibility information in this case would be frmGestordeactualizaciones.
As I wrote in my previous post, we are trying to separate as much as possible this kind of information from the scripts code. Apart from having classes for the common activities with applications, the text in windows, buttons, etc. is maintained in a separate file, ubuntu_constants.py that will be, eventually, the only file to be changed when porting the tests to a different language.
When scripting desktop tests is a common practice to rely too much on script recorders, that will script automatically your test, based on mouse and keyboard activity. The main problem with the resulting scripts is that they are too coupled with the desktop interface and, therefore, difficult to maintain.
One of the main objectives that we are persuading when creating a testing framework for Ubuntu desktop is to avoid scripts to know anything about the objects behind them. Definitively, these objects will still require to be maintained, but the logic of the scripts will remain the same.
One example. Let’s imagine that we had a regression test suite for Gedit that will edit, modify, open and save several files. Many. About a hundred. If any of the Gedit features changes its UI, only the Gedit class will be modified. All the scripts will still be valid.
Decoupling scripts and test objects will also keep the door opened to Model-based Testing.
To create our automated tests for the Desktop at Ubuntu we are using LDTP (Linux Desktop Testing Project), a framework that relies on the Accesibility layer to manipulate the desktop objects.
LDTP comes with a python wrapper to be able to create tests in python. One of the tests that we wrote was Gedit Chains that tries to open Gedit, writes a string chain, saves it, and compares the resulting file with an already saved oracle file.
We have two test cases for this test. The first one wrote an ASCII string, “This is a very basic string”, and it was successfully run against Intrepid Alpha3. The second one tried to write a string with Japanese characters, to test UTF-8 with Gedit. This test failed. The problem root was that python-ldtp communicates with LDTP server with XML, and the necessary encoding was not being done properly. Therefore, each time I tried to run this test I got the following error:
settextvalue _message = generatexml (command.SETTEXTVALUE, _requestId,
windowName, componentName, text) File
"/var/lib/python-support/python2.5/ldtp.py", line 84, in generatexml
_argDebug += ", " + str (args [index]) UnicodeEncodeError: 'ascii' codec
can't encode characters in position 27-30: ordinal not in range(128 )
I posted the error on the LDTP mailing list and 24 hours later Nagappan, one of the main LDTP developers, checked the patch into their GIT repository (Thanks!!). I tried again my script with the LDTP GIT version and this time it run smoothly.
I think it is a very important fix, as UTF-8 characters are likely to give errors in text editors, terminals or browsers. I have created PPA packages for Intrepid that include this fix so that anyone can try this test. Intrepid packages are available at https://launchpad.net/~apulido/+archive.