ui-automation

Java wrapper for ms-uiautomation

View on GitHub

The library is a programmers tool, in order to automate an application under tests developers will need to ‘explore’ the application to get the names of controls, etc. using a tool like inspect. It will also help to have the original source code, but it is not essential.

The MS UIAutomation Library is a COM library, and the classes that represent this have been wrapped using native JNA calls (currently using 5.6.0).

Maven

The library is held in the Sonatype OSS repository, so is available for download via Maven (or other tools, such as SBT for Scala).

As the library depends on jna (currently version 5.6.0), then there is no need to specify the JNA library version in YOUR project’s dependencies, the library will guide the dependencies and bring down the correct version.

<dependencies>
  <!-- Other dependencies -->
  <dependency>
    <groupId>com.github.mmarquee</groupId>
    <artifactId>ui-automation</artifactId>
    <version>0.7.0</version>
  </dependency>  
<dependencies>

Snapshot build

The ‘stable’ release of the library is 0.7.0, so the development branch is being snapshotted. To get the latest development version (currently 0.8.0-SNAPSHOT, if it exists), use the following in the pom.xml.

<dependencies>
  <!-- Other dependencies -->
  <dependency>
    <groupId>com.github.mmarquee</groupId>
    <artifactId>ui-automation</artifactId>
    <version>0.7.0-SNAPSHOT</version>
  </dependency>  
<dependencies>

Getting started

The ui-automation library is a wrapper for the UIAutomationClient library, which has been extracted using com4j. As the generated code is large and complex, this has been wrapped up in a number of classes, each providing classes that encapsulate part of this library (together with other utility methods as necessary).

Initialising

In order to get access to the automation API, an UIAutomation instance needs to be created, this is done as follows.

  UIAutomation automation = UIAutomation.getInstance();

Launching an application

The Application class provides functionality to start and attach to an application. There are 3 class methods provided to do this.

The snippet below will check whether Project1.exe is running, attaching to it if it is, or launch it if not.

  // Build the application details up, ready for launching
  Application application =
    new Application(
        new ElementBuilder()
                .automation(automation)
                .applicationPath("apps\\Project1.exe"));

  try {
    application.launchOrAttach();
  } catch (Throwable ex) {
    logger.warn("Failed to find application", ex);
  }

Getting hold of a window

The root element of the tree of automation elements is the user’s desktop.

To get a ‘desktop’ window (i.e. one that appears in the Windows tasks bar), then the AutomationDesktop class provides a class function that returns a AutomationWindow object.

  Window window = automation.getDesktopWindow("Form1");
  window.focus();

This will find (if it is there) a window that has the given title, and set focus to it. This window is independent of the overall application, and might not even be associated with the same application that is being automated.

Finding a control

Each control contained in a container (such as a window or panel) can be identified by the index of that control, sometimes (this depends on the control type), by the text associated with it, OR by the Automation Id. For example, in order to get the textbox associated with the connection window (and assuming that it is the 1st Edit box on the window), the following code will find the editbox, and change the text to be USER1.

  EditBox user = window.getEditBox(Search.getBuilder(0).build());
  user.setText("USER1");

Invoking actions on controls

In order to click the ‘OK’ button associated with a given window, it can be found by the text associated with the button, the following examples will all find the same button, and call the click event for it.

  // Get button by index
  Button button1 = window.getButton(Search.getBuilder(0).build());
  button1.click();

  // Get button by name
  Button button1 = window.getButton(Search.getBuilder("OK").build());
  button1.click();

  // Get button by automation id
  Button button1 = window.getButton(Search.getBuilder().automationId("btnOK").build());
  button1.click();

Current supported controls

The controls that have been implemented reflect the requirements for automating the applications that we are testing ourselves, so some controls have not been implemented, or only partially. The currently supported controls are …

Windows support details

Windows versions

Real world examples

Caching

Event Handling

Other languages