Alex is a Software Engineer working on Android development tools, especially Android Studio, at Google. His interests include Java, API design, OOP, IDEs and testing. Alex spends some of his spare time working on Open Source, blogging, writing technical articles, and speaking at international conferences. The opinions expressed here represent his own and not those of his employer. Alex is a DZone MVB and is not an employee of DZone and has posted 49 posts at DZone. You can read more from them at their website. View Full User Profile

Testing JavaFX UIs: Part 1 of ?

12.31.2008
| 6244 views |
  • submit to reddit

This is the first part of a series of blog entries where I plan to document our progress testing JavaFX UIs with FEST.

Although I had fun, my first attempt failed miserably. The main reason is that my assumptions were wrong. I thought that, since JavaFX uses Swing in its "desktop profile," testing a JavaFX UI that uses Swing would be similar to testing a regular Swing UI created in Java. This is partially right. In my previous try, I could successfully find a JButton using FEST-Swing. The problem was simulating a user clicking the found JButton. It seemed that FEST's Robot could not find the location of the component on the screen. That was pretty weird, since click simulation works perfectly for Swing-based apps. I couldn't understand why FEST could not find the coordinates of a #@$! JButton that in front of me!

Before I continue, I'd like to clarify that:

  • I'm just trying to figure out how to test JavaFX UIs
  • I only have access to the source code distributed with JavaFX
  • I'm not reverse-engineering/decompiling any of the JavaFX classes

I used the excellent tool Swing Explorer to visually inspect the JavaFX UI I wanted to test. Here is a screenshot of the JavaFX calculator example loaded in Swing Explorer:

 

Swing Explorer saved me a lot of time and effort! This wonderful tool can show you where Swing components are actually displayed in a JFrame (Swing Explorer can do a lot more than that, but this is the functionality I used.) I tried to find if the JButton that I wanted to click with FEST was being displayed in the JavaFX UI. For my surprise, the JButton (and actually any Swing component) is hidden! It seems that JavaFX paints a "fake" button instead. I don't know how exactly this works since Sun does not distribute the source code of the javafx-swing jar file.

Since the JButton is hidden, there should be a way JavaFX knows about the location where to paint it on the screen. In order to simulate a user clicking this JButton using FEST, I had to figure out how to obtain such location.

Figuring out some of the internals of JavaFX was a lot easier than what I expected (actually, I think I was lucky.) I started by writing a test. I first obtained a reference to the com.sun.javafx.scene.JSGPanelImpl, which hosts all the Swing components in the UI. Then, I found the method getScene, which returns a com.sun.scenario.scenegraph.SGNode. After that, it was a matter of looping through the children of its parent (and the children of the children, and so forth.) At last I found something useful: com.sun.scenario.scenegraph.fx.FXNode, a subclass of SGNode, provides the method getLeaf that returns another SGNode. This "leaf" node can be an instance of com.sun.scenario.scenegraph.SGComponent. And, finally, a SGComponent has a reference to the Swing component I've been looking for, through the method getComponent.

The following is the (ugly and hacky) code to find a node that has the button I'm looking for:

private static FXNode nodeWithButton(String text, SGParent root) 
{
for (SGNode child : root.getChildren())
{
if (child instanceof FXNode) {
FXNode fxNode = (FXNode) child;
SGNode leaf = fxNode.getLeaf();
if (leaf instanceof SGComponent) {
SGComponent componentNode = (SGComponent) leaf;
Component component = componentNode.getComponent();
if (component instanceof JButton) {
JButton button = (JButton) component;
if (text.equals(button.getText())) return fxNode;
}
}
}
if (child instanceof SGParent) {
SGParent newRoot = (SGParent) child;
FXNode fxNode = nodeWithButton(text, newRoot);
if (fxNode != null) return fxNode;
}
}
return null;
}

At this point I found the nodes that has Swing components as leaf nodes, but I still haven't found the location on the screen of the JButton. I took a closer look at the properties of FXNode and found the method getBoundsInScene, which has all the information I need!

Finally I found a way to obtain the location on the screen and click the JavaFX node containing the Swing component I was looking for:

private void click(FXNode node) 
{
moveMouseTo(node);
realRobot.mousePress(BUTTON1_MASK);
realRobot.mouseRelease(BUTTON1_MASK);
robot.waitForIdle();
}

private void moveMouseTo(FXNode fxNode)
{
Rectangle2D boundsInScene = fxNode.getBoundsInScene();
int centerX = (int)boundsInScene.getCenterX();
int centerY = (int)boundsInScene.getCenterY();
Point p = fxNode.getPanel().getLocationOnScreen();
p.translate(centerX, centerY);
realRobot.mouseMove(p.x, p.y);
}

The whole test can be found here.

Unfortunately, I confirmed that is not possible to test JavaFX UIs with FEST-Swing as it is today. In order to do so, we need to provide JavaFX-specific support, which IMHO will require the following work:

  1. Create a new component hierarchy based on JavaFX nodes, not on Swing components
  2. Create a new Robot that knows how to simulate user input on JavaFX nodes (we will still use the AWT Robot under the hood)
  3. Create new fixtures for JavaFX nodes

There are a couple of things that makes me uncomfortable:

  • We depend on classes that belong to com.sun.* packages, not javafx.* ones. This means that Sun can change implementation details at any time and our tool will stop working.
  • It is not clear (at least to me) the license for some jars provided with JavaFX. The worst-case scenario is that we might have to publish our code as GPL, instead of Apache 2.

Feedback is always appreciated

From http://www.jroller.com/alexRuiz

Published at DZone with permission of Alex Ruiz, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Carl Dea replied on Thu, 2009/01/01 - 1:41am

Alex,

Congratulations on the new release of FEST. I hope you will continue the effort with FEST as JavaFX becomes popular.  I am of the opinion of great software comes from great testing.  The GUI test looks really nice and easy. Although the uncomfortable feeling regarding the purity of JavaFX components vs underlying Sun components maybe be short lived. I think we will start seeing many skinnable and pure JavaFX components coming soon. Which doesn't make things any easier for FEST.

I'm assuming you were you talking about:

com.sun.javafx.scene.JSGPanelImpl;
com.sun.scenario.scenegraph.*;

While working with JavaFX I did notice the ability to get nodes by id and shape name.

http://java.sun.com/javafx/1/docs/api/javafx.fxd/javafx.fxd.FXDContent.html

Yep, licenses still boggle my mind.

 

Happy new year!

-Carl

Alex Ruiz replied on Fri, 2009/01/02 - 12:02am

Hi Carl,

Happy new year to you too!

Thanks for the kind words about FEST :)

True, the best way to get a reference to a node is through its id. Future posts will include that. Many thanks for the pointer!

Cheers,
-Alex

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.