Automate GUI testing using Sikuli – Part1

SikuliAutomationToolWhat is Sikuli –

As per the sikuli official site, “Sikuli automates anything you see on screen, it uses image recognition to identify and control GUI components.

  • It is an open source automation tool.
  • Used for automating GUI elements for desktop / web elements.

How it works –

Sikuli uses Visual image match method

First we have to capture all the required desktop / web elements in terms of images, and store into our project,

sikuli provides several methods to pass the images as parameters to perform actions like identifying, typing, clicking etc..

When to use –

  • Sikuli can be used in case if any application don’t have proper identifiers like id, name, class etc
  • It can be easily integrated with selenium webDriver & other tools
  • It can be used to automate Flash contents on web/desktop.

NOTE: We should consider Sikuli if the application GUI is stable and sometimes it can’t operate in proper manner if the captured image is of different resolution than the application elements while automating.

Where to download Sikuli – 

Download latest SikuliXSetup-x.x.x.jar from download Page.

Double click on the setup.jar to chose an option either to use as Sikuli IDE (for desktop) or integrate with java programming.


To automate desktop application:

Click on Pack1, chose Phython or Ruby to download the IDE. (click “yes” for each popup you get)


You will get sikulix.jar to automate desktop application.

Click on “Take screenshot” to capture images from your desktop screen and use the predefined methods to perform actions as shown below screenshot, then click on “Run” to see it in action.




Next post, we will be looking into the Pack1 option, Integrate Sikuli with Selenium WebDriver

Content posted is based on learning or working experience,
Share the post with others if find useful,
Any queries?, leave comments or discuss on our, Thanks!

1 Response

  1. August 3, 2016

    […] we have seen on our prev post Sikuli intro, how to download and automate desktop application, in this post we will be looking to integrating […]

Leave a Reply

Your email address will not be published. Required fields are marked *