Sikuli plays Hexiom for me

[Update: with some tweaking, my script sometimes succeeds with size-6, see at the end]

Yeah! Here is one of the first successful automatic runs of my Hexiom-playing bot. This is the combination of the solver I wrote about recently, and a bot coded with Sikuli.

Sikuli itself is a combination of OpenCV for the back-end, screen vision part, a Java middle layer and a scripting engine using Python as its language. It comes with an IDE that is quite easy to learn too.

What's great

The scripting part of Sikuli being Jython (Python implemented on top of the JVM) is great. It allows to code quickly and to fall back to Java for more preformance-sensitive parts. In my case, all the screen reading and playing logic is implemented in pure Python, but the problems are handed to Java for resolution.

OpenCV itself is great and fast enough.

And as previously said, the IDE is easy to learn and loaded with helpful tools like the screen capture utility and above all the pattern testing screen. It is invaluable to tweak the parameters of image matching, because small patterns like the digits of Hexiom can sometime lead to misses or false positives.

What's not so great

Sikuli, do no seem to offer a non-rectangular matching function. There's an open request for that on Launchpad, but from my quick investigation, it might be an OpenCV limitation. The only workaround is to match the biggest rectangular area that can fit in the preferred non-rectangular pattern. That is quite annoying: first, in Hexiom's case, all the patterns I'd like to use are hexagonal and second, using a smaller patterns leads to worse detection. After some tweaking, I succeeded in making the bot work every time on a size-4 puzzle, but it's not yet working on a size-6 one, because the board reading procedure making too many mistakes.

Wanna try?

Following these steps, you should be able to run the bot:

• Install Java, Sikuli and Firefox
• Fetch my code from github
• Compile java sources (in src/main/java to build/main/classes) or use Eclipse
• Open a Firefox window
• Set JAVA_HOME and SIKULI_HOME environment variables in a terminal
• Run ./autorun.sh 4 to solve a size-4 puzzle

You can also try with 6 instead of 4, but the results will probably be disappointing. Don't try 3 or 5: I haven't captured the appropriate digits yet.

Breaking news!

Sometimes, it works with size-6.

Daniel Hartmeier on his site with a somewhat different approach also succeeded, with a fully functional C program, see: http://www.benzedrine.cx/hexiom.html