GIMP has introduced a new platform for automated user-interface tests, designed to catch problems with menus, dialogs and tools before users encounter them. The tests run automatically each time contributor changes are merged.
Unlike existing unit tests, which check small parts of the code without opening a real window, UI tests start GIMP and carry out tasks through its interface. The project says it also had tests that interacted directly with core code, but those could break without detecting a bug and could miss entire categories of problems.
GIMP currently has 13 UI tests. They check that the application opens its main window; that the Welcome dialog appears on the correct tab and stays hidden when “Show on Start” is turned off; and that actions such as creating and exporting an image, swapping colors, editing layers, undoing and redoing, zooming, selecting and using a tool, saving a preference, and flattening an image work as expected.
The tests run GIMP inside a small, hidden desktop, so they can operate on continuous integration servers without a screen. They locate buttons and menus using accessibility information provided for tools such as screen readers, a method the project says also supports software accessibility.
The tests use dogtail, a Python tool for UI testing. GIMP contributors added a feature to dogtail to let it run GIMP on a server without a screen. The tests currently run only on Linux, using X11 or Wayland, because dogtail does not support Windows or macOs.
Harsh Verma wrote the UI tests and the new dogtail feature. Jehan reviewed the UI tests, and Michal Odehnal reviewed the dogtail feature. The changes are available in merge request !2909.
GIMP says the 13 tests are a start and invites contributors to write tests for the dialogs and tools they use. A guide to writing tests and investigating failures is available on developer.gimp.org. The project hopes the work will help find bugs earlier and keep GIMP working well for everyone.
