Learning User Interface Element Interactions

Degott, Christian and Pereira Borges Jr., Nataniel and Zeller, Andreas
(2019) Learning User Interface Element Interactions.
In: ACM SIGSOFT International Symposium on Software Testing and Analysis, July 15-19, 2019, Beijing, China.
Conference: ISSTA International Symposium on Software Testing and Analysis


Download (2MB) | Preview


When generating tests for graphical user interfaces, one central problem is to identify how individual UI elements can be interacted with---clicking, long- or right-clicking, swiping, dragging, typing, or more. We present an approach based on reinforcement learning that automatically learns which interactions can be used for which elements, and uses this information to guide test generation. We model the problem as an instance of the multi-armed bandit problem (MAB) from probability theory and show how its traditional solutions work on test generation, with and without relying on previous knowledge. The resulting guidance yields higher coverage. In our evaluation, our approach shows improvements in statement coverage between 18% (when not using any previous knowledge) and 20% (when reusing previously generated models).


Actions (login required)

View Item View Item