KR2023Proceedings of the 20th International Conference on Principles of Knowledge Representation and ReasoningProceedings of the 20th International Conference on Principles of Knowledge Representation and Reasoning

Rhodes, Greece. September 2-8, 2023.

Edited by

ISSN: 2334-1033
ISBN: 978-1-956792-02-7

Sponsored by
Published by

Copyright © 2023 International Joint Conferences on Artificial Intelligence Organization

From Vision to Execution: Enabling Knowledge Representation and Reasoning in Hybrid Intelligent Robots Playing Mobile Games

  1. Denise Angilica(University of Calabria)
  2. Mario Avolio(University of Calabria)
  3. Giovanni Beraldi(University of Calabria)
  4. Giovambattista Ianni(University of Calabria)
  5. Francesco Pacenza(University of Calabria)


  1. Applications of KR in robotics
  2. Applications of KR
  3. Integrating symbolic and sub-symbolic approaches
  4. KR related tools and systems


Automating acts on touch surfaces opens a range of possibilities for researching and experimenting with hybrid AI approaches.

In this paper, we propose a delta robot capable of playing match-3 games and ball-sorting puzzles by acting on mobile phones. The robot recognizes objects of different colors and shapes through a vision module, is capable of making strategic decisions based on declarative models of the game's rules and of the game playing strategy, and features an effector that executes moves on physical devices.

Our solution integrates multiple AI methods, including vision processing and answer set programming. Helpful and reusable infrastructure is provided: the vision task is facilitated, while robot motion control is inherently simplified by the usage of a delta robot layout.

We illustrate the components of our robotic application and how they were integrated. Then, we briefly showcase how recognition and general knowledge can be modeled and implemented, by overviewing the implementation of representative games.

We argue that our application provides potential for KR and robotics to be combined in creative ways, and offers itself as a general controlled environment where to experiment with forms of hybrid reasoning, while relieving from implementation details.