Festvortrag

"The Grammar of Action and Thought"?

Kicking off the Bremer Universit?tsgespr?che with his keynote, Professor Dr. Yiannis Aloimonos (University of Maryland) already had a serious warning for the audience at the historic Stadtwaage: "I believe, in the hands of the rich this technology will be a nightmare for us." He said a major reason for him to make this trip to Bremen was the fact that the democratization of robotics was a core part of the event's agenda.

He then addressed one of the most important questions regarding the development of autonomous robots: How can tasks be divided into small steps and translated into language, making it possible for computers to understand and execute even very vague instructions? With this goal in mind, Professor Aloimonos has developed a grammar that forms the basis of all activities. One source of inspiration was Noam Chomsky's X-bar theory which states that syntactic structures of all natural languages are based on common principles. According to Aloimonos, the same is true for actions.

This grammar is essential for the learning of tasks. "When you look at what somebody is doing and try to understand it, you need a set of rules," Aloimonos said. After all, even before humans had developed the ability to speak, they already needed to understand the structure underlying all actions they observed – e.g. "to realize when a guy steals something from their cave," the scientist explained. 

Robots, too, need to acquire this ability in order to learn from humans or from other machines. According to Aloimonos, actions consist of distinct factors such as objects, tools and hands, but also of the pursued goals and the underlying knowledge. "We cannot perform complex actions without information – in the form of language," he said, "for example playing the piano." His core message: We understand actions in a way that is similar to the way we understand language. Only with the help of the underlying grammar can instructions be given and specific questions be asked regarding a situation. However, to be very precise, grammar alone is not enough – the robot also needs to determine which problem the action is supposed to solve and which force is being applied, e.g. pushing, pulling, sliding, or aligning. Once the machine has figured all this out, it can include the observed action in its own repertoire of possible operations.