LAHA: Learning To Anticipate Human Actions

Our ability to anticipate the behaviour of others comes naturally to us. For example, an experienced driver can often predict the behaviour of other road users. Similarly, a good table tennis player can estimate the direction of the ball just by observing the movements of the opponent. This phenomenon is called Action Anticipation, the ability to recognise actions of others before it happens in the immediate future. This is so natural to us but how to develop a computational approach to do the same remains a challenge. It is critical to transfer this ability to computers so that robots may be able to react quickly by anticipating human actions like humans. Robots’ ability to understand what humans might do in the immediate future is important for the development of assistive robotics in domains such as manufacturing and healthcare. The objective of this project is to investigate a novel approach to anticipate human actions, specifically one to five seconds before action happens using visual information in human-robot engagement scenarios. Current action anticipation approaches primarily rely on new loss functions that allow us to tackle the uncertainty in future predictions. However, these approaches assume observed data contains rich information about the future human actions. Other approaches overcome this limitation by generating appearance information such as objects, environment and context for the future, and then classify those to anticipate future human actions. We argue that movement of objects and humans are the key indicators to anticipate human actions before they happen. Unfortunately, these approaches are not able to generate most useful information to anticipate actions because they only consider appearance and do not explicitly generate movement information such as motion.