The “Other Me”: Human-Centered AI Assistance In Situ

This research is supported by the National Research Foundation Singapore under
its AI Singapore Programme (Award Number: AISG2-RP-2020-0160). Duration : April 2021 to March 2025.

Imagine a situated AI assistant embedded in your smart glasses and collaborating with you to create a novel artifact, e.g., building an Arduino smart lamp. It communicates with you through the natural language to understand your goals and preferences. It works with you to develop and execute the solution. It sees what you see from a first-person view to understand your actions and offer useful tips. It connects with other AI assistants to scale up to complex tasks, e.g., assembling or disassembling an aircraft engine, which require a team solution. AI assistants become the “other me”, an external, but situated extension of our individual minds and senses.

Publications

Effective Multimodal Encoding for Image Paragraph Captioning
Thanh-Son Nguyen and Basura Fernando
IEEE Transactions on Image Processing (2022)
PDF Bibtex
3D Equivariant Graph Implicit Functions
Yunlu Chen and Basura Fernando and Hakan Bilen and Matthias Niessner and Efstratios Gavves
ECCV 2022
PDF Bibtex
Not All Relations are Equal: Mining Informative Labels for Scene Graph Generation
Arushi Goel, Basura Fernando, Frank Keller, and Hakan Bilen
CVPR 2022
PDF Bibtex

Team Members

1. 2022-** Dr. Rajendiran Ramanathan (Research Scientist)