The “Other Me”: Human-Centered AI Assistance In Situ

Imagine a situated AI assistant embedded in your smart glasses and collaborating with you to create a novel artifact, e.g., building an Arduino smart lamp. It communicates with you through the natural language to understand your goals and preferences. It works with you to develop and execute the solution. It sees what you see from a first-person view to understand your actions and offer useful tips. It connects with other AI assistants to scale up to complex tasks, e.g., assembling or disassembling an aircraft engine, which require a team solution. AI assistants become the “other me”, an external, but situated extension of our individual minds and senses.

Publications

	PointTFA: Training-Free Clustering Adaption for Large 3D Point Cloud Models Jinmeng Wu, Chong Cao, Hao Zhang, Basura Fernando, Yanbin Hao, and Hanyu Hong IJCAI (Accepted 2024) PDF
	Effective Multimodal Encoding for Image Paragraph Captioning Thanh-Son Nguyen and Basura Fernando IEEE Transactions on Image Processing (2022) PDF Bibtex @article{ son2022, title={Effective Multimodal Encoding for Image Paragraph Captioning}, author={Thanh-Son Nguyen and Basura Fernando}, journal={IEEE Transactions on Image Processing}, year={2022}, } ×
	3D Equivariant Graph Implicit Functions Yunlu Chen and Basura Fernando and Hakan Bilen and Matthias Niessner and Efstratios Gavves ECCV 2022 PDF Bibtex @inproceedings{ eccv22yunlu, title={3D Equivariant Graph Implicit Functions}, author={Yunlu Chen and Basura Fernando and Hakan Bilen and Matthias Niessner and Efstratios Gavves}, booktitle={ECCV}, year={2022}, } ×
	Not All Relations are Equal: Mining Informative Labels for Scene Graph Generation Arushi Goel, Basura Fernando, Frank Keller, and Hakan Bilen CVPR 2022 PDF Bibtex @inproceedings{ cvpr22, title={Not All Relations are Equal: Mining Informative Labels for Scene Graph Generation}, author={Arushi Goel and Basura Fernando and Frank Keller and Hakan Bilen}, booktitle={CVPR}, year={2022}, } ×