Challenge #1: Scenario Oriented Object Navigation (SOON)
The ability to navigate like a human towards a language-guided target from anywhere in a 3D embodied environment is one of the "holy grail" goals of intelligent robots. We introduce a Scenario Oriented Object Navigation (SOON) task, where an agent is required to navigate from an arbitrary position in a 3D embodied environment to localize a target following a scene description.
Challenge #2: REVERIE
The REVERIE task requires an intelligent agent to correctly localise a remote target object (can not be observed at the starting location) specified by a concise high-level natural language instruction, as shown by the demo above. Since the target object is in a different location from the starting one, the agent needs first to navigate to the goal location. When the agent determines to stop, it should select one object from a list of candidates provided by the simulator. The agent can attempt to localise the target at any step, which is totally up to algorithm design. But we only allow the agent output once in each episode, which means the agent only can guess the answer once in a single run. Please note that the interaction, such as 'check', with the target object is not required.
If your use our datasets, please cite:
@InProceedings{Zhu_2021_CVPR,
author = {Zhu, Fengda and Liang, Xiwen and Zhu, Yi and Yu, Qizhi and Chang, Xiaojun and Liang, Xiaodan},
title = {SOON: Scenario Oriented Object Navigation With Graph-Based Exploration},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2021},
pages = {12689-12699}
}
@InProceedings{Qi_2020_CVPR,
author = {Qi, Yuankai, Qi Wu, Peter Anderson, Xin Wang, William Yang Wang, Chunhua Shen, and Anton van den Hengel},
title = {Reverie: Remote embodied visual referring expression in real indoor environments},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020},
pages = {9982-9991}
}