Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

For EQA testing, how do you decide where to start the agent? #3

Open
danielgordon10 opened this issue Aug 8, 2018 · 3 comments
Open

For EQA testing, how do you decide where to start the agent? #3

danielgordon10 opened this issue Aug 8, 2018 · 3 comments

Comments

@danielgordon10
Copy link

This can have a dramatic effect on the accuracy of the agent. Do you pick a random start location, and if so, do you record that in the dataset so it can be replicated?
@abhshkdz
Copy link
Contributor

abhshkdz commented Aug 8, 2018

The set of shortest paths are available here: https://github.com/facebookresearch/EmbodiedQA#download-and-preprocess-the-dataset, and for evaluation, we spawn the agent 10, 30, 50 actions away from the target along these shortest paths: https://github.com/facebookresearch/EmbodiedQA/blob/master/training/train_nav.py#L102

@danielgordon10
Copy link
Author

Got it. Thanks.

@danielgordon10
Copy link
Author

Would it be possible for you to provide starting coordinates for each question for reproducibility?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@abhshkdz @danielgordon10