What happen if have a OR submission then a RL submission?
Which result will be showed on leaderboard? or both?
Hi @junjie_li,
Currently best of those submissions is available on the leaderboard.
But I agree with your question, it is possible that someone’s OR submission have better score and RL even better. Although, I guess we assumed (probably wrongly) that one team may not be working in both OR & RL submission.
cc: @MasterScrat for your views
1 Like
Is there any update on this topic? What if there are teams working on both RL and OR submissions?
1 Like
There is ongoing work in this direction. @shivam can tell you more.
It’s now live! you can filter the leaderboard to see either RL or non-RL submissions.
Eg to see only RL submissions: https://www.aicrowd.com/challenges/neurips-2020-flatland-challenge/leaderboards?challenge_leaderboard_extra_id=1&challenge_round_id=588
1 Like