EuRoC Forum

[rpretten]

Hi,

I have some questions concerning the evaluation process of task2.

I uploaded a bag-file to get my score of task2_1 and in the evaluation summary I saw an overall error-score of 4.0 points which has to be incorrect, because:
The positions false_free and true_free were both 0 and this caused an MCC score of -nan, which led to an overall score of 4 points. This also causes a score of 4 points for an empty occupancy grid.
Could you please fix this (for reliable contestant comparsion)?

It seems that my uploaded bag file contained either an empty occupancy grid or an occupancy grid that was totally mis-alligned. To find the error I serialized the octree to a binary file and opened it with octoviz. It looked fine (had occupied and free cells). I have no idea what went wrong.
Could you post the bounding box of the ground truth occupancy grid, it would be very helpful for me to find my mistake.
Or is there maybe a general problem with the evaluation system?

Thanks in advance,
Rudolf

[markusachtelik]

thanks for pointing us to this! Can you send me your team-name by email (the support mail address) and/or the bagfile, such that we can have a look at it?

[rpretten]

thanks for your reply,

I think I've already solved my issue.
It seems that my conversion of the octree to a binaryMap message was wrong (on the client the octree was fine, but on the server the map looked messed up).
Anyway, now my bagfile generates points and it seems to be correctly aligned.

regards,
Rudolf