Please post questions concerning the benchmarking task rules here and we will summarize these questions in the top post.
Please read the benchmarking task rules first. You can find the document in the euroc_c2 subversion in the 'rules' folder.
Please subscribe to this topic in order to be notified of updates.
General Questions:
- lighting conditions: will the blinds be closed during benchmark evaluation? The blinds will be closed.
- what is meant by "the time is stopped for the task" (p. 13 in the pdf rules)? We will start stopping the time of the task once you signal us you are ready for the first try.
- which is the maximum size for a marker? can a full table be covered? is there a limit on the number of markers? The size of markers and number are not limited.
- can the position of the markers change after the beginning of the 2 hour evaluation period? Yes you may change markers in between tasks, but not during the execution of one task. We will always place the dynamic objects after you have added your markers to the environment.
- markers may have a 3D volume or can they only be sheets of paper? Only flat 2D markers are allowed.
- how many people are inside the laboratory during the evaluation? Will they be in the robot moving area? At least one team member of DLR will monitor the evaluation. No people are allowed within the robot moving area.
- will teams have access (during the code camps) to complete setups (positions of SLCs, bolts, nuts and washers) of identical difficulty as the ones used for the start of task 1 and task 2? All teams have access to the same laboratory which is used during the evaluation during the code camps. Setting up the tasks during testing is the task of the individual teams.
- will teams have access to the other teams 2 hour videos for each task (after all teams have finished their evaluation week, of course)? We will not publish the videos.
- will the simulator be updated to have the same TF names and orientations as the real hardware system? The TFs in the simulator and the real system have different meaning as the real system contains calibrated transformations which are different from the ideal transformations of the simulator. Therefore we discourage using the same names and orientations.
- will the simulator be able to grasp objects? We provide the simulator "as is" as a basis for kinematic simulation. We do not support contact and dynamics.
Task1 Questions:
- Will the SLCs be empty or contain something inside? The SLCs will contain nuts and washers. Extra points are awarded for carrying the SLCs to the goal without dropping parts.
- can the fixtures for the assembly task be covered (possibly with a marker) during task 1? Yes.
- can you provide a better specification of the possible positions and orientations of the SLCs in the tables and shelf (eg: minimum distance to the wall, minimum distance between SLCs, can a SLC be stacked on another SLC, admissible orientations in the shelf, minimum distance to the corners of the shelf, shelf levels that will be used, ...)? We updated the specification (see p. 7).
- will the goal region always be half of the goal table? always the same table half? Yes. Yes.
- Which is the maximum height and weight of nuts and washers inside the SLCs? We updated the specification (see p. 7).
- Will the blue pad for the nuts and bolts of the task 2 be fixed to the pickup table during the task 1? Yes, they will remain fixed for task 1. You may cover the fixtures with 2D "markers" for task 1.
- Could the SLCs be located anywhere in the pickup table? Even on the blue pad area? The SLCs will not stand on the fixtures.
Task2 Questions:
- dimensions and possible positions of the "fixed area" for bolts, nuts and washers in task 2? We upated the specification (see p. 10).
- can you provide a better specification of the possible positions and orientations of the bolts, nuts and washers in the table (eg: minimum distance to the wall, minimum distance between parts, can nuts be placed on top of other nuts?...)? We upated the specification (see p. 10).
- which is the orientation of the washers (leaning to the wall or to the center of the room)? minimum distance between washers? The orientation of the washers will not be specified. The washers are not in contact.
- How should the assembled bolts be placed after assembly? Anywhere on the assembly table, the orientation can be arbitrary (lying or standing).
Freestyle Questions:
- does task setup (limited to 2 hours) also includes hardware changes to the robot platform? No, you are allowed to do hardware changes before the setup.
- the challenger team can only use one camera for recording the freestyle? No, you may record the freestyle with as many cameras as you would like to.
- can the team use non-fixed cameras for recording freestyle? Yes.
- can teams also submit an edited video (containing only recordings from the 2 hour freestyle benchmark) to be passed to the independent reviewers? Yes.