Initial contact forces in task 4

Dear all,
I'm testing the task 4, and I have noticed very different behaviors (and different scoring results) when I tested the same controller.
In my opinion, it depends on the initial contact forces/torques (and therefore on the initial joint positions) which vary very much from time to time.
My controller is able to reduce the contact forces in few cycles, but the RMS values is strongly related to the initial force/torque peaks.
Is this a desired behavior?

Regards,

Beaver

Dear all,

respect to my previous message in addition to the high variability of the score, I have also noticed a decay of the results re-launching exactly the same program sequentially.
Thus my question is: the log service starts exactly after call service /solve_t4 or there is a delay?
Because it seems that the variability of the score is strongly related with time instant of the beginning of the log service.
Regards,

Beaver

Hi Beaver,

We notice very different scores when solving the task just after receiving /t4_ready or after some time because t4_ready is send before gazebo has completely finished to start. In final evaluation, this will be changed, t4_ready will be called 30/40 seconds later.
Try to launch your solver only when gazebo is fully loaded.

Best,
Mathieu

Hi Mathieu,
thank you for your answer.
I tried to wait 30/40 seconds (in simulation time) before to call /solve_task4 service, but the measured forces/torques remains huge and unstable.
For instance, in the two last time stamps before calling /solve_task4, I receive the following data:

qL= -0.01478, -0.00314, -0.00311, +0.00952, +0.02064, -0.07128 (left robot joint positions)
qR= -0.00308, -0.00314, -0.00354, +0.01312, +0.01140, +0.03769 (right robot joint positions)
ForceL= +117.89, -524.93, +123.96, +31.68, +14.98, -15.91 (left robot forces/torques)
ForceR= +226.74, +151.91, +600.39, -22.13, -1.81, -28.29 (right robot forces/torques)

qL= -0.01481, -0.00313, -0.00285, +0.00751, +0.02143, +0.04875
qR= -0.00332, -0.00312, -0.00308, +0.01428, +0.01424, -0.00458
ForceL= -114.93, -155.20, -250.60, +22.96, +7.30, -12.49
ForceR= -227.17, -840.30, -499.54, +69.84, +12.78, +58.73


Due to the high force/torque values, the closed chain appears to be strongly deformed (as it is possible to see in the attached pictures)
Best,

Beaver

Attachments

task4.png (171.37 KiB) Viewed 911 times

Hi Beaver,

For now, log starts as soon as /solve_t4 is called. We will study the problem during final evaluation and adjust scoring if needed to be as fair as possible.
Please upload your logs on euroc-score.laas.fr so we can get an overview of scores.

Best,
Mathieu

Dear Mathieu,

thank you for the answer!
Due to this problem, I'm wondering if the final evaluation will be done in a "single shot" or making multiple simulations (e.g. considering the best score or the average of the tests)?

Beaver

Hi Beaver,

Final evaluation won't be a single shot, it will be the best score over three shots.

Best,
Mathieu

ok, thank you for the information, we agree with this solution.
Regards,
Beaver

Hi all,

we are testing our task 4 algorithm on different host machines, it seems that the scoring is strongly related to the VM host performances, just to clarify: on my laptop (Debian Host) the results is stable and repeatable, when I change host machine (e.g. I have tested the result on two machines Windows 7 and 8.1 Host) I have noticed a strong variability of the scoring and I'm not able to repeat the results obtained on my laptop.
Additional information: my laptop has lower performance then the other machines used. I didn't modify the VM settings.
I don't really know the reason, but I'm wondering if someone else have encountered this problem?
What will be the host machine for the final evaluation?
Regards,
Beaver

Hi Beaver,

I've also noticed this variability. I think it depends on the RT Factor you get on the simulator : the smaller the RT Factor is, the better the scores are. On final evaluation machine, simulator runs at about 0.2~0.25.

Best,
Mathieu