EuRoC Forum

[sebsch]

I am trying to set up a computer similar the final evaluation system so I can get a rough estimate of the computation time required by our solution on the final system. We have an Intel® Core2 6700 processor @ 2.66GHz with 4 GB RAM running Ubuntu 12.04 32 bit hosting the virtual machine.

Some things run as fast as expected but others are incredibly slow and I am trying to figure out whether it will also be that slow on the final system or whether I just made a mistake setting up our machine running virtualbox.

One drastic example: If I run the libelas demo on the physical machine it takes 567.9 ms for the cones image (first image). If I run exactly the same binary within the VM it takes 3806.7 ms.

I made sure that virtualization support is enabled in BIOS and the VBox.log file shows VT-x being enabled without any error messages.

Does anyone else have the same problem?

Thanks in advance,
Sebastian

[tracker]

We are experiencing similar problems.

As host computer we are using an Intel Core2 Quad QX6700 @ 2.66GHz with 4GB RAM running Ubuntu 12.04 32 bit. VT-x is enabled in BIOS and also detected in Virtualbox according to log.

When using the simserver VM and running one of our solutions for task 1 directly on the host (running on 2 CPUs), we get an average measured runtime of 45 msec (the same time measured in multiple runs).

Instead, when running the same solution for task1 on the simclient VM (with 2 CPUs) and using the host computer directly as server, we usually get a runtime of approx. 250 msec. However, the runtime is not constant in the VM. We already executed the same solution on the same dataset and had a runtime of 100msecs, and at another execution we had a runtime of 350msec.

We measured the exact computation time on the client, so the communication does not influence our times.

We found out that especially the parts implemented with SSE instructions are very slow in the VM.

For us it seems as such an old processor is just not optimized well for virtualization as, for example, a modern i7, where in a VM the performance loss is less and the runtimes are repeatable.

As others also seem to have similar problems, we would highly appreciate a modification of the final evaluation procedure! In our oppinion, solving problems of a VM on an old computer should not be the main challenge in this competition.

Thanks,
Thomas

[sebsch]

Thomas, thanks for your feedback. At least we are not the only ones with that problem.

I tried to use vmware to run a virtual machine on the same physical CPU and it turns out that vmware is much faster in the problematic cases.

I also posted my findings on the VirtualBox forums.

Dear organiers, could you check and tell us whether you experience a similar slowdown on the VM compared to the physical CPU on the evaluation machine?

Thanks,
Sebastian

P.S.: Here are some examples of measured times reported by the libelas demo:

Physical machine (Intel® Core2 6700 processor @ 2.66GHz)

Code: Select all

Processing: img/cones_left.pgm, img/cones_right.pgm
                    Descriptor   39.7 ms
               Support Matches   85.9 ms
        Delaunay Triangulation    9.6 ms
              Disparity Planes   11.2 ms
                          Grid   21.1 ms
                      Matching  221.7 ms
         L/R Consistency Check   22.4 ms
         Remove Small Segments   58.7 ms
             Gap Interpolation    9.4 ms
                 Adaptive Mean   93.7 ms
========================================
                    Total time  573.3 ms

VirtualBox (same hardware):

Code: Select all

Processing: img/cones_left.pgm, img/cones_right.pgm
                    Descriptor 1303.5 ms
               Support Matches   96.4 ms
        Delaunay Triangulation   67.7 ms
              Disparity Planes   13.4 ms
                          Grid  456.2 ms
                      Matching  468.6 ms
         L/R Consistency Check  250.4 ms
         Remove Small Segments  588.5 ms
             Gap Interpolation   12.0 ms
                 Adaptive Mean  550.0 ms
========================================
                    Total time 3806.7 ms


vmware (same hardware):

Code: Select all

Processing: img/cones_left.pgm, img/cones_right.pgm
                    Descriptor  230.3 ms
               Support Matches   90.3 ms
        Delaunay Triangulation   17.1 ms
              Disparity Planes   10.7 ms
                          Grid   84.4 ms
                      Matching  258.7 ms
         L/R Consistency Check   56.9 ms
         Remove Small Segments  146.6 ms
             Gap Interpolation   11.2 ms
                 Adaptive Mean  163.5 ms
========================================
                    Total time 1069.5 ms

[markusachtelik]

please have a look here: http://euroc.iphpbb3.com/forum/20112706nx57588/challenge-3-f5/trial-evaluation-evaluation-hardware-scenario-changes-t79.html