EuRoC Forum

[Alexander Reiter]

After a fresh installation of VirtualBox and importing the lastest VM image from the downloads section, the message below appears in the startup terminal of the simserv machine. The problem can be reproduced on different host systems, the specs of them are
Lenovo ThinkStation P300
Intel C226 Chipset
Intel Xeon E3-1246 V3 with integrated Intel HD Graphics P4600
4 GB RAM
Windows 8.1 Professional x64 (latest updates)

HP EliteBook 8560p
Intel QM67 Chipset
Intel Core i7-2620M with integrated HD Graphics 3000
8 GB RAM
Windows 7 Professional x64 (latest updates)

We would be grateful for any help with resolving the situation.

Code: Select all

...
1710
roscore was already running
... logging to /home/simguest/.ros/log/70a58c78-5dec-11e4-a792-08002749a783/roslaunch-EuRoCsimserv-1710.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

Unable to register with master node [http://192.168.172.1:11311]: master may not be running yet. Will keep trying.
started roslaunch server http://EuRoCsimserv:51499/
ros_comm version 1.9.54


SUMMARY
========

PARAMETERS
 * /rosdistro
 * /rosversion

NODES

auto-starting new master
process[master]: started with pid [1921]
ROS_MASTER_URI=http://EuRoCsimserv:11311/

setting /run_id to 70a58c78-5dec-11e4-a792-08002749a783
process[rosout-1]: started with pid [1952]
started core service [/rosout]
have 20 scenes:
 - task1_v1
 - task1_v2
 - task2_v1_1
 - task2_v1_2
 - task2_v1_3
 - task2_v2_1
 - task2_v2_2
 - task2_v2_3
 - task3_v1
 - task3_v2
 - task4_v1_1
 - task4_v1_2
 - task4_v1_3
 - task4_v2_1
 - task4_v2_2
 - task4_v2_3
 - task5_v1
 - task5_v2
 - task6_v1
 - task6_v2
EuRoC Challenge 2 Stage 1 task selector ready.
Received list scenes request
warning: overwrite /task_name
warning: overwrite /task_name
warning: overwrite /task_name
warning: overwrite /task_name
warning: overwrite /task_name
warning: overwrite /task_name
warning: overwrite /task_name
warning: overwrite /task_name
warning: overwrite /task_name
warning: overwrite /task_name
warning: overwrite /task_name
warning: overwrite /task_name
already stopped
start simulator with user_id 'demo user' and scene file '/opt/euroc_c2s1/scenes/task1_v1.yml'
extend two_axes_table/range
warning: overwrite /gui_camera_pose
Gazebo multi-robot simulator, version 2.2.1
Copyright (C) 2012-2013 Open Source Robotics Foundation.
Released under the Apache 2 License.
http://gazebosim.org

Msg Waiting for master
Msg Connected to gazebo master @ http://127.0.0.1:11345
Msg Publicized address: 10.0.2.15
gazebo server failed to startup: returncode: -4
gazebo server seems to be killed by signal 4 - try to upload core dumps...
upload_core version 1.0.1
please enter your simguest password to enable core-dump generation:
[sudo] password for simguest:


[fschmidt]

Code: Select all

gazebo server failed to startup: returncode: -4
gazebo server seems to be killed by signal 4 - try to upload core dumps...


as the output says: the gazebo server seems to be killed by signal 4 (SIGILL - Illegal Instruction).

the start script then tries to upload a coredump to our ftp server so that we can inspect what caused this. (but ubuntu disallowed users to generate core-dumps from their own running processes which is the reason why the script asks for the the simguest sudo password (which is "simguest" again).

but the password is actually only needed when the gazebo process is still running and we want to generate a coredump because of a startup-timeout. this is not the case in your situation.

i've changed the upload_core skript to not ask for a password in this situation. (will be in euroc-c2s1-simulator_1.0.16_i386.deb & euroc_c2s1_20141027183539.ova to be released tomorrow)

the simserv ubuntu should automatically generate a coredump when the gazebo process gets killed by this signal. this coredump will be written to /tmp/*core* within the simserv vm. the upload_core script would try to upload the coredump to the dlr ftp server(when your firewall settings allow this) so that we can investigate.

other challengers also reported those "illegal instruction" errors: http://euroc.iphpbb3.com/forum/20112706nx57588/challenge-2-f4/problem-running-demo-system-in-virtualbox-t29.html
it seems to be related with the virtualbox opengl implementation - we currently have no fix for this other than running the simulator in a native ubuntu 12.04 32bit environment as described in the above post and here: http://euroc.iphpbb3.com/forum/20112706nx57588/challenge-2-f4/native-simulator--simclient-vm-setup-t51.html

locally i can not reproduce this problem - from your post i see that all the hosts you mentioned are running a 64 bit version of windows - tomorrow i will try to find such a system and see whether i can reproduce it there.

here are three example hosts with which i can not reproduce this error:
- Dell Latitude E6230, Intel Core i5 with Ivy Bridge graphics, ubuntu 12.04 32bit
- Dell Precision T3400, Intel Core2 Q6600 with nVidia Quadro FX 570, suse linux enterprise 11 32bit
- Dell Precision T3500, Intel Xeon W3520 with nVidia Quadro FX 580, debian testing 32bit

virtualbox was choosen in the hope to eliminate all those hardware dependencies but sadly it seems that this is not the case.

could you please reproduce the error on one of your systems, and then after the error happened, within the simservVM press Ctrl-C in the "roscore... " terminal window and close the terminal (by pressing the orange "x" on the upper left of the window).

then open a new terminal (from the left menu-bar) and enter these commands:

to kill eventually running ros core/master procs:
$ killall rosmaster roscore

start the gazebo server without the task selector (will only work when the task_selector tried to start the server before):
$ /opt/euroc_c2s1/bin/test_start_gzserver

this should then at some point print the "illegal instruction (core dumped)" error message and a core-dump file should be created at
/tmp/core

can you please send us this core-dump? (notify us at support-challenge02@robotics-challenges.eu and provide us a link to this file as it would be too larde to attach it to an email)

[fschmidt]

to easily upload the generated core file from within the simservVM you can use the provided upload_core script. call it like this:

Code: Select all

$ /opt/euroc_c2s1/bin/upload_core my_user_id -f /tmp/core

this will try to upload the file /tmp/core to our ftp server.

[fschmidt]

this afternoon i will have access to a dell notebook with integrated intel graphics and Win7 64bit.

[fschmidt]

i managed to reproduce the problem on a
Dell Latitude E6420 with an Core i7 and a Nvidia GF119M GPU with Windows 7 sp1 64bit.
using virtualbox v4.1.18

[fschmidt]

this problem is not caused by our simulator software. it can be easily reproduced by calling the glxgears demo application from the mesa-utils package:

Code: Select all

simguest@EuRoCsimserv:/tmp$ dpkg -S glxgears
mesa-utils: /usr/bin/glxgears
mesa-utils: /usr/share/man/man1/glxgears.1.gz
simguest@EuRoCsimserv:/tmp$ glxgears
Illegal instruction (core dumped)


[fschmidt]

this seems to be related to a general problem with GLUT and virtualbox: https://forums.virtualbox.org/viewtopic.php?f=3&t=30964

[fschmidt]

virtualbox 4.3.18 has the same problem. also deinstalling the host-nvidia-graphics driver doesn't change anything.

[fschmidt]

i tested the Dell Latitude Notebook from above again, but this time with those virtualization options in the BIOS turned on:
- Virtualization Support
-- Virtualization
--- [x] Enable Intel Virtualization Technology
-- VT for Direct I/O
--- [x] Enable VT for Direct I/O
-- Trusted Execution
--- [x] Trusted Execution
with those settings glxgears works as expected and also the gazebo simulator starts up successfully.
can you please check your BIOS settings whethere you have similar options and turn them on?

then please test starting glxgears within the simservVM and test the gazebo simulator. hope this helps.

[fschmidt]

here are two other windows systems which are confirmed to work:

Intel Core i5 - nVidia NVS3100M - Win7 Sp1 32bit Home Premium (ThinkPad T410)
Intel Core i5 - nVidia GeForce GTX 570 - Win7 Sp1 64bit Professional (custom build)

(on the thinkpad it even worked with the virtualization options of the BIOS disabled)