Failed to spawn stage2 when /dev has noexec flag

[Kazurin-775]

It is common for modern Linux kernels (or distributions) to have /dev mounted with noexec flag to enhance security (such as the Ubuntu 20.04 LTS cloud image which we used in our tests). But this prevents VMSH from spawning the stage2 process, as the following kmsg states:

[   59.024633] stage1: failed to spawn stage2: errno=-13
[   59.033913] stage1: failed

... where -13 = Permission denied. Removing the noexec flag with mount -o remount,exec /dev resolves the problem.

Maybe some additional checks should be added to prevent stage1 from extracting stage2 binaries into unsuitable directories, such as those with ro or noexec?

[Mic92]

Yes. There should be definitely fallback directories like /tmp etc. The only tricky part is that we don't want to use a lot of kernel api for compatibility reasons. The easiest solution would be to process if execution fails here:

vmsh/src/stage1/src/lib.rs

Line 498 in cfbb612

let res = ffi::call_usermodehelper(

and than fallback to different directories.