Note if you are a Eurosys reviewer, we like to invite you to run the evaluation on our machines. Please send your ssh key to the paper author email address to obtain ssh access. If you run into problems you can write join the IRC channel #vmsh on libera for a live chat (there is also a webchat version at https://web.libera.chat/) or write an email for further questions.
The first step is to get the source code for vmsh:
$ git clone https://github.com/Mic92/vmsh
For convience we created an evaluation script (tests/reproduce.py) run all evaluation experiments from the paper.
The machine we used for evaluation had the following hardware configuration:
Output of [inxi](https://github.com/smxi/inxi)
System: Host: martha Kernel: 5.15.14 x86_64 bits: 64 compiler: gcc v: 10.3.0 Console: N/A
Distro: NixOS 21.11 (Porcupine)
Machine: Type: Desktop System: NOVATECH product: PC-BX19795 v: V1.0 serial: 7456935-002 Chassis:
type: 3 v: 1.0 serial: 7456935-002
Mobo: ASUSTeK model: PRIME Z390-P v: Rev X.0x serial: 190347447802077
UEFI: American Megatrends v: 2417 date: 06/03/2019
Memory: RAM: total: 62.54 GiB used: 17.55 GiB (28.1%)
Array-1: capacity: 64 GiB slots: 4 EC: None max-module-size: 16 GiB note: est.
Device-1: ChannelA-DIMM1 size: 16 GiB speed: 2666 MT/s type: DDR4 detail: synchronous
bus-width: 64 bits total: 64 bits manufacturer: Micron part-no: 16ATF2G64AZ-2G6E1
serial: 1E915F06
Device-2: ChannelA-DIMM2 size: 16 GiB speed: 2666 MT/s type: DDR4 detail: synchronous
bus-width: 64 bits total: 64 bits manufacturer: Micron part-no: 16ATF2G64AZ-2G6E1
serial: 1E915F22
Device-3: ChannelB-DIMM1 size: 16 GiB speed: 2666 MT/s type: DDR4 detail: synchronous
bus-width: 64 bits total: 64 bits manufacturer: Micron part-no: 16ATF2G64AZ-2G6E1
serial: 1E915EFC
Device-4: ChannelB-DIMM2 size: 16 GiB speed: 2666 MT/s type: DDR4 detail: synchronous
bus-width: 64 bits total: 64 bits manufacturer: Micron part-no: 16ATF2G64AZ-2G6E1
serial: 1E91605C
PCI Slots: Slot: 0 type: x1 PCI Express <BAD INDEX> status: Available length: Short
Slot: 1 type: x16 PCI Express <BAD INDEX> status: In Use length: Long
Slot: 2 type: x1 PCI Express PCIEX1_2 status: Available length: Short
Slot: 3 type: x16 PCI Express PCIEX16_2 status: Available length: Long
Slot: 4 type: x1 PCI Express PCIEX1_3 status: Available length: Short
Slot: 5 type: x1 PCI Express PCIEX1_4 status: In Use length: Short
CPU: Info: 8-Core model: Intel Core i9-9900K bits: 64 type: MT MCP arch: Kaby Lake
note: check rev: C cache: L1: 512 KiB L2: 16 MiB L3: 16 MiB
flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 115200
Speed: 4800 MHz min/max: 800/5000 MHz volts: 0.9 V ext-clock: 100 MHz
Core speeds (MHz): 1: 4800 2: 4793 3: 4802 4: 4785 5: 4801 6: 4758 7: 4800 8: 4780
9: 4801 10: 4801 11: 4801 12: 4800 13: 4802 14: 4797 15: 4799 16: 4801
Graphics: Device-1: Intel CoffeeLake-S GT2 [UHD Graphics 630] vendor: ASUSTeK driver: i915
v: kernel bus-ID: 00:02.0 chip-ID: 8086:3e98 class-ID: 0300
Display: server: No display server data found. Headless machine? tty: N/A
Message: Unable to show advanced data. Required tool glxinfo missing.
Audio: Device-1: Intel Cannon Lake PCH cAVS vendor: ASUSTeK driver: snd_hda_intel v: kernel
bus-ID: 00:1f.3 chip-ID: 8086:a348 class-ID: 0403
Sound Server-1: ALSA v: k5.15.14 running: yes
Network: Device-1: Intel Ethernet XL710 for 40GbE QSFP+ driver: i40e v: kernel port: efa0
bus-ID: 01:00.0 chip-ID: 8086:1583 class-ID: 0200
IF: enp1s0f0 state: up speed: 40000 Mbps duplex: full mac: 3c:fd:fe:9e:97:58
IP v4: 192.168.161.142/24 type: dynamic scope: global
IP v6: fd9a:5371:cd3f:0:3efd:feff:fe9e:9758/64 type: dynamic mngtmpaddr noprefixroute
scope: global
IP v6: fe80::3efd:feff:fe9e:9758/64 scope: link
Device-2: Intel Ethernet XL710 for 40GbE QSFP+ driver: i40e v: kernel port: efa0
bus-ID: 01:00.1 chip-ID: 8086:1583 class-ID: 0200
IF: enp1s0f1 state: up speed: 40000 Mbps duplex: full mac: 3c:fd:fe:9e:97:59
IP v4: 192.168.161.80/24 type: dynamic scope: global
IP v6: fd9a:5371:cd3f:0:3efd:feff:fe9e:9759/64 type: dynamic mngtmpaddr noprefixroute
scope: global
IP v6: fe80::3efd:feff:fe9e:9759/64 scope: link
Device-3: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet
vendor: ASUSTeK PRIME B450M-A driver: r8169 v: kernel port: 3000 bus-ID: 05:00.0
chip-ID: 10ec:8168 class-ID: 0200
IF: enp5s0 state: up speed: 1000 Mbps duplex: full mac: 04:d4:c4:04:4a:ba
IP v4: 129.215.165.53/23 type: dynamic scope: global
IP v6: 2001:630:3c1:164:6d4:c4ff:fe04:4aba/64 type: dynamic mngtmpaddr noprefixroute
scope: global
IP v6: fe80::6d4:c4ff:fe04:4aba/64 scope: link
IF-ID-1: docker0 state: down mac: 02:42:dd:21:89:e1
IP v4: 172.17.0.1/16 scope: global broadcast: 172.17.255.255
IF-ID-2: tinc.retiolum state: unknown speed: 10 Mbps duplex: full mac: N/A
IP v4: 10.243.29.179/12 scope: global broadcast: 10.255.255.255
IP v6: 42:0:3c46:5466:7dbe:f27a:673f:ea64/12 scope: global
IP v6: fe80::80d6:5c1a:258a:3e9/64 virtual: stable-privacy scope: link
WAN IP: 129.215.165.53
RAID: Device-1: zroot type: zfs status: ONLINE level: linear size: 888 GiB free: 571 GiB
allocated: 317 GiB
Components: Online: N/A
Drives: Local Storage: total: raw: 2.69 TiB usable: 3.56 TiB used: 293.46 GiB (8.1%)
ID-1: /dev/nvme0n1 vendor: Intel model: SSDPEDKE020T7 size: 1.82 TiB speed: 31.6 Gb/s
lanes: 4 rotation: SSD serial: PHLE844600FF2P0IGN rev: QDV101D1
ID-2: /dev/sda vendor: Kingston model: SA400S37960G size: 894.25 GiB speed: 6.0 Gb/s
rotation: SSD serial: 50026B7682C2E52B rev: 61F1 scheme: GPT
Partition: ID-1: / size: 827.88 GiB used: 292.92 GiB (35.4%) fs: zfs logical: zroot/root/nixos
ID-2: /boot size: 499.7 MiB used: 385.8 MiB (77.2%) fs: vfat dev: /dev/sda1
ID-3: /tmp size: 535.11 GiB used: 160.8 MiB (0.0%) fs: zfs logical: zroot/root/tmp
Swap: Alert: No swap data was found.
Sensors: Missing: Required tool sensors not installed. Check --recommends
Info: Processes: 326
Uptime: 14:50:01 up 1 day 5:04, 1 user, load average: 0.18, 0.07, 0.02 wakeups: 0
Init: systemd v: 249 target: multi-user.target Compilers: gcc: 10.3.0 Packages:
nix-sys: 470 Client: Sudo v: 1.9.7p2 inxi: 3.3.04
This script only depends on Python and Nix as referenced above. All other
dependencies will be loaded through nix. If the script fails at any point it can
be restarted and it will only re-run not-yet-completed builds and experiments.
Each command which the script runs will be printed during evaluation along with
environment variable set. Experiments store checkpoint results in
tests/measurements
, when re-run experiments might be therefore skipped based
on state stored in the same directory.
For disk benchmarks our scripts a assume a nvme block device which can be reformatted during
the benchmark. To configure the disk set the HOST_SSD
environment variable:
export HOST_SSD=/dev/nvme0n1 # the block device set here will be ereased prior to use!
To run the evaluation script use the following command:
$ cd vmsh
$ HOST_SSD=/dev/nvme0n1 nix develop -c python tests/reproduce.py
After the build is finished, it will start evaluations and generate graphs for
each afterwards. The graph files will be written to ./tests/graphs
.
The following figures are reproduced:
- 6.1 Robustness (xfstests)
- no graphs, just outputs how many tests passes
- 6.2 Generality, hypervisors
- no graphs, just outputs how many tests passes
- 6.2 Generality, kernels
- no graphs, just outputs how many tests passes
- Figure 5. Relative performance of vmsh-blk for the Phoronix Test Suite compared to qemu-blk.
- see tests/graphs/phoronix.pdf
- Figure 6. fio with different configurations featuring qemu-blk and vmsh-blk with direct IO, and file IO with qemu-9p.
- see
tests/graphs/fio-best-case-bw-seperate_overhead.pdf
- see
tests/graphs/fio-best-case-bw-seperate.pdf
- see
tests/graphs/fio-worst-case-iops-seperate_overhead.pdf
- see
tests/graphs/fio-worst-case-iops-seperate.pdf
- see
- Figure 7. Loki-console responsiveness compared to SSH
- see
tests/graphs/console.pdf
- see
- Figure 8. VM size reduction for the top-40 Docker images (average reduction: 60%).
- see
tests/graphs/docker-images.pdf
- see
- Usecase #2: : VM rescue system
- no graphs, just a successful unittest
- Usecase #3: : Package security scanner
- no graphs, just a successful unittest
For Usecase #1: : Serverless debug shell
see follow the instructions
here instead.
If you encounter this issue, there might be still a process using your disk.
Make sure lsof /mnt/nvme
does not list any process and you do not have any
shell open in this directory. If the issue still persist after the output of
lsof
is empty, try to reboot the machine.