ToroMicroVM is a unikernel dedicated to deploy microservices as microVMs. ToroMicroVM leverages on virtio-fs and virtio-vsocket to provide a minimalistic architecture. Microservices are deployed as Toro guests in which binaries and files are distributed in a Ceph cluster. The common fileystem allows to easely launch microvms from any node of the cluster.
- Support x86-64 architecture
- Support up to 512GB of RAM
- Support QEMU-KVM microvm and Firecracker
- Cooperative and I/O bound threading scheduler
- Support virtio-vsocket for networking
- Support virtio-fs for filesystem
- Fast boot up
- Tiny image
- Built-in gdbstub
You can quickly get a first taste of ToroMicroVM by running the HelloWorld example by building a docker image that includes all the required tools. To do so, execute the following commands in a console (These steps require KVM and Docker):
wget https://raw.githubusercontent.com/torokernel/torokernel/master/ci/Dockerfile
sudo docker build -t torokernel-dev .
sudo docker run --privileged --rm -it torokernel-dev
cd examples/HelloWorld
python3 ../CloudIt.py -a HelloWorld
If these commands execute successfully, you will get the output of the HelloWorld example. You can also pull the image from dockerhub instead of building it:
sudo docker pull torokernel/toro-kernel-dev-debian-10
sudo docker run --privileged --rm -it torokernel/toro-kernel-dev-debian-10
You can share a directory from the host by running:
sudo docker run --privileged --rm --mount type=bind,source="$(pwd)",target=/root/torokernel-host -it torokernel/toro-kernel-dev-debian-10
You will find $pwd from host at /root/torokernel-host in the container.
wget https://sourceforge.net/projects/lazarus/files/Lazarus%20Linux%20amd64%20DEB/Lazarus%202.0.10/fpc-laz_3.2.0-1_amd64.deb/download
mv download fpc-laz_3.2.0-1_amd64.deb
apt install ./fpc-laz_3.2.0-1_amd64.deb -y
apt-get update
apt-get install python3-pip make git libcap-dev libcap-ng-dev libcurl4-gnutls-dev libgtk-3-dev libglib2.0-dev libpixman-1-dev libseccomp-dev -y
pip3 install ninja
# uncomment to change PATH permanently
# echo 'export PATH=/home/debian/.local/bin:$PATH' >>~/.bashrc
export PATH="/home/debian/.local/bin:$PATH"
git clone https://github.com/qemu/qemu.git qemuforvmm
cd qemuforvmm
git checkout 51204c2f
mkdir build
cd build
../configure --target-list=x86_64-softmmu
make
git clone https://github.com/torokernel/torokernel.git
git clone https://github.com/torokernel/freepascal.git -b fpc-3.2.0 fpc-3.2.0
Note that Step 1, 2, 3 and 4 can be found in the script at ci/prepare_host.sh
.
Go to torokernel/examples
and edit CloudIt.py
to set the correct paths to Qemu and fpc. Optionally, you can install vsock-socat from here.
You have to go to examples/HelloWorld/
and execute:
python3 ../CloudIt.py -a HelloWorld
You can easily get the StaticWebServer up and running by following the tutorial at here. This would require only a Debian 10 installation. For example, you can get a s1-2 host from OVH. If you prefer to run it step by step, follow the next instructions. You have first to compile vsock-socat and virtiofds. The latter is built during the building of Qemu. The former can be built by executing:
git clone git@github.com:stefano-garzarella/socat-vsock.git
cd socat-vsock
autoreconf -fiv
./configure
make socat
Then, launch vsock-socat by executing:
./socat TCP4-LISTEN:4000,reuseaddr,fork VSOCK-CONNECT:5:80
In a second terminal, execute:
./virtiofsd -d --socket-path=/tmp/vhostqemu1 -o source=/root/qemulast/build/testdir/ -o cache=always
Replace source
with the directory to serve. Finally, launch the static webserver by executing:
python3 ../CloudIt.py -a StaticWebServer
This example shows how cores can communicate by using the VirtIOBus device. In this example, core #0 sends a packet to every core in the system with the ping string. Each core responds with a packet that contains the message pong. This example is configured to use three cores. To launch it, simply executes the following commands in the context of the container presented above:
python3 ../CloudIt.py -a InterCoreComm
You will get the following output:
First you have to follow this tutorial to get a FPC cross-compiler from Windows to Linux. Then, you have to execute the following script which compiles the RTL for Toro and outputs the generated files in the x86_64-linux directory. Note that this script overwrites the RTL for Linux, which is used when the -TLinux parameter is passed. This script requires three paths to set up:
- fpcrtlsource, which is the path to the repository from https://github.com/torokernel/freepascal
- fpcrtllinuxbin, which is the path to the cross-compiled linux RTL
- fpcbinlinux, which is the path to the fpc compiler.
fpcrtlsource="c:\Users\Matias\Desktop\fpc-3.2.0\rtl"
fpcrtllinuxbin="c:\fpcupdeluxefortoromicrovm\fpc\bin\x86_64-linux"
fpcbinlinux="c:\fpcupdeluxefortoromicrovm\fpc\bin\x86_64-win64"
$fpcbinlinux/ppcx64.exe -TLinux -dFPC_NO_DEFAULT_MEMORYMANAGER -dHAS_MEMORYMANAGER -uFPC_HAS_INDIRECT_ENTRY_INFORMATION -dx86_64 -I$fpcrtlsource/objpas/sysutils/ -I$fpcrtlsource/linux/x86_64/ -I$fpcrtlsource/x86_64/ -I$fpcrtlsource/linux/ -I$fpcrtlsource/inc/ -I$fpcrtlsource/unix/ -Fu$fpcrtlsource/unix/ -Fu$fpcrtlsource/linux/ -MObjfpc $fpcrtlsource/linux/si_prc.pp -Fu$fpcrtlsource/objpas -Fu$fpcrtlsource/inc -FE$fpcrtllinuxbin
$fpcbinlinux/ppcx64.exe -Us -TLinux -dx86_64 -I$fpcrtlsource/objpas/sysutils/ -I$fpcrtlsource/linux/x86_64/ -I$fpcrtlsource/x86_64/ -I$fpcrtlsource/linux/ -I$fpcrtlsource/inc/ -I$fpcrtlsource/unix/ -Fu$fpcrtlsource/unix -Fu$fpcrtlsource/linux -Fu$fpcrtlsource/objpas -Fu$fpcrtlsource/inc $fpcrtlsource/linux/system.pp -FE$fpcrtllinuxbin
Then, you have to go to Lazarus and open the project HelloWorld.lpi. You are able to compile the project from compile.
To create a CephFS cluster you can follow these instructions.
You have many ways to contribute to Toro. One of them is by joining the Google Group here. In addition, you can find more information here.
GPLv3
[0] A Dedicated Kernel named Toro. Matias Vara. FOSDEM 2015.
[1] Reducing CPU usage of a Toro Appliance. Matias Vara. FOSDEM 2018.
[2] Toro, a Dedicated Kernel for Microservices. Matias Vara and Cesar Bernardini. Open Source Summit Europe 2018.
[3] Speeding Up the Booting Time of a Toro Appliance. Matias Vara. FOSDEM 2019.
[4] Developing and Deploying Microservices with Toro Unikernel. Matias Vara. Open Source Summit Europe 2019.
[5] Leveraging Virtio-fs and Virtio-vsocket in Toro Unikernel. Matias Vara. DevConfCZ 2020.
[6] Building a Cloud Infrastructure to Deploy Microservices as Microvm Guests. Matias Vara. KVM Forum 2020.
[7] Running MPI applications on Toro unikernel. Matias Vara. FOSDEM 2023.
[8] Is Toro unikernel faster for MPI?. Matias Vara. FOSDEM 2024.