Update ci kernel

[quininer]

The kernel version currently running on ci is too old, preventing many features from being stable.

I fork cross image at the time to update the kernel, but it's too old now.
We need a better way to run tests with newer kernels.

[lperkov]

Hello @quininer!

The project seems like a good candidate for contributing. We are starting to use this project, and I am looking for ways we can help.

Would it be possible for you to provide more information about the build servers? I would like to know if it is possible to make a Replica.one image for this purpose.

Thanks,

Luka

cc @jpetrina

[quininer]

@lperkov

Thanks for your interest in this.

Currently ci uses cross + qemu to run the specified image. It's a bit complicated because it uses ssh to communicate with the host, which introduces a lot of dependencies. The tests for io-uring itself have no external dependencies, I think image just needs a sufficiently new kernel, glibc, shell to work.

qemu is a bit too complicated for me and I've been thinking about migrating to firecracker or some other lighter vm.

[jela1337]

Hello @quininer!

I have a couple of questions regarding this issue:

Is this the image with the outdated kernel you are referring to?
Does the OS/kernel of the host have any effect on the tests or is it just the one in the image?
Have you considered self-hosting your CI or GitHub Actions runners?

Thanks,

Alen

cc @lperkov @jpetrina

[quininer]

@jela1337

Yes, it is.
The kernel version of host has no effect on tests, we run the tests in vm.
Self-hosting is too complicated for me. I initially ran tests directly on github actions, but soon the kernel version became outdated, so I started running tests on vm.

[lperkov]

@quininer would it be okay with you if we managed the self-hosting? The only thing we would need is to setup the WebHooks to GitHub project so the self-hosted instance gets notified on pull requests & new commits.

We already have a similar setup and that would be easiest for us I think.

[quininer]

@lperkov This sounds like the fastest solution, but I still want a vm solution that can run anywhere.

[lperkov]

I don't think that's a problem. You or anyone else can run the VM manually...

@jela1337 & @jpetrina can you please make initial VM for testing so once that is functional we can start automating the process?

[FrankReh]

Is there an update on this? Am I correct in thinking the current ci is using a GitHub ci runner to run cargo cross and cargo cross is using qemu and qemu is pointed to a private linux image.

When I tried this on my ubuntu machine, with docker, cross test --target aarch64-unknown-linux-gnu
failed, not able to build libc v0.2.132.

[jela1337]

Hello,
sorry for the delay. Here are the VMs for testing, with instructions on how to run them.

Could you confirm if they satisfy your needs?

[quininer]

This looks great, but I don't have energy to do it right now. It would be great if someone would try it.

[BrokenWingsIcarus]

@quininer
I know a way to switch the kernel through wsl. But I'm not sure if it's easy to use. At present, it seems that it can run.
see https://github.com/WinSSitaly/WSL-Linux-Kernel/releases

[rushilmehra]

Is there a reason we can't just use a more recent version of Ubuntu and remove the VM dependency? Kinda like https://github.com/tokio-rs/tokio-uring/blob/master/.github/workflows/ci.yml#L19

[quininer]

@rushilmehra We can use host kernel, vm kernel version is outdated anyway. but even with host kernel, the latest features are still not available for testing on ci.