Testing with UPF mode enabled

[jovans2]

Hello,

When I follow the instructions from https://github.com/ease-lab/vhive/blob/main/docs/profiling.md and set flags (UPFEnabled, SnapshotEnabled, MetricsMode) I run into following issue:

• on first invocation I get the error: Task could not be killed.
• on subsequent invocations: "Tuntap IOCTL TUNSETIFF failed [0], errno device or resource busy", and "Bridge could not be created"

In general, I cannot run any of the tests with set UPF flag and I cannot come to the point where UPF statistics are written into file.

Also, when running all tests on a single node cluster (with and without UPF flag) I get the warning that kn8 could not be reached and default dns is used instead. Does it affect my experiments?

Do you know how could I solve this problem?
Thank you,
Jovan

[ustiugov]

hi @jovans2

it seems that your previous runs left some VMs running or some tap devices were not torn down. This usually happens when something crashed during the previous runs. You can try to use the cleaning script to clean all the resources:

./scripts/clean_fcctr.sh

The warning regarding DNS is not an issue.

In general, when reporting bugs, please include log snippets in your description and attach full logs as files if you think it's necessary. Also, please specify the host specs (HW and SW).

[ustiugov]

hi @jovans2, you can bind a VM to a particular core, as we do it here

[jovans2]

Hello @ustiugov,

Another question - to invoke many functions at the same time without waiting on the previous request to be completed - is it okay to use TestBenchParallelServe? If yes, do you know what is the upper bound of parallel invocations? The test fails whenever I set more than 50 parallel invocations.
The other thing, do I need separate VM for every invocation (which seems reasonable for me - every call is a different user and we need to provide isolation) or invocations can share VM?

Paper says "vHive follows the model of AWS Lambda, which deploys a single function inside a MicroVM that processes a single invocation at a time", is the call "resp, met, err := funcPool.Serve(context.Background(), vmIDString, imageName, "replay")" basically creation of a MicroVM or Pod?

Thank you!

[ustiugov]

hi @jovans2

1. feel free to modify the test as you feel the need for.
2. the upper bound depends on the number of concurrently booting.loading instances because this is a relatively lengthy and resource-consuming procedure, which often results in timeouts. If the traffic is steered to warm VMs, one can push a lot more requests.
3. No, you don't need a separate instance for each request although it's common to maintain strict FIFO order of requests, i.e., one at a time, steered to each instance (e.g., in AWS Lambda).
4. yes, vHive by default supports 1 function instance deployed in one MicroVM. However, you are free to steer many requests to the gRPC server inside. Basically, it depends on your benchmark's code. You can configure it in the way you like.

[jovans2]

Thank you for the clarification. So I can start an instance, keep it warm and send as many requests as I want? Also, is there a way to implement or use grace period?

[ustiugov]

if you skim the code of the benchmark, it does all of what you mentioned. if you don't shut a VM down explicitly (with RemoveInstance), it would stay and it can keep serving as long as you like.