[REQUEST]: Remote execution on different architectures

[partouf]

Before we launch an ARM instance, I would like to get ARM execution out of the way. I would like to implement this with remote execution on ARM instances. This will require a technique like #5370 tailored for the execution part, but maybe extendable to other parts.

Will require:

• Special endpoint to execute known buildpackages (for testing purposes only)
• Websockets
• Queued DB
• Mode to look in the queue (for the architecture) and execute
• Being able to make known that a certain architecture is ready for remote execution (properties? DB?)
• Making execution result available for the requesting instance
• Logging
• Live metrics

Will be making a plan to separate PR's so we don't have to get stuck with yet another long living draft.

[partouf]

#4315

[partouf]

#6413 merged

[partouf]

Follows that design of execution request in "message queue" (however we do that) should be at least this:

type NameValueThing = {
   name: string;
   value: string;
};

type ConfiguredTool = {
   name: string;
   options: NameValueThing[];
};

type ExecutionParams = {
   args: string[];
   stdin: string;
   runtimeTools: ConfiguredTool[];
};

Then the question is, what does the rest look like, maybe this?

type ExecutionMessage = ExecutionParams & {
   result_id: string; // unique identifier that we can lookup and store things under? maybe?
   execute_hash: string; // the hash the executable package has gotten after compilation - in the test endpoint this is a url parameter
   arch: string; // should be something like: `aarch64-linux`?
};

amd64-linux-nvidia-gpu / amd64-linux-amd-gpu / amd64-linux-intel-gpu?

[partouf]

It depends on how the selected queue would look like, but if you can only get or peek at 1 or a limited amount of messages, then you would need multiple queues? 1 per arch

[partouf]

It would be better to only use websockets and just work with DynamoDB, but there might be something something locking via queue or something. (depends on how the concurrency model of aws lambda websockets works, maybe its not needed)

[partouf]

Resources for websockets that I'll be using:

• https://docs.aws.amazon.com/apigateway/latest/developerguide/websocket-api-develop.html
• https://registry.terraform.io/providers/hashicorp/awscc/latest/docs/resources/apigatewayv2_api (awscc??)
• Websocket Example terraform-aws-modules/terraform-aws-apigateway-v2#8
• https://github.com/aws-samples/simple-websockets-chat-app

[partouf]

While writing a websocket I remember again why a naive websocket model is suboptimal.
But there's a solution somewhere