[POC] Compiler request queueing

[partouf]

Not really production code, but this is a proof of concept where compilations requests are rerouted to SQS first and then actually compiled elsewhere.

Sends back the result via a different queue, but not sure if that's the best solution. (there might be a message size limit)

Response time is quite fast with just square() example from my local pc (with aws creds).
It was really slow when I did not pass WaitTimeSeconds: 1 to receiveMessage when waiting for the result. Maybe throttling from Aws's side because of the while loop that keeps asking for messages?

Also learning that of course you cannot combine the functionality of queuing and compiling in the same app here when using SQS for the results as well, because once 1 client requests all the results, they aren't available for others anymore. But it works when just running 1 client locally as a POC.

[mattgodbolt]

Thanks for taking a look at this! I've sketched similar things and hit the same problem re: replies. Some things to think about/chat about sometime:

• I considered writing the response to S3: the "request" puts a UUID-based "and put the results here" field in the SQS request message, and then polls the response location. More latency. Could use redis though instead or something less latent (but requires standing up redis, which we can do)
• Once could have one SQS queue per worker, dynamically created, and then the "which queue to put result in" goes into the request. Gets messy

Worth also thinking about how we might do (breaking changes) updates, and separating environments: we (probably) want one queue per environment/type AND maybe version them so when we do a breaking change (if we ever do...) then old servers don't reply to new requests. or something like that.

[partouf]

Thanks for taking a look at this! I've sketched similar things and hit the same problem re: replies. Some things to think about/chat about sometime:

• I considered writing the response to S3: the "request" puts a UUID-based "and put the results here" field in the SQS request message, and then polls the response location. More latency. Could use redis though instead or something less latent (but requires standing up redis, which we can do)

Ahh yes, could use the original request's MessageId to put it in S3. But S3 is probably not made for polling... Ideally we'd have some kind of event service (observer/subscriber) that says like hey your order is ready, but not sure that exists.

• Once could have one SQS queue per worker, dynamically created, and then the "which queue to put result in" goes into the request. Gets messy

Hm, right, needs also cleanup for when an instance is shutdown/done/failing etc, doesn't sound great.

Worth also thinking about how we might do (breaking changes) updates, and separating environments: we (probably) want one queue per environment/type AND maybe version them so when we do a breaking change (if we ever do...) then old servers don't reply to new requests. or something like that.

Right, makes sense

[partouf]

Currently looking at https://us-east-1.console.aws.amazon.com/events/home?region=us-east-1#/eventbuses Amazon Eventbridge event buses

[partouf]

Currently looking at https://us-east-1.console.aws.amazon.com/events/home?region=us-east-1#/eventbuses Amazon Eventbridge event buses

Nope, can't seem to have custom listeners to the event bus, you need to write rules and let it be pushed to an API

[partouf]

Currently looking at https://us-east-1.console.aws.amazon.com/events/home?region=us-east-1#/eventbuses Amazon Eventbridge event buses

Nope, can't seem to have custom listeners to the event bus, you need to write rules and let it be pushed to an API

SNS seems to work the same, if you subscribe you have to pass an API endpoint

[partouf]

c2d9649

changes to use https://github.com/compiler-explorer/ce-eventserver-nodejs and an s3 bucket

worker:

1. stores result to s3
2. pushes an event by requestId

requester:

1. pushes request onto queue
2. subscribes to the event by requestId and waits for an alert
3. when an alert is received, pull result from s3, delete it, and return with the result

[partouf]

https://github.com/compiler-explorer/ce-eventserver-nodejs

Is however a little problematic. It's hard to catch socket errors. Perhaps a catch-all and ignore everything? or a rewrite in a language where we know how to catch and handle errors...

[partouf]

I did a tiny bit of research on how the aws websocket stuff should be working. It requires at a minimum 3 lambdas and 1 dynamodb table. Might be picking up on this again in the near future.