Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature request: awslambda.streamifyResponse support for lambda #10751

Open
1 task done
aravindjaimon opened this issue Apr 30, 2024 · 4 comments
Open
1 task done

feature request: awslambda.streamifyResponse support for lambda #10751

aravindjaimon opened this issue Apr 30, 2024 · 4 comments
Labels
aws:lambda AWS Lambda status: accepted Feature/enhancement request was accepted type: feature New feature, or improvement to an existing feature

Comments

@aravindjaimon
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Feature description

AWS introduced streaming responses.

https://docs.aws.amazon.com/lambda/latest/dg/configuration-response-streaming.html

This comes in handy when creating handlers working with LLM's.
Other middleware like Middy also supports response streaming.
https://middy.js.org/docs/intro/streamify-response/

As of now, these streamed lambdas are not working in localstack. Getting an internal server error for the same.

It would be good to have the feature as LLM-based applications are growing.

Another approach will be to use websockets. But a clean approach will always be to use streams for pushing out each response piece by piece for faster resolution.

🧑‍💻 Implementation

No response

Anything else?

No response

@aravindjaimon aravindjaimon added status: triage needed Requires evaluation by maintainers type: feature New feature, or improvement to an existing feature labels Apr 30, 2024
@localstack-bot
Copy link
Collaborator

Welcome to LocalStack! Thanks for reporting your first issue and our team will be working towards fixing the issue for you or reach out for more background information. We recommend joining our Slack Community for real-time help and drop a message to LocalStack Pro Support if you are a Pro user! If you are willing to contribute towards fixing this issue, please have a look at our contributing guidelines and our contributing guide.

@viren-nadkarni viren-nadkarni added the aws:lambda AWS Lambda label May 2, 2024
@joe4dev joe4dev added status: accepted Feature/enhancement request was accepted and removed status: triage needed Requires evaluation by maintainers labels May 6, 2024
@ptfpinho23
Copy link

ptfpinho23 commented May 15, 2024

I'd be happy to take this one as I'd really like to see this implemented & have been in the same situation

@joe4dev
Copy link
Member

joe4dev commented May 21, 2024

I'd be happy to take this one as I'd really like to see this implemented & have been in the same situation

Thank you for @ptfpinho23 for letting us know you are interested in this feature. We have an internal backlog item for this feature and are happy to prioritize highly requested features (also see support depending on our subscription plans).

This feature is quite complex because it requires a deep understanding of:

  • our custom Rolo HTTP framework
  • our custom lambda-runtime-init written in Golang
  • LocalStack development and Parity Testing in particular covered in CONTRIBUTING (ideally demonstrated through successfully merged PRs)
  • our LocalStack Lambda implementation

If you are confident in all these areas, feel free to give it a try and demonstrate a working implementation covered by an AWS Parity test.

@ptfpinho23
Copy link

Thank you for @ptfpinho23 for letting us know you are interested in this feature. We have an internal backlog item for this feature and are happy to prioritize highly requested features (also see support depending on our subscription plans).

This feature is quite complex because it requires a deep understanding of:

  • our custom Rolo HTTP framework
  • our custom lambda-runtime-init written in Golang
  • LocalStack development and Parity Testing in particular covered in CONTRIBUTING (ideally demonstrated through successfully merged PRs)
  • our LocalStack Lambda implementation

If you are confident in all these areas, feel free to give it a try and demonstrate a working implementation covered by an AWS Parity test.

Thanks for the breakdown @joe4dev . I've been taking a closer look in the meantime & will definitely give it a try.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
aws:lambda AWS Lambda status: accepted Feature/enhancement request was accepted type: feature New feature, or improvement to an existing feature
Projects
None yet
Development

No branches or pull requests

5 participants