Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File streams handling improvement for oc-azure-storage-adapter #62

Open
pbazydlo opened this issue May 3, 2018 · 0 comments
Open

File streams handling improvement for oc-azure-storage-adapter #62

pbazydlo opened this issue May 3, 2018 · 0 comments

Comments

@pbazydlo
Copy link
Collaborator

pbazydlo commented May 3, 2018

Implementing workaround/hack for the private/public file permissions (described in #61) have had a side effect when putting file content when content is provided as a stream.

After we have written the stream to private container, the stream is at its end. I have not been able to figure out how to reset this stream to beginning for writes to public container, I've tried the following:

  • use fs.read with empty buffer, read length of 0 and position set to 0 in an attempt to reset the stream
    • this had no effect on stream whatsoever
  • pipe through azure write stream
    • as one could expect azure write stream is not pass-through, so any further reads would basically hang

In order to be able to write the actual content to public container after using up original stream with file content, I set up additional stream from azure private container to azure public container. Even though it puts ugly and unnecessary strain on the network I opted in to do this because:

  • I couldn't solve original issue
  • I did not want to load files into memory, that would put unnecessary memory pressure on registry and I think it could potentially block event loop

How to resolve this issue

Figure out a way to either:
a) Solve #61
b) Reset the stream to it's beginning so that it can be reused

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant