Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@uppy/companion: upgrade redis to version 4.x #3589

Merged
merged 5 commits into from Jul 25, 2022

Conversation

aduh95
Copy link
Member

@aduh95 aduh95 commented Mar 22, 2022

Related to the socket.io upgrade. The new redis version is now using promises, which makes it a bit harder to interact with it from code that expects it to be synchronous. Fortunately, we were already using an util class to abstract that away, so I have hidden the promise interop boilerplate inside of it.

@aduh95 aduh95 requested a review from mifi March 22, 2022 11:53
@aduh95 aduh95 added safe to test Add this label on trustworthy PRs to spawn the e2e test suite and removed pending end-to-end tests labels Mar 22, 2022
@github-actions github-actions bot added pending end-to-end tests and removed safe to test Add this label on trustworthy PRs to spawn the e2e test suite labels Mar 22, 2022
Copy link
Contributor

@mifi mifi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice going. Left some feedback.

  • I have no problems with ditching node v10 but I think it would require a new major of companion, no?
  • Shouldn't we wrap all the error, pSubscribe, pUnsubscribe events in a node EventEmitter instead, that way we can be compliant with the interface we're trying to simulate, and it would allow multiple .on(), once() etc listeners to be attached to a single event as is expected, and if someone calls removeListener('x', handler) it will only remove that specific handler, and not all other handlers too?

packages/@uppy/companion/package.json Show resolved Hide resolved
packages/@uppy/companion/package.json Outdated Show resolved Hide resolved
packages/@uppy/companion/src/server/redis.js Show resolved Hide resolved
errorHandler && errorHandler(message) // eslint-disable-line no-unused-expressions
return
}
connectedPromise.then(() => publisher.publish(prefix + eventName, message)).catch(errorHandler)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this work? Shouldn't message be JSON.stringified?

@aduh95
Copy link
Member Author

aduh95 commented Mar 22, 2022

  • it would allow multiple .on(), once() etc listeners to be attached to a single event as is expected, and if someone calls removeListener('x', handler) it will only remove that specific handler, and not all other handlers too?

Isn't that already the case? We are already relying on RedisClient which I assume is an EventEmitter already. The error handler is a special case because that's what nrp is doing, although I suppose we don't have to stick to it if this is a breaking change anyway.

@mifi
Copy link
Contributor

mifi commented Mar 22, 2022

Isn't that already the case? We are already relying on RedisClient which I assume is an EventEmitter already. The error handler is a special case because that's what nrp is doing, although I suppose we don't have to stick to it if this is a breaking change anyway.

as i understand it, in the current implementation if someone calls:

const handler1 = (message) => console.log('listener1');
const handler2 = (message) => console.log('listener2');
emitter.on('some-event', handler1);
emitter.on('some-event', handler2);

emitter.removeListener('some-event', handler1)

then if some-event is received from redis, handler2 will never be called, although if it was a real event emitter, it would still call handler2 even though handler1 has been removed.

@mifi
Copy link
Contributor

mifi commented Mar 22, 2022

Actually looking at node-redis source code it does indeed seem to handle unsubscribing a specific listener already, if we pass it along. So if we change the removeListener implementation to pass the listener along to redis pubsubscribe, it should just work:

connectedPromise.then(() => subscriber.pUnsubscribe(`${prefix}${eventName}`)).catch(errorHandler)

@aduh95 aduh95 changed the base branch from main to 3.x May 25, 2022 15:06
@aduh95 aduh95 added Companion The auth server (for Instagram, GDrive, etc) and upload proxy (for S3) and removed pending end-to-end tests labels May 25, 2022
@aduh95 aduh95 force-pushed the 3.x branch 3 times, most recently from ea17f17 to 602f973 Compare May 30, 2022 16:33
@mifi
Copy link
Contributor

mifi commented Jul 25, 2022

i finished the implementation and tested and it seems to be working nicely

@mifi
Copy link
Contributor

mifi commented Jul 25, 2022

ohh, are we still supporting node14 in uppy v3? It would be nice if we could support only 16 onwards, as we can use more built in features

@aduh95
Copy link
Member Author

aduh95 commented Jul 25, 2022

ohh, are we still supporting node14 in uppy v3? It would be nice if we could support only 16 onwards, as we can use more built in features

We are supporting Node.js 14.x in Companion, we are dropping support for EOL versions of Node.js (10.x and 12.x). We can discuss dropping support for 14.x, but it won't be EOL until next April. Marking as draft to ensure this doesn't land without discussing that first.

@aduh95 aduh95 marked this pull request as draft July 25, 2022 07:50
@Murderlon
Copy link
Member

Murderlon commented Jul 25, 2022

I think we should definitely comply with supported LTS versions, so in my opinion we shouldn't drop 14.x for some syntactic sugar.

@aduh95 aduh95 marked this pull request as ready for review July 25, 2022 13:07
@aduh95 aduh95 merged commit aee4d3f into transloadit:3.x Jul 25, 2022
@aduh95 aduh95 deleted the upgrade-redis branch July 25, 2022 13:14
github-actions bot added a commit that referenced this pull request Jul 27, 2022
| Package                |      Version | Package                |      Version |
| ---------------------- | ------------ | ---------------------- | ------------ |
| @uppy/aws-s3           | 3.0.0-beta.2 | @uppy/react            | 3.0.0-beta.2 |
| @uppy/aws-s3-multipart | 3.0.0-beta.2 | @uppy/remote-sources   | 1.0.0-beta.2 |
| @uppy/companion        | 4.0.0-beta.2 | @uppy/store-redux      | 3.0.0-beta.2 |
| @uppy/compressor       | 1.0.0-beta.2 | @uppy/transloadit      | 3.0.0-beta.3 |
| @uppy/core             | 3.0.0-beta.2 | @uppy/webcam           | 3.0.0-beta.2 |
| @uppy/dashboard        | 3.0.0-beta.2 | @uppy/xhr-upload       | 3.0.0-beta.2 |
| @uppy/image-editor     | 2.0.0-beta.2 | @uppy/robodog          | 3.0.0-beta.3 |
| @uppy/locales          | 3.0.0-beta.3 | uppy                   | 3.0.0-beta.3 |

- @uppy/react: Fix exports in propTypes.js to fix website build (Murderlon)
- @uppy/dashboard,@uppy/webcam: Add support for `mobileNativeCamera` option to Webcam and Dashboard (Artur Paikin / #3844)
- @uppy/aws-s3-multipart: make `headers` part indexed too in `prepareUploadParts` (Merlijn Vos / #3895)
- @uppy/aws-s3,@uppy/core,@uppy/dashboard,@uppy/store-redux,@uppy/xhr-upload: upgrade `nanoid` to v4 (Antoine du Hamel / #3904)
- @uppy/companion: update minimal supported Node.js version in the docs (Antoine du Hamel / #3902)
- @uppy/companion: upgrade `redis` to version 4.x (Antoine du Hamel / #3589)
- @uppy/companion: remove unnecessary ts-ignores (Mikael Finstad / #3900)
- meta: use `node:` protocol when using Node.js built-in core modules (Antoine du Hamel / #3871)
- meta: upgrade to Vite v3 (Antoine du Hamel / #3882)
- @uppy/companion: remove `COMPANION_S3_GETKEY_SAFE_BEHAVIOR` env variable (Antoine du Hamel / #3869)
- meta: fix release script for major beta versions (Antoine du Hamel)
HeavenFox pushed a commit to docsend/uppy that referenced this pull request Jun 27, 2023
Co-authored-by: Mikael Finstad <finstaden@gmail.com>
HeavenFox pushed a commit to docsend/uppy that referenced this pull request Jun 27, 2023
| Package                |      Version | Package                |      Version |
| ---------------------- | ------------ | ---------------------- | ------------ |
| @uppy/aws-s3           | 3.0.0-beta.2 | @uppy/react            | 3.0.0-beta.2 |
| @uppy/aws-s3-multipart | 3.0.0-beta.2 | @uppy/remote-sources   | 1.0.0-beta.2 |
| @uppy/companion        | 4.0.0-beta.2 | @uppy/store-redux      | 3.0.0-beta.2 |
| @uppy/compressor       | 1.0.0-beta.2 | @uppy/transloadit      | 3.0.0-beta.3 |
| @uppy/core             | 3.0.0-beta.2 | @uppy/webcam           | 3.0.0-beta.2 |
| @uppy/dashboard        | 3.0.0-beta.2 | @uppy/xhr-upload       | 3.0.0-beta.2 |
| @uppy/image-editor     | 2.0.0-beta.2 | @uppy/robodog          | 3.0.0-beta.3 |
| @uppy/locales          | 3.0.0-beta.3 | uppy                   | 3.0.0-beta.3 |

- @uppy/react: Fix exports in propTypes.js to fix website build (Murderlon)
- @uppy/dashboard,@uppy/webcam: Add support for `mobileNativeCamera` option to Webcam and Dashboard (Artur Paikin / transloadit#3844)
- @uppy/aws-s3-multipart: make `headers` part indexed too in `prepareUploadParts` (Merlijn Vos / transloadit#3895)
- @uppy/aws-s3,@uppy/core,@uppy/dashboard,@uppy/store-redux,@uppy/xhr-upload: upgrade `nanoid` to v4 (Antoine du Hamel / transloadit#3904)
- @uppy/companion: update minimal supported Node.js version in the docs (Antoine du Hamel / transloadit#3902)
- @uppy/companion: upgrade `redis` to version 4.x (Antoine du Hamel / transloadit#3589)
- @uppy/companion: remove unnecessary ts-ignores (Mikael Finstad / transloadit#3900)
- meta: use `node:` protocol when using Node.js built-in core modules (Antoine du Hamel / transloadit#3871)
- meta: upgrade to Vite v3 (Antoine du Hamel / transloadit#3882)
- @uppy/companion: remove `COMPANION_S3_GETKEY_SAFE_BEHAVIOR` env variable (Antoine du Hamel / transloadit#3869)
- meta: fix release script for major beta versions (Antoine du Hamel)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Companion The auth server (for Instagram, GDrive, etc) and upload proxy (for S3) pending end-to-end tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants