Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discard vs. Drop #2103

Open
OlegDokuka opened this issue Apr 5, 2020 · 4 comments · May be fixed by #3240
Open

Discard vs. Drop #2103

OlegDokuka opened this issue Apr 5, 2020 · 4 comments · May be fixed by #3240
Labels
area/doOnDiscard This belongs to the doOnDiscard theme for/team-attention This issue needs team attention or action status/need-decision This needs a decision from the team status/need-design This needs more in depth design work type/enhancement A general enhancement
Milestone

Comments

@OlegDokuka
Copy link
Contributor

OlegDokuka commented Apr 5, 2020

problem

It seems that similar issues were raised in the past, but I feel I'm not convinced enough to agree.

  1. It seems (from @smaldini words from Better names for onNextDropped vs doOnDiscard hooks, compose/transform, doOnEach #1381) that onNextDropped is a spec violation which is incorrect.

    1. Let's consider FluxTake. It does take N elements, cancel subscription upon receiving the expected number of elements to take, and set the state to done. It means that in the case of racing between CANCEL and ON_NEXT more signals can appear (imaging .cancelOn up the stream which delays a little bit cancellation) which leads that few elements can be dropped easily (which is absolutely valid case due to 2.8).
    2. In the reactor, we are over protective so all the way we double-check different flags (either cancel or done) to ensure we are not sending something extra and dropping redundant.
    3. Taking into account everything said above, onNextDropped IS DESIGNED TO HANDLE unexpected along with racing between cancellation and onNext signals.
  2. Now we have another operator to handle filtered (discarded) elements on the level of the local stream, BUT it has significant design FLAW:

    1. Hooks are localized, so the discarded element is consumed only if the hook is present.
    2. It is SUPER expensive in terms of performance when it comes to Mono(or even from flux perspective) where we constantly have 3-4 objects overhead (FluxContextStart + ContextStartSubscriber + Consumer<? super R> discardHook + created Context) is WAY expensive (💔)

proposal

The only way to make it consistent is to merge both drop and discard APIs and make them interconnected, e.g if no local hooks discard hooks installed, we are free to lookup for global hooks. Since Operators.onNextDrop is, let's call it, internal API, I don't see any problem to UNIFY that in 3.4

@simonbasle simonbasle added area/doOnDiscard This belongs to the doOnDiscard theme for/team-attention This issue needs team attention or action status/need-decision This needs a decision from the team type/enhancement A general enhancement labels Apr 6, 2020
@simonbasle simonbasle added this to the 3.4 planning milestone Apr 6, 2020
@simonbasle simonbasle modified the milestones: 3.4 planning, 3.4.0-M1, 3.4.0-M2 May 5, 2020
@simonbasle simonbasle modified the milestones: 3.4.0-M2, 3.4.0 Aug 7, 2020
@simonbasle simonbasle added the status/need-design This needs more in depth design work label Oct 8, 2020
@simonbasle simonbasle modified the milestones: 3.4.0, 3.5 planning Oct 8, 2020
@simonbasle
Copy link
Member

Unfortunately, it seems too late to address that one in time for 3.4, considering how late we already are in the pre-release cycle. Thus I'm moving this to 3.5 planning.

@cavallium
Copy link

Any update?

@OlegDokuka
Copy link
Contributor Author

@simonbasle @chemicL I think it is a good oportunity to deprecate Operators.onNextDropped for 3.5.0 while making it private in 3.6

@simonbasle
Copy link
Member

a bit of context on #3229 being closed (as discussed off-band): this wasn't likely addressing the right issue or using the right angle.
the onNextDropped hook should be single-responsability: a way to track malformed onNext signals. It shouldn't double as a way to discard value, which is a responsability of onDiscard. but one issue is that there is no global hook for onDiscard.

so a preferable solution would be to introduce a dedicated global hook in Hooks that Operators.onDiscard can fallback on when no hook is defined locally in the Context. then, make sure that wherever onNextDropped is called, Operators.onDiscard is also called. => #3240

finally, convey the message that global Hooks.onNextDropped shouldn't be used for discarding, and to prefer Hooks.onDiscard. if onNextDropped continues to be used for discard AND onDiscard hook is set up, the double discarding shouldn't be an issue as such hooks are required to be idempotent already.

@violetagg violetagg modified the milestones: 3.5.0, 3.5.1 Nov 8, 2022
@violetagg violetagg modified the milestones: 3.5.1, 3.5.2 Dec 13, 2022
@OlegDokuka OlegDokuka modified the milestones: 3.5.2, 3.5.3 Jan 10, 2023
@chemicL chemicL removed this from the 3.5.3 milestone Feb 14, 2023
@chemicL chemicL added this to the 3.5.4 milestone Feb 14, 2023
@violetagg violetagg modified the milestones: 3.5.4, 3.5.5, 3.5.x backlog Mar 14, 2023
@chemicL chemicL modified the milestones: 3.5.x Backlog, 3.7 Planning Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/doOnDiscard This belongs to the doOnDiscard theme for/team-attention This issue needs team attention or action status/need-decision This needs a decision from the team status/need-design This needs more in depth design work type/enhancement A general enhancement
Projects
None yet
5 participants