feat: add `dedupe` flag #247

KATT · 2023-07-14T14:24:31Z

add dedupe flag that will make sure that complex objects only appear once in the output

KATT · 2023-07-14T14:25:18Z

src/plainer.ts

+        ? {
+            transformedValue: null,
+          }


It'd be nice with some symbol here (and for circular references fwiw) to show in the JSON that we have replaced it

Agree. I wonder if there's some other implementation that we can draw inspiration from 🤔

Apparently Node.js prints [Circular *1]:

Same with Deno. This might even be a v8 thing.

What if we did [Circular] and [Pruned]?

Skn0tt

Implementation looks solid, i'd like to pause a moment to think about the name + location of that flag.

src/index.test.ts

Skn0tt · 2023-07-14T14:39:51Z

src/plainer.ts

+        ? {
+            transformedValue: null,
+          }


Agree. I wonder if there's some other implementation that we can draw inspiration from 🤔

Apparently Node.js prints [Circular *1]:

Same with Deno. This might even be a v8 thing.

What if we did [Circular] and [Pruned]?

Skn0tt · 2023-07-14T14:44:29Z

src/index.ts

+  /**
+   * If true, SuperJSON will make sure only one instance of referentially equal objects are serialized and the rest are replaced with `null`.
+   */
+  public dedupe = false;


My gut instinct was to put this as an optional parameter to serialize, and call it something like pruneReferentialEqualities, in reference to meta.referentialEqualities. I'd also be fine with dedupeReferentialEqualities. It's a lot longer of a name, but more descriptive I guess? But also harder to parse, so i'm torn between the two. Any strong feelings from your end on this?

Adding an optional argument to serialize could be seen as a breaking change in cases like items.map(superjson.serialize.bind(superjson) 🙃

I see where you're coming from, but I don't necessarily agree with that strict of a definition of "breaking change" 😅

KATT · 2023-07-14T14:49:12Z

You decide everything and me / @juliusmarminge will just follow suit. :)

Feel free to just push to my PR or make another stab at it, I gotta go for today now!

KATT · 2023-07-14T14:49:35Z

FWIW, changing the circular to anything else but null could be seen as a breaking change - so maybe it's better to have both as null for now

Skn0tt · 2023-07-17T08:20:12Z

Published as part of https://github.com/blitz-js/superjson/releases/tag/v1.13.0

juliusmarminge · 2023-07-17T08:52:47Z

Published as part of https://github.com/blitz-js/superjson/releases/tag/v1.13.0

Awesome. Unfortunately, for some reason I get an error when running the compiled version yelling that SuperJSON isn't a constructor 🤔

Skn0tt · 2023-07-17T08:56:16Z

Can reproduce locally, let's see what's causing it 🤔

Skn0tt · 2023-07-17T09:03:58Z

It looks like it's related to how TypeScript emits the export default class. Seems like that bug has existed before this PR, but nobody tried doing new SuperJSON until now!

juliusmarminge · 2023-07-17T09:06:07Z

It looks like it's related to how TypeScript emits the export default class

Bundling javascript....

Seems like that bug has existed before this PR, but nobody tried doing new SuperJSON until now!

Seems reasonable. Nothing in this PR should cause difference in behavior in that regard

Skn0tt · 2023-07-17T09:07:36Z

As a workaround, could you try const SuperJSON = require(".").default? That fixed it for me locally.

juliusmarminge · 2023-07-17T09:11:37Z

As a workaround, could you try const SuperJSON = require(".").default? That fixed it for me locally.

How do you mean? The relative require('.') won't work outside of the local repo. I tried this without success though:

import type SuperJSON from 'superjson'

// eslint-disable-next-line @typescript-eslint/no-var-requires, unicorn/prefer-module
export const superjson: SuperJSON = require('superjson').default({
	dedupe: true,
})

Skn0tt · 2023-07-17T09:13:57Z

Oops, meant to replace the . with superjson. But you guessed right, that's what I was referring to - doesn't seem to work, though. I'll see what I can do to fix the import ...

juliusmarminge · 2023-07-17T09:22:05Z

Oops, meant to replace the . with superjson. But you guessed right, that's what I was referring to - doesn't seem to work, though. I'll see what I can do to fix the import ...

Made a temporary workaround that works if you use a named import instead of the default one in #250 🤷🏼‍♂️

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [superjson](https://togithub.com/blitz-js/superjson) | [`1.12.4` -> `1.13.1`](https://renovatebot.com/diffs/npm/superjson/1.12.4/1.13.1) | [![age](https://developer.mend.io/api/mc/badges/age/npm/superjson/1.13.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/npm/superjson/1.13.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/npm/superjson/1.12.4/1.13.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/superjson/1.12.4/1.13.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>blitz-js/superjson (superjson)</summary> ### [`v1.13.1`](https://togithub.com/blitz-js/superjson/releases/tag/v1.13.1) [Compare Source](https://togithub.com/blitz-js/superjson/compare/v1.13.0...v1.13.1) #### What's Changed - Bump json5 from 1.0.1 to 1.0.2 by [@dependabot](https://togithub.com/dependabot) in [blitz-js/superjson#218 - Bump minimatch from 3.0.4 to 3.1.2 by [@dependabot](https://togithub.com/dependabot) in [blitz-js/superjson#213 - Bump decode-uri-component from 0.2.0 to 0.2.2 by [@dependabot](https://togithub.com/dependabot) in [blitz-js/superjson#212 - Bump tough-cookie from 4.1.2 to 4.1.3 by [@dependabot](https://togithub.com/dependabot) in [blitz-js/superjson#248 - fix: TypeError: SuperJSON is not a constructor by [@juliusmarminge](https://togithub.com/juliusmarminge) in [blitz-js/superjson#250 #### New Contributors - [@juliusmarminge](https://togithub.com/juliusmarminge) made their first contribution in [blitz-js/superjson#250 **Full Changelog**: blitz-js/superjson@v1.13.0...v1.13.1 ### [`v1.13.0`](https://togithub.com/blitz-js/superjson/releases/tag/v1.13.0) [Compare Source](https://togithub.com/blitz-js/superjson/compare/v1.12.4...v1.13.0) #### What's Changed - fix: referential equalities should ignore child nodes by [@KATT](https://togithub.com/KATT) in [blitz-js/superjson#244 - feat: add `dedupe` flag by [@KATT](https://togithub.com/KATT) in [blitz-js/superjson#247 **Full Changelog**: blitz-js/superjson@v1.12.4...v1.13.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/Chia1104/chia1104.dev).

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [superjson](https://togithub.com/blitz-js/superjson) | [`1.12.4` -> `1.13.1`](https://renovatebot.com/diffs/npm/superjson/1.12.4/1.13.1) | [![age](https://developer.mend.io/api/mc/badges/age/npm/superjson/1.13.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/npm/superjson/1.13.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/npm/superjson/1.12.4/1.13.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/superjson/1.12.4/1.13.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>blitz-js/superjson (superjson)</summary> ### [`v1.13.1`](https://togithub.com/blitz-js/superjson/releases/tag/v1.13.1) [Compare Source](https://togithub.com/blitz-js/superjson/compare/v1.13.0...v1.13.1) #### What's Changed - Bump json5 from 1.0.1 to 1.0.2 by [@dependabot](https://togithub.com/dependabot) in [blitz-js/superjson#218 - Bump minimatch from 3.0.4 to 3.1.2 by [@dependabot](https://togithub.com/dependabot) in [blitz-js/superjson#213 - Bump decode-uri-component from 0.2.0 to 0.2.2 by [@dependabot](https://togithub.com/dependabot) in [blitz-js/superjson#212 - Bump tough-cookie from 4.1.2 to 4.1.3 by [@dependabot](https://togithub.com/dependabot) in [blitz-js/superjson#248 - fix: TypeError: SuperJSON is not a constructor by [@juliusmarminge](https://togithub.com/juliusmarminge) in [blitz-js/superjson#250 #### New Contributors - [@juliusmarminge](https://togithub.com/juliusmarminge) made their first contribution in [blitz-js/superjson#250 **Full Changelog**: blitz-js/superjson@v1.13.0...v1.13.1 ### [`v1.13.0`](https://togithub.com/blitz-js/superjson/releases/tag/v1.13.0) [Compare Source](https://togithub.com/blitz-js/superjson/compare/v1.12.4...v1.13.0) #### What's Changed - fix: referential equalities should ignore child nodes by [@KATT](https://togithub.com/KATT) in [blitz-js/superjson#244 - feat: add `dedupe` flag by [@KATT](https://togithub.com/KATT) in [blitz-js/superjson#247 **Full Changelog**: blitz-js/superjson@v1.12.4...v1.13.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/Chia1104/chia-stack).

KATT added 10 commits July 14, 2023 14:44

add failing test

0d95739

add failing test

b3f907f

tweak

2b4dbe6

fix test

7b0742d

Update src/plainer.ts

b1fb75a

Update src/plainer.ts

86f0295

less diff

1d134ee

Merge branch 'regression' of github.com:KATT/superjson into regression

ea9f0b0

feat: add dedupe flag

cb35784

feat: add dedupe flag

c18c4e6

KATT requested a review from Skn0tt as a code owner July 14, 2023 14:24

KATT commented Jul 14, 2023

View reviewed changes

Skn0tt reviewed Jul 14, 2023

View reviewed changes

KATT and others added 5 commits July 14, 2023 16:50

rm useless test

abf87c8

refactor: name dedupeReferentialEqualities

dc0e4b0

fix: dedupe is enough

cacdd60

fix: test

466de68

fix: test

308d15b

Skn0tt approved these changes Jul 17, 2023

View reviewed changes

Skn0tt merged commit dd3aeb5 into blitz-js:main Jul 17, 2023
3 checks passed

juliusmarminge mentioned this pull request Jul 17, 2023

fix: TypeError: SuperJSON is not a constructor #250

Merged

juliusmarminge mentioned this pull request Jul 17, 2023

fix: insert "symbols" instead of null for deduped objects #251

Open

KATT deleted the compression branch July 17, 2023 14:22

bullder mentioned this pull request May 24, 2024

[Snyk] Upgrade superjson from 1.12.2 to 1.13.3 bullder/realworld-nextjs-trpc-prisma#4

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add `dedupe` flag #247

feat: add `dedupe` flag #247

KATT commented Jul 14, 2023 •

edited

KATT Jul 14, 2023

Skn0tt Jul 14, 2023

Skn0tt left a comment

Skn0tt Jul 14, 2023

Skn0tt Jul 14, 2023

KATT Jul 14, 2023

Skn0tt Jul 17, 2023

KATT commented Jul 14, 2023

KATT commented Jul 14, 2023

Skn0tt commented Jul 17, 2023

juliusmarminge commented Jul 17, 2023

Skn0tt commented Jul 17, 2023

Skn0tt commented Jul 17, 2023

juliusmarminge commented Jul 17, 2023 •

edited

Skn0tt commented Jul 17, 2023

juliusmarminge commented Jul 17, 2023 •

edited

Skn0tt commented Jul 17, 2023

juliusmarminge commented Jul 17, 2023

feat: add dedupe flag #247

feat: add dedupe flag #247

Conversation

KATT commented Jul 14, 2023 • edited

KATT Jul 14, 2023

Choose a reason for hiding this comment

Skn0tt Jul 14, 2023

Choose a reason for hiding this comment

Skn0tt left a comment

Choose a reason for hiding this comment

Skn0tt Jul 14, 2023

Choose a reason for hiding this comment

Skn0tt Jul 14, 2023

Choose a reason for hiding this comment

KATT Jul 14, 2023

Choose a reason for hiding this comment

Skn0tt Jul 17, 2023

Choose a reason for hiding this comment

KATT commented Jul 14, 2023

KATT commented Jul 14, 2023

Skn0tt commented Jul 17, 2023

juliusmarminge commented Jul 17, 2023

Skn0tt commented Jul 17, 2023

Skn0tt commented Jul 17, 2023

juliusmarminge commented Jul 17, 2023 • edited

Skn0tt commented Jul 17, 2023

juliusmarminge commented Jul 17, 2023 • edited

Skn0tt commented Jul 17, 2023

juliusmarminge commented Jul 17, 2023

feat: add `dedupe` flag #247

feat: add `dedupe` flag #247

KATT commented Jul 14, 2023 •

edited

juliusmarminge commented Jul 17, 2023 •

edited

juliusmarminge commented Jul 17, 2023 •

edited