Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Faster Assign type #2594

Closed

Conversation

likewerealive
Copy link

@likewerealive likewerealive commented May 14, 2024

📝 Description

Update Assign type to use Omit to speed up type checking

⛳️ Current behavior (updates)

The current Assign can be very slow when type checking

🚀 New behavior

Assign is much faster. This mimics the type from ark-ui

💣 Is this a breaking change (Yes/No):

No

📝 Additional Information

If we have a ui-lib with a panda button component like:

const buttonRecipe = cva({
  base: {
    borderRadius: 'md',
    fontWeight: 'semibold',
    h: '10',
    px: '4',
  },
  variants: {
    visual: {
      solid: {
        bg: { base: 'colorPalette.500', _dark: 'colorPalette.300' },
        color: { base: 'white', _dark: 'gray.800' },
      },
      outline: {
        border: '1px solid',
        color: { base: 'colorPalette.600', _dark: 'colorPalette.200' },
        borderColor: 'currentColor',
      },
    },
  },
  defaultVariants: {
    visual: 'solid'
  }
});

type ButtonProps = HTMLStyledProps<'button'> & RecipeVariantProps<typeof buttonRecipe> & {
  isLoading?: boolean;
}

const StyledButton = styled('button', buttonRecipe);

export const Button = ({ isLoading, children, ...props }: ButtonProps) => {
  return <StyledButton {...props}>{isLoading ? 'Loading' : children}</StyledButton>
}

And we want to add some special variants to a single use-case of this Button in one product, we can do:

const ExtendedButton = styled(Button, {
  variants: {
    isSpecial: {
      true: {
        bg: 'red.500'
      }
    }
  }
})

This runs fine in the app, but when it comes to type checking this hangs for a long time. I've found that updating the Assign type to mimic Ark-UI fixes this! I had a file that only contained an override, like the above, take 151,008.080ms to type check. With this fix it took 162.014ms

Copy link

changeset-bot bot commented May 14, 2024

🦋 Changeset detected

Latest commit: 5ae1c61

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 19 packages
Name Type
@pandacss/generator Patch
@pandacss/studio Patch
@pandacss/types Patch
@pandacss/node Patch
@pandacss/parser Patch
@pandacss/dev Patch
@pandacss/config Patch
@pandacss/core Patch
@pandacss/logger Patch
@pandacss/preset-atlaskit Patch
@pandacss/preset-base Patch
@pandacss/preset-open-props Patch
@pandacss/preset-panda Patch
@pandacss/token-dictionary Patch
@pandacss/astro-plugin-studio Patch
@pandacss/postcss Patch
@pandacss/extractor Patch
@pandacss/is-valid-prop Patch
@pandacss/shared Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link

vercel bot commented May 14, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
panda-playground ✅ Ready (Inspect) Visit Preview May 22, 2024 11:42am
panda-studio ✅ Ready (Inspect) Visit Preview May 22, 2024 11:42am

Copy link

vercel bot commented May 14, 2024

@likewerealive is attempting to deploy a commit to the Chakra UI Team on Vercel.

A member of the Team first needs to authorize it.

@likewerealive
Copy link
Author

likewerealive commented May 14, 2024

I'm unsure of the fixes in playground here - I saw that it had the assign type from a generation, so i ran build:playground to update. I can pull those files out if not needed though!

@vercel vercel bot temporarily deployed to Preview – panda-playground May 14, 2024 15:23 Inactive
@vercel vercel bot temporarily deployed to Preview – panda-studio May 14, 2024 15:24 Inactive
@astahmer
Copy link
Collaborator

hey, sorry for the answer delay ! I'll look into this PR asap, I mostly need to benchmark stuff as this impact a lot of types indirectly

@astahmer
Copy link
Collaborator

astahmer commented May 22, 2024

tldr: so, I spent a fair amount of time on this and it doesn't seem like this makes it faster, on the contrary

Benchmark with @arktype/attest

Testing with @arktype/attest will give you deterministic results, meaning you always get the same result given the same input.

  • I've used a sandbox/codegen/__tests__/types.bench.tsx file with this as the content:
import { bench } from '@arktype/attest'

import { cva } from '../styled-system/css'
import { styled } from '../styled-system/jsx'
import { HTMLStyledProps, RecipeVariantProps } from '../styled-system/types'

bench('Omit', () => {
  const buttonRecipe = cva({
    base: {
      borderRadius: 'md',
      fontWeight: 'semibold',
      h: '10',
      px: '4',
    },
    variants: {
      visual: {
        solid: {
          bg: { base: 'colorPalette.500', _dark: 'colorPalette.300' },
          color: { base: 'white', _dark: 'gray.800' },
        },
        outline: {
          border: '1px solid',
          color: { base: 'colorPalette.600', _dark: 'colorPalette.200' },
          borderColor: 'currentColor',
        },
      },
    },
    defaultVariants: {
      visual: 'solid',
    },
  })

  type ButtonProps = HTMLStyledProps<'button'> &
    RecipeVariantProps<typeof buttonRecipe> & {
      isLoading?: boolean
    }

  const StyledButton = styled('button', buttonRecipe)

  const Button = ({ isLoading, children, ...props }: ButtonProps) => {
    return <StyledButton {...props}>{isLoading ? 'Loading' : children}</StyledButton>
  }
  const ExtendedButton = styled(Button, {
    variants: {
      isSpecial: {
        true: {
          bg: 'red.500',
        },
      },
    },
  })

  return {} as any as typeof ExtendedButton
}).types([58034, 'instantiations'])
"bench": "bun run ./__tests__/types.bench.tsx",
  • Then I ran 3 times the same command, updating the sandbox/codegen/styled-system/types/system-types.d.ts between each run with these 3 different Assign type:
// 1
// export type Assign<T, U> = {
//   [K in keyof T]: K extends keyof U ? U[K] : T[K]
// } & U

// 2
// export type Assign<T, U> = DistributiveOmit<T, keyof U> & U

// 3
export type Assign<T, U> = Omit<T, keyof U> & U

This is the result, in the above order (1 = Original, 2 = DistributeOmit, 3 = Omit):

panda/sandbox/codegen main*​​ ≡3s 
❯ bun bench
$ bun run ./__tests__/types.bench.tsx
🏌️  Omit
⛳ Result: 58034 instantiations
🎯 Baseline: 58034 instantiations
📊 Delta: 0.00%


panda/sandbox/codegen main*​​ ≡3s 
❯ bun bench
$ bun run ./__tests__/types.bench.tsx
🏌️  Omit
⛳ Result: 202002 instantiations
🎯 Baseline: 58034 instantiations
📈 'Omit' exceeded baseline by 248.08% (threshold is 20%).


panda/sandbox/codegen main*​​ ≡2s 
❯ bun bench
$ bun run ./__tests__/types.bench.tsx
🏌️  Omit
⛳ Result: 235239 instantiations
🎯 Baseline: 58034 instantiations
📈 'Omit' exceeded baseline by 305.35% (threshold is 20%).

It seems like the original (what we currently have in Panda) leads to less type instantiations

Typescript analyze-trace

Since the above benchmark is an isolated case, which could be far from the real-world, I thought of trying an actual tsc run in a real-world scenario: Park-UI is a component library that uses a lot of styled

So, in the park-ui repo I went in the website folder and ran this command 3 times again (same order, updated the website/styled-system/types/system-types.d.ts file and removed the trace-dir between each run)

bun tsc -b --generateTrace trace-dir 
bun x @typescript/analyze-trace trace-dir --forceMillis=100 --skipMillis=50

Below is a summary of the 3 runs (I removed most of the details):


name = Assign 1 (Original):
time = 9s
type =
export type Assign<T, U> = {
[K in keyof T]: K extends keyof U ? U[K] : T[K]
} & U

report =

Check file /website/src/components/demos/tags-input.demo.tsx (265ms)
Check file /website/src/components/ui/avatar.tsx (241ms)
Check file /website/src/lib/create-style-context.tsx (223ms)
Check file /plugins/panda/src/global-css.ts (183ms)
Check file /website/src/components/ui/number-input.tsx (177ms)
Check file /website/src/components/demos/toast.demo.tsx (175ms)
Check file /website/src/content/config.ts (129ms)
Check file /website/src/components/ui/text.tsx (128ms)


name = Assign 2 (DistributiveOmit):
time = 11s
type = export type Assign<T, U> = DistributiveOmit<T, keyof U> & U

report =

Check file /website/src/components/ui/avatar.tsx (435ms)
Check file /website/src/components/demos/tags-input.demo.tsx (311ms)
Check file /website/src/lib/create-style-context.tsx (243ms)
Check file /website/src/components/ui/form-label.tsx (221ms)
Check file /plugins/panda/src/global-css.ts (186ms)
Check file /website/src/components/ui/badge.tsx (179ms)
Check file /website/src/components/ui/textarea.tsx (168ms)
Check file /website/src/components/demos/toast.demo.tsx (166ms)
Check file /website/src/components/ui/link.tsx (154ms)
Check file /website/src/components/ui/color-picker.tsx (144ms)
Check file /website/src/components/ui/text.tsx (143ms)
Check file /website/src/components/ui/input.tsx (143ms)
Check file /website/src/components/ui/date-picker.tsx (140ms)
Check file /website/src/components/ui/button.tsx (136ms)
Check file /website/src/components/ui/code.tsx (127ms)
Check file /website/src/content/config.ts (126ms)


name = Assign 3 (Omit)
time = 13s
type = export type Assign<T, U> = Omit<T, keyof U> & U

report =

Check file /website/src/components/ui/badge.tsx (348ms)
Check file /website/src/components/ui/input.tsx (343ms)
Check file /website/src/components/ui/textarea.tsx (331ms)
Check file /website/src/components/ui/button.tsx (328ms)
Check file /website/src/components/ui/form-label.tsx (319ms)
Check file /website/src/components/ui/link.tsx (318ms)
Check file /website/src/components/ui/code.tsx (317ms)
Check file /website/src/components/ui/avatar.tsx (305ms)
Check file /website/src/components/demos/tags-input.demo.tsx (294ms)
Check file /website/src/components/demos/toast.demo.tsx (232ms)
Check file /website/src/components/ui/color-picker.tsx (223ms)
Check file /website/src/lib/create-style-context.tsx (215ms)
Check file /plugins/panda/src/global-css.ts (208ms)
Check file /website/src/components/ui/date-picker.tsx (186ms)
Check file /website/src/components/ui/combobox.tsx (178ms)
Check file /website/src/components/ui/text.tsx (131ms)
Check file /website/src/content/config.ts (118ms)
Check file /website/src/components/ui/select.tsx (112ms)
Check file /website/src/components/ui/menu.tsx (100ms)


Markdown Table of Time Taken for Each File

I asked gpt to make a pretty markdown table with those infos, here it is:

File Path Assign 1 (Original) Assign 2 (DistributiveOmit) Assign 3 (Omit)
/website/src/components/demos/tags-input.demo.tsx 265ms 311ms 294ms
/website/src/components/ui/avatar.tsx 241ms 435ms 305ms
/website/src/lib/create-style-context.tsx 223ms 243ms 215ms
/plugins/panda/src/global-css.ts 183ms 186ms 208ms
/website/src/components/ui/number-input.tsx 177ms - -
/website/src/components/demos/toast.demo.tsx 175ms 166ms 232ms
/website/src/content/config.ts 129ms 126ms 118ms
/website/src/components/ui/text.tsx 128ms 143ms 131ms
/website/src/components/ui/form-label.tsx - 221ms 319ms
/website/src/components/ui/badge.tsx - 179ms 348ms
/website/src/components/ui/textarea.tsx - 168ms 331ms
/website/src/components/ui/link.tsx - 154ms 318ms
/website/src/components/ui/color-picker.tsx - 144ms 223ms
/website/src/components/ui/input.tsx - 143ms 343ms
/website/src/components/ui/date-picker.tsx - 140ms 186ms
/website/src/components/ui/button.tsx - 136ms 328ms
/website/src/components/ui/code.tsx - 127ms 317ms
/website/src/components/ui/combobox.tsx - - 178ms
/website/src/components/ui/select.tsx - - 112ms
/website/src/components/ui/menu.tsx - - 100ms
Total Time Taken 1521ms 3022ms 4606ms

Conclusion

It seems what we currently have is the fastest of the 3, even though the proposed alternatives can be faster in some cases (like yours ?), so I think we can't merge this PR, unless I made a mistake somewhere

@likewerealive
Copy link
Author

tldr: so, I spent a fair amount of time on this and it doesn't seem like this makes it faster, on the contrary

Benchmark with @arktype/attest

Testing with @arktype/attest will give you deterministic results, meaning you always get the same result given the same input.

  • I've used a sandbox/codegen/__tests__/types.bench.tsx file with this as the content:
import { bench } from '@arktype/attest'

import { cva } from '../styled-system/css'
import { styled } from '../styled-system/jsx'
import { HTMLStyledProps, RecipeVariantProps } from '../styled-system/types'

bench('Omit', () => {
  const buttonRecipe = cva({
    base: {
      borderRadius: 'md',
      fontWeight: 'semibold',
      h: '10',
      px: '4',
    },
    variants: {
      visual: {
        solid: {
          bg: { base: 'colorPalette.500', _dark: 'colorPalette.300' },
          color: { base: 'white', _dark: 'gray.800' },
        },
        outline: {
          border: '1px solid',
          color: { base: 'colorPalette.600', _dark: 'colorPalette.200' },
          borderColor: 'currentColor',
        },
      },
    },
    defaultVariants: {
      visual: 'solid',
    },
  })

  type ButtonProps = HTMLStyledProps<'button'> &
    RecipeVariantProps<typeof buttonRecipe> & {
      isLoading?: boolean
    }

  const StyledButton = styled('button', buttonRecipe)

  const Button = ({ isLoading, children, ...props }: ButtonProps) => {
    return <StyledButton {...props}>{isLoading ? 'Loading' : children}</StyledButton>
  }
  const ExtendedButton = styled(Button, {
    variants: {
      isSpecial: {
        true: {
          bg: 'red.500',
        },
      },
    },
  })

  return {} as any as typeof ExtendedButton
}).types([58034, 'instantiations'])
"bench": "bun run ./__tests__/types.bench.tsx",
  • Then I ran 3 times the same command, updating the sandbox/codegen/styled-system/types/system-types.d.ts between each run with these 3 different Assign type:
// 1
// export type Assign<T, U> = {
//   [K in keyof T]: K extends keyof U ? U[K] : T[K]
// } & U

// 2
// export type Assign<T, U> = DistributiveOmit<T, keyof U> & U

// 3
export type Assign<T, U> = Omit<T, keyof U> & U

This is the result, in the above order (1 = Original, 2 = DistributeOmit, 3 = Omit):

panda/sandbox/codegen main*​​ ≡3s 
❯ bun bench
$ bun run ./__tests__/types.bench.tsx
🏌️  Omit
⛳ Result: 58034 instantiations
🎯 Baseline: 58034 instantiations
📊 Delta: 0.00%


panda/sandbox/codegen main*​​ ≡3s 
❯ bun bench
$ bun run ./__tests__/types.bench.tsx
🏌️  Omit
⛳ Result: 202002 instantiations
🎯 Baseline: 58034 instantiations
📈 'Omit' exceeded baseline by 248.08% (threshold is 20%).


panda/sandbox/codegen main*​​ ≡2s 
❯ bun bench
$ bun run ./__tests__/types.bench.tsx
🏌️  Omit
⛳ Result: 235239 instantiations
🎯 Baseline: 58034 instantiations
📈 'Omit' exceeded baseline by 305.35% (threshold is 20%).

It seems like the original (what we currently have in Panda) leads to less type instantiations

Typescript analyze-trace

Since the above benchmark is an isolated case, which could be far from the real-world, I thought of trying an actual tsc run in a real-world scenario: Park-UI is a component library that uses a lot of styled

So, in the park-ui repo I went in the website folder and ran this command 3 times again (same order, updated the website/styled-system/types/system-types.d.ts file and removed the trace-dir between each run)

bun tsc -b --generateTrace trace-dir 
bun x @typescript/analyze-trace trace-dir --forceMillis=100 --skipMillis=50

Below is a summary of the 3 runs (I removed most of the details):

name = Assign 1 (Original): time = 9s type = export type Assign<T, U> = { [K in keyof T]: K extends keyof U ? U[K] : T[K] } & U

report =

Check file /website/src/components/demos/tags-input.demo.tsx (265ms) Check file /website/src/components/ui/avatar.tsx (241ms) Check file /website/src/lib/create-style-context.tsx (223ms) Check file /plugins/panda/src/global-css.ts (183ms) Check file /website/src/components/ui/number-input.tsx (177ms) Check file /website/src/components/demos/toast.demo.tsx (175ms) Check file /website/src/content/config.ts (129ms) Check file /website/src/components/ui/text.tsx (128ms)

name = Assign 2 (DistributiveOmit): time = 11s type = export type Assign<T, U> = DistributiveOmit<T, keyof U> & U

report =

Check file /website/src/components/ui/avatar.tsx (435ms) Check file /website/src/components/demos/tags-input.demo.tsx (311ms) Check file /website/src/lib/create-style-context.tsx (243ms) Check file /website/src/components/ui/form-label.tsx (221ms) Check file /plugins/panda/src/global-css.ts (186ms) Check file /website/src/components/ui/badge.tsx (179ms) Check file /website/src/components/ui/textarea.tsx (168ms) Check file /website/src/components/demos/toast.demo.tsx (166ms) Check file /website/src/components/ui/link.tsx (154ms) Check file /website/src/components/ui/color-picker.tsx (144ms) Check file /website/src/components/ui/text.tsx (143ms) Check file /website/src/components/ui/input.tsx (143ms) Check file /website/src/components/ui/date-picker.tsx (140ms) Check file /website/src/components/ui/button.tsx (136ms) Check file /website/src/components/ui/code.tsx (127ms) Check file /website/src/content/config.ts (126ms)

name = Assign 3 (Omit) time = 13s type = export type Assign<T, U> = Omit<T, keyof U> & U

report =

Check file /website/src/components/ui/badge.tsx (348ms) Check file /website/src/components/ui/input.tsx (343ms) Check file /website/src/components/ui/textarea.tsx (331ms) Check file /website/src/components/ui/button.tsx (328ms) Check file /website/src/components/ui/form-label.tsx (319ms) Check file /website/src/components/ui/link.tsx (318ms) Check file /website/src/components/ui/code.tsx (317ms) Check file /website/src/components/ui/avatar.tsx (305ms) Check file /website/src/components/demos/tags-input.demo.tsx (294ms) Check file /website/src/components/demos/toast.demo.tsx (232ms) Check file /website/src/components/ui/color-picker.tsx (223ms) Check file /website/src/lib/create-style-context.tsx (215ms) Check file /plugins/panda/src/global-css.ts (208ms) Check file /website/src/components/ui/date-picker.tsx (186ms) Check file /website/src/components/ui/combobox.tsx (178ms) Check file /website/src/components/ui/text.tsx (131ms) Check file /website/src/content/config.ts (118ms) Check file /website/src/components/ui/select.tsx (112ms) Check file /website/src/components/ui/menu.tsx (100ms)

Markdown Table of Time Taken for Each File

I asked gpt to make a pretty markdown table with those infos, here it is:

File Path Assign 1 (Original) Assign 2 (DistributiveOmit) Assign 3 (Omit)
/website/src/components/demos/tags-input.demo.tsx 265ms 311ms 294ms
/website/src/components/ui/avatar.tsx 241ms 435ms 305ms
/website/src/lib/create-style-context.tsx 223ms 243ms 215ms
/plugins/panda/src/global-css.ts 183ms 186ms 208ms
/website/src/components/ui/number-input.tsx 177ms - -
/website/src/components/demos/toast.demo.tsx 175ms 166ms 232ms
/website/src/content/config.ts 129ms 126ms 118ms
/website/src/components/ui/text.tsx 128ms 143ms 131ms
/website/src/components/ui/form-label.tsx - 221ms 319ms
/website/src/components/ui/badge.tsx - 179ms 348ms
/website/src/components/ui/textarea.tsx - 168ms 331ms
/website/src/components/ui/link.tsx - 154ms 318ms
/website/src/components/ui/color-picker.tsx - 144ms 223ms
/website/src/components/ui/input.tsx - 143ms 343ms
/website/src/components/ui/date-picker.tsx - 140ms 186ms
/website/src/components/ui/button.tsx - 136ms 328ms
/website/src/components/ui/code.tsx - 127ms 317ms
/website/src/components/ui/combobox.tsx - - 178ms
/website/src/components/ui/select.tsx - - 112ms
/website/src/components/ui/menu.tsx - - 100ms
Total Time Taken 1521ms 3022ms 4606ms

Conclusion

It seems what we currently have is the fastest of the 3, even though the proposed alternatives can be faster in some cases (like yours ?), so I think we can't merge this PR, unless I made a mistake somewhere

Hey @astahmer !

Thanks so much for the detailed response here! I couldn't get attest working in the short time i looked into it, but your code examples here have actually cleared that up for me!
I was racking my brain as to why this worked fine for you and i found what was ultimately wrong in our codebase. We were still using the emitPackage version of the panda. I have today refactored our code to use the import map package version and the types are fast again! Apologies for any time wasted here!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants