Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LOD for images #3684

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

LOD for images #3684

wants to merge 11 commits into from

Conversation

steveruizok
Copy link
Collaborator

@steveruizok steveruizok commented May 2, 2024

This PR adds support for "level of detail" for image shapes. When an image is created, we create multiple versions of that image (each smaller than the last) and display the smallest image for the current zoom level.

Some thoughts here:

  • It works! and is quite nice.
  • Very hard to notice the switching between sizes, even with big jumps. (1, 1/2, 1/4, etc.)
  • This could make image-heavy rooms much more performant.
  • This could prevent crashes on image-heavy rooms on iOS / memory-sensitive devices

For shared rooms / hosted assets

  • the performance benefits are definitely worth the extra storage space for us on the backend
  • we could bump the maximum size for images on hosted rooms

For offline rooms / serialized assets

  • it's not clear that the performance benefits for offline are worth the extra storage space, because the extra image versions would be stored in the document JSON itself.
  • Offline projects should probably continue to use a single downscaled images

For the API

  • it would be nice to make the TLImageAsset.props.src be either a string or an array of scale / strings, so that this isn't a breaking change for anyone.
  • In this PR we have to compare the shape's width vs. the asset's width in order to determine how the image has been resized. The logic is a little complicated, maybe there's another way to do it, maybe not

Change Type

  • sdk — Changes the tldraw SDK
  • feature — New feature

@huppy-bot huppy-bot bot added feature sdk Affects the tldraw sdk labels May 2, 2024
Copy link

vercel bot commented May 2, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
examples ✅ Ready (Inspect) Visit Preview May 13, 2024 1:00pm
1 Ignored Deployment
Name Status Preview Updated (UTC)
tldraw-docs ⬜️ Ignored (Inspect) Visit Preview May 13, 2024 1:00pm

Comment on lines +86 to +89
await fetch(url, {
method: 'POST',
body: file,
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we tweak this to perform uploads in parallel?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we did this in parallel it feels like that might be potentially harder on bandwidth, perhaps for certain users?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair! tbh for this multiplayer one we could always look at using https://developers.cloudflare.com/images/transform-images/ if we wanted to really optimize this - just upload the original, let the server handle resizing on-demand (plus optimizing for smaller file size in a way that the browser won't out of the box)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hahah, Steve and I were indeed talking about this same thing this morning (albeit with Imgix, not Cloudflare's offering). I'll pull you aside and we can chat more offline on your thoughts on this.

Comment on lines 95 to 96
src: await FileHelpers.blobToDataUrl(
await downsizeImage(file, size.w, size.h, {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto re: doing some of this down-sizing in parallel

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hah, i have a similar concern here on parallelization with less powerful machines and large images :)

@@ -171,11 +180,15 @@ export class ImageShapeUtil extends BaseBoxShapeUtil<TLImageShape> {
}

override async toSvg(shape: TLImageShape) {
const asset = shape.props.assetId ? this.editor.getAsset(shape.props.assetId) : null

const zoomLevel = useValue('zoom level', () => this.editor.getZoomLevel(), [this.editor])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo svg exports shouldn't depend on the editor's zoom level - we usually just assume zoom = 100% for exports

@SomeHats
Copy link
Contributor

SomeHats commented May 8, 2024

When I tried a this on a multiplayer room preview i got an error when adding an image: At asset(type = image).props.sources: Expected an array, got undefined.

When I tried creating a room with an image locally and then sharing it, I got a similar error from the server

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature sdk Affects the tldraw sdk
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants