feat: image preprocessor (#10788)

.changeset/eighty-timers-exist.md

@@ -0,0 +1,5 @@
+---
+'@sveltejs/static-img': patch
+---
+
+feat: add experimental `@sveltejs/enhanced-img` package

.changeset/rare-owls-tease.md

@@ -0,0 +1,5 @@
+---
+'@sveltejs/kit': patch
+---
+
+fix: add vite.config.js to included files in generated tsconfig

documentation/docs/30-advanced/60-images.md

@@ -0,0 +1,150 @@
+---
+title: Images
+---
+
+Images can have a big impact on your app's performance. For best results, you should optimize them by doing the following:
+
+- generate optimal formats like `.avif` and `.webp`
+- create different sizes for different screens
+- ensure that assets can be cached effectively
+
+Doing this manually is tedious. There are a variety of techniques you can use, depending on your needs and preferences.
+
+## Vite's built-in handling
+
+[Vite will automatically process imported assets](https://vitejs.dev/guide/assets.html) for improved performance. This includes assets referenced via the CSS `url()` function. Hashes will be added to the filenames so that they can be cached, and assets smaller than `assetsInlineLimit` will be inlined. Vite's asset handling is most often used for images, but is also useful for video, audio, etc.
+
+```svelte
+<script>
+	import logo from '$lib/assets/logo.png';
+</script>
+
+<img alt="The project logo" src={logo} />
+```
+
+## @sveltejs/enhanced-img
+
+> **WARNING**: The `@sveltejs/enhanced-img` package is experimental. It uses pre-1.0 versioning and may introduce breaking changes with every minor version release.
+
+`@sveltejs/enhanced-img` builds on top of Vite's built-in asset handling. It offers plug and play image processing that serves smaller file formats like `avif` or `webp`, automatically sets the intrinsic `width` and `height` of the image to avoid layout shift, creates images of multiple sizes for various devices, and strips EXIF data for privacy. It will work in any Vite-based project including, but not limited to, SvelteKit projects.
+
+### Setup
+
+Install:
+
+```bash
+npm install --save-dev @sveltejs/enhanced-img
+```
+
+Adjust `vite.config.js`:
+
+```diff
+import { sveltekit } from '@sveltejs/kit/vite';
++import { enhancedImages } from '@sveltejs/enhanced-img';
+import { defineConfig } from 'vite';
+
+export default defineConfig({
+	plugins: [
++		enhancedImages(),
+		sveltekit()
+	]
+});
+```
+
+### Basic usage
+
+Use in your `.svelte` components by using `<enhanced:img>` rather than `<img>` and referencing the image file with a [Vite asset import](https://vitejs.dev/guide/assets.html#static-asset-handling) path:
+
+```svelte
+<enhanced:img src="./path/to/your/image.jpg" alt="An alt text" />
+```
+
+At build time, your `<enhanced:img>` tag will be replaced with an `<img>` wrapped by a `<picture>` providing multiple image types and sizes. It's only possible to downscale images without losing quality, which means that you should provide the highest resolution image that you need — smaller versions will be generated for the various device types that may request an image.
+
+You should provide your image at 2x resolution for HiDPI displays (a.k.a. retina displays). `<enhanced:img>` will automatically take care of serving smaller versions to smaller devices.
+
+If you wish to add styles to your `<enhanced:img>`, you should add a `class` and target that.
+
+### Dynamically choosing an image
+
+You can also manually import an image asset and pass it to an `<enhanced:img>`. This is useful when you have a collection of static images and would like to dynamically choose one or [iterate over them](https://github.com/sveltejs/kit/blob/master/sites/kit.svelte.dev/src/routes/home/Showcase.svelte). In this case you will need to update both the `import` statement and `<img>` element as shown below to indicate you'd like process them.
+
+```svelte
+<script>
+	import { MyImage } from './path/to/your/image.jpg?enhanced';
+</script>
+
+<enhanced:img src={MyImage} alt="Some alt text" />
+```
+
+You can also use [Vite's `import.meta.glob`](https://vitejs.dev/guide/features.html#glob-import). Note that you will have to specify `enhanced` via a [custom query](https://vitejs.dev/guide/features.html#custom-queries):
+
+```js
+const pictures = import.meta.glob(
+	'/path/to/assets/*.{avif,gif,heif,jpeg,jpg,png,tiff,webp}',
+	{
+		query: {
+			enhanced: true
+		}
+	}
+);
+```
+
+### Intrinsic Dimensions
+
+`width` and `height` are optional as they can be inferred from the source image and will be automatically added when the `<enhanced:img>` tag is preprocessed. With these attributes, the browser can reserve the correct amount of space, preventing [layout shift](https://web.dev/articles/cls). If you'd like to use a different `width` and `height` you can style the image with CSS. Because the preprocessor adds a `width` and `height` for you, if you'd like one of the dimensions to be automatically calculated then you will need to specify that:
+
+```svelte
+<style>
+	.hero-image img {
+		width: var(--size);
+		height: auto;
+	}
+</style>
+```
+
+### `srcset` and `sizes`
+
+If you have a large image, such as a hero image taking the width of the design, you should specify `sizes` so that smaller versions are requested on smaller devices. E.g. if you have a 1280px image you may want to specify something like:
+
+```svelte
+<enhanced:img src="./image.png" sizes="min(1280px, 100vw)"/>
+```
+
+If `sizes` is specified, `<enhanced:img>` will generate small images for smaller devices and populate the `srcset` attribute.
+
+The smallest picture generated automatically will have a width of 540px. If you'd like smaller images or would otherwise like to specify custom widths, you can do that with the `w` query parameter:
+```svelte
+<enhanced:img
+  src="./image.png?w=1280;640;400"
+  sizes="(min-width:1920px) 1280px, (min-width:1080px) 640px, (min-width:768px) 400px"
+/>
+```
+
+If `sizes` is not provided, then a HiDPI/Retina image and a standard resolution image will be generated. The image you provide should be 2x the resolution you wish to display so that the browser can display that image on devices with a high [device pixel ratio](https://developer.mozilla.org/en-US/docs/Web/API/Window/devicePixelRatio).
+
+### Per-image transforms
+
+By default, enhanced images will be transformed to more efficient formats. However, you may wish to apply other transforms such as a blur, quality, flatten, or rotate operation. You can run per-image transforms by appending a query string:
+
+```svelte
+<enhanced:img src="./path/to/your/image.jpg?blur=15" alt="An alt text" />
+```
+
+[See the imagetools repo for the full list of directives](https://github.com/JonasKruckenberg/imagetools/blob/main/docs/directives.md).
+
+## Loading images dynamically from a CDN
+
+In some cases, the images may not be accessible at build time — e.g. they may live inside a content management system or elsewhere.
+
+Using a content delivery network (CDN) can allow you to optimize these images dynamically, and provides more flexibility with regards to sizes, but it may involve some setup overhead and usage costs. Depending on caching strategy, the browser may not be able to use a cached copy of the asset until a [304 response](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/304) is received from the CDN. Building HTML to target CDNs may result in slightly smaller and simpler HTML because they can serve the appropriate file format for an `<img>` tag based on the `User-Agent` header whereas build-time optimizations must produce `<picture>` tags with multiple sources. Finally, some CDNs may generate images lazily, which could have a negative performance impact for sites with low traffic and frequently changing images. We do not currently offer any tools for dynamic image transforms, but we may offer such utilities in the future.
+
+## Best practices
+
+- For each image type, use the appropriate solution from those discussed above. You can mix and match all three solutions in one project. For example, you may use Vite's built-in handling to provide images for `<meta>` tags, display images on your homepage with `@sveltejs/enhanced-img`, and display user-submitted content with a dynamic approach.
+- Consider serving all images via CDN regardless of the image optimization types you use. CDNs reduce latency by distributing copies of static assets globally.
+- Your original images should have a good quality/resolution and should have 2x the width it will be displayed at to serve HiDPI devices. Image processing can size images down to save bandwidth when serving smaller screens, but it would be a waste of bandwidth to invent pixels to size images up.
+- For images which are much larger than the width of a mobile device (roughly 400px), such as a hero image taking the width of the page design, specify `sizes` so that smaller images can be served on smaller devices.
+- Choose one image per page which is the most important/largest one and give it `priority` so it loads faster. This gives you better web vitals scores (largest contentful paint in particular).
+- Give the image a container or styling so that it is constrained and does not jump around. `width` and `height` help the browser reserving space while the image is still loading. `@sveltejs/enhanced-img` will add a `width` and `height` for you.
+- Always provide a good `alt` text. The Svelte compiler will warn you if you don't do this.

packages/enhanced-img/README.md

@@ -0,0 +1,17 @@
+# `@sveltejs/enhanced-img`
+
+A Vite plugin which runs a Svelte preprocessor to locate images and then transform them at build-time.
+
+**WARNING**: This package is experimental. It uses pre-1.0 versioning and may introduce breaking changes with every minor version release.
+
+## Docs
+
+[Docs](https://kit.svelte.dev/docs/images)
+
+## Changelog
+
+[The Changelog for this package is available on GitHub](https://github.com/sveltejs/kit/blob/master/packages/enhanced-img/CHANGELOG.md).
+
+## Acknowledgements
+
+We'd like to thank the author of `svelte-preprocess-import-assets`, which this code is partially based off of. We'd also like to thank the authors of `vite-imagetools` which is used in `@sveltejs/enhanced-img`.

packages/enhanced-img/package.json

@@ -0,0 +1,42 @@
+{
+	"name": "@sveltejs/enhanced-img",
+	"version": "0.1.0",
+	"description": "Image optimization for your Svelte apps",
+	"repository": {
+		"type": "git",
+		"url": "https://github.com/sveltejs/kit",
+		"directory": "packages/image"
+	},
+	"license": "MIT",
+	"homepage": "https://kit.svelte.dev",
+	"type": "module",
+	"scripts": {
+		"lint": "prettier --check . --config ../../.prettierrc --ignore-path .gitignore",
+		"check": "tsc",
+		"format": "prettier --write . --config ../../.prettierrc --ignore-path .gitignore",
+		"test": "vitest"
+	},
+	"files": [
+		"src",
+		"types"
+	],
+	"exports": {
+		"types": "./types/index.d.ts",
+		"import": "./src/index.js"
+	},
+	"types": "types/index.d.ts",
+	"dependencies": {
+		"magic-string": "^0.30.0",
+		"svelte-parse-markup": "^0.1.1",
+		"vite-imagetools": "^6.2.3"
+	},
+	"devDependencies": {
+		"@types/estree": "^1.0.2",
+		"@types/node": "^16.18.6",
+		"estree-walker": "^3.0.3",
+		"svelte": "^4.0.5",
+		"typescript": "^4.9.4",
+		"vite": "^4.4.2",
+		"vitest": "^0.34.0"
+	}
+}

packages/enhanced-img/src/index.js

@@ -0,0 +1,135 @@
+import path from 'node:path';
+import { image } from './preprocessor.js';
+
+/**
+ * @returns {Promise<import('vite').Plugin[]>}
+ */
+export async function enhancedImages() {
+	const imagetools_plugin = await imagetools();
+	if (!imagetools_plugin) {
+		console.error(
+			'@sveltejs/enhanced-img: vite-imagetools is not installed. Skipping build-time optimizations'
+		);
+	}
+	return imagetools_plugin && !process.versions.webcontainer
+		? [image_plugin(imagetools_plugin), imagetools_plugin]
+		: [];
+}
+
+/**
+ * Creates the Svelte image plugin which provides the preprocessor.
+ * @param {import('vite').Plugin} imagetools_plugin
+ * @returns {import('vite').Plugin}
+ */
+function image_plugin(imagetools_plugin) {
+	/**
+	 * @type {{
+	 *   plugin_context: import('rollup').PluginContext
+	 *   imagetools_plugin: import('vite').Plugin
+	 * }}
+	 */
+	const opts = {
+		// @ts-expect-error populated when build starts so we cheat on type
+		plugin_context: undefined,
+		imagetools_plugin
+	};
+	const preprocessor = image(opts);
+
+	return {
+		name: 'vite-plugin-enhanced-img',
+		api: {
+			sveltePreprocess: preprocessor
+		},
+		buildStart() {
+			opts.plugin_context = this;
+		}
+	};
+}
+
+/** @type {Record<string,string>} */
+const fallback = {
+	'.avif': 'png',
+	'.gif': 'gif',
+	'.heif': 'jpg',
+	'.jpeg': 'jpg',
+	'.jpg': 'jpg',
+	'.png': 'png',
+	'.tiff': 'jpg',
+	'.webp': 'png'
+};
+
+async function imagetools() {
+	/** @type {typeof import('vite-imagetools').imagetools} */
+	let imagetools;
+	try {
+		({ imagetools } = await import('vite-imagetools'));
+	} catch (err) {
+		return;
+	}
+
+	/** @type {Partial<import('vite-imagetools').VitePluginOptions>} */
+	const imagetools_opts = {
+		defaultDirectives: async ({ pathname, searchParams: qs }, metadata) => {
+			if (!qs.has('enhanced')) return new URLSearchParams();
+
+			const img_width = qs.get('imgWidth');
+			const width = img_width ? parseInt(img_width) : (await metadata()).width;
+			if (!width) {
+				console.warn(`Could not determine width of image ${pathname}`);
+				return new URLSearchParams();
+			}
+
+			const { widths, kind } = get_widths(width, qs.get('imgSizes'));
+			return new URLSearchParams({
+				as: 'picture',
+				format: `avif;webp;${fallback[path.extname(pathname)] ?? 'png'}`,
+				w: widths.join(';'),
+				...(kind === 'x' && !qs.has('w') && { basePixels: widths[0].toString() })
+			});
+		},
+		namedExports: false
+	};
+
+	// TODO: should we make formats or sizes configurable besides just letting people override defaultDirectives?
+	// TODO: generate img rather than picture if only a single format is provided
+	//     by resolving the directives for the URL in the preprocessor
+	return imagetools(imagetools_opts);
+}
+
+/**
+ * @param {number} width
+ * @param {string | null} sizes
+ * @returns {{ widths: number[]; kind: 'w' | 'x' }}
+ */
+function get_widths(width, sizes) {
+	// We don't really know what the user wants here. But if they have an image that's really big
+	// then we can probably assume they're always displaying it full viewport/breakpoint.
+	// If the user is displaying a responsive image then the size usually doesn't change that much
+	// Instead, the number of columns in the design may reduce and the image may take a greater
+	// fraction of the screen.
+	// Assume if they're bothering to specify sizes that it's going to take most of the screen
+	// as that's the case where an image may be rendered at very different sizes. Otherwise, it's
+	// probably a responsive image and a single size is okay (two when accounting for HiDPI).
+	if (sizes) {
+		// Use common device sizes. Doesn't hurt to include larger sizes as the user will rarely
+		// provide an image that large.
+		// https://screensiz.es/
+		// https://gs.statcounter.com/screen-resolution-stats (note: logical. we want physical)
+		// Include 1080 because lighthouse uses a moto g4 with 360 logical pixels and 3x pixel ratio.
+		const widths = [540, 768, 1080, 1366, 1536, 1920, 2560, 3000, 4096, 5120];
+		widths.push(width);
+		return { widths, kind: 'w' };
+	}
+
+	// Don't need more than 2x resolution. Note that due to this optimization, pixel density
+	// descriptors will often end up being cheaper as many mobile devices have pixel density ratios
+	// near 3 which would cause larger images to be chosen on mobile when using sizes.
+
+	// Most OLED screens that say they are 3x resolution, are actually 3x in the green color, but
+	// only 1.5x in the red and blue colors. Showing a 3x resolution image in the app vs a 2x
+	// resolution image will be visually the same, though the 3x image takes significantly more
+	// data. Even true 3x resolution screens are wasteful as the human eye cannot see that level of
+	// detail without something like a magnifying glass.
+	// https://blog.twitter.com/engineering/en_us/topics/infrastructure/2019/capping-image-fidelity-on-ultra-high-resolution-devices.html
+	return { widths: [Math.round(width / 2), width], kind: 'x' };
+}