Array as keys?

[frederikhors]

Now I'm using svelte-query but I'm not happy with it: it has not-so-convenient APIs (@benbender is working on it, but the maintainer is not so active, so I doubt it will change in a short time).

After the "fix" for Svelte Kit I'm trying sswr.

The first "issue" is about the query keys. Svelte-Query also allows you to use an array of keys for use cases such as pagination or filtering, you know... https://sveltequery.vercel.app/guides/query-keys.

What do you think about?

Is it so hard to add?

[benbender]

quick note on my efforts regarding react-query: I'm thinking about abandoning my efforts as I start to realize that its not a great fit into the sveltekit-ecosystem in the first place. It replicates state-handling and reactivity where it isn't needed in svelte(kit) because both are built-in. And react-query is therefor quite heavy in relation to sveltekit itself...
Most use-cases of react-query should be achiveable with much less code.

I'm also generally start to questioning the whole idea of explicit caches for queries/mutations on an app-level, as it is basically a duplication of the builtin http-caching in browsers (at least for everything http-related). F.e. preloading is a breeze with sveltekits routing and even could be supported by the built-in support for service-workers. So there is basically only a small margin of cases where a solution on the app-level seems to be worth the effort anyways (given that the api, you are trying to reach, does http-caching right).

Oh, and I would love to see solutions for svelte which embraces the approach of sveltekits sample-todo-app, which leverages basic http to handle forms with progressive enhancement and are even working without js at all...

[frederikhors]

Wow, @benbender, thanks for your thoughts

In my work, I always try to find a way to speed up responses to the user of the application I'm building.

I think today we need a tool to do this.

Is it possible that we are the first to address this topic?

Is it possible that there isn't something already done that works regardless of the framework (Svelte, Vue, React)?

[frederikhors]

@benbender can you explain the below better, please?

preloading is a breeze with sveltekits routing and even could be supported by the built-in support for service-workers. So there is basically only a small margin of cases where a solution on the app-level seems to be worth the effort anyways (given that the api, you are trying to reach, does http-caching right).

Oh, and I would love to see solutions for svelte which embraces the approach of sveltekits sample-todo-app, which leverages basic http to handle forms with progressive enhancement and are even working without js at all...

[benbender]

Is it possible that there isn't something already done that works regardless of the framework (Svelte, Vue, React)?

The problem here is that it needs quite a bit of assumptions about the app and the api. It's basically out of scope of Svelte, Vue & React. And even Angular (here I'm not entirely sure), NuxtJS & NextJS are basically agnostic to the ways data is fetched and handled. Sveltekit has basics of it built in as it leverages the browsers' fetch-api.
In the end its a whole concept of handling data which is mostly done and thought about more on an app-level than a framework-level.

For details/implementations see:

prerendering:
https://github.com/sveltejs/kit/blob/master/packages/create-svelte/templates/default/src/lib/header/Header.svelte#L18

progressive enhancement of forms:
https://github.com/sveltejs/kit/blob/master/packages/create-svelte/templates/default/src/lib/form.ts

service-workers-api:
https://kit.svelte.dev/docs#service-workers

[frederikhors]

Yeah I know all that.

Now we need to understand how we make them work to have an effective SWR in every SvelteKit project.

[benbender]

Pseudo-code in my head could be something like the following: (problem: it has many opinions about the backend-code/api)

<script lang="ts" context="module">
  import type { Initial } from '$lib/package-naming-is-hard';
	import { serverQueryAndGetOptions, queryDataInBrowserIfNeededAndCreateForm } from '$lib/package-naming-is-hard';

	type Data = {
        title: string;
        content: string;
    };

	export const load = serverQueryAndGetOptions<Data>('/api/post/1');
</script>

<script lang="ts">
  // initially loaded data from the server-fetch, options for form-handling and maybe aditional config to have a single source //// of truth for handling crud
	export let initial: Initial<Data>;

    /* refetch, update, delete would be exposed methods to programattically manipulate the fetched data */
	const { status, data, form, refetch, update, delete }  = queryDataInBrowserIfNeededSetupSWRAndCreateForm<Data>(initial.url, initial.options);
</script>

/_ $data holds the fetched data, no browser request neccessary if initially done via ssr, should leverage swr-technics. _/
{#if $status === "loading"}
    <div>Loading...</div>
{:else if $status === "error"}
    <div>error...</div>
{:else}
    <div>
        <div>Title: {$post.title}</div>

<div>Content: {$post.content}</div>
<div>
{/if}

/_ should handle create and update (could be infered from method=), validation, handle optimistic updates, should work with js disabled, needs method & action to work without js _/
<form method="post" action="/api/post(/1?method=PATCH)" use:form({ onSuccess: (response) => { /* here $data & $status should already be reactivly updated */  }, onError: (error) => {}, onSubmitting: (data) => {} })>
    <input type="text" name="title" /> /* should have initial data, if any */
    <input type="text" name="content" /> /* should have initial data, if any */

    <button type="submit">Create/Update post</button>
</form>

/_ should handle optimistic updates etc _/
<form  method="post" action="/api/post/1?method=DELETE" use:form({ type: "delete" /* could be infered from method= */, onSuccess: (response) => { /* here $data should already be reactivly updated */  }, onError: (error) => {}, onSubmitting: (data) => {}  })>
    <button type="submit">Delete post</button>
</form>

[frederikhors]

Wow! We should open an RFC on sveltejs repo!

[benbender]

Nope, as I said: too opinionated. Besides that, let them finish v1 and go on from there ^^

[ConsoleTVs]

Is it possible that there isn't something already done that works regardless of the framework (Svelte, Vue, React)?

swrev 😢

Is it so hard to add?

I don't think it could take me that much time, I usually just use string as key with the needed info, I usually follow the https://swr.vercel.app/docs/pagination guide on pagination.

I'm going to be honest with you here, I think svelte-query is much more complex, it's trying to play nicely with react-query API. In contrast, sswr tries to play nicely with swr, in short, swr has a smaller API.

You're completly right tho that they both support array keys. I'm probably going to code some solution for that as well!!

This is probably going to be the next feature for this lib!

[benbender]

Back to topic: I don't think a lib should support an array as key(!). The HTTP-Spec is clear about the mapping between a URL and a resource. I think a caching lib should retain and respect that basic relationship. So a string would be sufficent.

But if I would want such a functionallity, to be able to, f.e., remove all blog-posts from a cache, I would add an additional bit of information. Something like tags, labels or hints (where a resource could have multiple - so an array would be correct here). This would allow an api like f.e. cache.filterByTag("post", (entry) => entry.remove()) or similar.

[ConsoleTVs]

Back to topic: I don't think a lib should support an array as key(!). The HTTP-Spec is clear about the mapping between a URL and a resource. I think a caching lib should retain and respect that basic relationship. So a string would be sufficent.

But if I would want such a functionallity, to be able to, f.e., remove all blog-posts from a cache, I would add an additional bit of information. Something like tags, labels or hints (where a resource could have multiple - so an array would be correct here). This would allow an api like f.e. cache.filterByTag("post", (entry) => entry.remove()) or similar.

I do strongly belive so as well. I'll explore oportunities in this field!

[benbender]

@ConsoleTVs I would advise to have a look at varnish or haproxy (http caching-proxies) and their caching-options as they should be very in line with the http-spec.

[frederikhors]

swrev 😢

I was talking about something like this amazing thing except I need it for my APIs and SvelteKit: https://philipwalton.com/articles/smaller-html-payloads-with-service-workers/

I usually follow the https://swr.vercel.app/docs/pagination guide on pagination.

I will try in a few hours!

This is probably going to be the next feature for this lib!

Please nope, there is something more urgent: localStorage or IndexedDB! 😄

[frederikhors]

@benbender what do you think about the amazing https://philipwalton.com/articles/smaller-html-payloads-with-service-workers/?

Can we use it for our purposes?

[benbender]

@frederikhors reminds me of turbolinks from rails back in the days ;)
But as the article itself states: this basic technique is already used by almost every SPA (incl. sveltekit) today. So I don't get your excitement... And I don't wanna be the person in charge for cache-invalidation with a self-building layer of cache in every users browser... At the end of the day: Keep it simple, stupid! ;)

[frederikhors]

I'm talking about something like that for API only avoiding JS runtime at all in my app. The service worker does the dirty work, instead of swr or svelte-query.

What do you think?

[frederikhors]

@ConsoleTVs the original SWR supports this: https://swr.vercel.app/docs/arguments#multiple-arguments.

[frederikhors]

@benbender https://sara-vanan.medium.com/here-is-why-we-did-not-use-service-worker-for-runtime-data-caching-927e5f2d15f8.

[benbender]

I'm talking about something like that for API only avoiding JS runtime at all in my app. The service worker does the dirty work, instead of swr or svelte-query.

So you want no SPA but a plain html-page. Go for it! Will be faster and leaner for sure. ;)

the original SWR

It's not "the original SWR". Stale-while-revalidate is a concept which was only popularized in the react-world by vercel.
See f.e. HTTP-Spec: https://datatracker.ietf.org/doc/html/rfc5861#section-3

supports this

That's exactly what I'm talking about. They reinvented the vary-header in a less powerful way...

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Vary

We are talking about data-fetching. The best way to speed up data-fetching is to don't do data-fetching (at least until necessary). This can be already achived with a smart architecture leveraging http- and browser-caching. (Espacially) in the react-world one was a bit overwhelmed by the new options in the frontend - forgetting about all the small, smart bits already invented. The reaction to this can be observed atm with developments like the remix-framework or svelte which are going a bit back to the roots and building upon the shoulders of the giants who built large parts of the internet.

For the rest: I can't answer any architectural questions without having any context. Many smart people (at least smarter than me) have thought about most of these problems for decades. And I sincerly doubt that any "one-size-fits-all"-solution will be brought up in this thread for any of the long-standing problems of the interwebs ;)

And always remember: "There are only two hard things in Computer Science: cache invalidation and naming things." (Phil Karlton) and "premature optimization is the root of all evil". Both statements are one of the few which are always true.

PS: This "SWR-extended in SW"-idea is basically ETags: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag

[frederikhors]

Thanks for your thoughts.

I wanna use SvelteKit together with Workbox (or something similar) to mimic the SWR behavior in my SPA app.

If in the future I can "fix" the backend with all the correct headers I'will be happier! 😄

[benbender]

It basically depends on the structure of your data. If it is constantly changing, I would generally advise against your approach because you will have a really hard time to get it under control and debug any errors by end-users... It basically sounds like the wrong end to fix your problems. If the api isn't under your control but you can estimate the needed fixes, use a proxy in between to do the needed alterations. This could also be done with js - f.e. embedded in Nginx: https://www.nginx.com/blog/harnessing-power-convenience-of-javascript-for-each-request-with-nginx-javascript-module/

[ConsoleTVs]

@frederikhors it does support miltiple arguments because of the limitation of react needing a lidt of dependencies. For example, vue or svelte dont need that.

[frederikhors]

limitation of react needing a lidt of dependencies. For example, vue or svelte dont need that.

Can you write an example to better understand, please?

Anyway I think we can close this because for now it's not a pressing urgency.

[ConsoleTVs]

In react, you need to specify a list of dependencies when you want to run an effect. That's why it needs a list of "dependencies". In vue, we have useEffect, that automatically scans internal dependencies same with svelte's $:.

https://swr.vercel.app/docs/arguments#multiple-arguments

When the fetcher requires some props to change, react does not know they changed thus they need an array to track the dependencies in a useEffect(). The dependencies are then pased to the fetcher function. However, I don't think we really need that imo.

[benbender]

@ConsoleTVs You're mixings things up here:

In React the exact opposite is true. Per default React tracks every prop in its virtual DOM and reruns the whole subtree when props are changing (actually there is a diff between the virtual DOM and the "real" DOM) - Every time and for the whole subtree. Only actual changes are written back to the actual DOM. React's useEffect is used to mitigate (speed-)issue that result from that default-behaviour and/or to gain more control over this defaults. You are effectively narrowing the prop-tracking for the particular piece of code inside useEffect to its dependency-array. Funfact: React itself isn't reactive in any way (but it can be with rxjs and similar).

For VUE we have a (in parts) reactive runtime and you are telling this runtime which vars to track in a reactive manner per useEffect(). (At least afaik, didn't used VUE much myself)

For Svelte we have a reactive compiler which needs to know upfront which parts of code are depending on each other. Most of this is automatically inferred, but you can basically hint the compiler to make a complete statement reactive with "$:".

For swr those multiple-args are simply used to build a cache where entries are uniquely identifiable. The fetcher is rerunning simply because React is rerendering on prop-change. So its an indirect effect of react's design and nothing special. Same is (f.e.) true for a simple console.log() without any additional inherent logic...

[frederikhors]

So maybe they support arrays as keys.

We need to, I think. It's super useful.

[ConsoleTVs]

@ConsoleTVs You're mixings things up here:

In React the exact opposite is true. Per default React tracks every prop in its virtual DOM and reruns the whole subtree when props are changing (actually there is a diff between the virtual DOM and the "real" DOM) - Every time and for the whole subtree. Only actual changes are written back to the actual DOM. React's useEffect is used to mitigate (speed-)issue that result from that default-behaviour and/or to gain more control over this defaults. You are effectively narrowing the prop-tracking for the particular piece of code inside useEffect to its dependency-array. Funfact: React itself isn't reactive in any way (but it can be with rxjs and similar).

For VUE we have a (in parts) reactive runtime and you are telling this runtime which vars to track in a reactive manner per useEffect(). (At least afaik, didn't used VUE much myself)

For Svelte we have a reactive compiler which needs to know upfront which parts of code are depending on each other. Most of this is automatically inferred, but you can basically hint the compiler to make a complete statement reactive with "$:".

For swr those multiple-args are simply used to build a cache where entries are uniquely identifiable. The fetcher is rerunning simply because React is rerendering on prop-change. So its an indirect effect of react's design and nothing special. Same is (f.e.) true for a simple console.log() without any additional inherent logic...

afaik your swr wont run again in react if some fetcher dependency changes. Only if you specify it on the key array and using the value provided in the fetcher function's param will re-trying that update.

Of course we could support a serializable array but I think this was introduced in swr just as a limitation and became something with some extra use. As noted, I will consider this as I think it might be an interesting thing for some people.

[benbender]

afaik your swr wont run again in react if some fetcher dependency changes. Only if you specify it on the key array and using the value provided in the fetcher function's param will re-trying that update.

Yes and no :) swr runs again, but it doesn't fetch again because the logic is wrapped in useCallback().

https://github.com/vercel/swr/blob/3bbbf5b86be0868963c12906816a7bc1ded84c15/src/use-swr.ts#L245

Thing is that all those "hooks" like useEffect(), useState(), useCallback() etc in React are basically methods to rip bits and pieces of code/state out of the normal react-rerender-flow to make them stateful. It's, more or less, a band-aid to have statefulness in functional components in react. Thats exactly what's happening here.

The beauty of svelte is that it gets rid of all those band-aid-concepts react introduced - and we are used to - for the past several years :)

About array-keys: they aren't only used for dependency-tracking but also to differentiate different results of requests and group requests with context. So there would be a benefit having something like that...