Does rqlite do any local caching for read-only nodes?

[acuozzo]

My use-case is best articulated by this old StackExchange post: https://serverfault.com/questions/556928/distributed-file-system-with-local-disk-cache

I have lots of little VMs which all connect back to a DB cluster across the internet. My goal is to make as best use of the resources available to them as possible, including the 8 or so GiB of available disk space they have.

I linked to the SE post above because I was (and still am) considering solving this at the filesystem level.

[otoolep]

Have you read the docs on read-only nodes? And on read-consistency?

I presume by "caching" what you really want to do is to serve reads from the read-only node, without that node going all the way back to the leader. If so, just select "none" for consistency, and read-only node will serve the data from the local copy of the SQLite database (which is in-memory) by default.

https://rqlite.io/docs/api/read-consistency/#none

So if you send your query to a local read-only node, using the following form:

curl -G 'localhost:4001/db/query?level=none&freshness=1s' --data-urlencode 'q=SELECT * FROM foo'

The read-only node will serve the data by reading the local SQLite database, also ensuring that the data is not more than 1 second old. If it is old, it will return an error (and then you can automatically redirect to the leader by re-issuing the command with weak consistency).

Does this answer your question?

[acuozzo]

It does. Thank you!

[otoolep]

But the disk and RAM usage of a read-only node will be the same as any other node in the cluster, read-only nodes are about avoiding network round trips, not about saving disk and RAM.

[gedw99]

ah 1 second - that useful to know.

Fly used a funny logic here. When a user writes to a node, and then asynchronously synchronises that write to the other servers...

For the next second they apply a session affinity logic of:

If it's the same user hitting the global load balancer it forces the read to the server they did the previous write to. So they see the worlds as complete. Their write is showing that they did a sub second ago.

BUT, If its a different user then let the global load balancer forward to any server, because they did not do the write, and so their view of the world is different.

After a second its all back to normal because the asynchronous write from a second ago is now on all servers.

This is similar what Google do in their papers - cant remember which one. They clock sync users to google time to account for leap years drift ( https://developers.google.com/time/smear ) . Then all writes coming in are buffered to account for who did what. They they do some sort of reordering of the buffer queue and then let the requests through. Its similar but different to fly approach..

Pretty interesting approach.. hope i explained it well enough..