Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite of the Redis extension #203

Closed
wants to merge 79 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
a2eb272
wip
tommoor Sep 21, 2021
6713c04
progress
tommoor Sep 22, 2021
173f294
messy, but passing messages
tommoor Sep 22, 2021
ce4026b
ci: fix code style linting errors
tommoor Sep 22, 2021
a8a944e
doc
tommoor Sep 22, 2021
64298e8
lift MessageReceiver
tommoor Sep 22, 2021
b44e331
test: Awareness propagation
tommoor Sep 22, 2021
a1ce43a
fix: persistWait not configurable, add onPersist test
tommoor Sep 22, 2021
d642db6
docs
tommoor Sep 22, 2021
99a5cb0
fix: Query awareness from other servers on load
tommoor Sep 23, 2021
d6fbfc2
fix: Change import path (for now)
tommoor Sep 23, 2021
e869e30
refactor
tommoor Sep 23, 2021
6999d64
fix: Sync step propagation
tommoor Sep 23, 2021
c607d53
docs
tommoor Sep 26, 2021
409a144
test: onDisconnect
tommoor Sep 26, 2021
5ebbf13
Add onCreatedDocument hook, reduce blocking
tommoor Sep 26, 2021
e9688a4
fix build? Prefer src if available
tommoor Sep 29, 2021
08b3336
Merge branch 'main' into tom/pubsub-178
tommoor Oct 29, 2021
43e52ee
ci: fix code style linting errors
tommoor Oct 29, 2021
41b1460
Merge branch 'main' into tom/pubsub-178
hanspagel Nov 18, 2021
66fba41
docs: update navigation
hanspagel Nov 18, 2021
df5f6b6
install dependencies
hanspagel Nov 18, 2021
e9155c3
update pubsub package.json, rename folder
hanspagel Nov 18, 2021
0631a1a
update imports to use package names
hanspagel Nov 18, 2021
671ad56
move pubsub class to separate class
hanspagel Nov 18, 2021
7786657
refactor tests
hanspagel Nov 18, 2021
c8ccfc1
fix tests
hanspagel Nov 18, 2021
239266b
Merge branch 'main' into tom/pubsub-178
hanspagel Nov 18, 2021
e8e9bef
fix package-lock.json
hanspagel Nov 18, 2021
ca52512
remove yarn lock file
hanspagel Nov 18, 2021
a493100
Merge branch 'main' into tom/pubsub-178
hanspagel Nov 18, 2021
6e5a75d
ci: fix code style linting errors
hanspagel Nov 18, 2021
676c4d1
update the ESLint config
hanspagel Nov 18, 2021
09bb1f4
Merge branch 'tom/pubsub-178' of github.com:ueberdosis/hocuspocus int…
hanspagel Nov 18, 2021
18ab170
add some debugging output to check what happens in CI
hanspagel Nov 19, 2021
4ec7f96
add more debugging output
hanspagel Nov 19, 2021
ea73780
enable all other tests
hanspagel Nov 19, 2021
24fe0fb
Merge branch 'main' into tom/pubsub-178
hanspagel Nov 30, 2021
98ad4aa
merge changes
hanspagel Nov 30, 2021
88b9b3c
Merge branch 'main' into tom/pubsub-178
hanspagel Nov 30, 2021
3be286a
database: story the recent state, and only restore from the recent state
hanspagel Jan 5, 2022
2c34824
rocksdb: revert change
hanspagel Jan 5, 2022
8747af1
update dependencies
hanspagel Jan 5, 2022
0a505c9
fix eslint config
hanspagel Jan 6, 2022
117496d
refactor tests
hanspagel Jan 6, 2022
75f8724
rewrite package-lock.json
hanspagel Jan 6, 2022
031951e
Merge branch 'main' into tom/pubsub-178
hanspagel Jan 6, 2022
9656485
rewrite package-lock.json
hanspagel Jan 6, 2022
89373fa
fix types
hanspagel Jan 6, 2022
ebd64c7
ci: fix code style linting errors
hanspagel Jan 6, 2022
bccc48d
wip
hanspagel Jan 6, 2022
fc671a6
fix tests
hanspagel Jan 6, 2022
a2cf826
Merge branch 'tom/pubsub-178' of github.com:ueberdosis/hocuspocus int…
hanspagel Jan 6, 2022
eb8d429
wip
hanspagel Jan 6, 2022
ed7aa74
upgrade redis to 4.0
hanspagel Jan 6, 2022
fe2fec2
update dependencies
hanspagel Jan 6, 2022
24297ba
refactoring
hanspagel Jan 6, 2022
220cade
clean up
hanspagel Jan 6, 2022
5b26205
replace old redis extension with the new approach
hanspagel Jan 6, 2022
f02273b
ci: fix code style linting errors
hanspagel Jan 6, 2022
3037627
rename test context
hanspagel Jan 6, 2022
f249e01
Merge branch 'tom/pubsub-178' of github.com:ueberdosis/hocuspocus int…
hanspagel Jan 6, 2022
a1d3e90
add new onStoreDocument hook
hanspagel Jan 6, 2022
cf5a431
add onStoreDocument to the logger extension
hanspagel Jan 6, 2022
1985328
make sure documents are removed from the memory after they are persis…
hanspagel Jan 6, 2022
f478da6
skip a failing test
hanspagel Jan 6, 2022
c1736d3
debounce the onStoreDocument hook
hanspagel Jan 6, 2022
b760f08
clean up
hanspagel Jan 6, 2022
bf5fb3c
wip
hanspagel Jan 6, 2022
a12e9b0
update database and sqlite extensions to use the new onStoreDocument …
hanspagel Jan 6, 2022
43d33cf
add two redis instances and two editors to the playground example
hanspagel Jan 6, 2022
90cba8c
fix status bar
hanspagel Jan 6, 2022
e736396
add onAwarenessUpdate hook
hanspagel Jan 6, 2022
34917b5
redis: use unique ids instead of the hostname
hanspagel Jan 6, 2022
8087bc3
introduce new afterStoreDocument hook, rename onLoadedDocument to aft…
hanspagel Jan 6, 2022
eae74d2
use global onStoreDocument hook
hanspagel Jan 6, 2022
4f2fd34
remove onPersist callback
hanspagel Jan 6, 2022
f454f99
add another test for promises in onStoreDocument
hanspagel Jan 6, 2022
a9465f3
wip
hanspagel Jan 6, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 2 additions & 1 deletion .eslintrc.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,11 @@ module.exports = {
'airbnb-base',
],
rules: {
"max-classes-per-file": 'off',
'no-unreachable-loop': 'off',
'no-promise-executor-return': 'off',
'default-param-last': 'off',
'vue/multi-word-component-names': 'off',
'no-promise-executor-return': 'off',

'no-alert': 'off',
'no-shadow': 'off',
Expand Down
19 changes: 15 additions & 4 deletions docs/api/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ tableOfContents: true
# Configuration

## Introduction
There’s only two settings to pass for now: A custom port and a connection timeout. Most things are controlled through [hooks](/api/hooks).
There are only a few settings to pass for now. Most things are controlled through [hooks](/api/hooks).

## Settings

Expand All @@ -20,22 +20,33 @@ Default: `80`
### timeout
A connection healthcheck interval in milliseconds.

Default: `30000`
Default: `30000` (= 30s)

### debounce
Debounces the call of the `onStoreDocument` hook for the given amount of time in ms. Otherwise every single update would be persisted.

Default: `2000` (= 2s)

### maxDebounce
Makes sure to call `onStoreDocument` at least in the given amount of time (ms).

Default: `10000` (= 10s)

### quiet
By default, the servers show a start screen. If passed false, the server will start quietly.

Default: `false`

## Usage

```js
import { Server } from '@hocuspocus/server'

const server = Server.configure({
name: 'hocuspocus-fra1-01',
port: 1234,
timout: 30000,
timeout: 30000,
debounce: 5000,
maxDebounce: 30000,
quiet: true,
})

Expand Down
42 changes: 9 additions & 33 deletions docs/api/extensions/redis.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,61 +5,37 @@ tableOfContents: true
# Redis

## Introduction

:::warning Work in progress
Currently, the Redis extension only syncs document changes. Awareness states, for example cursors, are not yet supported.
:::

hocuspocus can be scaled horizontally using the Redis extension. You can spawn multiple instances of the server behind a load balancer and sync changes between the instances through Redis pub/sub.
This extension allows horizontal scaling of Hocuspocus instances, allowing any
client to connect to any instance and share document state between them.

## Installation

Install the Redis package with:
Install the package with:

```bash
npm install @hocuspocus/extension-redis
```

## Configuration

For a full documentation on all available redis and redis cluster options, check out the [ioredis API docs](https://github.com/luin/ioredis/blob/master/API.md).

```js
import { Server } from '@hocuspocus/server'
import { Redis } from '@hocuspocus/extension-redis'

const server = Server.configure({
extensions: [
new Redis({
// [required] Hostname of your Redis instance
host: '127.0.0.1',

// [required] Port of your Redis instance
port: 6379,
})
host: process.env.REDIS_HOST,
port: process.env.REDIS_PORT,
onPersist: async ({ document }) => {
// Write the document to the database here.
}
}),
],
})

server.listen()
```

If you want to use a cluster instead of a single Redis instance, use the Redis cluster extension:

```js
import { Server } from '@hocuspocus/server'
import { RedisCluster } from '@hocuspocus/redis'

const server = Server.configure({
extensions: [
new RedisCluster({
scaleReads: 'all',
redisOptions: {
host: '127.0.0.1',
port: 6379,
},
})
],
})

server.listen()
```
9 changes: 9 additions & 0 deletions docs/api/hooks/on-awareness-update.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
tableOfContents: true
---

# onAwarenessUpdate

## Introduction

TODO
9 changes: 9 additions & 0 deletions docs/api/hooks/on-store-document.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
tableOfContents: true
---

# onStoreDocument

## Introduction

TODO
45 changes: 38 additions & 7 deletions docs/guide/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,7 @@ tableOfContents: true
# Configuration

## Introduction

hocuspocus provides a few useful options and configuring it is as easy as calling `configure` and passing it your custom configuration:
Hocuspocus provides a few useful options and configuring it is as easy as calling `configure` and passing it your custom configuration:

```js
import { Server } from '@hocuspocus/server'
Expand All @@ -21,29 +20,61 @@ server.listen()
## List of available settings

### port

hocuspocus listens on port `80`. But you're free to change it to whatever port you like:
Hocuspocus listens on port `80`. But you're free to change it to whatever port you like:

```js
import { Server } from '@hocuspocus/server'

const server = Server.configure({
port: 80,
port: 1234,
})

server.listen()
```

Default: `80`

### timeout
There is a default connection timeout of `30000ms`. After this timeout non responding websocket connections are automatically terminated.

```js
import { Server } from '@hocuspocus/server'

const server = Server.configure({
timeout: 15000,
})

server.listen()
```

Default: `30000` (= 30s)

There's a default connection timeout of `30000ms`. After this timeout non responding websocket connections are automatically terminated.
### debounce
Debounces the call of the `onStoreDocument` hook for the given amount of time in ms. Otherwise every single update would be persisted.

```js
import { Server } from '@hocuspocus/server'

const server = Server.configure({
timeout: 30000,
debounce: 5000,
})

server.listen()
```

Default: `2000` (= 2s)

### maxDebounce
Makes sure to call `onStoreDocument` at least in the given amount of time (ms).

```js
import { Server } from '@hocuspocus/server'

const server = Server.configure({
maxDebounce: 30000,
})

server.listen()
```

Default: `10000` (= 10s)
2 changes: 1 addition & 1 deletion docs/guide/extensions.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ We already created some very useful extensions you should check out for sure:

**[@hocuspocus/extension-monitor](/api/extensions/monitor)**: A beautiful dashboard to monitor and debug your Hocuspocus instance.

**[@hocuspocus/extension-redis](/api/extensions/redis)**: Scale Hocuspocus horizontally with redis.
**[@hocuspocus/extension-redis](/api/extensions/redis)**: Scale Hocuspocus horizontally with Redis.

**[@hocuspocus/extension-logger](/api/extensions/logger)**: Add logging to Hocuspocus.

Expand Down
20 changes: 12 additions & 8 deletions docs/links.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,12 @@
- title: Hooks
link: /api/hooks
items:
- title: onConfigure
link: /api/hooks/on-configure
- title: onListen
link: /api/hooks/on-listen
- title: onUpgrade
link: /api/hooks/on-upgrade
- title: onConnect
link: /api/hooks/on-connect
- title: onAuthenticate
Expand All @@ -79,18 +85,16 @@
link: /api/hooks/on-load-document
- title: onChange
link: /api/hooks/on-change
- title: onStoreDocument
link: /api/hooks/on-store-document
- title: onAwarenessUpdate
link: /api/hooks/on-awareness-update
- title: onRequest
link: /api/hooks/on-request
- title: onDisconnect
link: /api/hooks/on-disconnect
- title: onListen
link: /api/hooks/on-listen
- title: onDestroy
link: /api/hooks/on-destroy
- title: onConfigure
link: /api/hooks/on-configure
- title: onRequest
link: /api/hooks/on-request
- title: onUpgrade
link: /api/hooks/on-upgrade

- title: Extensions
link: /api/extensions
Expand Down