RFC: Collections (aka Atomic Creates)

[ntucker]

Motivation

Atomic mutations are one of the core values of Rest Hooks. However, while update was in the first pre-release, and delete quickly added for version 1, create has always been a bit awkward.

Often creating a new element you are on a view where you want to see it added to a list somewhere.

One of the challenges here is that there can be many distinct lists you view. Furthermore, where something is added to a list isn't always the same.

Use case patterns

• User is adding a new item to a list view.
  • The insertion location should be based on UI, and thus easily controllable from a component
• Collections can be nested inside other entities
  • There is sometimes an endpoint that maps to the same concept - to get all related items in that list. These should be the same.
• One list endpoint may have parameters that filter, sort, or do nothing interesting at all
• A creation block may be unaware of all other UI locations that contain that item
• Infinite scrolling pagination requires building up a list by fetching and removing the page/cursor argument
• List may contain Unions, not just Entities
• There may be many active lists where more than one should get the element added

Previously

https://resthooks.io/docs/concepts/atomic-mutations#create

const createUser = new Endpoint(postToUserFunction, {
  schema: User,
  update: (newUserId: string) => ({
    [userList.key()]: (users = []) => [newUserId, ...users],
  }),
});

Problems:

• Very awkward code we force users to write
• Only works on top level results (doesn't work with nesting)
• Forces users to enumerate all lists they might want to add - even if they aren't in store
  • Which means they may create a list that hasn't been populated

Solution

Inspiration: Backbone collections

Data

class Todo extends Entity{
  userId = 0;
  title = '';
  completed = false;

  static key = 'Todo';
}

class User {
  name = '';
  username = '';
  email = '';
  todos: Todo[] = [];

 static key = 'User';
 static schema = {
    todos: new schema.Collection(Todo, (parent, key) => ({ userId: parent.id })),
  }
}

Endpoints

Resources will have sane defaults making most use cases not need to specify. So we're going to zoom in and manually create endpoints so it's clear how they interface.

const getTodo = new RestEndpoint({
  path: '/todos',
  searchParams: {} as { userId?: string | number,  completed?: boolean } | undefined,
  schema: new schema.Collection(Todo, (urlParams, body) => urlParams)
});

const createTodo = getTodo.extend({
  method: 'POST',
  schema: getTodo.schema.push(),
});

Usage

const handleCreate = data => ctrl.fetch(createTodo, { userId: currentuser.id }, data);

This will add to these if they are already in store (but only if they already exist):

• User({ id: currentUser.id }).todos
• getTodo({ userId: currentUser.id })
• getTodo()

Collection

Collection's second argument is a key computation. By default it is the identity method. This is called with either ...args when used in an endpoint, or (parent, key) when used in another entity.

class Collection {
  constructor(schema, key = params => params) {
  }

  push(filter=defaultFilter) {
  }
  unshift(filter=defaultFilter) {
  }
  insert(cmp, filter=defaultFilter) {
  }
}

// this adds to any list *in store* that has same members as the urlParams
// so fetch(create, { userId: 'bob', completed: true }, data)
// would possibly add to {}, {userId: 'bob'}, {completed: true}, {userId: 'bob', completed: true } - but only those already in the store
// it ignores keys that start with sort as those are presumed to not filter results
const defaultFilter = (urlParams, body) => collectionKey =>
		Object.values(collectionKey).every(([key, value]) => key.startsWith('sort') || urlParams[key] === value || body?.[key] === value);

Store shape

{
  entities: {
    Todo: { '1': { id: '1', title: 'add collections to rest hooks', completed: false, userId: '5' } },
    User: { '5': { id: '5', username: 'ntucker', todos: '{"userId":"5"}' } },
  },
  collections: {
    Todo: {
      '{"userId":"5"}': ['1'],
    }
  }
}

Unions

Since unions don’t have keys like entities, we can have an optional argument to set collection key if it can’t grab it from the schema sent.

new schema.Collection(MyUnion, { name: 'MyUnion', key: ({ groupId }) => ({ groupId }) });

Ergonomic shortcuts

RestEndpoint push/unshift/insert

Since REST patterns often keep similarity between create and list endpoints, we can add special extenders. This let's us simply call getTodos.push instead of having to create a new endpoint for create.

Before:

const createTodo = getTodo.extend({
  method: 'POST',
  schema: getTodo.schema.push(),
});
const handleCreate = data => ctrl.fetch(createTodo, { userId: currentuser.id }, data);

After:

const handleCreate = data => ctrl.fetch(getTodos.push(), { userId: currentuser.id }, data);

Resources

Resource.create will simply exist as Resource.getList.push(). However, you can still use those manually:

const addToEnd = data => ctrl.fetch(TodoResource.create, { userId: currentuser.id }, data);

const addToBeginning = data => ctrl.fetch(TodoResource.getList.unshift(), { userId: currentuser.id }, data);

Collection.insert cmp argument

Can be a function, or a string referring to the key that should be compared. Or an Array referring to the keys that should be compared (order matters)

Open questions

Should we make Array a collection as well? Mostly concerned about the [Todo] type definitions.

• The Endpoint level key has a good default
• However, for inside an entity we'd have to make a lot of assumptions...
  • parent => ({ [ParentEntity.key.lowerFirst() + 'Id']: ParentEntity.pk(parent) })
  • This is fine to not match with other endpoint keys, as that simply means it will be updated distinctly, so worst case it won't be updated incorrectly, rather it might miss some updates.

I'm thinking maybe Collection should just be a type of entity. So it's stored in the same lookup table and most code paths won't need to change.

[ntucker]

Council notes

• insert is good name
• also should add insertAt if we just want to specify exact index to place
• push, etc being functions so we can override filter - perhaps we can have two names for each - push is just the schema, and pushWithFilter() is a function that lets you specify the filter

[ntucker]

Problem with using parent for args... Previously I was thinking that parent only had meaning in entities - but it will get set in any nesting including Object or Values. I found this because there was a test and I'm realizing this functionality makes sense. Only at the top level is it the input itself so kind of pointless - but entities are not the only way to nest. Furthermore, obviously it is a bit weird for the 'parent' arg to be 'args' even if it sort of makes sense.

This means pk might look like this pk(value, parent, key, args)

So if we are to keep parent the same, then is no obvious way to detect that we want to use the args rather than the parent for the instanceKey parameter. This would look like:

class User extends IDEntity {
  name = '';
  username = '';
  email = '';
  todos: Todo[] = [];

  static key = 'User';
  static schema = {
    todos: new schema.Collection(new schema.Array(Todo), (parent, key) => ({
      userId: parent.id,
    })),
  };
}

const userTodos = new schema.Collection(
  new schema.Array(Todo),
  (parent, key, args) => ({
    args[0].userId,
  }),
);

Instead we could make the second argument an object and use key to determine what the function should take:

class User extends IDEntity {
  name = '';
  username = '';
  email = '';
  todos: Todo[] = [];

  static key = 'User';
  static schema = {
    todos: new schema.Collection(new schema.Array(Todo), {
      nestKey: (parent, key) => ({ userId: parent.id })
    }),
  };
}

const userTodos = new schema.Collection(
  new schema.Array(Todo),
  {
    argsKey: ({ userId }) => ({ userId }),
  },
);

[ntucker]

Insertions

Should .push really push to every collection it adds? Perhaps it should only push to exact urlParams match, and others it should run insertion?

Insertion should be defined by collection. Must provide PK parameters to know exactly how to run it

[ntucker]

Implemented in #2593, released in endpoint@3.8 and rest@6.6