PUT should not switch resources between container and non-container

[michielbdejong]

When you ask me, I'm sure that the following two statements are true about Solid, but if you ask me how do I know that, then I don't really have a good answer to that. So let's clarify them both in the spec! They are:

1. If /foo/bar/ is a container, then a PUT to /foo/bar should fail
2. If /foo/bar is not a container, then a PUT to /foo/bar/baz should fail

[michielbdejong]

Also: After a PUT to https://example.com/some/path/to/a/resource, the newly created resource will be at that URL, and will be ldp-contained by https://example.com/some/path/to/a/, in general, remove the longest /-less suffix. Note that PUT to a URI that ends in a / makes no sense

[kjetilk]

Yeah, that'd be my hunch too...

[csarven]

If /foo/bar/ is a container, then a PUT to /foo/bar should fail
If /foo/bar is not a container, then a PUT to /foo/bar/baz should fail

They are constraints that may be used by implementations.

Resource owners can potentially allow any interaction model at those URIs.

After a PUT to https://example.com/some/path/to/a/resource, the newly created resource will be at that URL, and will be ldp-contained by https://example.com/some/path/to/a/

Why?

PUT to a URI that ends in a / makes no sense"

Why?

[dmitrizagidulin]

Note that PUT to a URI that ends in a / makes no sense

I definitely think that a PUT to a / url should result in container creation.

[csarven]

I definitely think that a PUT to a / url should result in container creation.

Why?

[csarven]

After a PUT to https://example.com/some/path/to/a/resource, the newly created resource will be at that URL, and will be ldp-contained by https://example.com/some/path/to/a/

The containment update goes against LDP's recommendation https://www.w3.org/TR/ldp/#ldpc-put-mbrprops :

LDP servers SHOULD NOT allow HTTP PUT to update a LDPC’s containment triples; if the server receives such a request, it SHOULD respond with a 409 (Conflict) status code.

[kjetilk]

After a PUT to https://example.com/some/path/to/a/resource, the newly created resource will be at that URL, and will be ldp-contained by https://example.com/some/path/to/a/

The containment update goes against LDP's recommendation https://www.w3.org/TR/ldp/#ldpc-put-mbrprops :

LDP servers SHOULD NOT allow HTTP PUT to update a LDPC’s containment triples; if the server receives such a request, it SHOULD respond with a 409 (Conflict) status code.

That is not how I interprete that statement. My interpretation is that if someone PUTs to the container itself with the containment triples, there should be a conflict. I interpreted @michielbdejong 's comment to mean that the containment triples of the container should be updated as a side-effect of PUTting an contained LDPR.

You know, I think the whole LDP R* stuff is just messy. These semantics should be entirely in-band in an ontology...

[csarven]

I have considered those interpretations however I don't think the interpretation is exclusive to the interaction being directly on the container ie. however PUT causes containment triples to be updated. The reasoning is rooted on the intended use of POST and PUT as per RFC 7231, as well as LDP's prescription of POST interactions and its effects on containers, in contrast to PUT being unspecified or severely underspecified for containers. If the intention of PUT around containers was to be equivalent to POST, I would've expected LDP to be more clear on that point. I don't assume that was an oversight on LDP's part.

I want to note here that LDP would essentially permit PUT to create LDPC, however it doesn't want PUT to manipulate the containment triples. So, there can be a detached container that's not contained in another container. LDP seems to suggest POST for creating resources that are intended to be contained in a container. Under this line of reasoning, if a client still wants to use PUT to create a container - which is permitted - and wants containment triples, it will need to follow up with PATCH to update container's containment triples.

[namedgraph]

Hmm, if only there was a Linked Data spec in which operations are defined formally, leaving no room for interpretation...

[acoburn]

One point of ambiguity around containment triples and PUT is whether the prohibition on manipulating containment triples refers to the target resource, the parent resource, or both. That is, consider a hierarchy of containers: /foo/bar/baz, in which the /foo resource includes </foo> ldp:contains </foo/bar> and the /foo/bar resource includes </foo/bar> ldp:contains </foo/bar/baz>

It seems pretty clear that a PUT to update /foo/bar shouldn't change the containment triples of /foo/bar: a client can't assert that /foo/bar now contains some other resources.

Likewise, a PUT to create a new resource /foo/bar2 cannot assert that this new resource suddenly ldp:contains some arbitrary set of resources (e.g. you can't have /foo/bar/baz contained by two separate resources)

What is unclear to me is whether a PUT to create a resource also does not update the ldp:contains triples of the implied parent resource (wherever that resource may happen to be). Is that an implementation decision or does this rise to the level of specified behavior. FWIW, Trellis can do either: it's controlled by a configuration switch.

[csarven]

AFAICT, there is no (implied) statement which expects PUT to do anything besides creating or replacing a target resource. An idempotent method. It is an implementation detail as to what else happens.

Information about the parent container (implied or not) is nowhere in sight. Note how this is also the exact issue we have already experienced in some cases about wanting to extract the context of a resource ie. child-to-parent relationship. Even if a client wanted to also update containment triples after creating an LDPR, it would neither know where, what or how to update. That justifiably prohibits the client from making a meaningful containment update anywhere. The required information is simply unavailable ie. neither the server or the client knows. On the other hand, the way to update is clearly paved via POST for servers in LDP.

A subtle point is that servers manage containment triples while clients are prohibited from managing.

Perhaps I've missed something but I'm defaulting to Occam's razor.

[csarven]

heya @csarven, re:

A subtle point is that servers manage containment triples while clients are prohibited from managing.

You also said this about PATCH:

Under this line of reasoning, if a client still wants to use PUT to create a container - which is permitted - and wants containment triples, it will need to follow up with PATCH to update container's containment triples.

I suppose PATCH may be an exception to the client rule with the constraint that it can only INSERT a containment triple. That however opens up the possibility for errors.

Interestingly, LDP only recommends: https://www.w3.org/TR/ldp/#ldpc-patch-req :

LDP servers are RECOMMENDED to support HTTP PATCH as the preferred method for updating a LDPC's minimal-container triples.

So, it is probably preferable that servers SHOULD NOT allow PATCH to update containment triples.

[csarven]

One more:

Under Containers->POST-> interaction models, https://www.w3.org/TR/ldp/#ldpc-post-createrdf notes:

A consequence of this is that LDPCs can be used to create LDPCs, if the server supports doing so.

POST is clearly the intended method to create LDPCs and consequently the containment triples.

[kjetilk]

I just wonder if we are looking too much to LDP with regards to container membership, Solid clearly has stronger assumptions around this, and it seems that we would simplify the discussion if we set them into MUSTs, they can then be relaxed later if needed. I think it is important though, that we should only mandate a strict subset of LDP in this context.

So, @csarven , I'm not quite sure what you meant with

A subtle point is that servers manage containment triples while clients are prohibited from managing.

so, I'll just spell out what I'm thinking, and then you are welcome to dissect and see if we are in agreement:

Whatever we think about sharing slash semantics between the server and client, internal to the server, we can surely say that the / has semantics, so that /foo/ is a container, and therefore, any LDPR with a path with starting with /foo/ but has at most one more slash is contained in /foo/. With that knowledge, it should be possible to leave maintainance of containment triples entirely up to the server. If we can do that, it simplifies a lot, since we can just reject any request that tries to update the containment triples, and having clients updating containment triples is risky, both from a atomicity perspective, and from a security perspective, as the server then has to check if the resource really is contained in /foo/.

As for @csarven saying:

Information about the parent container (implied or not) is nowhere in sight.

I'm then thinking about the RFC's statement:

A PUT request applied to the target resource can have side effects on
other resources. For example, an article might have a URI for
identifying "the current version" (a resource) that is separate from
the URIs identifying each particular version (different resources
that at one point shared the same state as the current version
resource). A successful PUT request on "the current version" URI
might therefore create a new version resource in addition to changing
the state of the target resource, and might also cause links to be
added between the related resources.

[my emphasis]. My interpretation of that final sentence is that this is applicable to containment triples, i.e. the side effect of adding containment triples to the LDPC is well within the recommended practice of RFC7231.

So, if we accept that the server is responsible for managing the containment triples, and the client should not touch them, then I think that the answer to @acoburn 's question:

What is unclear to me is whether a PUT to create a resource also does not update the ldp:contains triples of the implied parent resource

becomes that the server will also update the ldp:contains triples of the implied parent container.

[csarven]

Aside: if LDP conformance or proper extensibility is a non-concern, then we can skip a class of discussions. Virtually anything is possible if we have open discussion and reach to some consensus on what a spec should say. I'm hoping that we can arrive somewhere without breaking away from the Web architecture and core specs that Solid is trying to play along with.

I'm not quite sure what you meant with

A subtle point is that servers manage containment triples while clients are prohibited from managing.

LDP has the notion of "server-managed triples" that are triggered through POST and DELETE. PUT and PATCH are not intended for that management - if anything, suggested not to. LDP conforming servers can't be reliably directed to update membership/containment triples by clients.

With that knowledge, it should be possible to leave maintainance of containment triples entirely up to the server.

Possible but redundant.

If we can do that, it simplifies a lot

I reached a different conclusion: a brief list of issues or at least considerations that needs to be taken into account at solid/specification#35 (comment) under "essentially warrants URI string processing tied to LDP that entails". Taken in totality, I think it increases the complexity of the system.

having clients updating containment triples is risky, both from a atomicity perspective, and from a security perspective

Sure, I think that's LDP's position as well.

Requesting to create /foo/bar/baz while expecting /foo/ and /foo/bar/ to also be created (if not exists) along with creating containment triples is not at all atomic. Containment information doesn't need to be transmitted through the URI. LDP's interaction model does that already.

As for atomicity: you know "make each program do one thing well".. mkdir and touch are different programs. So, overloading PUT to do several changes based on the URI pattern would be like a clever shell script. It is instructing to act on two or more resources (creating and building relationships) that are besides the target resource.

Moreover, taking the URI slicing approach once again leads right back to building lazy clients where the server does the smarts based on some yet-another-API. I see that going in the opposite direction of what we expect from applications in the Solid ecosystem.

might also cause links to be added between the related resources.
My interpretation of that final sentence is that this is applicable to containment triples

That can indeed be one way to interpret. Most commonly, as you know, the intention of linking in that sentence is realised through mechanisms like eg. <link> or Link to connect two resources. Certainly that's not a strict constraint. It is just not particularly aligned with the intention of PUT as I see it - POST would make a bit more sense.

[kjetilk]

Aside: if LDP conformance or proper extensibility is a non-concern, then we can skip a class of discussions.

Yeah, but I'm not going that far :-) We should take care not to break LDP, and also ensure that any extensions are friendly, but it also seems to me that LDP allows for way more subtilities than we need, and what it seems to me Solid has thus far adopted.

Virtually anything is possible if we have open discussion and reach to some consensus on what a spec should say. I'm hoping that we can arrive somewhere without breaking away from the Web architecture and core specs that Solid is trying to play along with.

Sure!

I'm not quite sure what you meant with

A subtle point is that servers manage containment triples while clients are prohibited from managing.

LDP has the notion of "server-managed triples" that are triggered through POST and DELETE. PUT and PATCH are not intended for that management - if anything, suggested not to. LDP conforming servers can't be reliably directed to update membership/containment triples by clients.

But then, it seems the server needs to materialize the containment triples or keep track of how a certain resource was interacted with.

With that knowledge, it should be possible to leave maintainance of containment triples entirely up to the server.

Possible but redundant.

Mmmmmm, perhaps. Not entirely sure. So, the discussion we are getting into is really the one opened in solid/specification#68

In one sense, how exactly containment triples are managed is an implementation detail, but how the it is derived is not, if the client has to ensure that they are created, it has to be detailed.

If we can do that, it simplifies a lot

I reached a different conclusion: a brief list of issues or at least considerations that needs to be taken into account at solid/specification#35 (comment) under "essentially warrants URI string processing tied to LDP that entails". Taken in totality, I think it increases the complexity of the system.

My approach when we get to the point that we disagree on what is most complex is that we should prototype both :-) However, the discussion in specification#35 doesn't really apply here, as that discussion is about shared semantics between the client and the server. This discussion is only about the internals of the server, and the server mints the URIs and assign any semantics that it pleases, as long as the client doesn't need to care. The other issue is if the client should care, and I'm still conflicted about that. :-)

Anyway, in an attempt to illustrate what I mean by freedom afforded by LDP, take an LDP example of the freedom it seems like LDP affords, I think it would be valid to say

<http://example.org/c1/>
   a ldp:BasicContainer;
   dcterms:title "A very simple container";
   ldp:contains <http://example.org/somewhere/else>, <http://example.test/r2>, <http://example.foo/r3>.

That would still be a valid Basic Container, right? If that is the scenario we need to support, then just inspecting the URL would not suffice. But that is where I think Solid goes meh, and says that the system is hierarchical, and thus the structure of the URL can be used to infer the containment.

It is a constraint that can be relaxed later, at some implementation cost, but it seems to me that it is where Solid wants to go first.

having clients updating containment triples is risky, both from a atomicity perspective, and from a security perspective

Sure, I think that's LDP's position as well.

OK.

Requesting to create /foo/bar/baz while expecting /foo/ and /foo/bar/ to also be created (if not exists) along with creating containment triples is not at all atomic. Containment information doesn't need to be transmitted through the URI. LDP's interaction model does that already.

OK, so first, as much as it pains me that we let one request create more resources with representations other than the one that is being addressed, I think that depends on the implementation. I see two ways it could be atomic:

1. The server sees a request to create /foo/bar/baz, and locks the whole tree to root, creates the containers, add containment triples, unlocks, and only reports success if the resource and the containers were successfully created.
2. The server sees a request to create /foo/bar/baz and creates it. All good. Once there is a request for /foo/ or /foo/bar/, it looks up the URL structure and creates the containment triples. This is then (RDF) merged with any other triples of the container representation. Since containment triples in this scenario is never persisted, and always generated upon request from the structure of the URL space, which the server is free to manage, atomicity becomes a non-issue. But that implies that PUT and PATCH semantics shouldn't stand out, since you can't rely on the mere existence of resources in that case.

As for atomicity: you know "make each program do one thing well".. mkdir and touch are different programs. So, overloading PUT to do several changes based on the URI pattern would be like a clever shell script. It is instructing to act on two or more resources (creating and building relationships) that are besides the target resource.

Hold 👃 :-) But, I argue it would be easier anyway. I mean, what would it look like if the client had to be involved in ensuring that the operation over these three resources is atomic and (thus) isolated with HTTP only? If we had LOCK, then I could see it, but then, we'd create scalability issues. You could do something with ETags, I suppose, where the initial PUT returns a validator upon success, and then you is that to go If-Matches on the container updates/creates, but it gets messy if it doesn't succeed. It'd be a complex protocol.

[csarven]

But then, it seems the server needs to materialize the containment triples or keep track of how a certain resource was interacted with.

Already required by LDP (as per interaction model).

In one sense, how exactly containment triples are managed is an implementation detail, but how the it is derived is not, if the client has to ensure that they are created, it has to be detailed.

It is detailed. Do you mean something other than the section on POST and honouring client's desired interaction model(s): https://www.w3.org/TR/ldp/#ldpc-post-createrdf ?

eg. clients wanting to create /foo/bar/ (foo is a container and bar is a container in foo), can do:

POST /
Slug: foo/
Link: <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"

POST /foo/
Slug: bar/
Link: <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"

This discussion is only about the internals of the server

Implementation detail. Q.E.D. ? :)

Solid goes meh, and says that the system is hierarchical

It doesn't need to go meh. It is an implementation detail in that some servers may want to manage containments with container items using different URI paths.

and thus the structure of the URL can be used to infer the containment.

Can be but communication can happen elsewhere without conflicting established recommendations. We can explore alternatives.

For example, the above two POSTs could be condensed into single request - still creating containers foo and bar - with the use of a URI-Template header eg:

POST / HTTP/1.1
Host: example.org
Slug: foo/bar/
Link: <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"
URI-Template: {/slug}

HTTP/1.1 201 Created
Location: http://example.org/foo/bar/
Link: <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"

Template: {/slug}
Expansion: /foo/bar/

The leading slash (/) in {} signals "Path segments, slash-prefixed" as per https://tools.ietf.org/html/rfc6570#section-3.2.6 . Other separator strings can be used. The slug variable expands into foo/bar/. Hence, /foo/bar/.

There are plethora of implementations: https://github.com/uri-templates/uritemplate-spec/wiki/Implementations .. but for basic usage, once / is detected as the separator, splitting the string to get a list of containers is trivial.

Would something like that satisfy:

without breaking away from the Web architecture and core specs that Solid is trying to play along with

We should take care not to break LDP, and also ensure that any extensions are friendly

?

Is there anything else that needs to be signalled?

If the approach is ballpark meaningful, we can clean it up. Or consider other approaches. As I see it, POST + Slug may be starting points for recursive container creation.

I see two ways it could be atomic

I can wrap my mind around the first approach like this: request to create /foo/bar/baz should be done in at least two steps using the appropriate interaction models. First using the LDPC interaction model to create all the containers leading up to the resource (/foo/bar/), second to create the LDPR (baz) with LDPC parent (bar/).

Hold

All I wanted to say was that if we are going to use the success of Unix as an argument, let's see it through ;)

[kjetilk]

But then, it seems the server needs to materialize the containment triples or keep track of how a certain resource was interacted with.

Already required by LDP (as per interaction model).

That has somehow eluded me, could you provide a reference?

In one sense, how exactly containment triples are managed is an implementation detail, but how the it is derived is not, if the client has to ensure that they are created, it has to be detailed.

It is detailed. Do you mean something other than the section on POST and honouring client's desired interaction model(s): https://www.w3.org/TR/ldp/#ldpc-post-createrdf ?

Yes, because how about

POST /foo/
Slug: bar
Link: <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"

or

PUT /foo/bar
Link: <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"

or

PUT /foo/bar/
Link: <http://www.w3.org/ns/ldp#Resource>; rel="type"

which finally comes back to the original subject of the thread. In those cases, my interpretation of the discussion is that they shouldn't be honored, even though they are valid (in my interpretation) with the respective interaction models.

Solid goes meh, and says that the system is hierarchical

It doesn't need to go meh. It is an implementation detail in that some servers may want to manage containments with container items using different URI paths.

Yup, if we decide to go full LDP, it doesn't need to go meh, but I'm sensing that we are designing a constrained LDP, which I'm fine with, as long as it can be relaxed later.

and thus the structure of the URL can be used to infer the containment.

Can be but communication can happen elsewhere without conflicting established recommendations. We can explore alternatives.

For example, the above two POSTs could be condensed into single request - still creating containers foo and bar - with the use of a URI-Template header eg:

Hmmm, right. Didn't think about that one...

I can wrap my mind around the first approach like this: request to create /foo/bar/baz should be done in at least two steps using the appropriate interaction models. First using the LDPC interaction model to create all the containers leading up to the resource (/foo/bar/), second to create the LDPR (baz) with LDPC parent (bar/).

Two requests are still enough to split atoms and create uncontainable plasma ;-)

But, certainly, exploring different avenues is important. But I suspect there are other open issues where they are more appropriate?

[csarven]

I withdraw the URI-Template suggestion. Too much going on. There is a simplification to all of this but it need to only happen through POST.

That has somehow eluded me, could you provide a reference?

I think we slightly talked past each other there. What I meant was that https://www.w3.org/TR/ldp/#ldpc-post-createrdf states:

Servers [..] MUST honor the client's requested interaction model(s).

but how they exactly store and recall that information is an implementation detail.

which I'm fine with, as long as it can be relaxed later.

What do you think of the tentative proposal at solid/specification#39 (comment) which I think clarifies a bunch of issues and recommends POST as the only way to create resources that are intended to be managed in containers. With Slug and Link together for the interaction model, it can also handle the hint for (nested) containers to be created eg. slashes as separator in URI path.