Support for DSSE signatures over binary payloads

[laurentsimon]

The current API supports only intoto statement, and we'd like support for arbitrary DSSE blobs.

We discussed this in the past but the original DSSE issue has been closed, so creating this issue for tracking.

[woodruffw]

Thanks for filing @laurentsimon! I tweaked the issue title slightly so that I don't forget that this is about arbitrary blobs wrapped in a DSSE envelope 🙂

[laurentsimon]

Is there anything I can do to help with this feature?

[woodruffw]

If you want to take a stab at implementing it, that would be great!

Otherwise, I don't have any immediate bandwidth to try my hand at it 😅

[laurentsimon]

Would the change consist in updating https://github.com/sigstore/sigstore-python/blob/main/sigstore/sign.py#L192 by allowing input_: dsse.Statement | bytes, dsse_type: str (where bytes would be the payload and dsse_type is the type of the payload) + update https://github.com/sigstore/sigstore-python/blob/main/sigstore/dsse.py#L207 accordingly?

For verification, we'd add an optional dsse_type as argument to _verify.

Or do you have another idea?

[woodruffw]

Would the change consist in updating https://github.com/sigstore/sigstore-python/blob/main/sigstore/sign.py#L192 by allowing input_: dsse.Statement | bytes, dsse_type: str (where bytes would be the payload and dsse_type is the type of the payload) + update https://github.com/sigstore/sigstore-python/blob/main/sigstore/dsse.py#L207 accordingly?

That sounds pretty close to what I was thinking, although I have a (minor) concern that exposing the DSSE payload type like this will be misuse prone (e.g. users marking JSON as binary or vice versa). Do you have a concrete need for a payload type other than application/octet-stream or similar?

(Or as a related question: is there a fixed DSSE payload type for binary data already that we should use? I couldn't find one with some searching, but I might have missed it 😅)

For verification, we'd add an optional dsse_type as argument to _verify.

IIUC, this shouldn't be necessary: _verify takes an Envelope, which will have the payload_type you specify.

---

To take a step back a second: I realized that I'm not actually sure the rest of the Sigstore ecosystem even supports non-JSON DSSE bundles yet.

CC @haydentherapper: do you know if Rekor would accept these? I seem to recall there being a requirement that DSSE entries contain in-toto statements within Rekor itself.

Edit: this is the check I was thinking of: https://github.com/sigstore/rekor/blob/92cf4d5d3682ffcbf4138341f1bc5f7a66766549/pkg/types/dsse/v0.0.1/entry.go#L150-L156

[laurentsimon]

application/octet-stream

Our use case is model signing for multiple files in folder, so would be something like application/vnd.ai-model-manifest+json or maybe more generic application/vnd.path-manifest+json @mihaimaruseac to comment

[mihaimaruseac]

Oh, I haven't thought at all about this :(. Yes, I think something like application/...+json is what we'd need to have, but I'll have to think a little bit more about this

[haydentherapper]

Yep, json is all that Rekor supports currently, though we could support more than that. It means that other clients have to understand how to canonicalize non-JSON DSSEs, which I'm not sure if any would currently.

[laurentsimon]

Thanks all. For model signing, it will be json format.

CC @haydentherapper: do you know if Rekor would accept these? I seem to recall there being a requirement that DSSE entries contain in-toto statements within Rekor itself.

@haydentherapper can you confirm non-intoto DSSE payloads work with rekor?

[haydentherapper]

No, they don't currently. Rekor only understands intoto payloads - https://github.com/sigstore/rekor/blob/main/pkg/types/dsse/v0.0.1/entry.go#L113 and https://github.com/sigstore/rekor/blob/main/pkg/types/dsse/v0.0.1/entry.go#L159

[laurentsimon]

We had an offline chat with @haydentherapper and @mihaimaruseac to brainstorm. What came out is that Sigstore would like to not be in the business of litigating addition of new hashes that users may want to use. This mean (Hayden, Mihai please correct me if wrong) that #1018 can be closed in favor of this issue?

@haydentherapper mentioned that hashrekord (already supported in this library) could be a way forward, and we'd update https://github.com/sigstore/rekor/blob/92cf4d5d3682ffcbf4138341f1bc5f7a66766549/pkg/types/dsse/v0.0.1/entry.go#L150-L156 to accept a new DSSE payload type.

[laurentsimon]

/cc @susperius

[laurentsimon]

@haydentherapper can you explain why rekor needs to be aware of the hash type used in intoto subject? Why is the hash type used for DSSE signing not enough? I chatted with @susperius and this question came up.
Thanks