content: refactor threat diagram and add overview

[MarkLodato]

Preview: https://deploy-preview-1057--slsa.netlify.app/spec/v1.1/threats

Refactor the threat diagram to address clarity concerns and to expand it beyond tampering. The intent is that this threat model can be useful for any software supply chain security effort. Many of the threats previously called "out of scope" should be listed here, even if the SLSA ladder does not (yet!) cover them.

NOTE: this PR is a partial solution, but more work is needed. I want to at least merge what we have so far so that others can iterate on it.

This design is the result of much discussion on Slack. Thank you to @adityasaky, @arewm, @david-a-wheeler, @jkjell, @mlieberman85, @marcelamelara, and @trishankatdatadog for your contributions and suggestions.

Summary of major changes:

• Add threat indicators for Producer and Consumer, remove for Dependencies.
• Rename "Package" to "Distribution".
• Rewrite titles to describe the position rather than the tampering threat.

Detailed diagram changes:

• Update the threat markers:
  • Add a threat for Producer to cover malicious intent. For example, if you install malware, it's not tampering---the producer really did intend to write malware, and no amount of code review will "fix" that! (Previously this was called "out of scope".)
  • Add a threat for Consumer to cover...? It makes the diagram nicer and I assume we want something here, but I don't know what that is yet!
  • Remove the threat for Dependency, since it generated a lot of confusion. The intent was that it is recursive, so the hope is that the diagram and text make this clear enough.
  • Re-letter the threat markers accordingly.
  • Update the threat titles to describe the position rather than the tampering threat. The old titles generated a lot of disagreement, and they relied on understanding the non-obvious interpretation of the model. Now, the new titles just describe that model, which is hopefully more clear to everyone.
• Rename "Package" to "Distribution" to better reflect what that box means.
• Move the arrow from Distribution to Dependency to make it clear that (H) is also recursive.
  • Also make the arrow solid instead of dashed, but keep the dashed box around Dependency. The idea is that the use of the dependency is a real input to the build, while Dependency itself is really just another package.
• Move "build params" to "Build threats" instead of "Source threats", and add new "Usage threats" category.
• (minor) Use the same green color throughout, rather than having a slightly different green color for arrows.

Text changes:

• Add an overview section that explains a bit more about the threat model.
• Add Dependency Confusion.
• Update text to match the new diagram (only partially done).

[netlify]

✅ Deploy Preview for slsa ready!

Name	Link
🔨 Latest commit	8a0d8f8
🔍 Latest deploy log	https://app.netlify.com/sites/slsa/deploys/666089b580d3070008dab6c9
😎 Deploy Preview	https://deploy-preview-1057--slsa.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[MarkLodato]

Here are some options for the visualization of Z.

(Option 0) Don't add Z.

This is worth considering. Is the addition of Z too distracting?

(Option 1) Add Z off to the side with a curly bar.

(Option 2) Add Z at the bottom, replacing the "Source/Build/Dependency Threats" labels.

(Adding to to the top seemed too distracting.)

(Option 3) Add Z by itself under Consumer.

Also:

• Update the line from Package to Dependencies: (1) come out of the same line as H to make it clear that H also applies, and (2) make it solid since the dashed was confusing. The box around Dependencies is still dashed to show (hopefully!) that it's not a separate thing.
• Make J solid again, since the inverted colors were distracting.
• Say "(A-J recursively)" (not Z)
• Fix the colors (the lines used a slightly off color of green; I need to fix this regardless.)

[MarkLodato]

I might go back to (J) being the same color as the rest. It's kind of distracting with inverted colors.

[adityasaky]

I prefer the second image, just so it's clear that Z applies to all steps. I second some of the points that came up in today's SLSA spec meeting about Z, FWIW.

Transcribing some comments from Slack:

Me: F seems too specific in its naming; you could argue that a compromise of a package upload action to insert another artifact is technically a direct package upload, but those words to me specifically call out something like a developer running twine upload to submit something to PyPI.

@MarkLodato: Regarding F, yeah, I'd like a better name. Maybe "Misuse of upload credentials" or something like that? My intent was to call out the use upload API to upload arbitrary things, as opposed to tampering with a legitimate build (E) or, if a SLSA policy is in place, tampering with the inputs to build (D).

I think this is better!

Me: nit: I also wonder about consistency between F and H with "package" vs "artifact"

@MarkLodato: Regarding "package" vs "artifact", perhaps it's too subtle. My thinking was that in (H) someone selects a "package identifier" and then receives back an "artifact" (blob of data). But I agree that maybe it's just too confusing

I think this is a good point about the subtlety, but to me that goes back to the confusion we discussed last week about whether the wrong artifact being returned is actually part of G, since this considers the "processing of request for package" operation as well. Maybe I'm overthinking this specific connection though.

[MarkLodato]

I updated the comment above with an Option 3 incorporating some feedback from today's meeting.

It's also worth considering Option 0: not adding Z at all. I'd still like to somehow surface the risk of missing visibility, so we'd need an alternate proposal if we don't add a label on the diagram.

[MarkLodato]

I think I prefer the new line coming out of the right of Package, since it more clearly indicates that (H) is in scope for dependencies. But now it makes it seem like (I) is not in scope, whereas it really is.

One option for that would be to merge dependency confusion back into (H) like we had before because it makes the diagram nicer, but I don't like that because it's useful to talk about dependency confusion separately. And I don't want to move dependency confusion to J because it happens to consumers too, e.g. pip install some-typo-package.

Independently, I wonder if we should just merge visibility into (I) and expand it to all consumer threats. We kind of have that issue with (A) and (B), where there are a lot of sub-categories of threats within there that the diagram doesn't show. We could make (I) include:

• Use of unintended package
• Lack of visibility into supply chain (transitively)
• Lack of verification of integrity (or something like that)

I don't know how I feel about that.

[arewm]

Can you merge Consumer and Dependencies into one if we rename it something like Consumption? That makes the explicit connection between the two and we can clarify in text that consumption can be either the final artifact or a dependency input.

Or to make it a noun, just move consumer to replace dependencies.

[MarkLodato]

More diagrams, for the record. (These were all posted in Slack.)

Option 4: Add Producer (A) and Consumer (I), remove Dependency (J) and Visibility (Z)

The thinking is that J is confusing since it's really just a manifestation of all the other threats, so we can just visually show with the loop in the diagram. Also Z is confusing since it's not like hte others, so let's just omit it.

Option 5: Same as above, without I

Option 6: Arrow coming out of consumer

This is from @arewm. The arrow coming out of Consumer is to indicate that the build is just one type of consumer.

Here's a variant with dashed boxes:

And now no box on Dependencies:

Option 7: Same as Option 5 but with describing positions rather than threats

In Slack, Option 5 seemed to be the most popular (or at least I liked it the best!) but there was a lot of disagreement about what the labels should be. Rather than trying to describe threats at each point, we could instead just describe what the point means. This way we can talk about this in the model, and then talk about higher level attacks and point to the locations where they happened. This seemed to get at least some positive feedback.

[MarkLodato]

OK, I think this is ready for review. As I wrote in the top comment, it's not complete, but I'm hoping that we can at least agree on an intermediate state and merge it with TODOs, so that we can iterate.

[mlieberman85]

LGTM. I agree that generally this is in a reasonable state and we can iterate as needed.

[adityasaky]

Not a SLSA maintainer, but LGTM to merge, agree with iterating on TODOs rather than blocking! Thanks for working / coordinating this @MarkLodato!

[lehors]

It seems to me that we probably should change Source to Source Control or something like that, but I agree this is an improvement anyway and I'm happy for this to be merged as is.

[MarkLodato]

Thanks all. I'll wait for input from a few more folks, since this is a significant change in the main diagram that is used in a lot of places.

Note: I updated the "verification" diagram as well, since H is now out of scope.

[joshuagl]

This is excellent, thanks! I really like the major changes and fully agree with them. The minor changes look like an excellent start, too.

I made some minor comments, but this LGTM.

[marcelamelara]

Thanks @MarkLodato ! I have a few clarifying comments/questions, but I definitely think this is almost ready.

[marcelamelara]

I think this split is valuable, though I think the distinction between "producer submits bad code" and "authoring", i.e. someone introduces a code change through official interfaces. Is this about first- vs. third-party changes? A distinction I often draw between producer/entity threats and authoring threats is that producer threats are often tied credential compromises, while authoring is about the intentional introduction of bad/malicious code.

[MarkLodato]

That was not my intent, though I think your description is more in line with what @david-a-wheeler was saying. So we need to align on the model.

I rewrote the paragraph above to hopefully better explain the difference, and add a corresponding paragraph to (A). Does that help?

The reason why I think this is a useful split is because it's grouping by mitigation. In (A) code review won't help, in (B) it will to some degree.

[marcelamelara]

Yes, I think the distinction between (A) and (B) is a little clearer now, thanks!

The part about the producer practices in (A) is still a bit unclear, though. The first paragraph says "or the producer otherwise uses practices that are not deserving of the consumer's trust", while the threat description below says "Software producer intentionally submits "bad" code, following all proper processes". I think this is why I thought earlier that (A) was about bad first-party code, but it sounds like (A) rather covers producer threats that aren't related to code changes. In that case, I can see compromised producer credentials fitting as a threat category under (A).

That said, I'm fine with merging the current iteration of threats (A) and (B) with possible TODOs for further details and/or examples on the distinction between the two.

[MarkLodato]

Thanks. I agree it's still a bit fuzzy. I moved the TODO from (B) up to (A) and expanded it a bit to talk about compromised developer credentials, which I agree is unclear where it should go.

I'll submit now and we can hopefully improve over time :)

Thanks for the help!

[marcelamelara]

Thanks for all your work on this @MarkLodato . This lays a good foundation that we can keep refining.

[marcelamelara]

Yes, I think the distinction between (A) and (B) is a little clearer now, thanks!

The part about the producer practices in (A) is still a bit unclear, though. The first paragraph says "or the producer otherwise uses practices that are not deserving of the consumer's trust", while the threat description below says "Software producer intentionally submits "bad" code, following all proper processes". I think this is why I thought earlier that (A) was about bad first-party code, but it sounds like (A) rather covers producer threats that aren't related to code changes. In that case, I can see compromised producer credentials fitting as a threat category under (A).

That said, I'm fine with merging the current iteration of threats (A) and (B) with possible TODOs for further details and/or examples on the distinction between the two.