This actually isn't specific to Java, but applies to any system that needs to resolve dependencies to a single version. (Which I think is actually most package managers, with the notable exception of NPM [which will instead wantonly break the build by installing incompatible versions of a dependency even when they are part of the API.])

I agree with the MAJOR change rule being generally applicable to most package dependency systems. This new rule leads to a new kind of necessary "MAJOR version increment hell" where a commonly referenced project makes a breaking API change and requires all upstream dependencies libraries / systems to make changes and subsequently increment MAJOR as well.

In the cases of MINOR and PATCH, many dependency systems let you specify version ranges and in most cases (>= A.0.0) AND (< A+1.0.0) work fine. So I don't know if there is a general need to increment your MINOR & PATCH every time a transitive dependency posted a new version.

That would simplify the amendment down to something like:

Transitive dependencies are part of the public API thus the version of the user library SHOULD be incremented when a direct dependency makes an incompatible API change:

• for MAJOR change in a direct dependency we SHOULD increment MAJOR part of our version

This rule applies recursively for all transitive dependencies.

This is a "what is being versioned?" problem.

If you are shipping a dependency in your package, and that changes, your package version has to be bumped, but is it a patch, minor or major change? If it's the kind of package that works perfectly fine in side by side (SxS) installations and review/test shows little risk of breaking your customer's code, then it might just be a patch bump. It always comes down to what impact does this have on your customers? If it does not support SxS, then your customers must take the update to use your product, but this could break other products on their system that also rely on that package, so you need a matching minor or major version bump.

If you do not ship the dependency in your package and it changes without any corresponding changes in your code, no bump is needed. If you modify anything in your package, even a manifest, you have to bump something. Again, if you force your customers to take a change that could impact any other software installed on their system, you must bump either the minor or major version, depending on the level of changes made in that dependency.

I think the spec covers all of this, it's just that the full ramifications of it aren't always well understood. So I think I just talked myself into supporting some kind of change in the FAQ to cover these aspects, I don't think it would be Java specific at all.

@jwdonahue
My concern is mostly about JVM platform with a single classpath. If there is an option to use side-by-side deployment (like OSGi containers), then it might be possible to use different versions of transitive dependencies. However, this itself might be tricky.
In case of a single classpath JVM deployment, transitive dependency change is sufficient to break user's code, even if nothing has changed in the library's code. That's why the suggestion.

For example, let's take a look at http4s-core library:

org.http4s | http4s-core_2.11 | 0.16.5  | 11-Oct-2017 | pom  jar  javadoc.jar  sources.jar
org.http4s | http4s-core_2.11 | 0.16.5a | 11-Oct-2017 | pom  jar  javadoc.jar  sources.jar

Looking at this suffix a do you suspect any significant difference? If we take a look at the project's site, we'll find out that the difference is in a single transitive dependency - scalaz 7.2 or 7.1. If we research further (https://github.com/scalaz/scalaz/wiki/7.2.0), we'll find out that

This release is NOT binary compatible with 7.1.x and previous milestones, release candidate in the 7.2.x series

So switching 0.16.5 to 0.16.5a will cause major incompatibility in user's code despite that these versions have the same code except the transitive dependency.

That's why I think that this innocent looking suffix a should have been major version change (or at least 0.16 -> 0.17).

I see your point, can you give us any examples where the major version >=1 ? I fear it may be unfortunate that you chose an example where breaking changes are in fact to be expected from any minor change, because the major version is zero.

I should also point out that version labels you use as examples, do not adhere to the SemVer spec.

This is a real example we have seen in our project. I can't remember a real example with semver versions.

Indeed, the version labels are not SemVer. However, we might replace the version labels accordingly.

For scalaz versions should have been v7.1.x and v8.0.x (instead of 7.2). What version change should be reasonable for http4s? This issue says that it should have been major part change. For instance, 16.0.0 -> 17.0.0, ... 16.5.0 -> 17.5.0 (with just a transitive dependency version change). Two branches might evolve in parallel but with the explicit major change.

Ok, I think the spec already covers these cases. If you introduce a breaking change, you bump the major version. Should the FAQ cover every scenario that is a breaking change? Probably not, is Java worthy of a special mention? Perhaps.

Please do one or more of the following:

• Clone semver/semver.org, make the changes you propose and issue a pull request. I'm not certain you need to create an issue to link to over there or if you can just link the PR to this one (still a little green on GitHub).
• Close this issue at your earliest convenience.

@Primetalk, apologies for ignoring this for so long. I appreciate that you have expended the effort to issue a PR and I think that should probably be merged. Unfortunately, @haacked has been very busy over the past year and might not get to it for a while. I have created issue #468 in order to step back and take a holistic look at the whole API/Package/Dependencies issue. I've come to the conclusion that if we work this problem piecemeal, we could make it worse, but some additional FAQ's such as yours might help. I also suspect that the spec can't really be fixed without a 3.0.0 version of the spec.

ImgBot

On the original topic:

MAJOR change in d => increment MAJOR part of version A;

This is a way to go only if a new version of A stops working with old version of d.
If it works with both versions (i.e. breaking change in d doesn't break anything used by A) then there are no breaking changes for A, and no need to increase MAJOR.

1. If the new version of A works with old d, we might better stick with the old dependency?
2. It's fine that for A there is no breaking change. Unfortunately, in Java world there is typically a single classpath. So when A brings in d of the newer version, it might break some other code that is also using d. And the only way to reflect this danger is to explicitly state that A has major change.
   (Imagine a situation when d used to implement some interface I, that is not used in A. B depends on A that returns an instance from d, and B expects that instance to implement I. Unfortunately, in the newer version of d it's no longer the case. This leads to breakage in the larger classpath.)

Unfortunately, in Java world there is typically a single classpath

TL;DR: It's the job of your package manager/runtime to guarantee that the semver constraints for every package are satisfied.

Consider the following dependency graph.

Application:

• Framework
• Library@^1.0.0

Framework:

• Library@^1.0.0

Changing the major version to Library@^2.0.0 in Framework should not result in a different version in Application since the semver would not be satisfied. E.g. npm or yarn (or any package manager in the JS ecosystem) would pull in both version and ensure that Application still uses Library@^1.0.0 while Framework uses Library@^2.0.0

In general I disagree that transitive dependencies are part of the public API. The public API is what you define it is. If you you say that your library function returns an iterable then it is not breaking if you change your internal implementation (provided by a dependency) from a LinkedList to a TreeSet.

The consumer might find it important to know how many 3rd party dependencies exist but it's entirely up to the library maintainer to define this as part of the public API or not.

Edit:
The runtime is actually responsible for loading dependencies. The package manager is a prerequisite for the expected behavior in the runtime though.

In your example, both app and framework depend on library. And they require different versions. Strictly speaking this configuration should not be allowed at all.

Let's consider the following changes in Lib
A. Add implementation of a useful method foo to one of the returned objects.
B. Remove unsound method baz.
Since these are incompatible changes, Lib gets version 2.0.

There are the following possible failures:

1. If lib 1.0. is actually loaded.
   Lib 2.0 has change A. It returns an object that supports method foo. Framework expects that lib 2.0 implements the new feature and removes a workaround for it that it implemented for lib 1.0. It simply returns object provided by library (that has method foo) in one of it's methods bar.

Public API of framework doesn't change. In both versions method bar returns an object that implements method foo. Just implementation moved from framework to lib.

Tests of framework do not cover this case, because the results of lib are not used directly and instead returned to the application. Application, in it's turn, expects that Framework return correct results - bar will return an object that implements foo.

In this case if the actually used version is lib 1.0 application will fail.

2. If lib 2.0 is loaded, then application might fail on it's own, because lib 2.0 is incompatible with 1.0 (due to change B). They have simply removed a weird method baz. And this method was used by application...

There is no good decision that could be made based on these erroneous versions.

On the other hand, if we explicitly promote this difference in the version of framework, then it'll be obvious that application has to be upgraded.

Strictly speaking this configuration should not be allowed at all.

@Primetalk Why? I made that argument under the premise that your package manager (or runtime actually) should ensure that the semver constraints for each module is fulfilled.

I guess some runtimes don't do this? There's no useful argument to be had if you argue under the assumptions that runtimes are not responsible for semver constraints while I think they should. At that point we're arguing about implementation not specification.

Package managers and the node runtime, however, show that you can change transitive dependencies in a patch without violating semver for consumers of your package.

Package managers and the node runtime, however, show that you can change transitive dependencies in a patch without violating semver for consumers of your package.

The question is not if some package managers can resolve some transitive dependency updates, the questions are:

1. Can all (reasonably imagined or existing) package managers and runtimes resolve all transitive dependency updates without breaking consuming code?
2. If the answer to 1) is "No", should the SemVer spec be updated to recommend or mandate MAJOR version bumps in some or all transitive dependency changes?

The answer to 1) is very clearly "No". An easy example is library with a stateful singleton as only export. Either there will be a conflict in interfaces or the singleton nature will be violated if package manager/runtime provide differing instances.

This leaves us with 2), which really is a question about the definition of "(public) API" in the SemVer spec. The usual understanding in the developer community seems to be "the objects my package exports, their names, their interfaces and possibly their path relative to the package root". This is enforced by the answer to the FAQ "What should I do if I update my own dependencies without changing the public API?", but strictly speaking this FAQ doesn't actually make any claims about what "public API" actually means. It just asserts that if your public API is unchanged, SemVer does not mandate a MAJOR version bump.

I agree with @Primetalk that dependencies can be part of the public API of a piece of software and that the SemVer spec and the FAQ should be updated to improve verbiage and clarity around this topic. At the very least there should be a note that for many authors of libraries, what is versioned does not include dependencies.

My personal benchmark for "Is this a MAJOR change?" has always been "Could there be a consumer that uses my code as documented/intended who will need to change code (as opposed to package manager configuration) to consume my new package?".

This leaves us with 2), which really is a question about the definition of "(public) API" in the SemVer spec.

But that is what you define:

Software using Semantic Versioning MUST declare a public API.

-- https://semver.org/

SemVer does not define it for you. It sounds like this question is about what is implied by a certain language/runtime/ecosystem when defining a public API and not about transitive dependencies in general.

I agree with @Primetalk that dependencies can be part of the public API of a piece of software

Emphasis on can. If it is not always the case SemVer should not mandate a bump.

This leaves us with 2), which really is a question about the definition of "(public) API" in the SemVer spec.

But that is what you define:

Software using Semantic Versioning MUST declare a public API.

But this does not define what a public API is. Worse, the very next sentence is (emphasis by me)

This API could be declared in the code itself or exist strictly in documentation.

Now combine this with (emphasis by me)

In computer programming, an application programming interface (API) is a set of subroutine definitions, communication protocols, and tools for building software. In general terms, it is a set of clearly defined methods of communication among various components.

from https://en.wikipedia.org/wiki/Application_programming_interface and I am hard-pressed to find a way how API can not include the methods and objects of your dependencies you use.

I agree with @Primetalk that dependencies can be part of the public API of a piece of software

Emphasis on can. If it is not always the case SemVer should not mandate a bump.

But the current spec (or rather, the FAQ that are in the same .md file) does not even hint at can.

That would be considered compatible since it does not affect the public API.

In RFC2119 terms thats a SHOULD NOT at best, but it could easily interpreted as a MUST NOT.
I think the discussion here (and in other issues) has shown clearly that a narrow definition of "Public API" that excludes dependencies is problematic both for authors and for consumers.

Neither @Primetalk nor I want to mandate a bump. The PR @Primetalk opened uses SHOULD which is a recommendation, but not a mandate.
I would actually prefer to keep it even more broad and simply put a statement that authors need to declare explicitly if dependencies are considered part of the public API or not (and possibly caution consumers that packages that do not specify this one way or another should be treated as if dependencies are not part of what is being versioned).

The semantics of your public API are part of it. If a dependency version bump changes that, then semver applies. If it does not, then there's no observable change, so there's nothing to bump (besides what you'd normally increment for publishing).

I am hard-pressed to find a way how API can not include the methods and objects of your dependencies you use.

I gave you an example for this:

If you you say that your library function returns an iterable then it is not breaking if you change your internal implementation (provided by a dependency) from a LinkedList to a TreeSet.

I could conjure up plenty of examples for this. The important part is if a dependency is an implementation detail or not. I'm hard-pressed to accept that every dependency is part of your public API. If it's not public then it's not public. There's nothing special about dependencies. Even the mere existence of a dependency is an implementation detail. It doesn't matter if you have a dependency tree that is 3 levels deep or if every dependency is inlined. This is probably the best advice for evaluating transitive dependency impact on SemVer: "If every dependency would be inlined (removing the notion of dependecies) would my public API change?"

If you you say that your library function returns an iterable then it is not breaking if you change your internal implementation (provided by a dependency) from a LinkedList to a TreeSet.

You just shifted your dependency from e.g. Guava to the JVM version (Iterable was introduced in 1.5) without providing any further argument why runtime dependencies of your code should not be included in what is versioned. Iterable in Java 1.5 does not have a forEach method. So if your library offers a printAll(Iterable i) function and you start using forEach internally, you just broke for all your consumers that use javac 1.5 (and 1.6 and 1.7). Thanks for proving my point.

That is true whether you have transitive deps or not - which is why “whether a transitive dep change affects your version” is dependent on the actual change and not on the mere fact that the transitive dep changed.

which is why “whether a transitive dep change affects your version” is dependent on the actual change and not on the mere fact that the transitive dep changed.

I do not disagree with that, but the SemVer spec* does, because it says that

That would be considered compatible since it does not affect the public API

The sentences after that try to "soften" it a bit, but this statement implies a distinction between "public API" (which is versioned under SemVer) and dependencies.
Considering that the FAQ are part of the same document as the formal spec, it is important to have very clear language here, so that authors have the best possible guidance in determining SemVer impact of changes they make. The current verbiage is not clear, it implies a separation between versioned API and dependencies that does not exist and it leads to authors "under-reporting" breaking changes, which leads to a loss of trust in SemVer as a concept in consumers.

I find it very clear; the thing that’s part of your api is your usage of your deps, not the deps themselves.

Do you really believe that the authors who under report a breaking change are doing so because this verbose spec has multiple possible intepretations? If so, have you tried a PR with your suggested wording?

I find it very clear

If it was clear to everyone, this whole thread would not exist ;)

the thing that’s part of your api is your usage of your deps, not the deps themselves

Even that statement is too absolute. Singletons, especially stateful ones, are a good example where simply bumping the version of a dependency without any other code change may be a breaking change, e.g. in node where the singleton nature of a module depends on the absolute path and the resolution of name to path depends on the path of call-site. Nested dependencies compound this problem.

Do you really believe that the authors who under report a breaking change are doing so because this verbose spec has multiple possible intepretations?

Seen this more than once and at least one developer pointed me to that very FAQ to justify his patch level release when doing a major upgrade of a dependency shared. That is actually what lead me to find this discussion here.

If so, have you tried a PR with your suggested wording?

I have not. @Primetalk has however and the slow progress on that gives me little hope that a PR of my own would have much chance of being accepted at this time. I will think about it some more though.

@eps1lon
What you are talking is about unobservable changes. If that is possible - fine.
My example is different and shows the real problem.

This issue is about that on at least JVM platform there is a leak of transitive dependencies via unchanged public API. (The same problem exists on all platforms where only one version of a dependency can be used by the same runtime.)
If you take a look at the example above, you'll see that the problem is unavoidable unless we make transitive dependency part of public API.

My example is different and shows the real problem.

That is your opinion. You talk like this is some objective truth by tying SemVer to Java. I don't agree that this is the real problem. The real problem is that you changed your public API. How you did this is by not curating your transitive dependencies. SemVer should not dictate how changes are made but what changes are made. It is the responsibility of library authors to carefully define your public API.

unless we make transitive dependency part of public API.

Sounds like they already are. SemVer is not runtime specific and should therefore make no claims about how the runtime should behave.

Ok, great.

Sounds like they already are.

This is what the issue about. It describes that transitive dependencies are part of public API. And recommends to bump major version whenever transitive dependencies change major version.
(Also this issue specifically states that it is applicable to platforms where only one instance of dependency is used at runtime, like in JVM.)

You just shifted your dependency from e.g. Guava to the JVM version

@tkissing-work You just assumed this was Java. I didn't prove your point you strawmanned mine. This wasn't even concerned with Java. These concepts are language independent.

Seriously? Replace Guava with [wherever you get LinkedList from] and JVM with [wherever you get Iterable from]. Happy now?

Again, the point is not that you can conceive some scenario where bumping a dependency of your library has no impact on consumers, but that there are many scenarios where it does and where the impact is "consumers code breaks".

If they’re a part of your public API, then it’s a major bump when your public API has a breaking change. Whether the mechanism for that is that you bumped an improperly encapsulated dep or that you edited the code, it’s still not directly related to your transitive deps.

Just use inline dependencies as a mental model and you can make the decision.

That is precisely the wrong model for most languages/runtimes. Inlined code is isolated to its scope, dependencies are shared and in most runtimes that means all the code in the runtime needs to agree upon how to interact with the dependency and how the dependency itself may interact with the runtime environment.
This is has nothing to do with Java, you have the same problem in any language where one piece of code can include and use another piece of code by name.

That is precisely the wrong model for most languages/runtimes.

Could you name the languages where this is not the case?

This is has nothing to do with Java, you have the same problem in any language where one piece of code can include and use another piece of code by name.

Not true for javascript. The node runtime has solved this problem with node_modules/ layout and npm. Webpack or any other bundler uses the same standard. The deno runtime even has semver built-in when loading dependencies.

That is precisely the wrong model for most languages/runtimes.

Could you name the languages where this is not the case?

Languages for which import statements are not the same as inline code (partial list):

JavaScript, Python, Ruby, PHP, Java, C, C#

This is has nothing to do with Java, you have the same problem in any language where one piece of code can include and use another piece of code by name.

Not true for javascript. The node runtime has solved this problem with node_modules/ layout and npm.

Which only solves the problem for stateless modules. The moment your module does module.exports = new StatefulThing() or maintains state otherwise, things can and will break, because the two different paths mean there are now two different instances and their state will be independent. Singletons in node are only singletons as long as all files that require them want the same version.

Webpack or any other bundler uses the same standard.

Webpack just requires the app developer to resolve any conflicts as it flattens the dependency tree. Only one of the version will make it into the final bundle (unless you create intermediate bundles that hide the shared dependency, in which case you get the same singleton issue you have with node).

The deno runtime even has semver built-in when loading dependencies.

Same issue as regular node, if a module is supposed to be a stateful singleton, that assumption is only true as long all code agrees on the version.

Alright I guess I made my peace. You are very creative with constructing specific examples (now stateful modules) that break the public API but have in general nothing to do with being a dependency. Making transitive dependencies a MUST in the public API would be a breaking change for the SemVer spec. If this would be accepted I'm pretty sure any stateless library will not use that version of the spec.

I am not asking for MUST though, I am asking for MAY.
I am saying that the current wording is not precise and that it implies SHOULD NOT.
You seem to even prefer an interpretation that equates to MUST NOT.

https://github.com/tkissing-work/semver-dependencies-are-api is a very simple example of how even in node a the problem is not solved for all cases.

I feel like a broken record, but I will say it again:
I do not argue that ALL dependency updates have SemVer impact, I argue that some do and that the whole idea of SemVer getting us out of dependency hell requires SemVer to be reliable. To achieve that, the spec should be more clear in defining what "public API" means and remove wording from the FAQ that asserts that dependency updates are SemVer neutral.

The spec talks mostly about versioning API's, not packages, art or cars, yet the semantics can be applied to almost anything that requires versioning, if you are at least reasonably aware of exactly what the version number applies to. The word Package appears in the specification only once, but ten more times elsewhere in the document.

SemVer never set out to solve the diamond point, or any other dependency problem. It's not a tool, it's just a description of one way to version API's. Unfortunately, it also mentions packages. Well the only players in the game, that could successfully implement, support and promote SemVer, were the packaging tool owners, how could it possibly ignore them? Now it's the package tool owners who own the spec. Are any of them anxious to see the spec define things like how to version something with transitive dependencies? Probably not. Each of them owns one of the dominant packaging tools for their particular language or framework. They handle dependencies differently.

Thanks everyone for contributions, you're amazing 🎆 Did you find any consensus?

Closing as staled, feel free to re-open or create a new issue 👻

What will be the destiny of the PR - #414?
I thought that this issue is to be closed by merging that PR. Are there any objections to merge it?

2. It's fine that for **A** there is no breaking change. Unfortunately, in Java world there is typically a single classpath. So when **A** brings in **d** of the newer version, it might break some other code that is also using **d**.

Some other code is not responsibility of A's author. A library author's responsibility to indicate if change in a library breaks or does not break A's consumer.
If it happens that consumer also consumes d directly, then it is outside of A scope.

IMO, the main problem with this proposition is the difficulty of applying it in the real life.

If a transitive dependency uses SemVer then sure. I limit major version anyway, and I know that when they release a new major they are breaking something, so does me.

But if a transitive dependency doesn't use SemVer I cannot be sure whether their new version is a breaking change or not. It's not my code, I just use several features of it.
Consider Guava with their @Beta APIs as an example. Nobody sane wants to read their changelogs and decide whether each change should be considered as "breaking" or "non-breaking". And why it should be done by us, the users, anyway? It's them who should've used SemVer in the first place.

The proposition that could be really discussed is
Transitive dependencies that use SemVer are part of public API

No, if you use a dependency that doesn’t use semver, you’re responsible for pinning it so that consumers aren’t broken. every dependency of yours is part of your public api.

And why it should be done by us, the users, anyway?

Your users could ask the same question: Why should they have to read your changelog and all of your dependencies changelogs in order to tell whether your new version is a breaking change.

Keep in mind if some other library has conflicting requirements it would even be a problem for your users who don't use the transitive dependency directly.