3

I maintain a small (tiny!) .NET library.

It has a few ill-defined edge cases, which I call out explicitly in the docs. Places, where the "correct" behaviour is not self-evidently well-defined, so I explicitly said

For invocations like this, the method may return any of the viable outputs.

Note that in all of these 'may return any' cases, I suspect that it will always return the top-most value, but that will be dependant on .NET's implementation of various methods and is not in anyway guaranteed by this library! Frankly, any of these would be a mis-use of the library and could arguably throw instead.

I want to make a change, to fix a bug, which will still satisfy the first sentence above, but which will happen to switch to return the bottom-most value. (Probably).

I think that I could release this as a patch, and still be honouring SemVer. Because even though the in-practice functionality has changed, it is functionality which the docs explicitly instructed users not to rely upon.

Have I correctly interpretted SemVer?

Brondahl
  • 139
  • 1
    Please clarify whether I understood your question correctly: your Method does something and ends up with a list of things. It returns the first element now, regardless of how many there are. Ideally, it should throw an exception if that list does not have exactly 1 element because choosing the first of several does not yield consistent enough results across runtime environments. You now want to change it to return the last element of that list. Also, if that is indeed the case: why is the last element a better fit? Why is the last element reliably/consistently better than the first one? – marstato Jan 01 '20 at 22:48
  • 1
    obligatory xkcd – candied_orange Jan 02 '20 at 00:11
  • 1
    @marstato That change to the explicitly non-contractual behavior is a side-effect, not intentional. – Deduplicator Jan 02 '20 at 02:46
  • I don't think it's about interpretation of SemVer, nor how precise the API contract is, it's more about the existing usage of the API: any observed behavior can be part of the API as the Hyrum's Law states. – yining Jan 02 '20 at 04:04
  • @ReinstateMonicaBrohndahl then i agree with Deduplicators answer. SemVer doesn't care because the contract explicitly doesn't cover it. It's much more about how your users use the API and how much you care about not breaking code relying on the existing behaviour. – marstato Jan 02 '20 at 09:57

3 Answers3

11

Non-contractual behavior is non-contractual.

Strictly speaking, it thus isn't something to be concerned about at all, and should only bump the patch-number in SemVer.

Unfortunately, some people always think they are clever when they exploit happenstance, instead of coding to the interface, and far more people don't actually know the contract as well as they think, or simply don't care. The documentation-writers might even have "helpfully" documented it for you!

So, it will probably still break some users, even though you are doing The Right Thing™.

In the end, you have to decide how much back-compat you will give to happenstance and common law features, thus blocking bug-fixes and improvements.

Deduplicator
  • 9,031
  • This is a good answer. I would add that if > 90% of your customer base would be surprised by the new behavior, it should be a major bump. It's arguably more subjective if the surprise range is %60..90. It is always good to know your customer's usage patterns. – jwdonahue Jan 21 '20 at 18:38
3

SemVer isn't a rule about what you are allowed to change, it is a signal to your consumers about how safe it is to consume an update. If code breaks on a patch update, your customers aren't going to care that technically you were "allowed" to make that change, they are going to stop trusting your patch updates.

If you think consumers are going to want to intentionally depend on undefined behavior, then define it. If you think consumers are going to want to spend extra time to ensure they or a transitive dependency doesn't accidentally depend on undefined behavior, then version it accordingly.

Karl Bielefeldt
  • 147,435
1

This type of unspecified behavior seems to be beyond what is defined in Semantic Versioning:

For this system to work, you first need to declare a public API. This may consist of documentation or be enforced by the code itself. Regardless, it is important that this API be clear and precise.

The phrase "this system" refers to Semantic Versioning. The public API that your library provides is not only the implementation, but also includes the documentation of the API. Since this behavior is consistent with the documentation, on the surface it feels like this would be a backwards compatible change that would cause the PATCH version to be incremented.

However, note the last sentence in the description of what it takes SemVer to work - the API needs to be "clear and precise". Your API is not precise if the same version of your library may return different values in different environments. It seems like some change to the .NET framework libraries that are used will cause your library to behave very differently. As a user, I may see what I would think is a backwards incompatible change if I disregard your documentation. And this difference in perspective is why SemVer requires the precise API.

I'd recommend considering marstato's comment. Look at the design and implementation decisions that you made in the API. Is there a way to make this behavior consistent across environments that is still meaningful and then create a more clear and precise API?

Thomas Owens
  • 82,739
  • 1
    I would dispute that the API only having fully specified output if the input follows some simple (but potentially very costly to verify) rules stops it being "clear and precise". Of course, putting gratuitious restrictions on the input should be avoided. – Deduplicator Jan 02 '20 at 02:51