2

I am preparing a software package for internal consumption in my organization. It will be published on our internal non-public npm feed.

The used technology (npm) requires me to adhere to semantic versioning by format, though I would also like to adhere to it by idea, which is what this question aims at.

The content of the software package is not our own - it is a proprietary 3rd party software that we are licensing. We receive the JavaScript files by the producing entity for use in our products, and no npm package is provided. The necessity for packaging these files into an npm package is based on the desire to keep our internal development and build process streamlined.

How is the software versioned? The 3rd party software already adheres to semantic versioning; it is versioned with a 3-component scheme that matches the major.minor.patch idea.

How do we translate this into our package versions? This is where I see the problem: Intuitively, it would be best to just take over the 3rd party software's version and be happy. This has the advantage of making it obvious to consumers of the 3rd party package which version of the 3rd party software they are using, which release notes apply, and so on.

However, in a few situations, it is necessary to do "technical changes" to the package, in particular by reconfiguring things in the package.json file. These would cause an increase in the package version, as well, but there appears to be no component left for it.

Example Timeline:

  • 3rd party software version 2.4.1
  • 3rd party software version 2.4.2
  • 3rd party software version 2.4.2 with change 1 to package.json
  • 3rd party software version 2.4.3
  • 3rd party software version 2.4.3 with change 1 to package.json
  • 3rd party software version 2.4.3 with change 2 to package.json

Considered workarounds:

  • I have considered using the prerelease part of the semantic versioning number for this, but then each and every version of the package would technically have to be using a prerelease version, as at the time of upgrading to the new version of the 3rd party software, we do not yet know whether we will have to fix something about package.json.
  • The build number does not help, as by the SemVer definition, it is to be ignored in determining version precedence and serves only for information.
  • I could rely on the 3rd party software never releasing more than x patches and us never needing more than y package.json updates for any version. Then, I could encode two numbers in the patch number based on digits. E.g. 2.4.2003 to denote 3rd party software release 2.4.2, package.json update 3. This would result in slightly awkward looking version numbers, most of which end in 000.

Is there a common way to resolve this within the definitions of semantic versioning, or is this a situation where semantic versioning just does not fit with our needs and we must define on our own what the version numbers for our package mean?

F-H
  • 131
  • Why have you rejected using the build component of SemVer? – Philip Kendall Jul 29 '23 at 09:53
  • @PhilipKendall: You mean the one mentioned in item 10 in the SemVer spec, the one appended after a plus sign? Because the spec says: "Build metadata MUST be ignored when determining version precedence. Thus two versions that differ only in the build metadata, have the same precedence." But in our case, the package with the more recent package.json settings should supersede the older ones. – F-H Jul 29 '23 at 09:58
  • What change are you making to package.json? – Thomas Owens Jul 29 '23 at 11:44
  • @ThomasOwens: For example, changes to keywords, or description, changes to the publishConfig, etc. Granted, the mere metadata arguably isn't a change to the software and could count as build changes, but there appear to be options that influence how the package is assembled/included/whatever, so I cannot rule out that some of these might be relevant for the resulting application. – F-H Jul 29 '23 at 11:57
  • I don't understand why you would need to make these changes outside of releasing a new version. Especially since it's a third-party package that is being licensed. Why would you need to change these values without any other changes? This feels like it could be an XY problem, but there's not enough information about why you are doing what you are doing. – Thomas Owens Jul 29 '23 at 12:03
  • @ThomasOwens: "I don't understand why you would need to make these changes outside of releasing a new version." - because these changes may either have been forgotten at the time a new version of the 3rd party software got integrated, or they are unrelated to that new version and rather triggered by ourselves learning something new about npm, package.json, or the host for our internal npm feed (Azure DevOps). At that point, I see no reason for waiting until the supplier happens to release the next version of their 3rd party software, just so we can sneak in our other changes. – F-H Jul 29 '23 at 12:07
  • Why are you making any changes to the 3rd party software? Assuming this is allowed by the license, you're forking the 3rd party software. That means you have new software with a unique version and you should be choosing if and when to pull in upstream changes into your unique version and then incorporating your new version into the system you are making. Since your fork has its own version, it's likely that some of these package.json changes may cause you to update the patch version. Pulling in changes from upstream could be patch, minor, or major. – Thomas Owens Jul 29 '23 at 12:16
  • @ThomasOwens: "Why are you making any changes to the 3rd party software?" - where exactly do I claim I am doing this? "Since your fork has its own version, it's likely that some of these package.json changes may cause you to update the patch version" - how is that supposed to work? We receive version 2.4.1, so we publish this on our npm feed as package version 2.4.1. Then we update the package, e.g. because we realize the TypeScript .d.ts files were not properly included, and push this as what to our npm feed? 2.4.2? Then what happens when the supplier provides the actual 2.4.2 later on? – F-H Jul 29 '23 at 12:21
  • 1
    I have enough information to provide an answer. – Thomas Owens Jul 29 '23 at 12:24
  • 1
    @F-H "because these changes may either have been forgotten at the time a new version of the 3rd party software got integrated, or they are unrelated to that new version and rather triggered by ourselves learning something new about npm, package.json, or the host for our internal npm feed" This sounds to me like a claim that your company is introducing changes to a package which from its outset tries to only scope itself to the third party. This is the root cause of the conflict, you're trying to reuse a version number from a source with a different lifecycle than your package. – Flater Jul 29 '23 at 13:29
  • @Flater: I see where you're coming from, but what's a practical solution? Decouple the package version from the 3rd party software version? IMHO that would be a very poor decision from virtually all other angles (e.g. keeping track of which package version contains which version of the 3rd party software, which version of the 3rd party software is referenced in our own software, etc.). – F-H Jul 29 '23 at 13:56
  • 1
    Yeah. The "trigged by ourselves" is where I got the impression that you were making changes outside of just the package file, in which case I would decouple the versions. But that's not the case. The biggest problem is how NPM enforces semantic versioning. I'm not sure if you can disable that in some way, and if you could, what the broader ramifications of that when dealing with public package repos would be, so I wrote my answer avoiding that scenario entirely. – Thomas Owens Jul 29 '23 at 14:04
  • The third option looks good, the meaning of versioning is preserved and hopefully you won't make too many changes. – greenoldman Jul 30 '23 at 04:37

2 Answers2

4

From the standpoint of SemVer, I would recommend using the build metadata component of the semantic version to capture these changes. Beyond that, I would recommend using a timestamp of the format YYYYMMDD or YYYYMMDDHHMM or even YYYYMMDDHHMMSS to capture the time of creating the package. This would give you clear and unambiguous sorting for the package. If you receive 2.4.1 from your supplier and have two builds with differing metadata, you would be able to tell that 2.4.1-20230629 is an earlier build than 2.4.1-20230720.

However, the contents of package.json are supposed to be parseable by node-semver and node-semver doesn't (as far as I'm able to tell) handle build metadata. To work around this, you can use the prerelease tags instead of build metadata. That is, separate the date from the version using - instead of +. However, your dependent applications would need to explicitly allow for prerelease versions.

Other options are more specific to the Node ecosystem and may be violating some rules of Semantic Versioning.

Another option would be to use prerelease versions to test your build process against the latest version from your supplier. When your supplier provides you with 2.4.1, you would start to produce versions of the format 2.4.1-YYYYMMDDHHMM. Once you are satisfied that your build process is correct, you can build 2.4.1 and publish that to your repository. This would handle ensuring that errors in your build process, such as ensuring that your TypeScript definition files are properly included, are accounted for. You could also ensure that your other metadata is correct, but once you publish 2.4.1, you would be locked in until your supplier published 2.4.2 or later.

Depending on your tooling, though, you may have another option. You could unpublish and republish your package. Some tooling prevents publishing the same version twice, even if it has been unpublished, however. Depending on how often you wanted to unpublish and republish, you could avoid publishing versions with prerelease versions, or you could use prerelease versions, publish a release version when you are satisfied, and opt to unpublish and republish if the change was important enough.

My recommended approach would be the prerelease version approach. Use prerelease versions to test your build and packaging. Once you publish the build, lock down your changes until your supplier releases a new version. Errors in the build metadata won't stop anyone from using the dependency and you can fix them for future releases, using the prerelease versions there to test those metadata changes.

Thomas Owens
  • 82,739
  • 1
    The recommendation to use pre-release versions until you can verify that version is stable and functional is the right way to go. Every other solution is a kludge around a broken process, even if you aren't the one breaking it. You need a layer of insulation between you and the third party. – Greg Burghardt Jul 30 '23 at 20:33
  • "lock down your changes until your supplier releases a new version. Errors in the build metadata won't stop anyone from using the dependency and you can fix them for future releases" - this sounds like a solid policy; unfortunately, it seems impractical in my case. When a PR to the master branch is accepted, a pipeline will automatically push a new version of the npm package to the internal feed. Anything that requires exceptions in that, or additional steps like an intermediate space where changes amass until the next release from the supplier arrives, would just render the ... – F-H Jul 31 '23 at 13:21
  • ... process, and the pipelines we have to maintain, overly complex, given that it is merely meant to be a slightly structured replacement for just taking the 3rd party file and copying it into our codebase. Therefore, I have decided to mark the other answer as accepted for now and upvote this one, but in more "large-scale" cases, I will revisit this answer and try to apply it. – F-H Jul 31 '23 at 13:21
1

The linux world solves this same issue by not strictly following SemVer, having a pkgver and a pkgrel; eg 1.2.3-1 and 1.2.3-2 have the same content but have different packaging informations, or package release. Those are not prereleases, just different repackaging.

Technically, it is possible to have four-segment non-semver package versions that ignore non-compliant parts in semver (M.m.p.r)

Personally, If everything else fails, I would just add the re-packaging appending the release to end of the patch (M.m.pr) 1.2.30, 1.2.31 etc.

Jiulia
  • 207
  • 1
    Your first example doesn’t work, because when 1.2.4 is released it would be translated to 1.2.104, which is lower than 1.2.203. The latter example works though. – DisgruntledGoat Jul 30 '23 at 10:12
  • @DisgruntledGoat you're right, my bad. thanks for the correction – Jiulia Jul 30 '23 at 11:12
  • I think for the simple use case my question centers around, this is the most practical solution. – F-H Jul 31 '23 at 13:22