2

Given we have several data transformation applications, which e.g. load an image, do some calculation and save some output. They make use of several internal libraries (internal = developed at our company and not published at pypi).

When running these applications, shall we fix the versions of these internal libraries or rather always use the latest versions?

  • Pro using fixed versions:
    • Users of the applications get a 100% reproducible result
    • Developers of the applications (and users) don't need to worry about any bugs introduced in new versions.
  • Pro using latest versions:
    • Developers of the internal libraries get quick feedback about any unwanted bugs introduced (which will always happen despite being careful in my opinion). In my view, this leads to more robust libraries and therefore eventually also helps the users?!
gebbissimo
  • 241
  • Do you want to have the situation that shortly before an important demo of your software it turns out to be broken because some internal library received an incompatible or buggy update? – Bart van Ingen Schenau Oct 27 '21 at 12:17
  • 1
    By "fix versions" you mean to always use the same version, not "fix bugs,' right? I was a little confused at first. – Greg Burghardt Oct 27 '21 at 13:37

2 Answers2

4

Always pin the version. If you trust the library maintainer, you may be able to pin to a particular major-minor version and receive any patches with minimal friction. Since the library is built internally, you may opt to take this level of trust. If you don't trust the library maintainer or they don't use the commonly accepted definition of the changes that are considered major, minor, and patch, then you should pin to the most specific version possible.

Ensuring the reliability and stability of your product is the most important. However, it should be relatively easy to discover a new version of a library and pull it into a branch. If you have sufficient automated test coverage of your system, updating to the latest version of any library can become trivial, since a library upgrade shouldn't often result in changes to your system's behavior. A failing test would almost always indicate one of two things - your system has a defect or your system was relying on defective behavior in the library.

The biggest concern that I'd have with always using the latest library is that unexpected changes can add overhead. Consider that you are preparing for a release of your system and, in the last days or hours, a change to the library breaks your system. Your team is now scrambling to determine where the problem is and then has to make decisions about rolling back the library, working with the library maintainers to create and test and deploy a fix, or making additional changes to the system. Not the kind of thing I'd want to be forced onto a team.

Thomas Owens
  • 82,739
2

In my experience, always using the latest version of libraries under active development works well - and only works well - when the team or organizational unit who maintains the applications is the same which maintains the libraries.

Otherwise, you will get feedback about any unwanted bugs, or compatibility issues - in situations, where you really don't want those bugs to occur, and noone available for fixing them soon enough. Note that is not a "pro" argument, as you named it, quite the opposite. If the team who introduces changes into the libs, however, has also the goal of keeping all the applications alive which uses the libs, then you can be pretty sure they will be much more aware of which kind of lib changes run smoothly, and the ones which could cause issues in the applications.

Doc Brown
  • 206,877
  • Should have clarified that beforehand, sorry: Developers of the applications also maintain some/most of the internal libraries. And thanks for making the distinction, this makes perfect sense! – gebbissimo Oct 27 '21 at 17:14