Lockfiles and why JVM languages don't use them

Authored on 2024-11-12

Recently i got myself baited into a flamewar about lock files. The bait itself was sufficiently simple (which is why it was so effective): Python's build systems (as well as Ruby, Rust, Dart, etc) are all stupid for using lockfiles when Maven solved this issue 20 years ago.

I don't use Java, but I did spend half a decade writing Clojure for money and have opinions that I can now qualify as grounded in reality. And my opinion is that Java build tools (and Clojure's by proxy) are shooting themselves in the foot by not pushing for lockfiles as a first-class citizen.

But first, a question.

Does a package manager need a lockfile?

No.

Ehm..?!

Ok, the answer is "no, unless". More specifically, unless at least one of these is true:

• Package archives can be downloaded from multiple, "untrusted" sources i.e. storages not controlled by you, reader
• Versions of dependencies are mutable i.e. a version x.y.z can at different points in time be represented by different archives
• Version resolution of transitive dependencies is deterministic i.e. is guaranteed to provide same dependency graph even if newer version of [transitive] dependency is released

To absolutely no-one's surprise vast majority of programming language platforms fall into the "unless" category. I can't even name a single language that doesn't. So why lockfiles in the first place?

What are even lockfiles

Lockfiles at its core attempt to solve two fundamental issues with packaging software:

• Dependency integrity -- ensure package archive being downloaded for build N+1 still the same as it was at build N.
• Dependency graph stability -- if in between build N and N+1 a transitive dependency had a new release, ensure new version is not pulled in

Some package managers don't have the need for the latter, due to deterministic dependency version graph resolution with the most prominent example being Golang's mod. For mostly cultural reasons Golang core team prefers to not call their lockfiles "lockfiles" instead using the "go.sum" nomenclature yet I find it hard to argue that it's somehow different from a lockfile.

Some package managers don't offer integrity, with most prominent example being Gemfile.lock (although they're about to change that after 11 years of deliberation). This has the obvious downside from security point of view and as the push for SBOMs becomes more of a need rather than want the urgency to have integrity is increasing.

But one ecosystem in particular frowns upon lockfiles altogether -- JVM. Maven and Ant simply don't support them without plugins and Gradle has optional support for them, but treats integrity as a separate issue altogether, making for an exciting experience of editing XML if the project maintainer does want to have checksum verification.

Real men don't need lockfiles

So I'm going to focus on Maven mostly because it's the largest (in terms of active users) packaging tool without lockfile support in the core system and is often thought as the building tool for Java.

Maven is old, decades old in fact. And the idea of lockfiles is younger than that, although not by much. And Java community is historically resistant to breaking changes. So once POM was standardized circa 2004, the chances it would evolve into something different were essentially nil. Of course, there are attempt to address that (for example) but they barely got any traction in real world projects and as a consequence we see more and more projects switching to Gradle.

But what about integrity, do Java people at large just trust things downloaded from internet? Yes, yes they do! Well, some of them don't and they tried to change things for the better but their efforts failed.

So here we are, with a build system used by tens of millions across the globe built on trust and convention. Which is to say, an optimistic way of viewing the modern world of computing.

In lieu of conclusion

Friends don't let friends use Maven.