What do you think of lockfiles?

[luftywiranda13]

Hello Sindre!

In the latest npm (version 5), it creates package-lock.json by default. Yes same like yarn for now.
What do you think of it? Should we commit the lockfile to git or is it better to get it ignored?

And also what do you think about the latest npm itself? Does it miss something?

Thanks, great day!

[dangh]

The purpose of lockfile is to lock the explicit package version and checksum, since package.json doesn't do that (like lodash: ^3.0), so when you run npm install on the production server or on other dev machine you get the identical packages. So no "It work on my side" execuse. Also helped to prevent infected packages, like when your internet connection has been interfere and redirect to a hacked server, etc.
Definitely should commit it.

[sholladay]

The lockfile defeats the whole purpose of the caret ^ that is the default save behavior. And it prevents us from getting security patches immediately, which is insane. There are more good updates than bad updates, so it does more harm than good. The idea that it protects us from malicious code is silly because there's no way in hell that people are actually auditing the entire dependency graph when they do finally get around to updating the lockfile. It's a fallacy that leads to a false sense of security.

[sonicdoe]

Just FYI, Sindre let me know on Twitter that he doesn’t intend to commit the package-lock.json:

Only if you commit package-lock.json, which I don't intend to do for packages.

[luftywiranda13]

@sonicdoe yeah that's the reason i ask him here. maybe he can explain a bit more here

[dangh]

@luftywiranda13 I think he don't intent to commit it for the packages, and I think it shouldn't be also. The package should be flexible and shouldn't lock the user from using newer packages it depends on.
That's the application's responsibility, not the lib's.

[luftywiranda13]

@dangh but i have a specific concern behind this as the reason why post him a question here, but let's just wait for Sindre to answer first then maybe we can discuss it 😄

[sindresorhus]

Lockfiles for apps, but not for packages. Lockfiles are great for apps where you want a controlled reproducible environment, but for packages this doesn't make much sense. The package-lock.json files in dependencies are ignored, so the lockfile only applies when users run npm install in the package repo. If you use a lockfile for packages, your local dependency tree will not match the dependency tree of users having your package as a dependency. This can potentially cause problems if one of your package dependencies breaks your package in a patch release. The lockfile will prevent you from seeing the problem locally, but it will affect your users. I also don't like the diff churn lockfiles create, so I only use them when they're actually beneficial.

[sindresorhus]

And also what do you think about the latest npm itself? Does it miss something?

It's a much needed rewrite with many nice features, but it's currently way too buggy for actual usage: npm/npm#16991 I'm staying on npm v4 for now.

[sindresorhus]

If you 👎 my comment, please tell me how I'm wrong (with technical arguments).

[oligot]

I just found this article on the yarn blog regarding lockfiles : they explain why (in their opinion) lockfiles should also be committed for libraries.

[RayBenefield]

I think adding a lockfile for any project is extremely valuable as it reduces the amount of unknowns when debugging issues. More knowledge is more valuable than less in my opinion. Theoretically if someone comes in with a breaking change after installing a package as a dependency you can have them locally install the package (having the lockfile manage the dependencies), symlink to the local with the expected dependencies, and check if the problem still exists. If so, then you've easily narrowed down what the problem could be.

Granted there may be a better way to detect if versions being out of sync are the problem, but I still contest that there is value to knowing what depedencies are expected than not. Especially since when a bug is found and debugging needs to happen, there is a number of reasons the package collaborators won't have the same versions of dependencies at the time the bug was caught compared to the time when they released the last version of the package that is potentially introducing the bug. With a lockfile you can at least look at the dependencies at the time that version was released (as opposed to whatever it is you have installed locally). The amount of unknowns grows rapidly with the amount of unique collaborators and their environments. Plus if you also switched development environments regularly (work vs home computer), it is possible (though probably rare) that one environment falls out of sync with another environment when you are not using lockfiles which could potentially introduce a hard to track down bug during package development.

More known data makes inevitable failure easier to track down and squash. That's my philosophy.

[benwiley4000]

@sindresorhus I found your argument very compelling, and was opening a pull request to one of the libraries I maintain to .gitignore our package-lock.json (as it seems others have done), and then I read the yarn blog post @oligot shared, which was also compelling, and I cancelled my pull request. I'd be curious what you think after giving it a read.

[airtonix]

@mifi I don't think they're exactly like apps. You don't install apps with npm i -g. Also, npm ignores any package-lock.json in the package when installing it.

@sindresorhus Is this still your opinion in 2024?

Because there's two stories here (for one of them your point of view makes sense):

1. people that consume the library.
2. people that contribute to the library.

Consumers

Now lets first acknowledge that in story 1, the lockfile gets ignored by the consuming tools anyway.

How the consumers package manager behaves is driven by dependencies, peerDependencies, optionalDependencies`.

Contributers

This is you, your team mates and anyone that stops by to help out.

How will you ensure that time is not wasted on problems easily circumvented by simply pinning the hash of all your packages.

[sindresorhus]

How will you ensure that time is not wasted on problems easily circumvented by simply pinning the hash of all your packages.

I would rather know about issues with dependencies that users are having, even if it adds a slight burden on a contributors. As always, it's about trade-offs. I have gone without lockfiles for years, and contributors to my projects rarely have any issues. There have been a few cases of a lining rule changing behavior, but that was an easy fix. What has broken my project many times was changes in Node.js versions and the OS, or differences in the OS setup, which lockfiles would not help with. Lockfiles could actually cause the opposite problem here. A new Node.js version breaks something, a dependency gets fixed in a patch version, and contributors still experience the issue because the lockfile prevents using the new patch version of the dependency. It may be possible to pin Node.js version used in a project too, but you still cannot pin the operating system.

[ExplodingCabbage]

If you 👎 my comment, please tell me how I'm wrong (with technical arguments).

Sure.

Your argument is that using/committing lockfiles as a library developer is bad because you lose a chance to serendipitously catch issues caused by a new version of a dependency introducing a regression that makes it incompatible with your code. And yes, you do lose that.

But the flipside is that if you aren't using a lockfile (and are thus using the newest version of your dependencies that is compatible with your package.json's version ranges), then you lose the chance to serendipitously notice if you write some new code that relies on bugfixes (or even entire new features) from the latest versions of your dependencies, and hence is incompatible with older versions of those dependencies that you ostensibly (based on your package.json) are meant to support.

And it seems quite plausible that the latter is the more likely scenario. After all, newer versions are meant to fix bugs in older versions, and not meant to introduce new ones. If you are depending on "somelib": "^1.2.3", and you write some code that doesn't work in version 1.2.3 but does work in version 1.4.8 because the somelib devs fixed a bug, then that's... kind of what's supposed to happen? But if you're working without a lockfile, your ^1.2.3 version range will mean you install 1.4.8 during development and never discover that your library will be broken for the subset of your users who have old somelib versions that you are now incorrectly claiming to support.

Obviously the relative size of these two risks depends upon several variables, including the ratio of fixes vs regressions in somelib and the ratio of new vs incumbent code in your library.

Other arguments in favour of using lockfiles exist, of course, and have been raised here already. There's value in having a known-working set of dependency versions to start fixing things from if you come back to an abandoned project after years and find it no longer works with up-to-date dependencies. I also note the point by the Yarn devs that the vast majority of dependencies are dev dependencies, not runtime dependencies, and that the arguments you're raising simply don't apply to dev dependencies. I think these points both have weight. But the point I want to add to that here is that even if we focus solely on the chance of serendipitously discovering incompatibilities between your library's code and specific dependency versions (since this is what your argument is about), committing a lockfile will often be better (since developing using the oldest dependency versions your package.json says you are compatible with is ideal, and a lockfile gets you closer to that) and, given that the overall balance will vary case by case, it's not at all obvious that eschewing lockfiles is a better rule of thumb than always using them.

(Of course, ideally we'd all run tests with both the absolute oldest and absolute newest versions of our dependencies that our package.json version ranges permit, and make these arguments moot; it would be nice if this stuff didn't depend on serendipity! Does anyone do that? Do they have good stories of it catching bugs?)