[BUG] Overrides prevent `npm update` and `npm audit fix` from replacing eligible dependencies

[hashtagchris]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

Given a dependency tree like:

current package
|- a
|  |- b
|     |-c
|       |-d   <===== our out of date dependency
|-c
  |-d

npm update and npm audit fix won't update dependency d if this is an override directive related to c; Arborist's resolution will be KEEP. This is true even if the update of d would fall within c's semver range for d.

I've found one combination of overrides that exhibits this, but I don't know the exact requirements. I'm not confident that you have to have 4+ levels of dependencies.

Arborist's canReplaceWith returns false due this check:

cli/workspaces/arborist/lib/node.js

Lines 1009 to 1012 in f7da341

	// XXX need to check for two root nodes?
	if (node.overrides !== this.overrides) {
	return false
	}

Expected Behavior

npm update and npm audit fix update dependency d's version if the new version is compatible with c's semver range for d.

Steps To Reproduce

For this repro, we'll attempt to update nanoid to 3.3.8 or higher. nanoid is a dependency of postcss.

Setup

Start from https://github.com/hashtagchris/npm-test-packages/tree/hashtagchris-overrides-breaks-npm-update/workspaces/updateable-dependency, or do the following:

1. Create a private package for testing.
2. Add these dependencies and overrides:

  "dependencies": {
    "css-loader": "2.1.1",
    "postcss": "8.4.39"
  },
  "overrides": {
    "icss-utils": {
      "postcss": "8.4.39"
    }
  }

3. Run npm i to produce a package-lock.json and populate node_modules.
4. Run npm ls nanoid and verify 3.3.8 or higher was chosen for the fresh install.
5. Edit the package-lock file to downgrade to nanoid@3.3.7. Using yq: yq -i '(.packages["node_modules/nanoid"]) += {"version":"3.3.7", "resolved":"https://registry.npmjs.org/nanoid/-/nanoid-3.3.7.tgz", "integrity": "sha512-eSRppjcPIatRIMC1U6UngP8XFcz8MQWGQdt1MTBQ7NaAmvXDfvNxbvWV3x2y6CdEUciCSsDHDQZbhYaB8QEo2g=="}' package-lock.json
6. Run npm i to update node_modules
7. Run npm ls nanoid to verify 3.3.7 is now in use

Repro

1. Run npm update nanoid or npm update nanoid -ddd 2>&1 | grep 'placeDep ROOT'
2. Run npm ls nanoid to check if the version changed

Expected: nanoid is updated to 3.3.8 (again)
Actual: nanoid isn't updated

Environment

• npm: 11.0.0
• Node.js: v20.18.1
• OS Name: macOS
• System Model Name: M2 MacBook Air
• npm config:

% npm config ls 
; "user" config from /Users/hashtagchris/.npmrc

@github:registry = "https://registry.npmjs.org/"
//registry.npmjs.org/:_authToken = (protected)
logs-max = 1000

; node bin location = /Users/hashtagchris/.nvm/versions/node/v20.18.1/bin/node
; node version = v20.18.1
; npm local prefix = /Users/hashtagchris/r/hashtagchris/npm-test-packages/workspaces/updateable-dependency
; npm version = 11.0.0
; cwd = /Users/hashtagchris/r/hashtagchris/npm-test-packages/workspaces/updateable-dependency
; HOME = /Users/hashtagchris
; Run `npm config ls -l` to show all defaults.```

[dermasmid]

im also seeing some overrides issue, if i have one the npm install will hang with the log in the logfile:

19016 silly placeDep ROOT whatwg-url@14.1.0 REPLACE for: mongodb-connection-string-url@3.0.1 want: 14.1.0
19017 silly placeDep ROOT whatwg-url@14.1.0 REPLACE for: node-fetch@2.7.0 want: 14.1.0

[owlstronaut]

Running through the repro steps, I'm seeing this as fixed with the code from #8089. Specifically, the changes in canReplaceWith and findSpecificNode made it so the override's mere presence wasn't enough to block the update. Will land with 11.2.0 - #8074