Skip to content

ci: verify npm registry signatures for all dependencies - #12

Open
jakexcosme wants to merge 5 commits into
masterfrom
devin/1785268777-npm-audit-signatures
Open

ci: verify npm registry signatures for all dependencies#12
jakexcosme wants to merge 5 commits into
masterfrom
devin/1785268777-npm-audit-signatures

Conversation

@jakexcosme

@jakexcosme jakexcosme commented Jul 28, 2026

Copy link
Copy Markdown

Closes N/A — remediates code-scan finding sfind-55ecffda233344f5a2c2f8197a5d6799 (supply-chain hardening).

Describe your changes:

A security scan flagged the lodash@4.18.1 pin in package.json as a fabricated/nonexistent version (supply-chain tampering). Investigation showed this is a false positive: lodash resumed publishing in 2026 (4.17.23, 4.18.0, 4.18.1). Verified against registry.npmjs.org:

  • lodash@4.18.1 published 2026-04-01 by jdalton (lodash's author); tarball sha512 matches the lockfile integrity exactly.
  • npm registry ECDSA signature for lodash@4.18.1:<integrity> cryptographically verified against npm's public key (Verified OK).
  • Content diff vs 4.17.21 shows only upstream security hardening (prototype-pollution fixes GHSA-xxjr-mmjv-4gpg / GHSA-f23m-r3pf-42rh).

To address the scanner's underlying concern going forward, this PR adds a verify-dependency-signatures job to the PR workflow:

verify-dependency-signatures:
  - npm ci --no-audit --progress=false
  - npm audit signatures

npm audit signatures verifies every installed dependency's registry signature and provenance attestation against npm's public keys, so a tampered manifest/lockfile entry (a version or integrity hash not matching an authentically signed artifact) fails CI. Verified locally on Node 24.14.1: all 1045 packages have verified registry signatures; 103 have verified attestations.

All Submissions:

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Is this a notable change that will require a special callout in the release notes? For example, will this break compatibility with existing APIs or projects that consume these plugins?

Author Checklist

  • Changes address original issue?
  • Tests included and/or updated with changes?
  • Has this been smoke tested?
  • Have you associated this PR with a type: label? Note: this is not necessarily the same as the original issue.
  • Have you associated a milestone with this PR? Note: leave blank if unsure.
  • Testing instructions included in associated issue OR is this a dependency/testcase change?

Link to Devin session: https://app.devin.ai/sessions/bcb81305e16144a8bd23b950f9652d61
Requested by: @jakexcosme


Devin Review

Status Commit
⚪ Not started

Run Devin Review

💡 Connect your GitHub account to enable automatic code reviews.

Open in Devin Review (Staging)
Open in Devin Review

Co-Authored-By: Jake Cosme <jake@cognition.ai>
@jakexcosme jakexcosme self-assigned this Jul 28, 2026
@devin-ai-integration

Copy link
Copy Markdown

Prompt hidden (unlisted session)

@devin-ai-integration

Copy link
Copy Markdown

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

devin-ai-integration[bot]

This comment was marked as resolved.

@devin-ai-integration devin-ai-integration Bot added type:maintenance Maintenance/CI change no milestone PR intentionally has no milestone labels Jul 28, 2026
Co-Authored-By: Jake Cosme <jake@cognition.ai>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

Open in Devin Review

Comment thread .github/workflows/pr.yml
Comment on lines +90 to +95
- run: npm ci --no-audit --progress=false
- run: npm audit signatures
- run: npm ci --no-audit --progress=false
working-directory: e2e
- run: npm audit signatures
working-directory: e2e

@devin-ai-integration devin-ai-integration Bot Jul 28, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Committed lockfiles are stripped of most integrity/resolved fields, which weakens signature verification

The new job's stated goal is to catch a tampered manifest/lockfile entry via npm audit signatures. However, the committed lockfiles are largely stripped of resolved/integrity metadata: the root package-lock.json has 947 package entries but only ~45 resolved and ~44 integrity fields, and e2e/package-lock.json has 104 entries with only 4 each. npm audit signatures verifies signatures against the installed tree (which npm ci re-fetches and re-computes), not against the committed lockfile's integrity hashes. Because most integrity hashes are absent from the committed lockfile, a tampered integrity value in the committed lockfile would generally be re-resolved rather than compared, so the effectiveness of the intended supply-chain check is narrower than the PR description implies. It also implies npm ci is re-resolving many packages from the registry rather than pinning them, which is a reproducibility concern (pre-existing, not introduced here). Worth confirming the check actually fails on a tampered entry as claimed.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified empirically: with a deliberately tampered integrity for node_modules/lodash in the committed lockfile, npm ci fails with EINTEGRITY ("integrity checksum failed ... wanted sha512-AAAA... but got sha512-dMIn..."), so this job fails on tampered entries that carry integrity fields. You're right that ~903/947 entries in the committed lockfile lack resolved/integrity (pre-existing condition, not introduced here) — for those, npm ci re-resolves from the registry and npm audit signatures then verifies the fetched artifacts against npm's registry signing keys, which is exactly the layer this job adds: a non-authentic artifact fails signature verification even without a committed hash. Regenerating the lockfile to restore full resolved/integrity pinning is a worthwhile follow-up but out of scope for this CI-hardening PR; flagging it to the maintainers.

Comment thread .github/workflows/pr.yml
Co-Authored-By: Jake Cosme <jake@cognition.ai>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

Open in Devin Review

Comment thread .github/workflows/pr.yml
Comment on lines +95 to +98
- run: npm ci --no-audit --progress=false
working-directory: e2e
- run: npm audit signatures
working-directory: e2e

@devin-ai-integration devin-ai-integration Bot Jul 28, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 e2e signature verification steps are redundant (e2e is a workspace)

e2e is declared as an npm workspace of the root package (package.json workspaces: ["e2e"]). Because of this, the root npm ci at .github/workflows/pr.yml:93 already installs the e2e workspace's dependencies, and the root npm audit signatures at .github/workflows/pr.yml:94 already verifies signatures for those packages. The additional working-directory: e2e steps at lines 95-98 therefore re-install and re-audit largely the same dependency set. This is not incorrect, but it roughly doubles install time for little additional coverage. If the intent is specifically to audit the e2e sub-package in isolation, note that the standalone npm ci in e2e/ will not read the repository root .npmrc (ignore-scripts=true, min-release-age=7), so the isolated install runs under different constraints than the workspace install.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right — e2e is an npm workspace, so the root npm ci + npm audit signatures already covers the e2e dependency tree (the 1045-package root audit includes the e2e/node_modules/* entries). Removed the redundant standalone e2e steps (and the now-unneeded cache-dependency-path override) in 9174c31.

Comment thread .github/workflows/pr.yml
Comment on lines +82 to +98
verify-dependency-signatures:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
cache: npm
cache-dependency-path: |
package-lock.json
e2e/package-lock.json
- run: npm ci --no-audit --progress=false
- run: npm audit signatures
- run: npm ci --no-audit --progress=false
working-directory: e2e
- run: npm audit signatures
working-directory: e2e

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 New job is not wired into any needs/branch-protection chain

The verify-dependency-signatures job (.github/workflows/pr.yml:82-98) is standalone and is not listed in the needs: of any downstream job (lint, unit-test, e2e-test). It will still run and report pass/fail on the commit, but unless branch protection explicitly requires this check, a signature-verification failure would not by itself block a merge. Worth confirming the check is added to required status checks so the supply-chain gate is actually enforced.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct — the job is intentionally standalone (it needs a clean, uncached npm ci so signature verification isn't satisfied from a warm cache) and, like the repo's other gate jobs (lint, unit-test), enforcement relies on branch protection required status checks. I don't have permission to read/modify branch protection here; a repo admin should add verify-dependency-signatures to the required checks for master to make the gate blocking.

Co-Authored-By: Jake Cosme <jake@cognition.ai>
devin-ai-integration[bot]

This comment was marked as resolved.

Co-Authored-By: Jake Cosme <jake@cognition.ai>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Open in Devin Review

Comment thread .github/workflows/pr.yml
Comment on lines +95 to +101
# e2e is an npm workspace covered by the root audit above, but its
# standalone lockfile (used for standalone e2e installs) is verified
# separately so tampering there is also caught.
- run: npm ci --no-audit --progress=false
working-directory: e2e
- run: npm audit signatures
working-directory: e2e

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Standalone npm ci inside e2e workspace member relies on npm using the local lockfile

The new job runs npm ci at the root (which installs the e2e workspace) and then again inside working-directory: e2e (.github/workflows/pr.yml:98-99). Because e2e/package.json exists, npm's prefix resolution treats e2e as its own project root and uses e2e/package-lock.json (a self-contained lockfileVersion 3), so the standalone install and subsequent npm audit signatures do cover the standalone lockfile as the comment claims. This is correct today, but it depends on npm's prefix/workspace-detection behavior; if a future npm version walks up to the parent workspace root, the second install/audit would silently re-verify the root lockfile instead of the standalone one. Worth a note but not a current bug.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acknowledged — confirmed locally that the standalone npm ci in e2e/ installs from e2e/package-lock.json (104 packages audited vs 1045 at root), so the standalone lockfile is covered today. No change needed; the inline comment documents the intent if npm's prefix behavior ever changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no milestone PR intentionally has no milestone type:maintenance Maintenance/CI change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant