mirror of
https://github.com/webrecorder/browsertrix-crawler.git
synced 2026-06-18 03:50:34 +00:00
2e9ad14454
## Summary Dependency security maintenance: `yarn audit` reported 33 unique advisories (2 critical, 19 high); this PR resolves all but one (details below) and removes dev tooling from the published Docker image. ### Direct dependency upgrades - `undici` ^7.16.0 → ^7.18.0 (resolves 7.27.2) — CVE-2026-1525, CVE-2026-22036 - `ws` ^7.4.4 → ^8.20.1 — CVE-2026-45736 - `uuid` 8.3.2 → ^11.1.1 — CVE-2026-41907 (`@types/uuid` dropped; types are bundled since v10) - `@aws-sdk/client-s3` / `lib-storage` refreshed in-range — pulls `fast-xml-parser` 5.7.3 (CVE-2026-25896 critical, + 5 more) ### Transitive re-resolutions (lockfile only, within existing ranges) `fast-xml-parser` 4.5.5+, `tar-fs`, `ip-address` 10.x (via `socks`), `minimatch`, `picomatch`, `ajv`, `js-yaml`, `@babel/helpers`, `basic-ftp`, `flatted`, `lodash`, `qs`, `follow-redirects`, `brace-expansion`. One resolutions entry forces `minimatch` 9.0.7 for `@typescript-eslint/typescript-estree` (pinned exact at 9.0.3). ### ws v8 runtime fix + new test ws v8 no longer exposes `Server` on the default export — the screencast server crashed at startup with `ws.Server is not a constructor` (tsc did not catch it). Fixed by using the `WebSocketServer` named export, and added `tests/screencast.test.ts`, which runs a crawl with `--screencastPort` and asserts a WebSocket client receives the `init` message — this path previously had no test coverage and the new test catches the regression. ### Docker image: prune devDependencies The image ran a plain `yarn install`, shipping jest, eslint, lighthouse, http-server and their transitive trees in production. The Dockerfile now prunes to production dependencies after `tsc` and cleans the yarn cache (runtime code only imports production deps — verified). Smaller image, smaller vulnerability surface. ### Known remaining advisory (accepted) `fast-xml-parser` CVE-2026-41650 (moderate) via `@webrecorder/wabac`: the fix only exists on the 5.x line and wabac latest still pins `^4.5.4`. Will resolve when wabac moves to fxp 5. ## Testing - `yarn audit`: 33 unique advisories → 1 moderate (above) - `yarn tsc`, `eslint`, `prettier` clean - e2e against the rebuilt image: `basic_crawl.test.ts`, new `screencast.test.ts`, plus S3 upload (versitygw), SOCKS5 proxy (3proxy), and yaml config crawls verified |
||
|---|---|---|
| .. | ||
| custom-behaviors | ||
| fixtures | ||
| invalid-behaviors | ||
| adblockrules.test.ts | ||
| add-exclusion.test.ts | ||
| basic_crawl.test.ts | ||
| blockrules.test.ts | ||
| brave-query-redir.test.ts | ||
| collection_name.test.ts | ||
| config_file.test.ts | ||
| config_stdin.test.ts | ||
| crawl_overwrite.test.ts | ||
| custom-behavior-flow.test.ts | ||
| custom-behavior.test.ts | ||
| custom_driver.test.ts | ||
| custom_selector.test.ts | ||
| dedupe-basic.test.ts | ||
| dedupe-page.test.ts | ||
| dryrun.test.ts | ||
| exclude-redirected.test.ts | ||
| extra_hops_depth.test.ts | ||
| file_stats.test.ts | ||
| http-auth.test.ts | ||
| lang-code.test.ts | ||
| limit_reached.test.ts | ||
| log_filtering.test.ts | ||
| mult_url_crawl_with_favicon.test.ts | ||
| multi-instance-crawl.test.ts | ||
| non-html-crawl.test.ts | ||
| norm-test.test.ts | ||
| pageinfo-records.test.ts | ||
| profiles.test.ts | ||
| proxy.test.ts | ||
| qa_compare.test.ts | ||
| retry-failed.test.ts | ||
| robots_txt.test.ts | ||
| rollover-writer.test.ts | ||
| saved-state.test.ts | ||
| scopes.test.ts | ||
| screencast.test.ts | ||
| screenshot.test.ts | ||
| seeds.test.ts | ||
| sitemap-parse.test.ts | ||
| skipped_pages.test.ts | ||
| storage.test.ts | ||
| text-extract.test.ts | ||
| upload-wacz.test.ts | ||
| url-normalize.test.ts | ||
| url_file_list.test.ts | ||
| utils.ts | ||
| vnc-serve.test.ts | ||
| warcinfo.test.ts | ||