Stowage/browsertrix-crawler
2
0
Fork 0
mirror of https://github.com/webrecorder/browsertrix-crawler.git synced 2026-04-18 07:00:22 +00:00
Code Issues Projects Releases Packages Wiki Activity
browsertrix-crawler/src
History
Exact
Ilya Kreymer cee501a20a
add reference to external WACZ per revisit record (#1009) 
- store in `WARC-Refers-To-Container` with file://<WACZ filename> as per
discussions in iipc/warc-specifications#111
- wabac.js 2.26.0 will use this header for prioritizing the specified
WACZ for looking up the original.
- also clears the per-WACZ dependency key `...:duperef` after current
WACZ is finished, so future WACZ files don't use stale dependencies
- fixes #1008
- version: bump to 1.12.4
2026-03-31 17:39:06 -07:00
..
util add reference to external WACZ per revisit record (#1009) 2026-03-31 17:39:06 -07:00
crawler.ts track crawlIds included in each --collection directory (#1005) 2026-03-30 10:21:15 -07:00
create-login-profile.ts Fix default user-agent to not include minor version + set sec-ua-ch-* headers (#962) 2026-02-04 16:06:28 -08:00
indexer.ts Make fatal() and interrupt() async (#994) 2026-03-11 14:53:01 -07:00
main.ts fix signal handling edge-cases: (#951) 2026-01-19 13:31:27 -08:00
replaycrawler.ts qa: don't add pages from WACZ files that have fromDependency set (#1010) 2026-03-31 11:13:41 -07:00