Stowage/browsertrix-crawler
2
0
Fork 0
mirror of https://github.com/webrecorder/browsertrix-crawler.git synced 2025-12-08 06:09:48 +00:00
Code Issues Projects Releases Packages Wiki Activity
566 commits 63 branches 154 tags 87 MiB
Commit graph

3 commits

Author SHA1 Message Date
Ilya Kreymer
eb374fa835
base: bump to brave 1.80.113 (#857) 
version: bump to 1.7.0-beta.0
tests: update deprecated command to work with latest minio
 2025-06-30 19:55:38 -07:00
Ilya Kreymer
fe6199eebd
pages redis: include 'depth', 'seed' and 'favIconUrl' in page data added to redis (#749) 
follow-up to #747
 2025-01-30 11:18:59 -08:00
Ilya Kreymer
457d07aea4
if uploading wacz files, compute waczfile name on load to be able to … (#748) 
…store filename along with page data:

- set filename on crawler load, if not already set, otherwise use
existing
- store filename per crawler instance in <crawlid>:nextWacz
- add 'filename' field to page when writing pages to redis
- clear wacz filename when wacz is uploaded to set a new one
- fixes #747

---------

Co-authored-by: Tessa Walsh <tessa@bitarchivist.net>
 2025-01-29 18:15:28 -08:00