Stowage/browsertrix-crawler
2
0
Fork 0
mirror of https://github.com/webrecorder/browsertrix-crawler.git synced 2025-10-26 17:54:11 +00:00
Code Issues Projects Releases Packages Wiki Activity
505 commits 53 branches 144 tags 82 MiB
Commit graph

505 commits

This branch
This branch
All branches
Author SHA1 Message Date
Ilya Kreymer
8f740d4e24 support custom crawl directory with --cwd flag, default to /crawls 
update README
 2020-11-02 15:28:19 +00:00
Ilya Kreymer
e2bce2f30d
README tweaks 2020-11-01 21:43:52 -08:00
Ilya Kreymer
a875aa90d3 Dockerfile: switch to cmd 'crawl', instead of entrypoint to support running 'pywb' also 
update README with docker-compose and docker run examples, update commandline example
default output to './crawls' subdirectory
 2020-11-01 21:35:00 -08:00
Ilya Kreymer
91b8994a08 refactor crawler and default driver: 
- add extensible defaultDriver, wrap crawling functionality in Crawler class
- support headless/non-headless, custom driver
- support custom collection name for pywb, generate-cdx option
- autoplay: add slightly delay for splash loading
 2020-11-01 19:53:47 -08:00
Ilya Kreymer
ded83b52b3 initial commit after split from zimit 2020-10-31 13:16:37 -07:00