Stowage/browsertrix-crawler
2
0
Fork 0
mirror of https://github.com/webrecorder/browsertrix-crawler.git synced 2025-12-08 06:09:48 +00:00
Code Issues Projects Releases Packages Wiki Activity
browsertrix-crawler/docs/docs
History
Exact
Tessa Walsh ff5619e624
Rename robots flag to --useRobots, keep --robots as alias (#932) 
Follow-up to
https://github.com/webrecorder/browsertrix-crawler/issues/631

Based on feedback from
https://github.com/webrecorder/browsertrix/pull/3029

Renaming `--robots` to `--useRobots` will allow us to keep the
Browsertrix backend API more consistent with similar flags like
`--useSitemap`. Keeping `--robots` as it's a nice shorthand alias.
2025-12-02 15:55:25 -08:00
..
assets Add MKDocs documentation site for Browsertrix Crawler 1.0.0 (#494) 2024-03-16 14:59:32 -07:00
develop Add MKDocs documentation site for Browsertrix Crawler 1.0.0 (#494) 2024-03-16 14:59:32 -07:00
overrides Add MKDocs documentation site for Browsertrix Crawler 1.0.0 (#494) 2024-03-16 14:59:32 -07:00
stylesheets docs: Update header font (#785) 2025-03-05 14:21:00 -08:00
user-guide Rename robots flag to --useRobots, keep --robots as alias (#932) 2025-12-02 15:55:25 -08:00
CNAME CNAME: keep CNAME in docs/docs for mkdocs 2024-03-16 15:24:54 -07:00
index.md Add crawler QA docs (#551) 2024-04-18 16:18:22 -04:00