cleanup: remove dead pywb code from argparser and docs (#847)

The value of `--dedupPolicy` was once passed to pywb (see
https://pywb.readthedocs.io/en/latest/manual/configuring.html#dedup-options-for-recording).
Now that pywb has been dropped, there is no need to keep this option
around.

In fact, I know multiple users that have been confused by the mention of
this option in the docs (myself included).

(for historical context, see
https://github.com/webrecorder/browsertrix-crawler/pull/332)
This commit is contained in:
Rijnder Wever 2025-06-16 18:36:32 +02:00 committed by GitHub
parent e09d10c582
commit fa26f05f66
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 0 additions and 9 deletions

View file

@ -161,8 +161,6 @@ Options:
econds) after behaviors before movin
g on to next page
[number] [default: 0]
--dedupPolicy Deduplication policy
[string] [choices: "skip", "revisit", "keep"] [default: "skip"]
--profile Path or HTTP(S) URL to tar.gz file w
hich contains the browser profile di
rectory [string]

View file

@ -392,13 +392,6 @@ class ArgParser {
type: "number",
},
dedupPolicy: {
describe: "Deduplication policy",
default: "skip",
type: "string",
choices: ["skip", "revisit", "keep"],
},
profile: {
describe:
"Path or HTTP(S) URL to tar.gz file which contains the browser profile directory",