Fixes#563
This PR makes a few changes to fix a regression in behavior around
`failOnFailedSeed` for the 1.x releases:
- Fail with exit code 1, not 17, when pages are unreachable due to DNS
not resolving or other network errors if the page is a seed and
`failOnFailedSeed` is set
- Extend tests, add test to ensure crawl succeeds on 404 seed status
code if `failOnINvalidStatus` isn't set
This adds prettier to the repo, and sets up the pre-commit hook to
auto-format as well as lint.
Also updates ignores files to exclude crawls, test-crawls, scratch, dist as needed.
Allow for some seeds to be invalid unless failOnFailedSeed is set
Fail crawl if not valid seeds are provided
Co-authored-by: Ilya Kreymer <ikreymer@gmail.com>
