mirror of
https://github.com/webrecorder/browsertrix-crawler.git
synced 2025-10-19 06:23:16 +00:00
Logging and Behavior Tweaks (#229)
- Ensure page is included in all logging details - Update logging messages to be a single string, with variables added in the details - Always wait for all pending wait requests to finish (unless counter <0) - Don't set puppeteer-cluster timeout (prep for removing puppeeteer-cluster) - Add behaviorTimeout to running behaviors in crawler, in addition to in behaviors themselves. - Add logging for behavior start, finish and timeout - Move writeStats() logging to beginning of each page as well as at the end, to avoid confusion about pending pages. - For events from frames, use frameUrl along with current page - deps: bump browsertrix-behaviors to 0.4.2 - version: bump to 0.8.1
This commit is contained in:
parent
a4358f4622
commit
5da379cb5f
6 changed files with 116 additions and 82 deletions
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "browsertrix-crawler",
|
||||
"version": "0.8.0-beta.2",
|
||||
"version": "0.8.1",
|
||||
"main": "browsertrix-crawler",
|
||||
"type": "module",
|
||||
"repository": "https://github.com/webrecorder/browsertrix-crawler",
|
||||
|
@ -13,7 +13,7 @@
|
|||
"dependencies": {
|
||||
"@novnc/novnc": "1.4.0-beta",
|
||||
"abort-controller": "^3.0.0",
|
||||
"browsertrix-behaviors": "^0.4.1",
|
||||
"browsertrix-behaviors": "^0.4.2",
|
||||
"get-folder-size": "^4.0.0",
|
||||
"ioredis": "^4.27.1",
|
||||
"js-yaml": "^4.1.0",
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue