Stowage/zimit
2
0
Fork 0
mirror of https://github.com/openzim/zimit.git synced 2025-12-31 04:23:15 +00:00
Code Issues Projects Releases Packages Wiki Activity
474 commits 8 branches 44 tags 1.3 MiB
Commit graph

42 commits

Author SHA1 Message Date
Uchechukwu Orji
a9805c84c2 set proper outputs name 2025-10-10 10:38:38 +01:00
Uchechukwu Orji
ad09665c4a add workflow dispatch to update-offliner ci 2025-10-10 10:22:29 +01:00
Uchechukwu Orji
4ec47cd6dd use base64 string as argument to workflow call 2025-10-08 04:25:12 +01:00
Uchechukwu Orji
5624cbf081 set up offliner definitions 2025-10-07 04:08:14 +01:00
benoit74
7bfb4b25f0
Remove confusion between zimit, warc2zim and crawler stats filenames 2025-02-14 14:27:28 +00:00
benoit74
ed1a8a0aa9
Use preferred Browsertrix Crawler arguments and fix multiple/file seeds support 2025-02-14 14:27:26 +00:00
benoit74
101fb71a0b
Better processing of crawler exit codes with soft/hard limits 2025-02-13 10:51:14 +00:00
benoit74
5af981c01c
Remove ARM64 job temporarily, still not working 2025-02-07 08:07:23 +00:00
benoit74
b4c0495f48
Fix arm runner selector 2025-02-06 21:19:08 +00:00
benoit74
cea10bd3b5
Add second build job on native arch for ARM64 2025-02-06 21:17:46 +00:00
benoit74
4ef9a0d380
Remove support for ARM64, this is not working anymore and was painfully slow 2025-02-06 21:11:40 +00:00
benoit74
8d42a8dd93
Move integration tests to test website 2025-01-09 10:41:05 +00:00
benoit74
0e6919300b
Fix help and add test case to check this important feature 2024-10-08 11:57:31 +00:00
benoit74
b056c6dc4f
Add logic to skip Youtube test on-demand 2024-09-27 09:09:01 +00:00
benoit74
5fe6539201
Archive ZIM built as artifact 2024-09-23 06:59:46 +00:00
benoit74
6d078c4dcf
Automate daily tests of ZIM behavior - Youtube only for now 2024-08-07 10:34:19 +00:00
benoit74
c67ccb9528
Allow to run dev image update manually + use main warc2zim branch for zimit dev versions 2024-06-04 15:17:33 +00:00
benoit74
8fdad5954e
Bump Github CI Actions versions 2024-05-24 14:16:53 +00:00
benoit74
9e6c998816
Bump zimit to 2.0.0-dev5 + use warc2zim2 branch + remove zimit2 image workflow 2024-05-24 14:10:19 +00:00
benoit74
c0ffb74d8c
Adopt Python bootstrap conventions 2024-01-18 13:31:00 +01:00
benoit74
a352c0c402
Add temporary Github Actions workflow to build zimit2 image 2024-01-15 08:06:50 +01:00
renaud gaudin
c9c7e7a26f
Fixed #178: publish images for arm64 2023-08-23 12:14:12 +00:00
renaud gaudin
722306d3bf
Using a dedicated venv for zimit in image 
zimit dependencies conflicts with crawler's python ones
 2023-08-02 14:31:42 +00:00
renaud gaudin
b7265b49b6 updated to crawler 0.9 (b1) 2023-03-24 07:26:33 +00:00
renaud gaudin
b8714d1260 removed references to docker.io 2023-03-22 13:55:07 +00:00
Emmanuel Engelhart
79d444e7ea
Update GitHub workflow actions 2023-02-07 14:24:29 +01:00
Kelson
511fccdc56
"main" is the new default branch 2022-12-21 11:07:37 +01:00
renaud gaudin
dffc81860e updated docker publish action 2022-06-21 17:06:40 +00:00
renaud gaudin
932f97c999 updated tests for crawler and warc2zim 2022-06-21 16:55:32 +00:00
renaud gaudin
e29b6f3ad6 CI on push is suffiscient 2022-06-13 10:02:35 +00:00
renaud gaudin
885e1763a1 updated CI test website URL 2022-06-13 09:57:37 +00:00
renaud gaudin
610ecc7e5c using docker publish v5 2021-01-14 18:27:07 +00:00
renaud gaudin
f4c11dc948 using published version of action 2020-12-22 15:48:12 +00:00
renaud gaudin
f72caad35c added Docker publish GA 2020-12-22 11:10:53 +00:00
renaud gaudin
85fad62b61 Updated test to new stats files 
- verify output of crawl, warc2zim and zimit file
- using a simpler tag for CI test image as to not confuse it with public image
 2020-12-10 10:44:49 +00:00
Ilya Kreymer
5084c54af6 stats: add support for stats output after every page crawled, fixes #39 
tests: integration test checks for stats.json
 2020-12-02 16:28:25 +00:00
Ilya Kreymer
4723376ebc ci: add --keep to keep warc files 2020-11-14 20:33:36 +00:00
Ilya Kreymer
5e4b3d80b3 ci: path fix 2020-11-14 20:30:15 +00:00
Ilya Kreymer
82f0fae959 update to warc2zim 1.3.2 
fix ci test command
 2020-11-14 20:27:43 +00:00
Ilya Kreymer
a930542af8 mobile + user agent support: 
- add support for custom user agent suffix +Zimit with email address specifyable via --adminEmail cmd arg #38
- add ability to crawl as mobile device with --mobileDevice flag (default to iPhone X)
add integration tests runnable in docker via github actions
logging: print temp dir, flush print statements for immediate logging
 2020-11-14 20:10:16 +00:00
Ilya Kreymer
5b640f2f8b main page redirect check: check if specified URL is a redirect, and use final URL if it is. Reject if redirect goes to a different domain, as suggested in #42 2020-11-10 06:07:27 +00:00
Ilya Kreymer
88a280bc58 ci: add simple github action for building image, running crawl, verifying zim exists 2020-11-10 03:55:33 +00:00