Stowage/clamav
2
0
Fork 0
mirror of https://github.com/Cisco-Talos/clamav.git synced 2025-10-19 18:33:16 +00:00
Code Issues Projects Releases Packages Wiki Activity
clamav/unit_tests/freshclam_test.py

895 lines
37 KiB
Python
Raw Normal View History

Bump copyright for 2023
2023-02-07 19:35:18 -08:00
# Copyright (C) 2020-2023 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
CMake: Add CTest support to match Autotools checks An ENABLE_TESTS CMake option is provided so that users can disable testing if they don't want it. Instructions for how to use this included in the INSTALL.cmake.md file. If you run `ctest`, each testcase will write out a log file to the <build>/unit_tests directory. As with Autotools' make check, the test files are from test/.split and unit_tests/.split files, but for CMake these are generated at build time instead of at test time. On Posix systems, sets the LD_LIBRARY_PATH so that ClamAV-compiled libraries can be loaded when running tests. On Windows systems, CTest will identify and collect all library dependencies and assemble a temporarily install under the build/unit_tests directory so that the libraries can be loaded when running tests. The same feature is used on Windows when using CMake to install to collect all DLL dependencies so that users don't have to install them manually afterwards. Each of the CTest tests are run using a custom wrapper around Python's unittest framework, which is also responsible for finding and inserting valgrind into the valgrind tests on Posix systems. Unlike with Autotools, the CMake CTest Valgrind-tests are enabled by default, if Valgrind can be found. There's no need to set VG=1. CTest's memcheck module is NOT supported, because we use Python to orchestrate our tests. Added a bunch of Windows compatibility changes to the unit tests. These were primarily changing / to PATHSEP and making adjustments to use Win32 C headers and ifdef out the POSIX ones which aren't available on Windows. Also disabled a bunch of tests on Win32 that don't work on Windows, notably the mmap ones and FD-passing (i.e. FILEDES) ones. Add JSON_C_HAVE_INTTYPES_H definition to clamav-config.h to eliminate warnings on Windows where json.h is included after inttypes.h because json-c's inttypes replacement relies on it. This is a it of a hack and may be removed if json-c fixes their inttypes header stuff in the future. Add preprocessor definitions on Windows to disable MSVC warnings about CRT secure and nonstandard functions. While there may be a better solution, this is needed to be able to see other more serious warnings. Add missing file comment block and copyright statement for clamsubmit.c. Also change json-c/json.h include filename to json.h in clamsubmit.c. The directory name is not required. Changed the hash table data integer type from long, which is poorly defined, to size_t -- which is capable of storing a pointer. Fixed a bunch of casts regarding this variable to eliminate warnings. Fixed two bugs causing utf8 encoding unit tests to fail on Windows: - The in_size variable should be the number of bytes, not the character count. This was was causing the SHIFT_JIS (japanese codepage) to UTF8 transcoding test to only transcode half the bytes. - It turns out that the MultiByteToWideChar() API can't transcode UTF16-BE to UTF16-LE. The solution is to just iterate over the buffer and flip the bytes on each uint16_t. This but was causing the UTF16-BE to UTF8 tests to fail. I also split up the utf8 transcoding tests into separate tests so I could see all of the failures instead of just the first one. Added a flags parameter to the unit test function to open testfiles because it turns out that on Windows if a file contains the \r\n it will replace it with just \n if you opened the file as a text file instead of as binary. However, if we open the CBC files as binary, then a bunch of bytecode tests fail. So I've changed the tests to open the CBC files in the bytecode tests as text files and open all other files as binary. Ported the feature tests from shell scripts to Python using a modified version of our QA test-framework, which is largely compatible and will allow us to migrate some QA tests into this repo. I'd like to add GitHub Actions pipelines in the future so that all public PR's get some testing before anyone has to manually review them. The clamd --log option was missing from the help string, though it definitely works. I've added it in this commit. It appears that clamd.c was never clang-format'd, so this commit also reformats clamd.c. Some of the check_clamd tests expected the path returned by clamd to match character for character with original path sent to clamd. However, as we now evaluate real paths before a scan, the path returned by clamd isn't going to match the relative (and possibly symlink-ridden) path passed to clamdscan. I fixed this test by changing the test to search for the basename: <signature> FOUND within the response instead of matching the exact path. Autotools: Link check_clamd with libclamav so we can use our utility functions in check_clamd.c.
2020-08-25 23:14:23 -07:00
"""
Run freshclam tests
"""
Tests: Enable freshclam tests to pass as root If you test in a docker container and don't create a user/switch to run as something other than root, than the tests will fail. Default freshclam behavior is to drop to run as the "clamav" user when run as root. This change sets the DatabaseOwner to the current user, so it the tests will pass when run as root.
2021-04-23 05:20:26 -07:00
import getpass
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
from multiprocessing import Process, Pipe
CMake: Add CTest support to match Autotools checks An ENABLE_TESTS CMake option is provided so that users can disable testing if they don't want it. Instructions for how to use this included in the INSTALL.cmake.md file. If you run `ctest`, each testcase will write out a log file to the <build>/unit_tests directory. As with Autotools' make check, the test files are from test/.split and unit_tests/.split files, but for CMake these are generated at build time instead of at test time. On Posix systems, sets the LD_LIBRARY_PATH so that ClamAV-compiled libraries can be loaded when running tests. On Windows systems, CTest will identify and collect all library dependencies and assemble a temporarily install under the build/unit_tests directory so that the libraries can be loaded when running tests. The same feature is used on Windows when using CMake to install to collect all DLL dependencies so that users don't have to install them manually afterwards. Each of the CTest tests are run using a custom wrapper around Python's unittest framework, which is also responsible for finding and inserting valgrind into the valgrind tests on Posix systems. Unlike with Autotools, the CMake CTest Valgrind-tests are enabled by default, if Valgrind can be found. There's no need to set VG=1. CTest's memcheck module is NOT supported, because we use Python to orchestrate our tests. Added a bunch of Windows compatibility changes to the unit tests. These were primarily changing / to PATHSEP and making adjustments to use Win32 C headers and ifdef out the POSIX ones which aren't available on Windows. Also disabled a bunch of tests on Win32 that don't work on Windows, notably the mmap ones and FD-passing (i.e. FILEDES) ones. Add JSON_C_HAVE_INTTYPES_H definition to clamav-config.h to eliminate warnings on Windows where json.h is included after inttypes.h because json-c's inttypes replacement relies on it. This is a it of a hack and may be removed if json-c fixes their inttypes header stuff in the future. Add preprocessor definitions on Windows to disable MSVC warnings about CRT secure and nonstandard functions. While there may be a better solution, this is needed to be able to see other more serious warnings. Add missing file comment block and copyright statement for clamsubmit.c. Also change json-c/json.h include filename to json.h in clamsubmit.c. The directory name is not required. Changed the hash table data integer type from long, which is poorly defined, to size_t -- which is capable of storing a pointer. Fixed a bunch of casts regarding this variable to eliminate warnings. Fixed two bugs causing utf8 encoding unit tests to fail on Windows: - The in_size variable should be the number of bytes, not the character count. This was was causing the SHIFT_JIS (japanese codepage) to UTF8 transcoding test to only transcode half the bytes. - It turns out that the MultiByteToWideChar() API can't transcode UTF16-BE to UTF16-LE. The solution is to just iterate over the buffer and flip the bytes on each uint16_t. This but was causing the UTF16-BE to UTF8 tests to fail. I also split up the utf8 transcoding tests into separate tests so I could see all of the failures instead of just the first one. Added a flags parameter to the unit test function to open testfiles because it turns out that on Windows if a file contains the \r\n it will replace it with just \n if you opened the file as a text file instead of as binary. However, if we open the CBC files as binary, then a bunch of bytecode tests fail. So I've changed the tests to open the CBC files in the bytecode tests as text files and open all other files as binary. Ported the feature tests from shell scripts to Python using a modified version of our QA test-framework, which is largely compatible and will allow us to migrate some QA tests into this repo. I'd like to add GitHub Actions pipelines in the future so that all public PR's get some testing before anyone has to manually review them. The clamd --log option was missing from the help string, though it definitely works. I've added it in this commit. It appears that clamd.c was never clang-format'd, so this commit also reformats clamd.c. Some of the check_clamd tests expected the path returned by clamd to match character for character with original path sent to clamd. However, as we now evaluate real paths before a scan, the path returned by clamd isn't going to match the relative (and possibly symlink-ridden) path passed to clamdscan. I fixed this test by changing the test to search for the basename: <signature> FOUND within the response instead of matching the exact path. Autotools: Link check_clamd with libclamav so we can use our utility functions in check_clamd.c.
2020-08-25 23:14:23 -07:00
import os
from pathlib import Path
import platform
import shutil
import subprocess
import sys
import time
import unittest
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
from functools import partial
CMake: Add CTest support to match Autotools checks An ENABLE_TESTS CMake option is provided so that users can disable testing if they don't want it. Instructions for how to use this included in the INSTALL.cmake.md file. If you run `ctest`, each testcase will write out a log file to the <build>/unit_tests directory. As with Autotools' make check, the test files are from test/.split and unit_tests/.split files, but for CMake these are generated at build time instead of at test time. On Posix systems, sets the LD_LIBRARY_PATH so that ClamAV-compiled libraries can be loaded when running tests. On Windows systems, CTest will identify and collect all library dependencies and assemble a temporarily install under the build/unit_tests directory so that the libraries can be loaded when running tests. The same feature is used on Windows when using CMake to install to collect all DLL dependencies so that users don't have to install them manually afterwards. Each of the CTest tests are run using a custom wrapper around Python's unittest framework, which is also responsible for finding and inserting valgrind into the valgrind tests on Posix systems. Unlike with Autotools, the CMake CTest Valgrind-tests are enabled by default, if Valgrind can be found. There's no need to set VG=1. CTest's memcheck module is NOT supported, because we use Python to orchestrate our tests. Added a bunch of Windows compatibility changes to the unit tests. These were primarily changing / to PATHSEP and making adjustments to use Win32 C headers and ifdef out the POSIX ones which aren't available on Windows. Also disabled a bunch of tests on Win32 that don't work on Windows, notably the mmap ones and FD-passing (i.e. FILEDES) ones. Add JSON_C_HAVE_INTTYPES_H definition to clamav-config.h to eliminate warnings on Windows where json.h is included after inttypes.h because json-c's inttypes replacement relies on it. This is a it of a hack and may be removed if json-c fixes their inttypes header stuff in the future. Add preprocessor definitions on Windows to disable MSVC warnings about CRT secure and nonstandard functions. While there may be a better solution, this is needed to be able to see other more serious warnings. Add missing file comment block and copyright statement for clamsubmit.c. Also change json-c/json.h include filename to json.h in clamsubmit.c. The directory name is not required. Changed the hash table data integer type from long, which is poorly defined, to size_t -- which is capable of storing a pointer. Fixed a bunch of casts regarding this variable to eliminate warnings. Fixed two bugs causing utf8 encoding unit tests to fail on Windows: - The in_size variable should be the number of bytes, not the character count. This was was causing the SHIFT_JIS (japanese codepage) to UTF8 transcoding test to only transcode half the bytes. - It turns out that the MultiByteToWideChar() API can't transcode UTF16-BE to UTF16-LE. The solution is to just iterate over the buffer and flip the bytes on each uint16_t. This but was causing the UTF16-BE to UTF8 tests to fail. I also split up the utf8 transcoding tests into separate tests so I could see all of the failures instead of just the first one. Added a flags parameter to the unit test function to open testfiles because it turns out that on Windows if a file contains the \r\n it will replace it with just \n if you opened the file as a text file instead of as binary. However, if we open the CBC files as binary, then a bunch of bytecode tests fail. So I've changed the tests to open the CBC files in the bytecode tests as text files and open all other files as binary. Ported the feature tests from shell scripts to Python using a modified version of our QA test-framework, which is largely compatible and will allow us to migrate some QA tests into this repo. I'd like to add GitHub Actions pipelines in the future so that all public PR's get some testing before anyone has to manually review them. The clamd --log option was missing from the help string, though it definitely works. I've added it in this commit. It appears that clamd.c was never clang-format'd, so this commit also reformats clamd.c. Some of the check_clamd tests expected the path returned by clamd to match character for character with original path sent to clamd. However, as we now evaluate real paths before a scan, the path returned by clamd isn't going to match the relative (and possibly symlink-ridden) path passed to clamdscan. I fixed this test by changing the test to search for the basename: <signature> FOUND within the response instead of matching the exact path. Autotools: Link check_clamd with libclamav so we can use our utility functions in check_clamd.c.
2020-08-25 23:14:23 -07:00
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
from http.server import HTTPServer, BaseHTTPRequestHandler
import cgi
CMake: Add CTest support to match Autotools checks An ENABLE_TESTS CMake option is provided so that users can disable testing if they don't want it. Instructions for how to use this included in the INSTALL.cmake.md file. If you run `ctest`, each testcase will write out a log file to the <build>/unit_tests directory. As with Autotools' make check, the test files are from test/.split and unit_tests/.split files, but for CMake these are generated at build time instead of at test time. On Posix systems, sets the LD_LIBRARY_PATH so that ClamAV-compiled libraries can be loaded when running tests. On Windows systems, CTest will identify and collect all library dependencies and assemble a temporarily install under the build/unit_tests directory so that the libraries can be loaded when running tests. The same feature is used on Windows when using CMake to install to collect all DLL dependencies so that users don't have to install them manually afterwards. Each of the CTest tests are run using a custom wrapper around Python's unittest framework, which is also responsible for finding and inserting valgrind into the valgrind tests on Posix systems. Unlike with Autotools, the CMake CTest Valgrind-tests are enabled by default, if Valgrind can be found. There's no need to set VG=1. CTest's memcheck module is NOT supported, because we use Python to orchestrate our tests. Added a bunch of Windows compatibility changes to the unit tests. These were primarily changing / to PATHSEP and making adjustments to use Win32 C headers and ifdef out the POSIX ones which aren't available on Windows. Also disabled a bunch of tests on Win32 that don't work on Windows, notably the mmap ones and FD-passing (i.e. FILEDES) ones. Add JSON_C_HAVE_INTTYPES_H definition to clamav-config.h to eliminate warnings on Windows where json.h is included after inttypes.h because json-c's inttypes replacement relies on it. This is a it of a hack and may be removed if json-c fixes their inttypes header stuff in the future. Add preprocessor definitions on Windows to disable MSVC warnings about CRT secure and nonstandard functions. While there may be a better solution, this is needed to be able to see other more serious warnings. Add missing file comment block and copyright statement for clamsubmit.c. Also change json-c/json.h include filename to json.h in clamsubmit.c. The directory name is not required. Changed the hash table data integer type from long, which is poorly defined, to size_t -- which is capable of storing a pointer. Fixed a bunch of casts regarding this variable to eliminate warnings. Fixed two bugs causing utf8 encoding unit tests to fail on Windows: - The in_size variable should be the number of bytes, not the character count. This was was causing the SHIFT_JIS (japanese codepage) to UTF8 transcoding test to only transcode half the bytes. - It turns out that the MultiByteToWideChar() API can't transcode UTF16-BE to UTF16-LE. The solution is to just iterate over the buffer and flip the bytes on each uint16_t. This but was causing the UTF16-BE to UTF8 tests to fail. I also split up the utf8 transcoding tests into separate tests so I could see all of the failures instead of just the first one. Added a flags parameter to the unit test function to open testfiles because it turns out that on Windows if a file contains the \r\n it will replace it with just \n if you opened the file as a text file instead of as binary. However, if we open the CBC files as binary, then a bunch of bytecode tests fail. So I've changed the tests to open the CBC files in the bytecode tests as text files and open all other files as binary. Ported the feature tests from shell scripts to Python using a modified version of our QA test-framework, which is largely compatible and will allow us to migrate some QA tests into this repo. I'd like to add GitHub Actions pipelines in the future so that all public PR's get some testing before anyone has to manually review them. The clamd --log option was missing from the help string, though it definitely works. I've added it in this commit. It appears that clamd.c was never clang-format'd, so this commit also reformats clamd.c. Some of the check_clamd tests expected the path returned by clamd to match character for character with original path sent to clamd. However, as we now evaluate real paths before a scan, the path returned by clamd isn't going to match the relative (and possibly symlink-ridden) path passed to clamdscan. I fixed this test by changing the test to search for the basename: <signature> FOUND within the response instead of matching the exact path. Autotools: Link check_clamd with libclamav so we can use our utility functions in check_clamd.c.
2020-08-25 23:14:23 -07:00
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
import testcase
CMake: Add CTest support to match Autotools checks An ENABLE_TESTS CMake option is provided so that users can disable testing if they don't want it. Instructions for how to use this included in the INSTALL.cmake.md file. If you run `ctest`, each testcase will write out a log file to the <build>/unit_tests directory. As with Autotools' make check, the test files are from test/.split and unit_tests/.split files, but for CMake these are generated at build time instead of at test time. On Posix systems, sets the LD_LIBRARY_PATH so that ClamAV-compiled libraries can be loaded when running tests. On Windows systems, CTest will identify and collect all library dependencies and assemble a temporarily install under the build/unit_tests directory so that the libraries can be loaded when running tests. The same feature is used on Windows when using CMake to install to collect all DLL dependencies so that users don't have to install them manually afterwards. Each of the CTest tests are run using a custom wrapper around Python's unittest framework, which is also responsible for finding and inserting valgrind into the valgrind tests on Posix systems. Unlike with Autotools, the CMake CTest Valgrind-tests are enabled by default, if Valgrind can be found. There's no need to set VG=1. CTest's memcheck module is NOT supported, because we use Python to orchestrate our tests. Added a bunch of Windows compatibility changes to the unit tests. These were primarily changing / to PATHSEP and making adjustments to use Win32 C headers and ifdef out the POSIX ones which aren't available on Windows. Also disabled a bunch of tests on Win32 that don't work on Windows, notably the mmap ones and FD-passing (i.e. FILEDES) ones. Add JSON_C_HAVE_INTTYPES_H definition to clamav-config.h to eliminate warnings on Windows where json.h is included after inttypes.h because json-c's inttypes replacement relies on it. This is a it of a hack and may be removed if json-c fixes their inttypes header stuff in the future. Add preprocessor definitions on Windows to disable MSVC warnings about CRT secure and nonstandard functions. While there may be a better solution, this is needed to be able to see other more serious warnings. Add missing file comment block and copyright statement for clamsubmit.c. Also change json-c/json.h include filename to json.h in clamsubmit.c. The directory name is not required. Changed the hash table data integer type from long, which is poorly defined, to size_t -- which is capable of storing a pointer. Fixed a bunch of casts regarding this variable to eliminate warnings. Fixed two bugs causing utf8 encoding unit tests to fail on Windows: - The in_size variable should be the number of bytes, not the character count. This was was causing the SHIFT_JIS (japanese codepage) to UTF8 transcoding test to only transcode half the bytes. - It turns out that the MultiByteToWideChar() API can't transcode UTF16-BE to UTF16-LE. The solution is to just iterate over the buffer and flip the bytes on each uint16_t. This but was causing the UTF16-BE to UTF8 tests to fail. I also split up the utf8 transcoding tests into separate tests so I could see all of the failures instead of just the first one. Added a flags parameter to the unit test function to open testfiles because it turns out that on Windows if a file contains the \r\n it will replace it with just \n if you opened the file as a text file instead of as binary. However, if we open the CBC files as binary, then a bunch of bytecode tests fail. So I've changed the tests to open the CBC files in the bytecode tests as text files and open all other files as binary. Ported the feature tests from shell scripts to Python using a modified version of our QA test-framework, which is largely compatible and will allow us to migrate some QA tests into this repo. I'd like to add GitHub Actions pipelines in the future so that all public PR's get some testing before anyone has to manually review them. The clamd --log option was missing from the help string, though it definitely works. I've added it in this commit. It appears that clamd.c was never clang-format'd, so this commit also reformats clamd.c. Some of the check_clamd tests expected the path returned by clamd to match character for character with original path sent to clamd. However, as we now evaluate real paths before a scan, the path returned by clamd isn't going to match the relative (and possibly symlink-ridden) path passed to clamdscan. I fixed this test by changing the test to search for the basename: <signature> FOUND within the response instead of matching the exact path. Autotools: Link check_clamd with libclamav so we can use our utility functions in check_clamd.c.
2020-08-25 23:14:23 -07:00
os_platform = platform.platform()
operating_system = os_platform.split('-')[0].lower()
class TC(testcase.TestCase):
@classmethod
def setUpClass(cls):
super(TC, cls).setUpClass()
# Prepare a directory to host our test databases
TC.path_www = Path(TC.path_tmp, 'www')
TC.path_www.mkdir()
TC.path_db = Path(TC.path_tmp, 'database')
test: freshclam cdiff test tuning There is a bug where freshclam fails to detect if a downloaded CDIFF is empty. In 0.103 this, combined with a CDN caching issue could result in freshclam downloading a daily.cvd but failing to update, putting it in a sort of infinite loop. In 0.104 this issue manifests slightly differently, requiring freshclam to run up to 3x before you get over the empty-CVD hump and are back to normal updates. This commit updates an existing cdiff test with the zero-byte cdiff + an out-of-date CVD to confirm the bug. The following commit will fix it.
2021-09-27 19:00:14 -07:00
TC.path_db.mkdir()
CMake: Add CTest support to match Autotools checks An ENABLE_TESTS CMake option is provided so that users can disable testing if they don't want it. Instructions for how to use this included in the INSTALL.cmake.md file. If you run `ctest`, each testcase will write out a log file to the <build>/unit_tests directory. As with Autotools' make check, the test files are from test/.split and unit_tests/.split files, but for CMake these are generated at build time instead of at test time. On Posix systems, sets the LD_LIBRARY_PATH so that ClamAV-compiled libraries can be loaded when running tests. On Windows systems, CTest will identify and collect all library dependencies and assemble a temporarily install under the build/unit_tests directory so that the libraries can be loaded when running tests. The same feature is used on Windows when using CMake to install to collect all DLL dependencies so that users don't have to install them manually afterwards. Each of the CTest tests are run using a custom wrapper around Python's unittest framework, which is also responsible for finding and inserting valgrind into the valgrind tests on Posix systems. Unlike with Autotools, the CMake CTest Valgrind-tests are enabled by default, if Valgrind can be found. There's no need to set VG=1. CTest's memcheck module is NOT supported, because we use Python to orchestrate our tests. Added a bunch of Windows compatibility changes to the unit tests. These were primarily changing / to PATHSEP and making adjustments to use Win32 C headers and ifdef out the POSIX ones which aren't available on Windows. Also disabled a bunch of tests on Win32 that don't work on Windows, notably the mmap ones and FD-passing (i.e. FILEDES) ones. Add JSON_C_HAVE_INTTYPES_H definition to clamav-config.h to eliminate warnings on Windows where json.h is included after inttypes.h because json-c's inttypes replacement relies on it. This is a it of a hack and may be removed if json-c fixes their inttypes header stuff in the future. Add preprocessor definitions on Windows to disable MSVC warnings about CRT secure and nonstandard functions. While there may be a better solution, this is needed to be able to see other more serious warnings. Add missing file comment block and copyright statement for clamsubmit.c. Also change json-c/json.h include filename to json.h in clamsubmit.c. The directory name is not required. Changed the hash table data integer type from long, which is poorly defined, to size_t -- which is capable of storing a pointer. Fixed a bunch of casts regarding this variable to eliminate warnings. Fixed two bugs causing utf8 encoding unit tests to fail on Windows: - The in_size variable should be the number of bytes, not the character count. This was was causing the SHIFT_JIS (japanese codepage) to UTF8 transcoding test to only transcode half the bytes. - It turns out that the MultiByteToWideChar() API can't transcode UTF16-BE to UTF16-LE. The solution is to just iterate over the buffer and flip the bytes on each uint16_t. This but was causing the UTF16-BE to UTF8 tests to fail. I also split up the utf8 transcoding tests into separate tests so I could see all of the failures instead of just the first one. Added a flags parameter to the unit test function to open testfiles because it turns out that on Windows if a file contains the \r\n it will replace it with just \n if you opened the file as a text file instead of as binary. However, if we open the CBC files as binary, then a bunch of bytecode tests fail. So I've changed the tests to open the CBC files in the bytecode tests as text files and open all other files as binary. Ported the feature tests from shell scripts to Python using a modified version of our QA test-framework, which is largely compatible and will allow us to migrate some QA tests into this repo. I'd like to add GitHub Actions pipelines in the future so that all public PR's get some testing before anyone has to manually review them. The clamd --log option was missing from the help string, though it definitely works. I've added it in this commit. It appears that clamd.c was never clang-format'd, so this commit also reformats clamd.c. Some of the check_clamd tests expected the path returned by clamd to match character for character with original path sent to clamd. However, as we now evaluate real paths before a scan, the path returned by clamd isn't going to match the relative (and possibly symlink-ridden) path passed to clamdscan. I fixed this test by changing the test to search for the basename: <signature> FOUND within the response instead of matching the exact path. Autotools: Link check_clamd with libclamav so we can use our utility functions in check_clamd.c.
2020-08-25 23:14:23 -07:00
TC.freshclam_pid = Path(TC.path_tmp, 'freshclam-test.pid')
TC.freshclam_config = Path(TC.path_tmp, 'freshclam-test.conf')
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
TC.mock_mirror_port = 8001 # Chosen instead of 8000 because CVD-Update tool serves on 8000 by default.
# TODO: Ideally we'd find an open port to use for these tests instead of crossing our fingers.
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
TC.mock_mirror = None
CMake: Add CTest support to match Autotools checks An ENABLE_TESTS CMake option is provided so that users can disable testing if they don't want it. Instructions for how to use this included in the INSTALL.cmake.md file. If you run `ctest`, each testcase will write out a log file to the <build>/unit_tests directory. As with Autotools' make check, the test files are from test/.split and unit_tests/.split files, but for CMake these are generated at build time instead of at test time. On Posix systems, sets the LD_LIBRARY_PATH so that ClamAV-compiled libraries can be loaded when running tests. On Windows systems, CTest will identify and collect all library dependencies and assemble a temporarily install under the build/unit_tests directory so that the libraries can be loaded when running tests. The same feature is used on Windows when using CMake to install to collect all DLL dependencies so that users don't have to install them manually afterwards. Each of the CTest tests are run using a custom wrapper around Python's unittest framework, which is also responsible for finding and inserting valgrind into the valgrind tests on Posix systems. Unlike with Autotools, the CMake CTest Valgrind-tests are enabled by default, if Valgrind can be found. There's no need to set VG=1. CTest's memcheck module is NOT supported, because we use Python to orchestrate our tests. Added a bunch of Windows compatibility changes to the unit tests. These were primarily changing / to PATHSEP and making adjustments to use Win32 C headers and ifdef out the POSIX ones which aren't available on Windows. Also disabled a bunch of tests on Win32 that don't work on Windows, notably the mmap ones and FD-passing (i.e. FILEDES) ones. Add JSON_C_HAVE_INTTYPES_H definition to clamav-config.h to eliminate warnings on Windows where json.h is included after inttypes.h because json-c's inttypes replacement relies on it. This is a it of a hack and may be removed if json-c fixes their inttypes header stuff in the future. Add preprocessor definitions on Windows to disable MSVC warnings about CRT secure and nonstandard functions. While there may be a better solution, this is needed to be able to see other more serious warnings. Add missing file comment block and copyright statement for clamsubmit.c. Also change json-c/json.h include filename to json.h in clamsubmit.c. The directory name is not required. Changed the hash table data integer type from long, which is poorly defined, to size_t -- which is capable of storing a pointer. Fixed a bunch of casts regarding this variable to eliminate warnings. Fixed two bugs causing utf8 encoding unit tests to fail on Windows: - The in_size variable should be the number of bytes, not the character count. This was was causing the SHIFT_JIS (japanese codepage) to UTF8 transcoding test to only transcode half the bytes. - It turns out that the MultiByteToWideChar() API can't transcode UTF16-BE to UTF16-LE. The solution is to just iterate over the buffer and flip the bytes on each uint16_t. This but was causing the UTF16-BE to UTF8 tests to fail. I also split up the utf8 transcoding tests into separate tests so I could see all of the failures instead of just the first one. Added a flags parameter to the unit test function to open testfiles because it turns out that on Windows if a file contains the \r\n it will replace it with just \n if you opened the file as a text file instead of as binary. However, if we open the CBC files as binary, then a bunch of bytecode tests fail. So I've changed the tests to open the CBC files in the bytecode tests as text files and open all other files as binary. Ported the feature tests from shell scripts to Python using a modified version of our QA test-framework, which is largely compatible and will allow us to migrate some QA tests into this repo. I'd like to add GitHub Actions pipelines in the future so that all public PR's get some testing before anyone has to manually review them. The clamd --log option was missing from the help string, though it definitely works. I've added it in this commit. It appears that clamd.c was never clang-format'd, so this commit also reformats clamd.c. Some of the check_clamd tests expected the path returned by clamd to match character for character with original path sent to clamd. However, as we now evaluate real paths before a scan, the path returned by clamd isn't going to match the relative (and possibly symlink-ridden) path passed to clamdscan. I fixed this test by changing the test to search for the basename: <signature> FOUND within the response instead of matching the exact path. Autotools: Link check_clamd with libclamav so we can use our utility functions in check_clamd.c.
2020-08-25 23:14:23 -07:00
@classmethod
def tearDownClass(cls):
super(TC, cls).tearDownClass()
def setUp(self):
super(TC, self).setUp()
def tearDown(self):
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
if TC.mock_mirror != None:
TC.mock_mirror.terminate()
TC.mock_mirror = None
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
# Clear the database directory
try:
Test: python 3.5 compatibility fix Python 3.5 compatibility fixes for Debian 9, etc that lack 3.6+. Change a python f-string to an old-style `"".format()`. Convert Path objects to strings for older `shutil` APIs that don't accept Paths.
2021-07-17 11:31:56 -07:00
shutil.rmtree(str(self.path_db))
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
except Exception:
pass
self.path_db.mkdir()
# Clear the www directory
try:
Test: python 3.5 compatibility fix Python 3.5 compatibility fixes for Debian 9, etc that lack 3.6+. Change a python f-string to an old-style `"".format()`. Convert Path objects to strings for older `shutil` APIs that don't accept Paths.
2021-07-17 11:31:56 -07:00
shutil.rmtree(str(self.path_www))
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
except Exception:
pass
self.path_www.mkdir()
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
CMake: Add CTest support to match Autotools checks An ENABLE_TESTS CMake option is provided so that users can disable testing if they don't want it. Instructions for how to use this included in the INSTALL.cmake.md file. If you run `ctest`, each testcase will write out a log file to the <build>/unit_tests directory. As with Autotools' make check, the test files are from test/.split and unit_tests/.split files, but for CMake these are generated at build time instead of at test time. On Posix systems, sets the LD_LIBRARY_PATH so that ClamAV-compiled libraries can be loaded when running tests. On Windows systems, CTest will identify and collect all library dependencies and assemble a temporarily install under the build/unit_tests directory so that the libraries can be loaded when running tests. The same feature is used on Windows when using CMake to install to collect all DLL dependencies so that users don't have to install them manually afterwards. Each of the CTest tests are run using a custom wrapper around Python's unittest framework, which is also responsible for finding and inserting valgrind into the valgrind tests on Posix systems. Unlike with Autotools, the CMake CTest Valgrind-tests are enabled by default, if Valgrind can be found. There's no need to set VG=1. CTest's memcheck module is NOT supported, because we use Python to orchestrate our tests. Added a bunch of Windows compatibility changes to the unit tests. These were primarily changing / to PATHSEP and making adjustments to use Win32 C headers and ifdef out the POSIX ones which aren't available on Windows. Also disabled a bunch of tests on Win32 that don't work on Windows, notably the mmap ones and FD-passing (i.e. FILEDES) ones. Add JSON_C_HAVE_INTTYPES_H definition to clamav-config.h to eliminate warnings on Windows where json.h is included after inttypes.h because json-c's inttypes replacement relies on it. This is a it of a hack and may be removed if json-c fixes their inttypes header stuff in the future. Add preprocessor definitions on Windows to disable MSVC warnings about CRT secure and nonstandard functions. While there may be a better solution, this is needed to be able to see other more serious warnings. Add missing file comment block and copyright statement for clamsubmit.c. Also change json-c/json.h include filename to json.h in clamsubmit.c. The directory name is not required. Changed the hash table data integer type from long, which is poorly defined, to size_t -- which is capable of storing a pointer. Fixed a bunch of casts regarding this variable to eliminate warnings. Fixed two bugs causing utf8 encoding unit tests to fail on Windows: - The in_size variable should be the number of bytes, not the character count. This was was causing the SHIFT_JIS (japanese codepage) to UTF8 transcoding test to only transcode half the bytes. - It turns out that the MultiByteToWideChar() API can't transcode UTF16-BE to UTF16-LE. The solution is to just iterate over the buffer and flip the bytes on each uint16_t. This but was causing the UTF16-BE to UTF8 tests to fail. I also split up the utf8 transcoding tests into separate tests so I could see all of the failures instead of just the first one. Added a flags parameter to the unit test function to open testfiles because it turns out that on Windows if a file contains the \r\n it will replace it with just \n if you opened the file as a text file instead of as binary. However, if we open the CBC files as binary, then a bunch of bytecode tests fail. So I've changed the tests to open the CBC files in the bytecode tests as text files and open all other files as binary. Ported the feature tests from shell scripts to Python using a modified version of our QA test-framework, which is largely compatible and will allow us to migrate some QA tests into this repo. I'd like to add GitHub Actions pipelines in the future so that all public PR's get some testing before anyone has to manually review them. The clamd --log option was missing from the help string, though it definitely works. I've added it in this commit. It appears that clamd.c was never clang-format'd, so this commit also reformats clamd.c. Some of the check_clamd tests expected the path returned by clamd to match character for character with original path sent to clamd. However, as we now evaluate real paths before a scan, the path returned by clamd isn't going to match the relative (and possibly symlink-ridden) path passed to clamdscan. I fixed this test by changing the test to search for the basename: <signature> FOUND within the response instead of matching the exact path. Autotools: Link check_clamd with libclamav so we can use our utility functions in check_clamd.c.
2020-08-25 23:14:23 -07:00
super(TC, self).tearDown()
self.verify_valgrind_log()
def test_freshclam_00_version(self):
self.step_name('freshclam version test')
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
if TC.freshclam_config.exists():
os.remove(str(TC.freshclam_config))
TC.freshclam_config.write_text('''
FreshClam: Fix tests, configs for IPv6-only systems Some config settings and some tests hardcoded 127.0.0.1. This switches to localhost, they'll work for systems that don't support IPv4.
2021-05-27 16:52:58 -07:00
DatabaseMirror http://localhost:{port}
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
'''.format(
freshclam_pid=TC.freshclam_pid,
path_db=TC.path_db,
FreshClam: Fix tests, configs for IPv6-only systems Some config settings and some tests hardcoded 127.0.0.1. This switches to localhost, they'll work for systems that don't support IPv4.
2021-05-27 16:52:58 -07:00
port=TC.mock_mirror_port,
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
))
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} -V'.format(
ctest: Support Python 3.5 Python 3.6 is not available on Debian 9 and other older LTS releases. This patch removes use of Python f-strings which were introduced in Python 3.6 so as to support Python 3.5. TODO: Revert this commit when Debian 9 dies or gets f-string support (whichver comes first).
2021-01-20 07:27:04 -08:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
CMake: Add CTest support to match Autotools checks An ENABLE_TESTS CMake option is provided so that users can disable testing if they don't want it. Instructions for how to use this included in the INSTALL.cmake.md file. If you run `ctest`, each testcase will write out a log file to the <build>/unit_tests directory. As with Autotools' make check, the test files are from test/.split and unit_tests/.split files, but for CMake these are generated at build time instead of at test time. On Posix systems, sets the LD_LIBRARY_PATH so that ClamAV-compiled libraries can be loaded when running tests. On Windows systems, CTest will identify and collect all library dependencies and assemble a temporarily install under the build/unit_tests directory so that the libraries can be loaded when running tests. The same feature is used on Windows when using CMake to install to collect all DLL dependencies so that users don't have to install them manually afterwards. Each of the CTest tests are run using a custom wrapper around Python's unittest framework, which is also responsible for finding and inserting valgrind into the valgrind tests on Posix systems. Unlike with Autotools, the CMake CTest Valgrind-tests are enabled by default, if Valgrind can be found. There's no need to set VG=1. CTest's memcheck module is NOT supported, because we use Python to orchestrate our tests. Added a bunch of Windows compatibility changes to the unit tests. These were primarily changing / to PATHSEP and making adjustments to use Win32 C headers and ifdef out the POSIX ones which aren't available on Windows. Also disabled a bunch of tests on Win32 that don't work on Windows, notably the mmap ones and FD-passing (i.e. FILEDES) ones. Add JSON_C_HAVE_INTTYPES_H definition to clamav-config.h to eliminate warnings on Windows where json.h is included after inttypes.h because json-c's inttypes replacement relies on it. This is a it of a hack and may be removed if json-c fixes their inttypes header stuff in the future. Add preprocessor definitions on Windows to disable MSVC warnings about CRT secure and nonstandard functions. While there may be a better solution, this is needed to be able to see other more serious warnings. Add missing file comment block and copyright statement for clamsubmit.c. Also change json-c/json.h include filename to json.h in clamsubmit.c. The directory name is not required. Changed the hash table data integer type from long, which is poorly defined, to size_t -- which is capable of storing a pointer. Fixed a bunch of casts regarding this variable to eliminate warnings. Fixed two bugs causing utf8 encoding unit tests to fail on Windows: - The in_size variable should be the number of bytes, not the character count. This was was causing the SHIFT_JIS (japanese codepage) to UTF8 transcoding test to only transcode half the bytes. - It turns out that the MultiByteToWideChar() API can't transcode UTF16-BE to UTF16-LE. The solution is to just iterate over the buffer and flip the bytes on each uint16_t. This but was causing the UTF16-BE to UTF8 tests to fail. I also split up the utf8 transcoding tests into separate tests so I could see all of the failures instead of just the first one. Added a flags parameter to the unit test function to open testfiles because it turns out that on Windows if a file contains the \r\n it will replace it with just \n if you opened the file as a text file instead of as binary. However, if we open the CBC files as binary, then a bunch of bytecode tests fail. So I've changed the tests to open the CBC files in the bytecode tests as text files and open all other files as binary. Ported the feature tests from shell scripts to Python using a modified version of our QA test-framework, which is largely compatible and will allow us to migrate some QA tests into this repo. I'd like to add GitHub Actions pipelines in the future so that all public PR's get some testing before anyone has to manually review them. The clamd --log option was missing from the help string, though it definitely works. I've added it in this commit. It appears that clamd.c was never clang-format'd, so this commit also reformats clamd.c. Some of the check_clamd tests expected the path returned by clamd to match character for character with original path sent to clamd. However, as we now evaluate real paths before a scan, the path returned by clamd isn't going to match the relative (and possibly symlink-ridden) path passed to clamdscan. I fixed this test by changing the test to search for the basename: <signature> FOUND within the response instead of matching the exact path. Autotools: Link check_clamd with libclamav so we can use our utility functions in check_clamd.c.
2020-08-25 23:14:23 -07:00
output = self.execute_command(command)
assert output.ec == 0 # success
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
expected_stdout = [
ctest: Support Python 3.5 Python 3.6 is not available on Debian 9 and other older LTS releases. This patch removes use of Python f-strings which were introduced in Python 3.6 so as to support Python 3.5. TODO: Revert this commit when Debian 9 dies or gets f-string support (whichver comes first).
2021-01-20 07:27:04 -08:00
'ClamAV {}'.format(TC.version),
CMake: Add CTest support to match Autotools checks An ENABLE_TESTS CMake option is provided so that users can disable testing if they don't want it. Instructions for how to use this included in the INSTALL.cmake.md file. If you run `ctest`, each testcase will write out a log file to the <build>/unit_tests directory. As with Autotools' make check, the test files are from test/.split and unit_tests/.split files, but for CMake these are generated at build time instead of at test time. On Posix systems, sets the LD_LIBRARY_PATH so that ClamAV-compiled libraries can be loaded when running tests. On Windows systems, CTest will identify and collect all library dependencies and assemble a temporarily install under the build/unit_tests directory so that the libraries can be loaded when running tests. The same feature is used on Windows when using CMake to install to collect all DLL dependencies so that users don't have to install them manually afterwards. Each of the CTest tests are run using a custom wrapper around Python's unittest framework, which is also responsible for finding and inserting valgrind into the valgrind tests on Posix systems. Unlike with Autotools, the CMake CTest Valgrind-tests are enabled by default, if Valgrind can be found. There's no need to set VG=1. CTest's memcheck module is NOT supported, because we use Python to orchestrate our tests. Added a bunch of Windows compatibility changes to the unit tests. These were primarily changing / to PATHSEP and making adjustments to use Win32 C headers and ifdef out the POSIX ones which aren't available on Windows. Also disabled a bunch of tests on Win32 that don't work on Windows, notably the mmap ones and FD-passing (i.e. FILEDES) ones. Add JSON_C_HAVE_INTTYPES_H definition to clamav-config.h to eliminate warnings on Windows where json.h is included after inttypes.h because json-c's inttypes replacement relies on it. This is a it of a hack and may be removed if json-c fixes their inttypes header stuff in the future. Add preprocessor definitions on Windows to disable MSVC warnings about CRT secure and nonstandard functions. While there may be a better solution, this is needed to be able to see other more serious warnings. Add missing file comment block and copyright statement for clamsubmit.c. Also change json-c/json.h include filename to json.h in clamsubmit.c. The directory name is not required. Changed the hash table data integer type from long, which is poorly defined, to size_t -- which is capable of storing a pointer. Fixed a bunch of casts regarding this variable to eliminate warnings. Fixed two bugs causing utf8 encoding unit tests to fail on Windows: - The in_size variable should be the number of bytes, not the character count. This was was causing the SHIFT_JIS (japanese codepage) to UTF8 transcoding test to only transcode half the bytes. - It turns out that the MultiByteToWideChar() API can't transcode UTF16-BE to UTF16-LE. The solution is to just iterate over the buffer and flip the bytes on each uint16_t. This but was causing the UTF16-BE to UTF8 tests to fail. I also split up the utf8 transcoding tests into separate tests so I could see all of the failures instead of just the first one. Added a flags parameter to the unit test function to open testfiles because it turns out that on Windows if a file contains the \r\n it will replace it with just \n if you opened the file as a text file instead of as binary. However, if we open the CBC files as binary, then a bunch of bytecode tests fail. So I've changed the tests to open the CBC files in the bytecode tests as text files and open all other files as binary. Ported the feature tests from shell scripts to Python using a modified version of our QA test-framework, which is largely compatible and will allow us to migrate some QA tests into this repo. I'd like to add GitHub Actions pipelines in the future so that all public PR's get some testing before anyone has to manually review them. The clamd --log option was missing from the help string, though it definitely works. I've added it in this commit. It appears that clamd.c was never clang-format'd, so this commit also reformats clamd.c. Some of the check_clamd tests expected the path returned by clamd to match character for character with original path sent to clamd. However, as we now evaluate real paths before a scan, the path returned by clamd isn't going to match the relative (and possibly symlink-ridden) path passed to clamdscan. I fixed this test by changing the test to search for the basename: <signature> FOUND within the response instead of matching the exact path. Autotools: Link check_clamd with libclamav so we can use our utility functions in check_clamd.c.
2020-08-25 23:14:23 -07:00
]
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
# verify stdout
self.verify_output(output.out, expected=expected_stdout)
CMake: Add CTest support to match Autotools checks An ENABLE_TESTS CMake option is provided so that users can disable testing if they don't want it. Instructions for how to use this included in the INSTALL.cmake.md file. If you run `ctest`, each testcase will write out a log file to the <build>/unit_tests directory. As with Autotools' make check, the test files are from test/.split and unit_tests/.split files, but for CMake these are generated at build time instead of at test time. On Posix systems, sets the LD_LIBRARY_PATH so that ClamAV-compiled libraries can be loaded when running tests. On Windows systems, CTest will identify and collect all library dependencies and assemble a temporarily install under the build/unit_tests directory so that the libraries can be loaded when running tests. The same feature is used on Windows when using CMake to install to collect all DLL dependencies so that users don't have to install them manually afterwards. Each of the CTest tests are run using a custom wrapper around Python's unittest framework, which is also responsible for finding and inserting valgrind into the valgrind tests on Posix systems. Unlike with Autotools, the CMake CTest Valgrind-tests are enabled by default, if Valgrind can be found. There's no need to set VG=1. CTest's memcheck module is NOT supported, because we use Python to orchestrate our tests. Added a bunch of Windows compatibility changes to the unit tests. These were primarily changing / to PATHSEP and making adjustments to use Win32 C headers and ifdef out the POSIX ones which aren't available on Windows. Also disabled a bunch of tests on Win32 that don't work on Windows, notably the mmap ones and FD-passing (i.e. FILEDES) ones. Add JSON_C_HAVE_INTTYPES_H definition to clamav-config.h to eliminate warnings on Windows where json.h is included after inttypes.h because json-c's inttypes replacement relies on it. This is a it of a hack and may be removed if json-c fixes their inttypes header stuff in the future. Add preprocessor definitions on Windows to disable MSVC warnings about CRT secure and nonstandard functions. While there may be a better solution, this is needed to be able to see other more serious warnings. Add missing file comment block and copyright statement for clamsubmit.c. Also change json-c/json.h include filename to json.h in clamsubmit.c. The directory name is not required. Changed the hash table data integer type from long, which is poorly defined, to size_t -- which is capable of storing a pointer. Fixed a bunch of casts regarding this variable to eliminate warnings. Fixed two bugs causing utf8 encoding unit tests to fail on Windows: - The in_size variable should be the number of bytes, not the character count. This was was causing the SHIFT_JIS (japanese codepage) to UTF8 transcoding test to only transcode half the bytes. - It turns out that the MultiByteToWideChar() API can't transcode UTF16-BE to UTF16-LE. The solution is to just iterate over the buffer and flip the bytes on each uint16_t. This but was causing the UTF16-BE to UTF8 tests to fail. I also split up the utf8 transcoding tests into separate tests so I could see all of the failures instead of just the first one. Added a flags parameter to the unit test function to open testfiles because it turns out that on Windows if a file contains the \r\n it will replace it with just \n if you opened the file as a text file instead of as binary. However, if we open the CBC files as binary, then a bunch of bytecode tests fail. So I've changed the tests to open the CBC files in the bytecode tests as text files and open all other files as binary. Ported the feature tests from shell scripts to Python using a modified version of our QA test-framework, which is largely compatible and will allow us to migrate some QA tests into this repo. I'd like to add GitHub Actions pipelines in the future so that all public PR's get some testing before anyone has to manually review them. The clamd --log option was missing from the help string, though it definitely works. I've added it in this commit. It appears that clamd.c was never clang-format'd, so this commit also reformats clamd.c. Some of the check_clamd tests expected the path returned by clamd to match character for character with original path sent to clamd. However, as we now evaluate real paths before a scan, the path returned by clamd isn't going to match the relative (and possibly symlink-ridden) path passed to clamdscan. I fixed this test by changing the test to search for the basename: <signature> FOUND within the response instead of matching the exact path. Autotools: Link check_clamd with libclamav so we can use our utility functions in check_clamd.c.
2020-08-25 23:14:23 -07:00
def test_freshclam_01_file_copy(self):
self.step_name('Basic freshclam test using file:// to "download" clamav.hdb')
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
if TC.freshclam_config.exists():
os.remove(str(TC.freshclam_config))
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
# Select database files for test
shutil.copy(
XOR test files; clean up tests directory The split test files are flagged by some AV's because they look like broken executables. Instead of splitting the test files to prevent detections, we should encrypt them. This commit replaces the "reassemble testfiles" script with a basic "XOR testfiles" script that can be used to encrypt or decrypt test files. This commit also of course then replaces all the split files with xor'ed files. The test and unit_tests directories were a bit of a mess, so I reorganized them all into unit_tests with all of the test files placed under "unit_tests/input" using subdirectories for different types of files.
2021-07-16 13:43:28 -07:00
str(TC.path_build / 'unit_tests' / 'input' / 'clamav.hdb'),
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
str(TC.path_www),
)
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
TC.freshclam_config.write_text('''
DatabaseMirror http://localhost:{port}
PidFile {freshclam_pid}
LogVerbose yes
LogFileMaxSize 0
LogTime yes
DatabaseDirectory {path_db}
DatabaseCustomURL file://{file_db}
ExcludeDatabase daily
ExcludeDatabase main
ExcludeDatabase bytecode
Tests: Enable freshclam tests to pass as root If you test in a docker container and don't create a user/switch to run as something other than root, than the tests will fail. Default freshclam behavior is to drop to run as the "clamav" user when run as root. This change sets the DatabaseOwner to the current user, so it the tests will pass when run as root.
2021-04-23 05:20:26 -07:00
DatabaseOwner {user}
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
'''.format(
freshclam_pid=TC.freshclam_pid,
path_db=TC.path_db,
file_db=TC.path_www / "clamav.hdb",
port=TC.mock_mirror_port,
Tests: Enable freshclam tests to pass as root If you test in a docker container and don't create a user/switch to run as something other than root, than the tests will fail. Default freshclam behavior is to drop to run as the "clamav" user when run as root. This change sets the DatabaseOwner to the current user, so it the tests will pass when run as root.
2021-04-23 05:20:26 -07:00
user=getpass.getuser(),
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
))
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config}'.format(
ctest: Support Python 3.5 Python 3.6 is not available on Debian 9 and other older LTS releases. This patch removes use of Python f-strings which were introduced in Python 3.6 so as to support Python 3.5. TODO: Revert this commit when Debian 9 dies or gets f-string support (whichver comes first).
2021-01-20 07:27:04 -08:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
CMake: Add CTest support to match Autotools checks An ENABLE_TESTS CMake option is provided so that users can disable testing if they don't want it. Instructions for how to use this included in the INSTALL.cmake.md file. If you run `ctest`, each testcase will write out a log file to the <build>/unit_tests directory. As with Autotools' make check, the test files are from test/.split and unit_tests/.split files, but for CMake these are generated at build time instead of at test time. On Posix systems, sets the LD_LIBRARY_PATH so that ClamAV-compiled libraries can be loaded when running tests. On Windows systems, CTest will identify and collect all library dependencies and assemble a temporarily install under the build/unit_tests directory so that the libraries can be loaded when running tests. The same feature is used on Windows when using CMake to install to collect all DLL dependencies so that users don't have to install them manually afterwards. Each of the CTest tests are run using a custom wrapper around Python's unittest framework, which is also responsible for finding and inserting valgrind into the valgrind tests on Posix systems. Unlike with Autotools, the CMake CTest Valgrind-tests are enabled by default, if Valgrind can be found. There's no need to set VG=1. CTest's memcheck module is NOT supported, because we use Python to orchestrate our tests. Added a bunch of Windows compatibility changes to the unit tests. These were primarily changing / to PATHSEP and making adjustments to use Win32 C headers and ifdef out the POSIX ones which aren't available on Windows. Also disabled a bunch of tests on Win32 that don't work on Windows, notably the mmap ones and FD-passing (i.e. FILEDES) ones. Add JSON_C_HAVE_INTTYPES_H definition to clamav-config.h to eliminate warnings on Windows where json.h is included after inttypes.h because json-c's inttypes replacement relies on it. This is a it of a hack and may be removed if json-c fixes their inttypes header stuff in the future. Add preprocessor definitions on Windows to disable MSVC warnings about CRT secure and nonstandard functions. While there may be a better solution, this is needed to be able to see other more serious warnings. Add missing file comment block and copyright statement for clamsubmit.c. Also change json-c/json.h include filename to json.h in clamsubmit.c. The directory name is not required. Changed the hash table data integer type from long, which is poorly defined, to size_t -- which is capable of storing a pointer. Fixed a bunch of casts regarding this variable to eliminate warnings. Fixed two bugs causing utf8 encoding unit tests to fail on Windows: - The in_size variable should be the number of bytes, not the character count. This was was causing the SHIFT_JIS (japanese codepage) to UTF8 transcoding test to only transcode half the bytes. - It turns out that the MultiByteToWideChar() API can't transcode UTF16-BE to UTF16-LE. The solution is to just iterate over the buffer and flip the bytes on each uint16_t. This but was causing the UTF16-BE to UTF8 tests to fail. I also split up the utf8 transcoding tests into separate tests so I could see all of the failures instead of just the first one. Added a flags parameter to the unit test function to open testfiles because it turns out that on Windows if a file contains the \r\n it will replace it with just \n if you opened the file as a text file instead of as binary. However, if we open the CBC files as binary, then a bunch of bytecode tests fail. So I've changed the tests to open the CBC files in the bytecode tests as text files and open all other files as binary. Ported the feature tests from shell scripts to Python using a modified version of our QA test-framework, which is largely compatible and will allow us to migrate some QA tests into this repo. I'd like to add GitHub Actions pipelines in the future so that all public PR's get some testing before anyone has to manually review them. The clamd --log option was missing from the help string, though it definitely works. I've added it in this commit. It appears that clamd.c was never clang-format'd, so this commit also reformats clamd.c. Some of the check_clamd tests expected the path returned by clamd to match character for character with original path sent to clamd. However, as we now evaluate real paths before a scan, the path returned by clamd isn't going to match the relative (and possibly symlink-ridden) path passed to clamdscan. I fixed this test by changing the test to search for the basename: <signature> FOUND within the response instead of matching the exact path. Autotools: Link check_clamd with libclamav so we can use our utility functions in check_clamd.c.
2020-08-25 23:14:23 -07:00
output = self.execute_command(command)
assert output.ec == 0 # success
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
expected_stdout = [
ctest: Support Python 3.5 Python 3.6 is not available on Debian 9 and other older LTS releases. This patch removes use of Python f-strings which were introduced in Python 3.6 so as to support Python 3.5. TODO: Revert this commit when Debian 9 dies or gets f-string support (whichver comes first).
2021-01-20 07:27:04 -08:00
'Downloading clamav.hdb',
'Database test passed.',
'clamav.hdb updated',
CMake: Add CTest support to match Autotools checks An ENABLE_TESTS CMake option is provided so that users can disable testing if they don't want it. Instructions for how to use this included in the INSTALL.cmake.md file. If you run `ctest`, each testcase will write out a log file to the <build>/unit_tests directory. As with Autotools' make check, the test files are from test/.split and unit_tests/.split files, but for CMake these are generated at build time instead of at test time. On Posix systems, sets the LD_LIBRARY_PATH so that ClamAV-compiled libraries can be loaded when running tests. On Windows systems, CTest will identify and collect all library dependencies and assemble a temporarily install under the build/unit_tests directory so that the libraries can be loaded when running tests. The same feature is used on Windows when using CMake to install to collect all DLL dependencies so that users don't have to install them manually afterwards. Each of the CTest tests are run using a custom wrapper around Python's unittest framework, which is also responsible for finding and inserting valgrind into the valgrind tests on Posix systems. Unlike with Autotools, the CMake CTest Valgrind-tests are enabled by default, if Valgrind can be found. There's no need to set VG=1. CTest's memcheck module is NOT supported, because we use Python to orchestrate our tests. Added a bunch of Windows compatibility changes to the unit tests. These were primarily changing / to PATHSEP and making adjustments to use Win32 C headers and ifdef out the POSIX ones which aren't available on Windows. Also disabled a bunch of tests on Win32 that don't work on Windows, notably the mmap ones and FD-passing (i.e. FILEDES) ones. Add JSON_C_HAVE_INTTYPES_H definition to clamav-config.h to eliminate warnings on Windows where json.h is included after inttypes.h because json-c's inttypes replacement relies on it. This is a it of a hack and may be removed if json-c fixes their inttypes header stuff in the future. Add preprocessor definitions on Windows to disable MSVC warnings about CRT secure and nonstandard functions. While there may be a better solution, this is needed to be able to see other more serious warnings. Add missing file comment block and copyright statement for clamsubmit.c. Also change json-c/json.h include filename to json.h in clamsubmit.c. The directory name is not required. Changed the hash table data integer type from long, which is poorly defined, to size_t -- which is capable of storing a pointer. Fixed a bunch of casts regarding this variable to eliminate warnings. Fixed two bugs causing utf8 encoding unit tests to fail on Windows: - The in_size variable should be the number of bytes, not the character count. This was was causing the SHIFT_JIS (japanese codepage) to UTF8 transcoding test to only transcode half the bytes. - It turns out that the MultiByteToWideChar() API can't transcode UTF16-BE to UTF16-LE. The solution is to just iterate over the buffer and flip the bytes on each uint16_t. This but was causing the UTF16-BE to UTF8 tests to fail. I also split up the utf8 transcoding tests into separate tests so I could see all of the failures instead of just the first one. Added a flags parameter to the unit test function to open testfiles because it turns out that on Windows if a file contains the \r\n it will replace it with just \n if you opened the file as a text file instead of as binary. However, if we open the CBC files as binary, then a bunch of bytecode tests fail. So I've changed the tests to open the CBC files in the bytecode tests as text files and open all other files as binary. Ported the feature tests from shell scripts to Python using a modified version of our QA test-framework, which is largely compatible and will allow us to migrate some QA tests into this repo. I'd like to add GitHub Actions pipelines in the future so that all public PR's get some testing before anyone has to manually review them. The clamd --log option was missing from the help string, though it definitely works. I've added it in this commit. It appears that clamd.c was never clang-format'd, so this commit also reformats clamd.c. Some of the check_clamd tests expected the path returned by clamd to match character for character with original path sent to clamd. However, as we now evaluate real paths before a scan, the path returned by clamd isn't going to match the relative (and possibly symlink-ridden) path passed to clamdscan. I fixed this test by changing the test to search for the basename: <signature> FOUND within the response instead of matching the exact path. Autotools: Link check_clamd with libclamav so we can use our utility functions in check_clamd.c.
2020-08-25 23:14:23 -07:00
]
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
# verify stdout
self.verify_output(output.out, expected=expected_stdout)
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
def test_freshclam_02_http_403(self):
self.step_name('Verify correct behavior when receiving 403 (forbidden)')
# Start our mock database mirror.
Tests: fix python warnings in freshclam tests The WebServerHandler classes used for the tests have a "Test_" prefix which causes pytest to print some warnings thinking they're special test classes. This commit gets rid of the warnings.
2021-05-11 15:36:22 -07:00
TC.mock_mirror = Process(target=mock_database_mirror, args=(WebServerHandler_02,))
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
TC.mock_mirror.start()
if TC.freshclam_config.exists():
os.remove(str(TC.freshclam_config))
TC.freshclam_config.write_text('''
DatabaseMirror http://localhost:{port}
DNSDatabaseInfo no
PidFile {freshclam_pid}
LogVerbose yes
LogFileMaxSize 0
LogTime yes
DatabaseDirectory {path_db}
Tests: Enable freshclam tests to pass as root If you test in a docker container and don't create a user/switch to run as something other than root, than the tests will fail. Default freshclam behavior is to drop to run as the "clamav" user when run as root. This change sets the DatabaseOwner to the current user, so it the tests will pass when run as root.
2021-04-23 05:20:26 -07:00
DatabaseOwner {user}
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
'''.format(
freshclam_pid=TC.freshclam_pid,
path_db=TC.path_db,
port=TC.mock_mirror_port,
Tests: Enable freshclam tests to pass as root If you test in a docker container and don't create a user/switch to run as something other than root, than the tests will fail. Default freshclam behavior is to drop to run as the "clamav" user when run as root. This change sets the DatabaseOwner to the current user, so it the tests will pass when run as root.
2021-04-23 05:20:26 -07:00
user=getpass.getuser(),
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
))
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} --update-db=daily'.format(
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
output = self.execute_command(command)
assert output.ec == 17 # forbidden
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
expected_stderr = [
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
'FreshClam received error code 403',
'Forbidden',
]
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
# verify stderr
self.verify_output(output.err, expected=expected_stderr)
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} --update-db=daily'.format(
test: extend freshclam 403 test to test cool-down Extends the existing freshclam 403 test to check if a repeated freshclam 403 response outputs the cool-down message instead of the original 403 response message.
2021-10-06 17:09:13 -07:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
output = self.execute_command(command)
assert output.ec == 0 # "fine" (on cooldown, refusing to try again for now)
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
expected_stderr = [
test: extend freshclam 403 test to test cool-down Extends the existing freshclam 403 test to check if a repeated freshclam 403 response outputs the cool-down message instead of the original 403 response message.
2021-10-06 17:09:13 -07:00
'FreshClam previously received error code 429 or 403',
'You are still on cool-down until after',
]
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
# verify stderr
self.verify_output(output.err, expected=expected_stderr)
test: extend freshclam 403 test to test cool-down Extends the existing freshclam 403 test to check if a repeated freshclam 403 response outputs the cool-down message instead of the original 403 response message.
2021-10-06 17:09:13 -07:00
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
def test_freshclam_03_http_403_daemonized(self):
self.step_name('Verify correct behavior when receiving 403 (forbidden) and daemonized')
# Start our mock database mirror.
Tests: fix python warnings in freshclam tests The WebServerHandler classes used for the tests have a "Test_" prefix which causes pytest to print some warnings thinking they're special test classes. This commit gets rid of the warnings.
2021-05-11 15:36:22 -07:00
TC.mock_mirror = Process(target=mock_database_mirror, args=(WebServerHandler_02,))
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
TC.mock_mirror.start()
if TC.freshclam_config.exists():
os.remove(str(TC.freshclam_config))
TC.freshclam_config.write_text('''
DatabaseMirror http://localhost:{port}
DNSDatabaseInfo no
PidFile {freshclam_pid}
LogVerbose yes
LogFileMaxSize 0
LogTime yes
DatabaseDirectory {path_db}
Tests: Enable freshclam tests to pass as root If you test in a docker container and don't create a user/switch to run as something other than root, than the tests will fail. Default freshclam behavior is to drop to run as the "clamav" user when run as root. This change sets the DatabaseOwner to the current user, so it the tests will pass when run as root.
2021-04-23 05:20:26 -07:00
DatabaseOwner {user}
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
'''.format(
freshclam_pid=TC.freshclam_pid,
path_db=TC.path_db,
port=TC.mock_mirror_port,
Tests: Enable freshclam tests to pass as root If you test in a docker container and don't create a user/switch to run as something other than root, than the tests will fail. Default freshclam behavior is to drop to run as the "clamav" user when run as root. This change sets the DatabaseOwner to the current user, so it the tests will pass when run as root.
2021-04-23 05:20:26 -07:00
user=getpass.getuser(),
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
))
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} --update-db=daily --daemon -F'.format(
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
output = self.execute_command(command)
assert output.ec == 17 # forbidden
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
expected_stderr = [
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
'FreshClam received error code 403',
'Forbidden',
]
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
# verify stderr
self.verify_output(output.err, expected=expected_stderr)
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
def test_freshclam_04_http_429(self):
self.step_name('Verify correct behavior when receiving 429 (too-many-requests)')
# Start our mock database mirror.
Tests: fix python warnings in freshclam tests The WebServerHandler classes used for the tests have a "Test_" prefix which causes pytest to print some warnings thinking they're special test classes. This commit gets rid of the warnings.
2021-05-11 15:36:22 -07:00
TC.mock_mirror = Process(target=mock_database_mirror, args=(WebServerHandler_04,TC.mock_mirror_port))
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
TC.mock_mirror.start()
if TC.freshclam_config.exists():
os.remove(str(TC.freshclam_config))
TC.freshclam_config.write_text('''
DatabaseMirror http://localhost:{port}
DNSDatabaseInfo no
PidFile {freshclam_pid}
LogVerbose yes
LogFileMaxSize 0
LogTime yes
DatabaseDirectory {path_db}
Tests: Enable freshclam tests to pass as root If you test in a docker container and don't create a user/switch to run as something other than root, than the tests will fail. Default freshclam behavior is to drop to run as the "clamav" user when run as root. This change sets the DatabaseOwner to the current user, so it the tests will pass when run as root.
2021-04-23 05:20:26 -07:00
DatabaseOwner {user}
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
'''.format(
freshclam_pid=TC.freshclam_pid,
path_db=TC.path_db,
port=TC.mock_mirror_port,
Tests: Enable freshclam tests to pass as root If you test in a docker container and don't create a user/switch to run as something other than root, than the tests will fail. Default freshclam behavior is to drop to run as the "clamav" user when run as root. This change sets the DatabaseOwner to the current user, so it the tests will pass when run as root.
2021-04-23 05:20:26 -07:00
user=getpass.getuser(),
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
))
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} --update-db=daily'.format(
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
output = self.execute_command(command)
assert output.ec == 0 # success
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
expected_stderr = [
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
'FreshClam received error code 429',
'You are on cool-down',
]
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
# verify stderr
self.verify_output(output.err, expected=expected_stderr)
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
def test_freshclam_05_cdiff_update(self):
self.step_name('Verify that freshclam can update from an older CVD to a newer with CDIFF patches')
# start with this CVD
Test: python 3.5 compatibility fix Python 3.5 compatibility fixes for Debian 9, etc that lack 3.6+. Change a python f-string to an old-style `"".format()`. Convert Path objects to strings for older `shutil` APIs that don't accept Paths.
2021-07-17 11:31:56 -07:00
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-1.cvd'), str(TC.path_db / 'test.cvd'))
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
# advertise this CVD (by sending the header response to Range requests)
Test: python 3.5 compatibility fix Python 3.5 compatibility fixes for Debian 9, etc that lack 3.6+. Change a python f-string to an old-style `"".format()`. Convert Path objects to strings for older `shutil` APIs that don't accept Paths.
2021-07-17 11:31:56 -07:00
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-6.cvd'), str(TC.path_www / 'test.cvd.advertised'))
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
# using these CDIFFs
Test: python 3.5 compatibility fix Python 3.5 compatibility fixes for Debian 9, etc that lack 3.6+. Change a python f-string to an old-style `"".format()`. Convert Path objects to strings for older `shutil` APIs that don't accept Paths.
2021-07-17 11:31:56 -07:00
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-2.cdiff'), str(TC.path_www))
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-3.cdiff'), str(TC.path_www))
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-4.cdiff'), str(TC.path_www))
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-5.cdiff'), str(TC.path_www))
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-6.cdiff'), str(TC.path_www))
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
handler = partial(WebServerHandler_WWW, TC.path_www)
TC.mock_mirror = Process(target=mock_database_mirror, args=(handler, TC.mock_mirror_port))
TC.mock_mirror.start()
if TC.freshclam_config.exists():
os.remove(str(TC.freshclam_config))
TC.freshclam_config.write_text('''
DatabaseMirror http://localhost:{port}
DNSDatabaseInfo no
PidFile {freshclam_pid}
LogVerbose yes
LogFileMaxSize 0
LogTime yes
DatabaseDirectory {path_db}
DatabaseOwner {user}
'''.format(
freshclam_pid=TC.freshclam_pid,
path_db=TC.path_db,
port=TC.mock_mirror_port,
user=getpass.getuser(),
))
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} --update-db=test'.format(
Test: CDIFF update with UNC paths (Windows) This is a regression test for https://github.com/Cisco-Talos/clamav/pull/226
2021-08-16 11:05:01 -07:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
output = self.execute_command(command)
assert output.ec == 0 # success
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
expected_stdout = [
Test: CDIFF update with UNC paths (Windows) This is a regression test for https://github.com/Cisco-Talos/clamav/pull/226
2021-08-16 11:05:01 -07:00
'test.cld updated',
]
unexpected_results = [
'already up-to-date'
]
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
# verify stdout
self.verify_output(output.out, expected=expected_stdout, unexpected=unexpected_results)
# verify stderr
self.verify_output(output.err, unexpected=unexpected_results)
Test: CDIFF update with UNC paths (Windows) This is a regression test for https://github.com/Cisco-Talos/clamav/pull/226
2021-08-16 11:05:01 -07:00
@unittest.skipIf(operating_system != 'windows', 'This test is specific to Windows.')
def test_freshclam_05_cdiff_update_UNC(self):
# This is a regression test for https://github.com/Cisco-Talos/clamav/pull/226
self.step_name('Verify that freshclam can update from an older CVD to a newer with CDIFF patches with UNC paths')
# start with this CVD
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-1.cvd'), str(TC.path_db / 'test.cvd'))
# advertise this CVD (by sending the header response to Range requests)
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-6.cvd'), str(TC.path_www / 'test.cvd.advertised'))
# using these CDIFFs
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-2.cdiff'), str(TC.path_www))
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-3.cdiff'), str(TC.path_www))
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-4.cdiff'), str(TC.path_www))
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-5.cdiff'), str(TC.path_www))
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-6.cdiff'), str(TC.path_www))
handler = partial(WebServerHandler_WWW, TC.path_www)
TC.mock_mirror = Process(target=mock_database_mirror, args=(handler, TC.mock_mirror_port))
TC.mock_mirror.start()
if TC.freshclam_config.exists():
os.remove(str(TC.freshclam_config))
path_db_unc = str(TC.path_db).replace('C:\\', '\\\\localhost\\c$\\').replace('D:\\', '\\\\localhost\\d$\\').replace('E:\\', '\\\\localhost\\e$\\')
TC.freshclam_config.write_text('''
DatabaseMirror http://localhost:{port}
DNSDatabaseInfo no
PidFile {freshclam_pid}
LogVerbose yes
LogFileMaxSize 0
LogTime yes
DatabaseDirectory {path_db}
DatabaseOwner {user}
'''.format(
freshclam_pid=TC.freshclam_pid,
path_db=path_db_unc,
port=TC.mock_mirror_port,
user=getpass.getuser(),
))
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} --update-db=test'.format(
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
output = self.execute_command(command)
assert output.ec == 0 # success
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
expected_stdout = [
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
'test.cld updated',
]
unexpected_results = [
'already up-to-date'
]
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
# verify stdout
self.verify_output(output.out, expected=expected_stdout, unexpected=unexpected_results)
# verify stderr
self.verify_output(output.err, unexpected=unexpected_results)
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
def test_freshclam_06_cdiff_partial_minus_1(self):
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
self.step_name('Verify that freshclam will accept a partial update with 1 missing cdiff')
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
# start with this CVD
Test: python 3.5 compatibility fix Python 3.5 compatibility fixes for Debian 9, etc that lack 3.6+. Change a python f-string to an old-style `"".format()`. Convert Path objects to strings for older `shutil` APIs that don't accept Paths.
2021-07-17 11:31:56 -07:00
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-3.cvd'), str(TC.path_db / 'test.cvd'))
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
# advertise this CVD (by sending the header response to Range requests)
Test: python 3.5 compatibility fix Python 3.5 compatibility fixes for Debian 9, etc that lack 3.6+. Change a python f-string to an old-style `"".format()`. Convert Path objects to strings for older `shutil` APIs that don't accept Paths.
2021-07-17 11:31:56 -07:00
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-6.cvd'), str(TC.path_www / 'test.cvd.advertised'))
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
# using these CDIFFs
Test: python 3.5 compatibility fix Python 3.5 compatibility fixes for Debian 9, etc that lack 3.6+. Change a python f-string to an old-style `"".format()`. Convert Path objects to strings for older `shutil` APIs that don't accept Paths.
2021-07-17 11:31:56 -07:00
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-4.cdiff'), str(TC.path_www))
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-5.cdiff'), str(TC.path_www))
#shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-6.cdiff'), str(TC.path_www)) # <-- don't give them the last CDIFF
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
handler = partial(WebServerHandler_WWW, TC.path_www)
TC.mock_mirror = Process(target=mock_database_mirror, args=(handler, TC.mock_mirror_port))
TC.mock_mirror.start()
if TC.freshclam_config.exists():
os.remove(str(TC.freshclam_config))
TC.freshclam_config.write_text('''
DatabaseMirror http://localhost:{port}
DNSDatabaseInfo no
PidFile {freshclam_pid}
LogVerbose yes
LogFileMaxSize 0
LogTime yes
DatabaseDirectory {path_db}
DatabaseOwner {user}
'''.format(
freshclam_pid=TC.freshclam_pid,
path_db=TC.path_db,
port=TC.mock_mirror_port,
user=getpass.getuser(),
))
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} --update-db=test'.format(
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
output = self.execute_command(command)
assert output.ec == 0 # success
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
expected_stdout = [
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
'Downloaded 2 patches for test, which is fewer than the 3 expected patches',
'We\'ll settle for this partial-update, at least for now',
'test.cld updated',
]
unexpected_results = [
'already up-to-date'
]
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
# verify stdout
self.verify_output(output.out, expected=expected_stdout, unexpected=unexpected_results)
# verify stderr
self.verify_output(output.err, unexpected=unexpected_results)
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
#
# Try again, we should be 1 behind which is tolerable and should not trigger a full CVD download
#
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} --update-db=test'.format(
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
output = self.execute_command(command)
assert output.ec == 0 # success
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
expected_stdout = [
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
'The database server doesn\'t have the latest patch',
'The server will likely have updated if you check again in a few hours',
]
unexpected_results = [
'test.cld updated',
'test.cvd updated',
]
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
# verify stdout
self.verify_output(output.out, expected=expected_stdout, unexpected=unexpected_results)
# verify stderr
self.verify_output(output.err, unexpected=unexpected_results)
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
def test_freshclam_07_cdiff_partial_minus_2(self):
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
self.step_name('Verify that freshclam behavior with 2 missing cdiffs')
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
# start with this CVD
Test: python 3.5 compatibility fix Python 3.5 compatibility fixes for Debian 9, etc that lack 3.6+. Change a python f-string to an old-style `"".format()`. Convert Path objects to strings for older `shutil` APIs that don't accept Paths.
2021-07-17 11:31:56 -07:00
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-3.cvd'), str(TC.path_db / 'test.cvd'))
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
# advertise this CVD (by sending the header response to Range requests)
Test: python 3.5 compatibility fix Python 3.5 compatibility fixes for Debian 9, etc that lack 3.6+. Change a python f-string to an old-style `"".format()`. Convert Path objects to strings for older `shutil` APIs that don't accept Paths.
2021-07-17 11:31:56 -07:00
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-6.cvd'), str(TC.path_www / 'test.cvd.advertised'))
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
# serve this CVD when requested instead of the advertised one
Test: python 3.5 compatibility fix Python 3.5 compatibility fixes for Debian 9, etc that lack 3.6+. Change a python f-string to an old-style `"".format()`. Convert Path objects to strings for older `shutil` APIs that don't accept Paths.
2021-07-17 11:31:56 -07:00
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-6.cvd'), str(TC.path_www / 'test.cvd.served'))
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
# using these CDIFFs
Test: python 3.5 compatibility fix Python 3.5 compatibility fixes for Debian 9, etc that lack 3.6+. Change a python f-string to an old-style `"".format()`. Convert Path objects to strings for older `shutil` APIs that don't accept Paths.
2021-07-17 11:31:56 -07:00
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-4.cdiff'), str(TC.path_www))
# shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-5.cdiff'), str(TC.path_www)) <--- dont' give them the second to last, either!
# shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-6.cdiff'), str(TC.path_www)) <--- don't give them the last CDIFF
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
handler = partial(WebServerHandler_WWW, TC.path_www)
TC.mock_mirror = Process(target=mock_database_mirror, args=(handler, TC.mock_mirror_port))
TC.mock_mirror.start()
if TC.freshclam_config.exists():
os.remove(str(TC.freshclam_config))
TC.freshclam_config.write_text('''
DatabaseMirror http://localhost:{port}
DNSDatabaseInfo no
PidFile {freshclam_pid}
LogVerbose yes
LogFileMaxSize 0
LogTime yes
DatabaseDirectory {path_db}
DatabaseOwner {user}
'''.format(
freshclam_pid=TC.freshclam_pid,
path_db=TC.path_db,
port=TC.mock_mirror_port,
user=getpass.getuser(),
))
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} --update-db=test'.format(
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
output = self.execute_command(command)
assert output.ec == 0 # success
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
expected_stdout = [
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
'Downloaded 1 patches for test, which is fewer than the 3 expected patches',
'We\'ll settle for this partial-update, at least for now',
'test.cld updated',
]
unexpected_results = [
'already up-to-date'
]
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
# verify stdout
self.verify_output(output.out, expected=expected_stdout, unexpected=unexpected_results)
# verify stderr
self.verify_output(output.err, unexpected=unexpected_results)
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
#
# Try again, we should be 2 behind which is NOT tolerable and SHOULD trigger a full CVD download
#
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} --update-db=test'.format(
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
output = self.execute_command(command)
assert output.ec == 0 # success
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
expected_stdout = ['test.cvd updated']
expected_stderr = ['Incremental update failed, trying to download test.cvd']
unexpected_results = ['test.cld updated']
# verify stdout
self.verify_output(output.out, expected=expected_stdout, unexpected=unexpected_results)
# verify stderr
self.verify_output(output.err, expected=expected_stderr, unexpected=unexpected_results)
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
def test_freshclam_07_no_cdiff_out_of_date_cvd(self):
self.step_name('Verify that freshclam will properly handle an out-of-date CVD update after a zero-byte CDIFF')
test: freshclam cdiff test tuning There is a bug where freshclam fails to detect if a downloaded CDIFF is empty. In 0.103 this, combined with a CDN caching issue could result in freshclam downloading a daily.cvd but failing to update, putting it in a sort of infinite loop. In 0.104 this issue manifests slightly differently, requiring freshclam to run up to 3x before you get over the empty-CVD hump and are back to normal updates. This commit updates an existing cdiff test with the zero-byte cdiff + an out-of-date CVD to confirm the bug. The following commit will fix it.
2021-09-27 19:00:14 -07:00
# start with CVD 1
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-1.cvd'), str(TC.path_db / 'test.cvd'))
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
test: freshclam cdiff test tuning There is a bug where freshclam fails to detect if a downloaded CDIFF is empty. In 0.103 this, combined with a CDN caching issue could result in freshclam downloading a daily.cvd but failing to update, putting it in a sort of infinite loop. In 0.104 this issue manifests slightly differently, requiring freshclam to run up to 3x before you get over the empty-CVD hump and are back to normal updates. This commit updates an existing cdiff test with the zero-byte cdiff + an out-of-date CVD to confirm the bug. The following commit will fix it.
2021-09-27 19:00:14 -07:00
# advertise CVD 6 (by sending the header response to Range requests)
Test: python 3.5 compatibility fix Python 3.5 compatibility fixes for Debian 9, etc that lack 3.6+. Change a python f-string to an old-style `"".format()`. Convert Path objects to strings for older `shutil` APIs that don't accept Paths.
2021-07-17 11:31:56 -07:00
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-6.cvd'), str(TC.path_www / 'test.cvd.advertised'))
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
test: freshclam cdiff test tuning There is a bug where freshclam fails to detect if a downloaded CDIFF is empty. In 0.103 this, combined with a CDN caching issue could result in freshclam downloading a daily.cvd but failing to update, putting it in a sort of infinite loop. In 0.104 this issue manifests slightly differently, requiring freshclam to run up to 3x before you get over the empty-CVD hump and are back to normal updates. This commit updates an existing cdiff test with the zero-byte cdiff + an out-of-date CVD to confirm the bug. The following commit will fix it.
2021-09-27 19:00:14 -07:00
# Serve the patches 2.
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-2.cdiff'), str(TC.path_www))
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
test: freshclam cdiff test tuning There is a bug where freshclam fails to detect if a downloaded CDIFF is empty. In 0.103 this, combined with a CDN caching issue could result in freshclam downloading a daily.cvd but failing to update, putting it in a sort of infinite loop. In 0.104 this issue manifests slightly differently, requiring freshclam to run up to 3x before you get over the empty-CVD hump and are back to normal updates. This commit updates an existing cdiff test with the zero-byte cdiff + an out-of-date CVD to confirm the bug. The following commit will fix it.
2021-09-27 19:00:14 -07:00
# Serve a zero-byte file instead of test-3.cdiff. This should trigger a whole CVD download.
with (TC.path_www / 'test-3.cdiff').open('w') as fp:
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
pass
test: freshclam cdiff test tuning There is a bug where freshclam fails to detect if a downloaded CDIFF is empty. In 0.103 this, combined with a CDN caching issue could result in freshclam downloading a daily.cvd but failing to update, putting it in a sort of infinite loop. In 0.104 this issue manifests slightly differently, requiring freshclam to run up to 3x before you get over the empty-CVD hump and are back to normal updates. This commit updates an existing cdiff test with the zero-byte cdiff + an out-of-date CVD to confirm the bug. The following commit will fix it.
2021-09-27 19:00:14 -07:00
# Serve CVD 5 when test.cvd is requested instead of 6 (the advertised one). This should trigger an incremental update, starting a test-4.cvd + patches 5-6.
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-5.cvd'), str(TC.path_www / 'test.cvd.served'))
# Serve the patches 5 - 6. Patch 4 should never be requested.
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-5.cdiff'), str(TC.path_www))
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-6.cdiff'), str(TC.path_www))
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
handler = partial(WebServerHandler_WWW, TC.path_www)
TC.mock_mirror = Process(target=mock_database_mirror, args=(handler, TC.mock_mirror_port))
TC.mock_mirror.start()
if TC.freshclam_config.exists():
os.remove(str(TC.freshclam_config))
TC.freshclam_config.write_text('''
DatabaseMirror http://localhost:{port}
DNSDatabaseInfo no
PidFile {freshclam_pid}
LogVerbose yes
LogFileMaxSize 0
LogTime yes
DatabaseDirectory {path_db}
DatabaseOwner {user}
'''.format(
freshclam_pid=TC.freshclam_pid,
path_db=TC.path_db,
port=TC.mock_mirror_port,
user=getpass.getuser(),
))
test: freshclam cdiff test tuning There is a bug where freshclam fails to detect if a downloaded CDIFF is empty. In 0.103 this, combined with a CDN caching issue could result in freshclam downloading a daily.cvd but failing to update, putting it in a sort of infinite loop. In 0.104 this issue manifests slightly differently, requiring freshclam to run up to 3x before you get over the empty-CVD hump and are back to normal updates. This commit updates an existing cdiff test with the zero-byte cdiff + an out-of-date CVD to confirm the bug. The following commit will fix it.
2021-09-27 19:00:14 -07:00
#
# 1st attempt
#
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} --update-db=test'.format(
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
output = self.execute_command(command)
assert output.ec == 0 # success
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
expected_stdout = ['test.cvd updated \\(version: 5']
expected_stderr = ['Received an older test CVD than was advertised. Incremental updates either failed or ']
unexpected_results = ['already up-to-date']
# verify stdout
self.verify_output(output.out, expected=expected_stdout, unexpected=unexpected_results)
# verify stderr
self.verify_output(output.err, expected=expected_stderr, unexpected=unexpected_results)
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
test: freshclam cdiff test tuning There is a bug where freshclam fails to detect if a downloaded CDIFF is empty. In 0.103 this, combined with a CDN caching issue could result in freshclam downloading a daily.cvd but failing to update, putting it in a sort of infinite loop. In 0.104 this issue manifests slightly differently, requiring freshclam to run up to 3x before you get over the empty-CVD hump and are back to normal updates. This commit updates an existing cdiff test with the zero-byte cdiff + an out-of-date CVD to confirm the bug. The following commit will fix it.
2021-09-27 19:00:14 -07:00
#
# 2nd attempt
#
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} --update-db=test'.format(
test: freshclam cdiff test tuning There is a bug where freshclam fails to detect if a downloaded CDIFF is empty. In 0.103 this, combined with a CDN caching issue could result in freshclam downloading a daily.cvd but failing to update, putting it in a sort of infinite loop. In 0.104 this issue manifests slightly differently, requiring freshclam to run up to 3x before you get over the empty-CVD hump and are back to normal updates. This commit updates an existing cdiff test with the zero-byte cdiff + an out-of-date CVD to confirm the bug. The following commit will fix it.
2021-09-27 19:00:14 -07:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
output = self.execute_command(command)
assert output.ec == 0 # success
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
expected_stdout = [
test: freshclam cdiff test tuning There is a bug where freshclam fails to detect if a downloaded CDIFF is empty. In 0.103 this, combined with a CDN caching issue could result in freshclam downloading a daily.cvd but failing to update, putting it in a sort of infinite loop. In 0.104 this issue manifests slightly differently, requiring freshclam to run up to 3x before you get over the empty-CVD hump and are back to normal updates. This commit updates an existing cdiff test with the zero-byte cdiff + an out-of-date CVD to confirm the bug. The following commit will fix it.
2021-09-27 19:00:14 -07:00
'test.cld updated \\(version: 6',
]
unexpected_results = [
'already up-to-date',
'Received an older test CVD than was advertised',
'cdiff_apply: lseek\\(desc, -350, SEEK_END\\) failed',
'Incremental update failed, trying to download test.cvd',
]
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
# verify stdout
self.verify_output(output.out, expected=expected_stdout, unexpected=unexpected_results)
# verify stderr
self.verify_output(output.err, unexpected=unexpected_results)
test: freshclam cdiff test tuning There is a bug where freshclam fails to detect if a downloaded CDIFF is empty. In 0.103 this, combined with a CDN caching issue could result in freshclam downloading a daily.cvd but failing to update, putting it in a sort of infinite loop. In 0.104 this issue manifests slightly differently, requiring freshclam to run up to 3x before you get over the empty-CVD hump and are back to normal updates. This commit updates an existing cdiff test with the zero-byte cdiff + an out-of-date CVD to confirm the bug. The following commit will fix it.
2021-09-27 19:00:14 -07:00
#
# 3rd attempt
#
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} --update-db=test'.format(
test: freshclam cdiff test tuning There is a bug where freshclam fails to detect if a downloaded CDIFF is empty. In 0.103 this, combined with a CDN caching issue could result in freshclam downloading a daily.cvd but failing to update, putting it in a sort of infinite loop. In 0.104 this issue manifests slightly differently, requiring freshclam to run up to 3x before you get over the empty-CVD hump and are back to normal updates. This commit updates an existing cdiff test with the zero-byte cdiff + an out-of-date CVD to confirm the bug. The following commit will fix it.
2021-09-27 19:00:14 -07:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
output = self.execute_command(command)
assert output.ec == 0 # success
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
expected_stdout = [
test: freshclam cdiff test tuning There is a bug where freshclam fails to detect if a downloaded CDIFF is empty. In 0.103 this, combined with a CDN caching issue could result in freshclam downloading a daily.cvd but failing to update, putting it in a sort of infinite loop. In 0.104 this issue manifests slightly differently, requiring freshclam to run up to 3x before you get over the empty-CVD hump and are back to normal updates. This commit updates an existing cdiff test with the zero-byte cdiff + an out-of-date CVD to confirm the bug. The following commit will fix it.
2021-09-27 19:00:14 -07:00
'already up-to-date',
]
unexpected_results = [
'test.cld updated \\(version: 6',
'Received an older test CVD than was advertised',
'cdiff_apply: lseek\\(desc, -350, SEEK_END\\) failed',
'Incremental update failed, trying to download test.cvd',
]
Correct stdout/stderr expectations for freshclam tests Improvements to the logg() API appear to have corrected a bug wherein debug/warning/error messages were writing to stdout instead of stderr. Now that they're going to stderr correctly, this fixes the test to match.
2022-02-19 14:34:37 -08:00
# verify stdout
self.verify_output(output.out, expected=expected_stdout, unexpected=unexpected_results)
# verify stderr
self.verify_output(output.err, unexpected=unexpected_results)
test: freshclam cdiff test tuning There is a bug where freshclam fails to detect if a downloaded CDIFF is empty. In 0.103 this, combined with a CDN caching issue could result in freshclam downloading a daily.cvd but failing to update, putting it in a sort of infinite loop. In 0.104 this issue manifests slightly differently, requiring freshclam to run up to 3x before you get over the empty-CVD hump and are back to normal updates. This commit updates an existing cdiff test with the zero-byte cdiff + an out-of-date CVD to confirm the bug. The following commit will fix it.
2021-09-27 19:00:14 -07:00
Test: Add test for incremental update twice in a row This is a regression test to ensure that freshclam can update from a CVD to a CLD, and then update from the CLD to a (newer) CLD.
2022-11-01 15:45:52 -07:00
def test_freshclam_08_cdiff_update_twice(self):
self.step_name('Verify that freshclam can update from an older CVD to a newer with CDIFF patches, and then update from an older CLD to the latest with more CDIFF patches')
# start with this CVD
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-1.cvd'), str(TC.path_db / 'test.cvd'))
# advertise this CVD FIRST (by sending the header response to Range requests)
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-3.cvd'), str(TC.path_www / 'test.cvd.advertised'))
# using these CDIFFs
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-2.cdiff'), str(TC.path_www))
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-3.cdiff'), str(TC.path_www))
handler = partial(WebServerHandler_WWW, TC.path_www)
TC.mock_mirror = Process(target=mock_database_mirror, args=(handler, TC.mock_mirror_port))
TC.mock_mirror.start()
if TC.freshclam_config.exists():
os.remove(str(TC.freshclam_config))
TC.freshclam_config.write_text('''
DatabaseMirror http://localhost:{port}
DNSDatabaseInfo no
PidFile {freshclam_pid}
LogVerbose yes
LogFileMaxSize 0
LogTime yes
DatabaseDirectory {path_db}
DatabaseOwner {user}
'''.format(
freshclam_pid=TC.freshclam_pid,
path_db=TC.path_db,
port=TC.mock_mirror_port,
user=getpass.getuser(),
))
#
# Now run the update for the first set up updates.
#
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} --update-db=test'.format(
Test: Add test for incremental update twice in a row This is a regression test to ensure that freshclam can update from a CVD to a CLD, and then update from the CLD to a (newer) CLD.
2022-11-01 15:45:52 -07:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
output = self.execute_command(command)
assert output.ec == 0 # success
expected_stdout = [
'test.cld updated',
]
unexpected_results = [
'already up-to-date'
]
# verify stdout
self.verify_output(output.out, expected=expected_stdout, unexpected=unexpected_results)
# verify stderr
self.verify_output(output.err, unexpected=unexpected_results)
# advertise this newer CVD SECOND (by sending the header response to Range requests)
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-6.cvd'), str(TC.path_www / 'test.cvd.advertised'))
# using these CDIFFs
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-4.cdiff'), str(TC.path_www))
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-5.cdiff'), str(TC.path_www))
shutil.copy(str(TC.path_source / 'unit_tests' / 'input' / 'freshclam_testfiles' /'test-6.cdiff'), str(TC.path_www))
#
# Now re-run the update for more updates.
#
Tests: Add sigtool tests for correct CDIFF UNLINK functionality Also includes: - A sigtool test to verify that Rust log macros are working. - Changing the freshclam tests to use --no-dns so they run faster when DNS isn't working (e.g. no internet).
2023-04-12 22:42:31 -07:00
command = '{valgrind} {valgrind_args} {freshclam} --no-dns --config-file={freshclam_config} --update-db=test'.format(
Test: Add test for incremental update twice in a row This is a regression test to ensure that freshclam can update from a CVD to a CLD, and then update from the CLD to a (newer) CLD.
2022-11-01 15:45:52 -07:00
valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, freshclam=TC.freshclam, freshclam_config=TC.freshclam_config
)
output = self.execute_command(command)
assert output.ec == 0 # success
expected_stdout = [
'test.cld updated',
]
unexpected_results = [
'already up-to-date'
]
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
def mock_database_mirror(handler, port=8001):
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
'''
Process entry point for our HTTP Server to mock a database mirror.
'''
try:
server = HTTPServer(('', port), handler)
print("Web server is running on port {}".format(port))
server.serve_forever()
except KeyboardInterrupt:
print("^C entered, stopping web server...")
server.socket.close()
Tests: fix python warnings in freshclam tests The WebServerHandler classes used for the tests have a "Test_" prefix which causes pytest to print some warnings thinking they're special test classes. This commit gets rid of the warnings.
2021-05-11 15:36:22 -07:00
class WebServerHandler_02(BaseHTTPRequestHandler):
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
'''
Web server handler to send 403 (Forbidden) if a whole file is requested.
Will send a CVD header if a Range-requeset is received.
'''
def do_GET(self):
if 'Range' in self.headers:
# HACK: This will send a CVD header so FreshClam thinks there is an update.
# This is needed so we can operate with `DNSDatabaseInfo no` in case
# someone wants to run these tests without internet access.
self.send_response(206) # Partial file
self.send_header('Content-type', 'application/octet-stream')
page =b'ClamAV-VDB:21 Sep 2020 09-52 -0400:25934:4320797:63:2ee5a3e4285b496656117ae3809b6040:gMj7NXhxfew0+bToOF8GX7xPHPGXhOSD+CSuf3E7SHhLmVZCJUVhPS01h42I0W1py7L+BmM2yhPIW8t/oGPFw8+hdD4DU/ceET15wnPWU4lsJJeRkl46Z4D8INe9Oq36ixT1xEIkERogPE3qr6wszmjT2Xe2VcmydTXN2GfPQX:raynman:1600696324 '
else:
# Send the 403 FORBIDDEN header.
self.send_response(403) # Forbidden (blocked)
self.send_header('Content-type', 'text/html')
page= b'''<html><body>
No CVD for you!
</body></html>'''
Tests: Hoist repeated code outside if statement
2021-11-22 11:25:18 +08:00
self.end_headers()
self.wfile.write(page)
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
Tests: fix python warnings in freshclam tests The WebServerHandler classes used for the tests have a "Test_" prefix which causes pytest to print some warnings thinking they're special test classes. This commit gets rid of the warnings.
2021-05-11 15:36:22 -07:00
class WebServerHandler_04(BaseHTTPRequestHandler):
Test: freshclam 429 and 403 behavior When running ctest, Windows users may be prompted to allow the freshclam test to access the network, so that it can host the mock database mirror.
2021-03-22 16:40:47 -07:00
'''
Web server handler to send 429 (Too-Many-Requests) if a whole file is requested.
Will send a CVD header if a Range-requeset is received.
'''
def do_GET(self):
if 'Range' in self.headers:
# HACK: This will send a CVD header so FreshClam thinks there is an update.
# This is needed so we can operate with `DNSDatabaseInfo no` in case
# someone wants to run these tests without internet access.
self.send_response(206) # Partial file
self.send_header('Content-type', 'application/octet-stream')
page =b'ClamAV-VDB:21 Sep 2020 09-52 -0400:25934:4320797:63:2ee5a3e4285b496656117ae3809b6040:gMj7NXhxfew0+bToOF8GX7xPHPGXhOSD+CSuf3E7SHhLmVZCJUVhPS01h42I0W1py7L+BmM2yhPIW8t/oGPFw8+hdD4DU/ceET15wnPWU4lsJJeRkl46Z4D8INe9Oq36ixT1xEIkERogPE3qr6wszmjT2Xe2VcmydTXN2GfPQX:raynman:1600696324 '
else:
# Send the 429 Too-Many-Requests header.
self.send_response(429) # Too-Many-Requests (rate limiting)
self.send_header('Content-type', 'text/html')
self.send_header('Retry-After', '60') # Try again in a minute ;-)!
page= b'''<html><body>
Retry later please!
</body></html>'''
Tests: Hoist repeated code outside if statement
2021-11-22 11:25:18 +08:00
self.end_headers()
self.wfile.write(page)
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
class WebServerHandler_WWW(BaseHTTPRequestHandler):
'''
Make an HTTP server handler that has a configurable directory for hosting files.
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
Server handler to send a CVD header of `test.cvd.advertised` indicating an update is available,
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
and then to serve up CDIFFs that should allow the test to do an incremental update.
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
If `test.cvd` is requested, it will serve up `test.cvd.served` (not `test.cvd.advertised`)
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
'''
def __init__(self, path_www, *args, **kwargs):
self.path_www = path_www
super().__init__(*args, **kwargs)
def do_GET(self):
requested_file = self.path_www / self.path.lstrip('/')
print("Mock Server: Test requested: {}".format(requested_file))
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
if 'Range' in self.headers:
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
# This will send a CVD header so FreshClam thinks there is an update.
(range_start, range_end) = self.headers['Range'].split('=')[-1].split('-')
print("Mock Server: But they only want bytes {} through {} ...".format(range_start, range_end))
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
if requested_file.name.endswith('.cvd'):
Test: python 3.5 compatibility fix Python 3.5 compatibility fixes for Debian 9, etc that lack 3.6+. Change a python f-string to an old-style `"".format()`. Convert Path objects to strings for older `shutil` APIs that don't accept Paths.
2021-07-17 11:31:56 -07:00
response_file = requested_file.parent / '{}.advertised'.format(requested_file)
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
else:
response_file = requested_file
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
if not response_file.exists():
self.send_error(404, "{} Not Found".format(self.path.lstrip('/')))
else:
with response_file.open('rb') as the_file:
self.send_response(206) # Partial file
self.send_header('Content-type', 'application/octet-stream')
self.end_headers()
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
the_file.seek(int(range_start))
page = the_file.read(int(range_end) - int(range_start) + 1)
bytes_written = self.wfile.write(page)
print("Mock Server: Sending {} bytes back to client.".format(bytes_written))
Test: Basic freshclam CDIFF tests Adds 3 tests to validate that: 1. a CDIFF update works 2. a CDIFF partial update (with 1 missing CDIFF) works and that a subsequent update is ok with being 1 behind 3. a CDIFF partial update (with 2 missing CDIFFs) works and that a subsequent update will try to get the WHOLE CVD - because being 2+ CDIFFs behind without any update isn't good enough. Also fixed a minor bug so that the database name is properly displayed when a partial update occurs instead of displaying "(null)". Also changed the freshclam test port to 8001 to deconflict with CVD-Update, in case that's running in the background. TODO: Make the tests smarter so they find an open port instead of hoping that 8001 is available.
2021-05-26 22:37:32 -07:00
else:
# Send back some whole files
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
if requested_file.name.endswith('.cvd'):
Test: python 3.5 compatibility fix Python 3.5 compatibility fixes for Debian 9, etc that lack 3.6+. Change a python f-string to an old-style `"".format()`. Convert Path objects to strings for older `shutil` APIs that don't accept Paths.
2021-07-17 11:31:56 -07:00
response_file = requested_file.parent / '{}.served'.format(requested_file)
Test: freshclam w/ zero-byte cdiff & cvd out-of-date Add a test where freshclam received a zero-byte cdiff to trigger a whole CVD database download, and the CVD served is older than advertised. This is a regression test for a bug found & fixed by Andrew Williams.
2021-06-23 13:24:53 -07:00
else:
response_file = requested_file
if not response_file.exists():
self.send_error(404, "{} Not Found".format(self.path.lstrip('/')))
else:
with response_file.open('rb') as the_file:
self.send_response(200) # Partial file
self.send_header('Content-type', 'application/octet-stream')
self.end_headers()
page = the_file.read()
bytes_written = self.wfile.write(page)
print("Mock Server: Sending {} bytes back to client.".format(bytes_written))
Reference in a new issue Copy permalink