E.g. this will also be valid:
```freshclam.conf
DatabaseMirror http://localhost:8000<tab># My private server.
```
Also:
- Fix compile warning regarding unnecessarily const string variable.
- Add parenthesis to resolve inconsistency between clang-format and
vscode auto-format.
- Add a macro defining the max config line length.
The config parser will now treat a '#' character in a config file as an
inline comment, allowing users to write configs like this:
```freshclam.conf
DatabaseMirror http://localhost:8000 # My private server
```
The .sign test files have the min flevel set to 220.
It should be 230.
Also upgrade clamav-signature-util to v1.2.4 for fix so new .sign
files will have the correct min flevel.
The logic for loading an icon matcher assumes that only one .idb file is
loaded. If a second is loaded, the first is forgotten (memory leak).
This commit checks to see if `engine->iconcheck` is already allocated
and if so it will use that instead of allocating a new one.
I also cleaned up the error handling in this function, using goto-done
error handling.
I added proper cleanup for freeing the matcher in case of an idb
signature load error, copied from `cl_engine_free()`.
We observed build failures on Ubuntu 20.04 ARM64 because the Rust code
saw extra OpenSSL dependencies in the OPENSSL_LIBS environment variable
and was confused.
This change switches from using OPENSSL_LIBRARIES, which may have extra
dependencies for libcrypto/libssl, to only use OPENSSL_CRYPTO_LIBRARY
and OPENSSL_SSL_LIBRARY.
The memory allocated for certs_directory in the ClamAV engine is not
free'd when the engine is free'd. This isn't readily apparent when using
a mempool because the mempool itself is free'd and the issue is masked.
Also: I found that the location in `cl_engine_free()` where we estimate
the number of tasks incorrectly placed the fuzzy_hashmap free task in
the block where we free the `test_root`, rather than up in the
`engine->root` for-loop. Fixed.
We were signing with the signing key + signing cert and verifying
with the intermediate cert + root cert. However, we should have been
signing with the signing key + signing cert + intermediate cert, and
verifying with just the root cert.
To fix this, I...
1. Provided new certs and test file .sign files to use the correct
signing method.
2. Restructured the `unit_tests/input/signing` directory to highlight
which files are for signing and which are for verification.
There is a multi-arch build issue because I previously used i8 to
represent a C character. I switched it to c_char, which should fix the
clamav-debian multi-arch Docker image build.
It turns out we weren't failing out when signing if one of the provided
intermediate certificate paths is incorrect. Instead of using
`filter_map()`, I switched to just iterate the list to populate the
vector of intermediate certs.
This is just preliminary support for identifying an assortment of
different AI model files.
So far, this detects the following types:
- GGML GGUF (.gguf)
- ONNX AI (.onnx)
- TensorFlow Lite (.tflite)
Additional types to consider:
- SafeTensors (.safetensors)
- TensorFlow (.pb, .ckpt, .tfrecords)
- Keras (.keras)
- pickle (.pkl)
- numpy (.npy, .npz)
- coreml (.coreml)
- PyTorch (.pt, .pth, .bin, .mar, .pte, .pt2, .ptl)
Outside of being able to differentiate by file type, the scanner
will treat CL_TYPE_AI_MODEL the same as CL_TYPE_BINARY_DATA.
We're not adding parsers to further process these files, for now.
Add X509 certificate chain based signing with PKCS7-PEM external
signatures distributed alongside CVD's in a custom .cvd.sign format.
This new signing and verification mechanism is primarily in support
of FIPS compliance.
Fixes: https://github.com/Cisco-Talos/clamav/issues/564
Add a Rust implementation for parsing, verifying, and unpacking CVD
files.
Now installs a 'certs' directory in the app config directory
(e.g. <prefix>/etc/certs). The install location is configurable.
The CMake option to configure the CVD certs directory is:
`-D CVD_CERTS_DIRECTORY=PATH`
New options to set an alternative CVD certs directory:
- Commandline for freshclam, clamd, clamscan, and sigtool is:
`--cvdcertsdir PATH`
- Env variable for freshclam, clamd, clamscan, and sigtool is:
`CVD_CERTS_DIR`
- Config option for freshclam and clamd is:
`CVDCertsDirectory PATH`
Sigtool:
- Add sign/verify commands.
- Also verify CDIFF external digital signatures when applying CDIFFs.
- Place commonly used commands at the top of --help string.
- Fix up manpage.
Freshclam:
- Will try to download .sign files to verify CVDs and CDIFFs.
- Fix an issue where making a CLD would only include the CFG file for
daily and not if patching any other database.
libclamav.so:
- Bump version to 13:0:1 (aka 12.1.0).
- Also remove libclamav.map versioning.
Resolves: https://github.com/Cisco-Talos/clamav/issues/1304
- Add two new API's to the public clamav.h header:
```c
extern cl_error_t cl_cvdverify_ex(const char *file,
const char *certs_directory);
extern cl_error_t cl_cvdunpack_ex(const char *file,
const char *dir,
bool dont_verify,
const char *certs_directory);
```
The original `cl_cvdverify` and `cl_cvdunpack` are deprecated.
- Add `cl_engine_field` enum option `CL_ENGINE_CVDCERTSDIR`.
You may set this option with `cl_engine_set_str` and get it
with `cl_engine_get_str`, to override the compiled in default
CVD certs directory.
libfreshclam.so: Bump version to 4:0:0 (aka 4.0.0).
Add sigtool sign/verify tests and test certs.
Make it so downloadFile doesn't throw a warning if the server
doesn't have the .sign file.
Replace use of md5-based FP signatures in the unit tests with
sha256-based FP signatures because the md5 implementation used
by Python may be disabled in FIPS mode.
Fixes: https://github.com/Cisco-Talos/clamav/issues/1411
CMake: Add logic to enable the Rust openssl-sys / openssl-rs crates
to build against the same OpenSSL library as is used for the C build.
The Rust unit test application must also link directly with libcrypto
and libssl.
Fix some log messages with missing new lines.
Fix missing environment variable notes in --help messages and manpages.
Deconflict CONFDIR/DATADIR/CERTSDIR variable names that are defined in
clamav-config.h.in for libclamav from variable that had the same name
for use in clamav applications that use the optparser.
The 'clamav-test' certs for the unit tests will live for 10 years.
The 'clamav-beta.crt' public cert will only live for 120 days and will
be replaced before the stable release with a production 'clamav.crt'.
Fixes:
- We need to look at the local headers if no central directory headers are
found. Restructured the main `cli_unzip()` function to allocate an empty
zip catalogue when we can't use a central directory at all.
- In `index_local_file_headers_within_bounds()`, we must decrement the
`coff` variable after adding the size of a file entry using
`parse_local_file_header()`, to account for the increment when it loops
around. If we don't, the next entry won't be at 'PK\x03\x04', it will be
at 'K\0x03\x04'.
- Attempt to unzip when encrypted if we don't have a valid password.
This may enable extraction for files where a header lies about encryption.
- The `fmap_need_off()` call to get the `compressed_data` pointer used the
wrong size, checking if there was enough data for a header instead of
for the compressed data that follows the header. I stumbled across this
older bug when testing extraction of a zip where the file entries are
tiny and I'd stripped off the central directory. As a result, there
wasn't enough data for a whole file header and my test failed.
Cleanup:
- Initialize status variables as CL_ERROR and only assign to CL_SUCCESS if
successful. This is to protect against future changes in case someone
accidentally goes-to-done without setting the status.
- Remove legacy use of CL_CLEAN. Not a functional change.
This mostly a stylistic preference.
- Use calloc instead of malloc + memset in a couple places.
Make use of the new allocation macros with goto-done error handling.
- Some opinionated format changes such as shifting some longer function
arguments all to a new line so they're no so far to the right.
- Auto-format with clang-format.
When the --move or --remove options are used, ClamAV carefully traverses
the file path one layer at a time so as to avoid following a directory
that is a symlink or reparse point.
We do this for directories, but could also do it for files.
Only an admin should be able to create a reparse point for a file,
but it is better to be consistent.
Thank you to Maxim Suhanov for reporting this issue.
The ClamAV inflate64 module is based on zlib 1.2.3 source code with
significant changes to support extracting zip64 and some addressing
code quality issues.
This commit adds a zlib v1.2.9 fix for possible undefined behavior:
6a043145ca
Thank you to TITAN Team for reporting this issue.
ClamD's STATS API reports process memory stats on systems that
provide the `mallinfo()` system call.
This feature is used by ClamDTOP to show process memory usage.
When we switched to the CMake build system, we neglected to add the
check for the `mallinfo()` system call and so broke ClamD memory
usage reporting.
This commit adds the CMake check for `mallinfo()` and sets
HAVE_MALLINFO, if found.
Fixes: https://github.com/Cisco-Talos/clamav/issues/706
Jira: CLAM-2742
The bounds check for the loop iterating an OLE2 block during decryption
may have an integer unerflow if the `leftover + bytesToWrite` is less
than 16. That results in a significant buffer over read and a segfault.
The fix is simply to do addition on the left side of the check instead
of subtraction on the right.
Fixes https://issues.oss-fuzz.com/issues/372544101
At install, the CMake build may fail if it detects the same library
dependency in two locations. This happened for us with the following
error:
CMake Error at libfreshclam/cmake_install.cmake:157 (file):
file Multiple conflicting paths found for libcrypto-3-x64.dll:
C:/Users/clamav_jenkins_svc.TALOS/clam_dependencies/x64/lib/libcrypto-3-x64.dll
C:/WINDOWS/system32/libcrypto-3-x64.dll
C:\WINDOWS\system32/libcrypto-3-x64.dll
Call Stack (most recent call first):
cmake_install.cmake:96 (include)
This happens when system provided DLL names match exactly with the ones
we provide. ClamAV woudld't prefer that DLL at load time, because it
looks in the EXE directory first. But it does confuse the `file()`
command used to locate build dependencies.
The fix in this commit uses a regex to exclude all libraries found under
C:\Windows
