Stowage/clamav
2
0
Fork 0
mirror of https://github.com/Cisco-Talos/clamav.git synced 2025-10-19 10:23:17 +00:00
Code Issues Projects Releases Packages Wiki Activity
clamav/libfreshclam/libfreshclam_internal.h

103 lines
3 KiB
C
Raw Permalink Normal View History

Freshclam refresh. This update makes libcurl a hard requirement for ClamAV. New features added to freshclam: - Update signature definitions over HTTPS. - Support for HTTP protocol v1.1 (formerly v1.0). - New libfreshclam library with an all new API and versioning separate from libclamav (v2.0.0). This library is now build and installed alongside libclamav as a hard dependency of freshclam. - The ability to opt-in and opt-out of standard and optional official ClamAV databases (ExtraDatabase, ExcludeDatabase) - The option to specify the protocol and port number of official and private mirror servers. - Support for additional types of proxy servers beyond plain HTTP (SOCKS 4, SOCKS 5). Features removed from freshclam: - Mirror management (mirrors.dat) file. This feature is no longer needed as official signature databases are distributed using a paid content delivery network (Cloudflare). This commit also adds the following features for Windows users: - The clamsubmit tool. - The json-c library dependency, which will enable the --gen-json option in clamscan. - Third party libraries under the win32/3rdparty directory have been removed. Developers will need to build the libraries separately from ClamAV and provide the headers and lib/dll library files the same way they do for OpenSSL. This includes libxml2, pthread-win32, bzip2, zlib, pcre2 as well as new dependencies: curl, json-c. Developers are encouraged to use the build tool Mussels to simplify this task.
2019-03-26 15:09:52 -04:00
/*
Bump copyright dates for 2025
2025-02-14 10:24:30 -05:00
* Copyright (C) 2013-2025 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
Freshclam refresh. This update makes libcurl a hard requirement for ClamAV. New features added to freshclam: - Update signature definitions over HTTPS. - Support for HTTP protocol v1.1 (formerly v1.0). - New libfreshclam library with an all new API and versioning separate from libclamav (v2.0.0). This library is now build and installed alongside libclamav as a hard dependency of freshclam. - The ability to opt-in and opt-out of standard and optional official ClamAV databases (ExtraDatabase, ExcludeDatabase) - The option to specify the protocol and port number of official and private mirror servers. - Support for additional types of proxy servers beyond plain HTTP (SOCKS 4, SOCKS 5). Features removed from freshclam: - Mirror management (mirrors.dat) file. This feature is no longer needed as official signature databases are distributed using a paid content delivery network (Cloudflare). This commit also adds the following features for Windows users: - The clamsubmit tool. - The json-c library dependency, which will enable the --gen-json option in clamscan. - Third party libraries under the win32/3rdparty directory have been removed. Developers will need to build the libraries separately from ClamAV and provide the headers and lib/dll library files the same way they do for OpenSSL. This includes libxml2, pthread-win32, bzip2, zlib, pcre2 as well as new dependencies: curl, json-c. Developers are encouraged to use the build tool Mussels to simplify this task.
2019-03-26 15:09:52 -04:00
* Copyright (C) 2007-2013 Sourcefire, Inc.
* Copyright (C) 2002-2007 Tomasz Kojm <tkojm@clamav.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
* MA 02110-1301, USA.
*/
#ifndef __LIBFRESHCLAM_INTERNAL_H
#define __LIBFRESHCLAM_INTERNAL_H
#include "clamav-types.h"
// clang-format off
#define DNS_UPDATEINFO_NEWVERSION 0
#define DNS_UPDATEINFO_RECORDTIME 3
#define DNS_UPDATEINFO_VERSIONWARNING 4
#define DNS_UPDATEINFO_REMOTEFLEVEL 5
#define DNS_EXTRADBINFO_RECORDTIME 1
// clang-format on
FreshClam: mirrors.dat, User-Agent UUID, cool-down Add back the mirrors.dat file to the database directory. This new version of mirros.dat will store: - A randomly generated UUID for the FreshClam User-Agent. - A retry-after timestamp that so FreshClam won't try to update after having received an HTTP 429 response until the Retry-After timeout has expired. Also: FreshClam will now exit with a failure in daemon mode if an HTTP 403 (Forbidden) was received, because retrying later won't help any. The FreshClam user will have to take actions to get unblocked.
2021-03-21 19:47:21 -07:00
#define SIZEOF_UUID_V4 37 /** For uuid_v4_gen(), includes NULL byte */
FreshClam: rename mirrors.dat to freshclam.dat Some users have scripts set up from long ago to delete mirrors.dat if FreshClam failed. We used to recommend this if people had technical issues because mirrors.dat would store a bunch of entries indicating that all of their regional mirrors were failing and then FreshClam would give up. The new freshclam DAT file no longer stores that kind of information. Deleting the DAT file is no longer sound advice. We very much want the UUID, which is generated when creating the DAT file, to persist between runs. So unless people go and change the scripts to delete freshclam.dat instead, this commit should resolve the concern.
2021-05-24 18:37:33 -07:00
#define MIRRORS_DAT_MAGIC "FreshClamData" /** Magic bytes for freshclam.dat found before freshclam_dat_v1_t */
typedef struct _freshclam_dat_v1 {
FreshClam: mirrors.dat, User-Agent UUID, cool-down Add back the mirrors.dat file to the database directory. This new version of mirros.dat will store: - A randomly generated UUID for the FreshClam User-Agent. - A retry-after timestamp that so FreshClam won't try to update after having received an HTTP 429 response until the Retry-After timeout has expired. Also: FreshClam will now exit with a failure in daemon mode if an HTTP 403 (Forbidden) was received, because retrying later won't help any. The FreshClam user will have to take actions to get unblocked.
2021-03-21 19:47:21 -07:00
uint32_t version; /** version of this dat format */
char uuid[SIZEOF_UUID_V4]; /** uuid to be used in user-agent */
time_t retry_after; /** retry date. If > 0, don't update until after this date */
FreshClam: rename mirrors.dat to freshclam.dat Some users have scripts set up from long ago to delete mirrors.dat if FreshClam failed. We used to recommend this if people had technical issues because mirrors.dat would store a bunch of entries indicating that all of their regional mirrors were failing and then FreshClam would give up. The new freshclam DAT file no longer stores that kind of information. Deleting the DAT file is no longer sound advice. We very much want the UUID, which is generated when creating the DAT file, to persist between runs. So unless people go and change the scripts to delete freshclam.dat instead, this commit should resolve the concern.
2021-05-24 18:37:33 -07:00
} freshclam_dat_v1_t;
FreshClam: mirrors.dat, User-Agent UUID, cool-down Add back the mirrors.dat file to the database directory. This new version of mirros.dat will store: - A randomly generated UUID for the FreshClam User-Agent. - A retry-after timestamp that so FreshClam won't try to update after having received an HTTP 429 response until the Retry-After timeout has expired. Also: FreshClam will now exit with a failure in daemon mode if an HTTP 403 (Forbidden) was received, because retrying later won't help any. The FreshClam user will have to take actions to get unblocked.
2021-03-21 19:47:21 -07:00
Undo accidental change to libfreshclam public header libfreshclam.h defines the public API. CFRAY_LEN is not for internal use. Moved to the top of the libfreshclam_internal.h header instead.
2024-04-24 18:17:00 -04:00
/*Length of a cf-ray id.*/
#define CFRAY_LEN 20
Freshclam refresh. This update makes libcurl a hard requirement for ClamAV. New features added to freshclam: - Update signature definitions over HTTPS. - Support for HTTP protocol v1.1 (formerly v1.0). - New libfreshclam library with an all new API and versioning separate from libclamav (v2.0.0). This library is now build and installed alongside libclamav as a hard dependency of freshclam. - The ability to opt-in and opt-out of standard and optional official ClamAV databases (ExtraDatabase, ExcludeDatabase) - The option to specify the protocol and port number of official and private mirror servers. - Support for additional types of proxy servers beyond plain HTTP (SOCKS 4, SOCKS 5). Features removed from freshclam: - Mirror management (mirrors.dat) file. This feature is no longer needed as official signature databases are distributed using a paid content delivery network (Cloudflare). This commit also adds the following features for Windows users: - The clamsubmit tool. - The json-c library dependency, which will enable the --gen-json option in clamscan. - Third party libraries under the win32/3rdparty directory have been removed. Developers will need to build the libraries separately from ClamAV and provide the headers and lib/dll library files the same way they do for OpenSSL. This includes libxml2, pthread-win32, bzip2, zlib, pcre2 as well as new dependencies: curl, json-c. Developers are encouraged to use the build tool Mussels to simplify this task.
2019-03-26 15:09:52 -04:00
/* ----------------------------------------------------------------------------
* Internal libfreshclam globals
*/
extern fccb_download_complete g_cb_download_complete;
extern char *g_localIP;
extern char *g_userAgent;
extern char *g_proxyServer;
extern uint16_t g_proxyPort;
extern char *g_proxyUsername;
extern char *g_proxyPassword;
extern char *g_tempDirectory;
extern char *g_databaseDirectory;
FIPS-compliant CVD signing and verification Add X509 certificate chain based signing with PKCS7-PEM external signatures distributed alongside CVD's in a custom .cvd.sign format. This new signing and verification mechanism is primarily in support of FIPS compliance. Fixes: https://github.com/Cisco-Talos/clamav/issues/564 Add a Rust implementation for parsing, verifying, and unpacking CVD files. Now installs a 'certs' directory in the app config directory (e.g. <prefix>/etc/certs). The install location is configurable. The CMake option to configure the CVD certs directory is: `-D CVD_CERTS_DIRECTORY=PATH` New options to set an alternative CVD certs directory: - Commandline for freshclam, clamd, clamscan, and sigtool is: `--cvdcertsdir PATH` - Env variable for freshclam, clamd, clamscan, and sigtool is: `CVD_CERTS_DIR` - Config option for freshclam and clamd is: `CVDCertsDirectory PATH` Sigtool: - Add sign/verify commands. - Also verify CDIFF external digital signatures when applying CDIFFs. - Place commonly used commands at the top of --help string. - Fix up manpage. Freshclam: - Will try to download .sign files to verify CVDs and CDIFFs. - Fix an issue where making a CLD would only include the CFG file for daily and not if patching any other database. libclamav.so: - Bump version to 13:0:1 (aka 12.1.0). - Also remove libclamav.map versioning. Resolves: https://github.com/Cisco-Talos/clamav/issues/1304 - Add two new API's to the public clamav.h header: ```c extern cl_error_t cl_cvdverify_ex(const char *file, const char *certs_directory); extern cl_error_t cl_cvdunpack_ex(const char *file, const char *dir, bool dont_verify, const char *certs_directory); ``` The original `cl_cvdverify` and `cl_cvdunpack` are deprecated. - Add `cl_engine_field` enum option `CL_ENGINE_CVDCERTSDIR`. You may set this option with `cl_engine_set_str` and get it with `cl_engine_get_str`, to override the compiled in default CVD certs directory. libfreshclam.so: Bump version to 4:0:0 (aka 4.0.0). Add sigtool sign/verify tests and test certs. Make it so downloadFile doesn't throw a warning if the server doesn't have the .sign file. Replace use of md5-based FP signatures in the unit tests with sha256-based FP signatures because the md5 implementation used by Python may be disabled in FIPS mode. Fixes: https://github.com/Cisco-Talos/clamav/issues/1411 CMake: Add logic to enable the Rust openssl-sys / openssl-rs crates to build against the same OpenSSL library as is used for the C build. The Rust unit test application must also link directly with libcrypto and libssl. Fix some log messages with missing new lines. Fix missing environment variable notes in --help messages and manpages. Deconflict CONFDIR/DATADIR/CERTSDIR variable names that are defined in clamav-config.h.in for libclamav from variable that had the same name for use in clamav applications that use the optparser. The 'clamav-test' certs for the unit tests will live for 10 years. The 'clamav-beta.crt' public cert will only live for 120 days and will be replaced before the stable release with a production 'clamav.crt'.
2024-11-21 14:01:09 -05:00
extern void *g_signVerifier;
Freshclam refresh. This update makes libcurl a hard requirement for ClamAV. New features added to freshclam: - Update signature definitions over HTTPS. - Support for HTTP protocol v1.1 (formerly v1.0). - New libfreshclam library with an all new API and versioning separate from libclamav (v2.0.0). This library is now build and installed alongside libclamav as a hard dependency of freshclam. - The ability to opt-in and opt-out of standard and optional official ClamAV databases (ExtraDatabase, ExcludeDatabase) - The option to specify the protocol and port number of official and private mirror servers. - Support for additional types of proxy servers beyond plain HTTP (SOCKS 4, SOCKS 5). Features removed from freshclam: - Mirror management (mirrors.dat) file. This feature is no longer needed as official signature databases are distributed using a paid content delivery network (Cloudflare). This commit also adds the following features for Windows users: - The clamsubmit tool. - The json-c library dependency, which will enable the --gen-json option in clamscan. - Third party libraries under the win32/3rdparty directory have been removed. Developers will need to build the libraries separately from ClamAV and provide the headers and lib/dll library files the same way they do for OpenSSL. This includes libxml2, pthread-win32, bzip2, zlib, pcre2 as well as new dependencies: curl, json-c. Developers are encouraged to use the build tool Mussels to simplify this task.
2019-03-26 15:09:52 -04:00
extern uint32_t g_maxAttempts;
extern uint32_t g_connectTimeout;
extern uint32_t g_requestTimeout;
extern uint32_t g_bCompressLocalDatabase;
FreshClam: rename mirrors.dat to freshclam.dat Some users have scripts set up from long ago to delete mirrors.dat if FreshClam failed. We used to recommend this if people had technical issues because mirrors.dat would store a bunch of entries indicating that all of their regional mirrors were failing and then FreshClam would give up. The new freshclam DAT file no longer stores that kind of information. Deleting the DAT file is no longer sound advice. We very much want the UUID, which is generated when creating the DAT file, to persist between runs. So unless people go and change the scripts to delete freshclam.dat instead, this commit should resolve the concern.
2021-05-24 18:37:33 -07:00
extern freshclam_dat_v1_t *g_freshclamDat;
Provide cf-ray ids to user when freshclam fails Providing cf-ray ids to the user when freshclam fails will skip the step of having to ask them for the ids. This should save some time when the user is not in the same time zone.
2024-03-15 16:40:20 -04:00
extern uint8_t g_lastRay[CFRAY_LEN + 1];
FreshClam: mirrors.dat, User-Agent UUID, cool-down Add back the mirrors.dat file to the database directory. This new version of mirros.dat will store: - A randomly generated UUID for the FreshClam User-Agent. - A retry-after timestamp that so FreshClam won't try to update after having received an HTTP 429 response until the Retry-After timeout has expired. Also: FreshClam will now exit with a failure in daemon mode if an HTTP 403 (Forbidden) was received, because retrying later won't help any. The FreshClam user will have to take actions to get unblocked.
2021-03-21 19:47:21 -07:00
FIPS & FIPS-like limits on hash algs for cryptographic uses ClamAV will not function when using a FIPS-enabled OpenSSL 3.x. This is because ClamAV uses MD5 and SHA1 algorithms for a variety of purposes including matching for malware detection, matching to prevent false positives on known-clean files, and for verification of MD5-based RSA digital signatures for determining CVD (signature database archive) authenticity. Interestingly, FIPS had been intentionally bypassed when creating hashes based whole buffers and whole files (by descriptor or `FILE`-pointer): https://github.com/Cisco-Talos/clamav/commit/78d4a9985a06a418dd1338c94ee5db461035d75b Note: this bypassed FIPS the 1.x way with: `EVP_MD_CTX_set_flags(ctx, EVP_MD_CTX_FLAG_NON_FIPS_ALLOW);` It was NOT disabled when using `cl_hash_init()` / `cl_update_hash()` / `cl_finish_hash()`. That likely worked by coincidence in that the hash was already calculated most of the time. It certainly would have made use of those functions if the hash had not been calculated prior: https://github.com/Cisco-Talos/clamav/blob/78d4a9985a06a418dd1338c94ee5db461035d75b/libclamav/matcher.c#L743 Regardless, bypassing FIPS entirely is not the correct solution. The FIPS restrictions against using MD5 and SHA1 are valid, particularly when verifying CVD digital siganatures, but also I think when using a hash to determine if the file is known-clean (i.e. the "clean cache" and also MD5-based and SHA1-based FP signatures). This commit extends the work to bypass FIPS using the newer 3.x method: `md = EVP_MD_fetch(NULL, alg, "-fips");` It does this for the legacy `cl_hash*()` functions including `cl_hash_init()` / `cl_update_hash()` / `cl_finish_hash()`. It also introduces extended versions that allow the caller to choose if they want to bypass FIPS: - `cl_hash_data_ex()` - `cl_hash_init_ex()` - `cl_update_hash_ex()` - `cl_finish_hash_ex()` - `cl_hash_destroy_ex()` - `cl_hash_file_fd_ex()` See the `flags` parameter for each. Ironically, this commit does NOT use the new functions at this time. The rational is that ClamAV may need MD5, SHA1, and SHA-256 hashes of the same files both for determining if the file is malware, and for determining if the file is clean. So instead, this commit will do a checks when: 1. Creating a new ClamAV scanning engine. If FIPS-mode enabled, it will automatically toggle the "FIPS limits" engine option. When loading signatures, if the engine "FIPS limits" option is enabled, then MD5 and SHA1 FP signatures will be skipped. 2. Before verifying a CVD (e.g. also for loading, unpacking when verification enabled). If "FIPS limits" or FIPS-mode are enabled, then the legacy MD5-based RSA method is disabled. Note: This commit also refactors the interface for `cl_cvdverify_ex()` and `cl_cvdunpack_ex()` so they take a `flags` parameters, rather than a single `bool`. As these functions are new in this version, it does not break the ABI. The cache was already switched to use SHA2-256, so that's not a concern for checking FIPS-mode / FIPS limits options. This adds an option for `freshclam.conf` and `clamd.conf`: FIPSCryptoHashLimits yes And an equivalent command-line option for `clamscan` and `sigtool`: --fips-limits You may programmatically enable FIPS-limits for a ClamAV engine like this: ```C cl_engine_set_num(engine, CL_ENGINE_FIPS_LIMITS, 1); ``` CLAM-2792
2025-07-01 20:41:47 -04:00
extern bool g_bFipsLimits;
FreshClam: rename mirrors.dat to freshclam.dat Some users have scripts set up from long ago to delete mirrors.dat if FreshClam failed. We used to recommend this if people had technical issues because mirrors.dat would store a bunch of entries indicating that all of their regional mirrors were failing and then FreshClam would give up. The new freshclam DAT file no longer stores that kind of information. Deleting the DAT file is no longer sound advice. We very much want the UUID, which is generated when creating the DAT file, to persist between runs. So unless people go and change the scripts to delete freshclam.dat instead, this commit should resolve the concern.
2021-05-24 18:37:33 -07:00
fc_error_t load_freshclam_dat(void);
fc_error_t save_freshclam_dat(void);
fc_error_t new_freshclam_dat(void);
FreshClam: mirrors.dat, User-Agent UUID, cool-down Add back the mirrors.dat file to the database directory. This new version of mirros.dat will store: - A randomly generated UUID for the FreshClam User-Agent. - A retry-after timestamp that so FreshClam won't try to update after having received an HTTP 429 response until the Retry-After timeout has expired. Also: FreshClam will now exit with a failure in daemon mode if an HTTP 403 (Forbidden) was received, because retrying later won't help any. The FreshClam user will have to take actions to get unblocked.
2021-03-21 19:47:21 -07:00
Freshclam refresh. This update makes libcurl a hard requirement for ClamAV. New features added to freshclam: - Update signature definitions over HTTPS. - Support for HTTP protocol v1.1 (formerly v1.0). - New libfreshclam library with an all new API and versioning separate from libclamav (v2.0.0). This library is now build and installed alongside libclamav as a hard dependency of freshclam. - The ability to opt-in and opt-out of standard and optional official ClamAV databases (ExtraDatabase, ExcludeDatabase) - The option to specify the protocol and port number of official and private mirror servers. - Support for additional types of proxy servers beyond plain HTTP (SOCKS 4, SOCKS 5). Features removed from freshclam: - Mirror management (mirrors.dat) file. This feature is no longer needed as official signature databases are distributed using a paid content delivery network (Cloudflare). This commit also adds the following features for Windows users: - The clamsubmit tool. - The json-c library dependency, which will enable the --gen-json option in clamscan. - Third party libraries under the win32/3rdparty directory have been removed. Developers will need to build the libraries separately from ClamAV and provide the headers and lib/dll library files the same way they do for OpenSSL. This includes libxml2, pthread-win32, bzip2, zlib, pcre2 as well as new dependencies: curl, json-c. Developers are encouraged to use the build tool Mussels to simplify this task.
2019-03-26 15:09:52 -04:00
fc_error_t updatedb(
const char *database,
const char *dnsUpdateInfo,
char *server,
int bPrivateMirror,
void *context,
int bScriptedUpdates,
int logerr,
int *signo,
char **dbFilename,
int *bUpdated);
fc_error_t updatecustomdb(
const char *url,
void *context,
int logerr,
int *signo,
char **dbFilename,
int *bUpdated);
Freshclam: raise stale DNS warning threshold Change the threshold for the age of the DNS entry before warning that the DNS entry is too old. Raises it from 3 hours to 12 hours.
2022-07-12 15:04:49 -04:00
#define DNS_WARNING_THRESHOLD_HOURS 12
#define DNS_WARNING_THRESHOLD_SECONDS (DNS_WARNING_THRESHOLD_HOURS * 60 * 60)
Freshclam refresh. This update makes libcurl a hard requirement for ClamAV. New features added to freshclam: - Update signature definitions over HTTPS. - Support for HTTP protocol v1.1 (formerly v1.0). - New libfreshclam library with an all new API and versioning separate from libclamav (v2.0.0). This library is now build and installed alongside libclamav as a hard dependency of freshclam. - The ability to opt-in and opt-out of standard and optional official ClamAV databases (ExtraDatabase, ExcludeDatabase) - The option to specify the protocol and port number of official and private mirror servers. - Support for additional types of proxy servers beyond plain HTTP (SOCKS 4, SOCKS 5). Features removed from freshclam: - Mirror management (mirrors.dat) file. This feature is no longer needed as official signature databases are distributed using a paid content delivery network (Cloudflare). This commit also adds the following features for Windows users: - The clamsubmit tool. - The json-c library dependency, which will enable the --gen-json option in clamscan. - Third party libraries under the win32/3rdparty directory have been removed. Developers will need to build the libraries separately from ClamAV and provide the headers and lib/dll library files the same way they do for OpenSSL. This includes libxml2, pthread-win32, bzip2, zlib, pcre2 as well as new dependencies: curl, json-c. Developers are encouraged to use the build tool Mussels to simplify this task.
2019-03-26 15:09:52 -04:00
#endif // __LIBFRESHCLAM_INTERNAL_H
Reference in a new issue Copy permalink