Refactor vuln analysis task to be more efficient

* For portfolio analysis, don't fetch *all* projects at once, but in batches of 100.
* Regularly evict objects from the ORM's L1 cache to prevent excessive cache size.
* For project analysis, don't fetch *all* components at once, but in batches of 1000.
* Limit the fields being fetched from the database to only those needed for vulnerability analysis.
* When triggering a project analysis, only transmit the project's UUID via events, instead of all of its components.
* Deprecate `QueryManager#getAllProjects` for removal. This method is never the right choice, but is still used in a few places.

In addition, as byproducts of this refactoring:

* Add MDC variables to provide more context in logs emitted during vulnerability analysis.
* Remove event subscription for specific scanner events. Scanners are only ever invoked directly, never through the event system.
* Gracefully handle thread interrupts.
* Fix `CacheableScanTask#applyAnalysisFromCache` never being invoked.

Signed-off-by: nscuro <nscuro@protonmail.com>
This commit is contained in:
nscuro 2025-02-06 18:31:32 +01:00
parent 78364cf0ef
commit 70e0ac2b05
No known key found for this signature in database
37 changed files with 777 additions and 572 deletions

View file

@ -182,6 +182,7 @@ final class ProjectQueryManager extends QueryManager implements IQueryManager {
* @return a List of Projects
*/
@Override
@Deprecated(forRemoval = true, since = "4.12.4")
public List<Project> getAllProjects() {
return getAllProjects(false);
}
@ -192,6 +193,7 @@ final class ProjectQueryManager extends QueryManager implements IQueryManager {
* @return a List of Projects
*/
@Override
@Deprecated(forRemoval = true, since = "4.12.4")
public List<Project> getAllProjects(boolean excludeInactive) {
final Query<Project> query = pm.newQuery(Project.class);
if (excludeInactive) {