forgejo/modules
Earl Warren 238ecfdeb8 fix: garbage collect lingering actions logs (#10009)
If, for any reason (e.g. server crash), a task is recorded as done in the database but the logs are still in the database instead of being in storage, they need to be collected.

The log_in_storage field is only set to true after the logs have been transfered to storage and can be relied upon to reflect which tasks have lingering logs.

A cron job collects lingering logs every day, 3000 at a time, sleeping one second between them. In normal circumstances there will be only a few of them, even on a large instance, and there is no need to collect them as quickly as possible.

When there are a lot of them for some reason, garbage collection must happen at a rate that is not too hard on storage I/O.

Refs https://codeberg.org/forgejo/forgejo/issues/9999

---

Note on backports: the v11 backport is done manually because of minor conflicts. https://codeberg.org/forgejo/forgejo/pulls/10024

## Checklist

The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).

### Tests

- I added test coverage for Go changes...
  - [x] in their respective `*_test.go` for unit tests.
  - [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
  - [ ] in `web_src/js/*.test.js` if it can be unit tested.
  - [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).

### Documentation

- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.

### Release notes

- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.

<!--start release-notes-assistant-->

## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Bug fixes
  - [PR](https://codeberg.org/forgejo/forgejo/pulls/10009): <!--number 10009 --><!--line 0 --><!--description Z2FyYmFnZSBjb2xsZWN0IGxpbmdlcmluZyBhY3Rpb25zIGxvZ3M=-->garbage collect lingering actions logs<!--description-->
<!--end release-notes-assistant-->

Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10009
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
2025-11-18 18:59:01 +01:00
..
actions fix: garbage collect lingering actions logs (#10009) 2025-11-18 18:59:01 +01:00
activitypub fix: assorted ActivityPub code only refactors (#8708) 2025-07-28 15:17:29 +02:00
analyze Rename code_langauge.go to code_language.go (#26377) 2023-08-07 15:00:53 -04:00
assetfs fix: follow symlinks for local assets (#8596) 2025-07-22 15:02:47 +02:00
auth chore(cleanup): replaces unnecessary calls to formatting functions by non-formatting equivalents (#7994) 2025-05-29 17:34:29 +02:00
avatar feat: strip EXIF information from uploaded avatars (#9638) 2025-10-13 23:16:17 +02:00
base git/commit: re-implement submodules file reader (#8438) 2025-07-15 00:20:00 +02:00
cache fix: prevent Remove(key)...Get*(key) from returning a value computed before the Remove(key) 2025-11-16 08:53:37 -07:00
card chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
charset feat: update ambigious characters (#7988) 2025-05-29 10:00:12 +02:00
container chore: add new functions to container.Set 2025-10-14 14:40:49 -06:00
csv Update module github.com/golangci/golangci-lint/cmd/golangci-lint to v2 (forgejo) (#7367) 2025-03-28 22:22:21 +00:00
emoji chore(cleanup): replaces unnecessary calls to formatting functions by non-formatting equivalents (#7994) 2025-05-29 17:34:29 +02:00
eventsource chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
forgefed Sent user activities to distant federated server (#8792) 2025-08-06 16:16:13 +02:00
generate chore: unify the usage of CryptoRandomString (#10110) 2025-11-15 13:24:53 +01:00
git chore: unify the usage of CryptoRandomString (#10110) 2025-11-15 13:24:53 +01:00
gitrepo Update module github.com/golangci/golangci-lint/v2/cmd/golangci-lint to v2.6.1 (forgejo) (#10053) 2025-11-11 07:04:35 +01:00
graceful feat: enable H2C for the HTTP server (#8861) 2025-08-16 21:00:20 +02:00
hcaptcha chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
highlight Update module github.com/golangci/golangci-lint/cmd/golangci-lint to v2 (forgejo) (#7367) 2025-03-28 22:22:21 +00:00
hostmatcher Support allowed hosts for migrations to work with proxy (#32025) 2024-09-14 17:52:54 +02:00
html Refactor backend SVG package and add tests (#26335) 2023-08-05 04:34:59 +00:00
httpcache chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
httplib feat: detect Interlisp sources as text (#8377) 2025-07-02 07:38:46 +02:00
indexer Update module github.com/meilisearch/meilisearch-go to v0.34.0 (forgejo) (#9275) 2025-09-14 13:22:15 +02:00
issue/template chore: replace gopkg.in/yaml.v3 with go.yaml.in/yaml/v3 (#8956) 2025-08-20 15:31:12 +02:00
json Replace interface{} with any (#25686) 2023-07-04 18:36:08 +00:00
keying feat: cache derived keys for faster keying (#10114) 2025-11-16 14:29:14 +01:00
label chore: replace gopkg.in/yaml.v3 with go.yaml.in/yaml/v3 (#8956) 2025-08-20 15:31:12 +02:00
lfs chore: add unit test for SearchPointerBlobs 2025-10-03 14:37:24 +02:00
log feat(log): better parseable and configurable ssh-logs (#9056) 2025-09-11 18:59:24 +02:00
markup fix: use scrollHeight for rendered iframe if offsetHeight is unavailable (#9508) 2025-10-16 15:51:57 +02:00
mcaptcha chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
metrics chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
migration chore: replace gopkg.in/yaml.v3 with go.yaml.in/yaml/v3 (#8956) 2025-08-20 15:31:12 +02:00
nosql chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
optional chore: replace gopkg.in/yaml.v3 with go.yaml.in/yaml/v3 (#8956) 2025-08-20 15:31:12 +02:00
options chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
packages chore: replace gopkg.in/yaml.v3 with go.yaml.in/yaml/v3 (#8956) 2025-08-20 15:31:12 +02:00
paginator Use more specific test methods (#24265) 2023-04-22 17:56:27 -04:00
pprof chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
private feat(log): better parseable and configurable ssh-logs (#9056) 2025-09-11 18:59:24 +02:00
process Update module github.com/golangci/golangci-lint/cmd/golangci-lint to v2 (forgejo) (#7367) 2025-03-28 22:22:21 +00:00
proxy chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
proxyprotocol chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
public add model viewer for .glb (GLTF) model in file view (#8111) 2025-06-21 14:42:35 +02:00
queue test: fix modules/queue tests to use TEST_REDIS_SERVER when present (#10139) 2025-11-16 21:51:32 +01:00
recaptcha chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
references fix: pull request cross references (#7979) 2025-05-28 14:50:05 +02:00
regexplru Update module github.com/golangci/golangci-lint/cmd/golangci-lint to v2 (forgejo) (#7367) 2025-03-28 22:22:21 +00:00
repository fix: set tag message on tag addition (#9913) 2025-10-31 07:04:28 +01:00
secret Update module github.com/golangci/golangci-lint/cmd/golangci-lint to v1.64.6 (forgejo) (#7118) 2025-03-04 21:38:35 +00:00
session chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
setting fix: get new session from enginegroup instead of masterengine (#10140) 2025-11-17 14:42:56 +01:00
sitemap Add testifylint to lint checks (#4535) 2024-07-30 19:41:10 +00:00
ssh feat: ensure only expected ssh public keys are in authorized_keys file (#10010) 2025-11-09 01:06:04 +01:00
storage fix: minio initialization can freeze indefinitely if misconfigured (#8897) 2025-08-15 21:03:51 +02:00
structs Update module github.com/golangci/golangci-lint/v2/cmd/golangci-lint to v2.6.1 (forgejo) (#10053) 2025-11-11 07:04:35 +01:00
svg chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
sync chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
system Update module github.com/golangci/golangci-lint/cmd/golangci-lint to v2 (forgejo) (#7367) 2025-03-28 22:22:21 +00:00
templates chore: remove not working PREFERRED_TIMESTAMP_TENSE setting (#9490) 2025-10-01 15:16:01 +02:00
test fix(ui): document token validity in key verification view (#9002) 2025-11-14 23:40:03 +01:00
testlogger feat: use XORM EngineGroup instead of single Engine connection (#7212) 2025-03-30 11:34:02 +00:00
timeutil Update module github.com/golangci/golangci-lint/cmd/golangci-lint to v2 (forgejo) (#7367) 2025-03-28 22:22:21 +00:00
translation frontend: generic lazy loader for webcomponents (#8510) 2025-07-23 04:10:50 +02:00
turnstile chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
typesniffer feat: detect Interlisp sources as text (#8377) 2025-07-02 07:38:46 +02:00
updatechecker chore: branding import path (#7337) 2025-03-27 19:40:14 +00:00
uri Add testifylint to lint checks (#4535) 2024-07-30 19:41:10 +00:00
user Drop SSPI auth support and more Windows files (#7148) 2025-03-08 00:43:41 +00:00
util chore: unify the usage of CryptoRandomString (#10110) 2025-11-15 13:24:53 +01:00
validation chore: add email blocklist unit test 2025-08-30 09:45:19 +02:00
web feat: make Forgejo Actions server logs less noisy (#7986) 2025-05-29 10:06:30 +02:00
webhook Actions Failure, Succes, Recover Webhooks (#7508) 2025-06-03 14:29:19 +02:00
zstd Cache generated binary across jobs 2024-08-26 23:43:09 +02:00