Commit graph

12182 commits

Author SHA1 Message Date
das
9cbeba2b39 fix tests part 2 2025-10-17 12:09:33 +02:00
das
d4de59d7ce fix tests part 1 2025-10-17 12:07:41 +02:00
das
f3308a062d wip 5 2025-10-17 11:11:29 +02:00
das
74eb86ac21 wip 4 2025-10-17 11:10:04 +02:00
das
094c9169ba wip 3 2025-10-17 11:08:11 +02:00
map
3e01b42a23 print initial training times 2025-10-17 10:55:38 +02:00
das
b6092704bb wip 2 2025-10-17 10:22:02 +02:00
das
397b644ced wip 2025-10-17 10:15:45 +02:00
abp
525e695995
adjust updateSpamClassificationData 2025-10-17 10:02:26 +02:00
das
7d51f72a42 wip 2025-10-16 16:30:14 +02:00
abp
46939cfb09
update TutanotaModelV96 2025-10-16 16:03:09 +02:00
sug
48ddeb1484 wip: fix for testing in mobile 2025-10-16 14:49:59 +02:00
abp
1f8a297849
fix failing tests
Co-authored-by: map <mpfau@users.noreply.github.com>
2025-10-15 17:46:05 +02:00
abp
3d1a84caf9
use the confidence on the mail instead of computing on the client
Co-authored-by: map <mpfau@users.noreply.github.com>
2025-10-15 16:42:09 +02:00
sug
7c1c8e3a7e wip: test 2025-10-15 16:33:29 +02:00
das
b4a1a24a35 wip 2025-10-15 15:57:35 +02:00
abp
cdcee340c9
wip 2025-10-15 15:56:06 +02:00
sug
3cdcb88e81
wip: test 2025-10-15 15:56:05 +02:00
das
ae570ccf9b
Delete spam training data on mail deletion 2025-10-15 15:56:04 +02:00
sug
5b8b49d316
wip: Add connectivity model to spam handler 2025-10-15 15:56:04 +02:00
sug
3f09b0f5db
wip: tests2 2025-10-15 15:56:03 +02:00
sug
1bc8ebdc4e
wip: tests 2025-10-15 15:56:03 +02:00
das
b0d1c2bd43
Retrain spamclassifier from scratch, deleting past X days of data 2025-10-15 15:56:02 +02:00
sug
44ff6ab394
ask: why? 2025-10-15 15:56:01 +02:00
sug
9923c918ef
wip: test 3 2025-10-15 15:56:01 +02:00
jhm
bb6a0c5634
wip add fixme
Co-authored-by: abp <abp@tutao.de>
2025-10-15 15:56:00 +02:00
sug
f48bd25fea
wip: test 2 2025-10-15 15:55:59 +02:00
jhm
4ab2a07b2e
fix not successfully decrypting instances on the entityUpdate
When trying to decrypt instances that were either not encrypted or
did not have an _ownerEncSessionKey (but a bucketKey), we failed to
set the instance on the entityUpdateData. Instead of checking for
existence of the _ownerEncSessionKey, we now resolve the optional
sessionKey using the cryptoFacade.

Co-authored-by: abp <abp@tutao.de>
2025-10-15 15:55:58 +02:00
jhm
04cfbc36c8
improve inbox rule handling
Instead of applying inbox rules based on the unread mail state in the
inbox folder, we introduce the new isInboxRuleApplied boolean flag on
the mail type. If a mail has been processed by the leader client, which
is checking for matching inbox rules, the isInboxRuleApplied flag is
set to true. If there is a matching rule the flag is updated through the
MoveMailService, if there is no matching rule, the flag is updated
using the IsInboxRuleAppliedStateService. Both requests are
throttled / debounced.

This new inbox rule handing solves the following two problems:
 - when clicking on a notification it could still happen,
   that sometimes the inbox rules where not applied
 - when the inbox folder had a lot of unread mails, the loading time did
   massively increase, since inbox rules were re-applied on every load

Co-authored-by: abp <abp@tutao.de>
2025-10-15 15:55:57 +02:00
jhm
88ec43159d
add TutanotaModelV96
Co-authored-by: abp <abp@tutao.de>
2025-10-15 15:55:57 +02:00
sug
af7c05011c wip: test 2025-10-14 15:53:36 +02:00
sug
23e628dd86 wip 2025-10-14 14:03:37 +02:00
sug
b27e2166e2 [antispam] Do spam prediction after inbox rule handeling
Co-authored-by: amm <amm@tutao.de>
Co-authored-by: Nick <nif@tutao.de>
2025-10-14 13:56:49 +02:00
das
b3c2dabfa7 [antispam] Add client-side local spam filtering
Implement a local machine learning model for client-side spam filtering.
The local model is implemented using tensorflow "LayersModel" to train
separate models in all available mailboxes, resulting in one model
per ownerGroup (i.e. mailbox).

Initially, the training data is aggregated from the last 30 days of
received mails, and the data is stored in a separate offline database
table named spam_classification_training_data. The trained model is
stored in the table spam_classification_model. The initial training
starts after indexing, with periodic training happening
every 30 minutes and on each subsequent login.

The model will predict on incoming mails once we have received the
entity event for said mail, moving it to either inbox or spam folder.
When users move mails, we update the training data labels accordingly,
by adjusting the isSpam classification and isSpamConfidence values in
the offline database. The MoveMailService now contains a moveReason,
which indicates that the mail has been moved by our spam filter.

Client-side spam filtering can be activated using the
SpamClientClassification feature flag, and is for now only
available on the desktop client.

Co-authored-by: sug <sug@tutao.de>
Co-authored-by: kib <104761667+kibibytium@users.noreply.github.com>
Co-authored-by: abp <abp@tutao.de>
Co-authored-by: map <mpfau@users.noreply.github.com>
Co-authored-by: jhm <17314077+jomapp@users.noreply.github.com>
Co-authored-by: frm <frm@tutao.de>
Co-authored-by: das <das@tutao.de>
Co-authored-by: nif <nif@tutao.de>
Co-authored-by: amm <amm@tutao.de>
2025-10-14 12:32:17 +02:00
das
f69466a713 Add tensorflow.js library
Reviewed by jhm, abp, das.

The library is added to support training a spam classification model on
client. A custom rollupTensorFlow was created to only use the needed
methods on the final file, which also has multiple stubs which remove
code which is unwanted such as Requests.

Co-authored-by: jhm <17314077+jomapp@users.noreply.github.com>
Co-authored-by: abp <abp@tutao.de>
2025-10-14 12:32:04 +02:00
map
e6b90e5bbb allow downloading blobs of same file from multiple archives
Co-authored-by: abp <abp@tutao.de>
2025-10-14 11:52:56 +02:00
jenkins build server
b2ecf92a77 v310.251008.0 2025-10-08 15:10:33 +02:00
jenkins build server
043f032df2 update translations 2025-10-08 15:08:19 +02:00
toj
d1b9093d7b Add usage test with switched revo/legend positions 2025-10-08 14:48:48 +02:00
hrb-hub
218d3e196b [mobile] Fix JSON serialization error caused by whitelabel theme version
Close #9790

Co-authored-by: bir <bir@tutao.de>
2025-10-08 12:48:24 +02:00
jenkins build server
dc7a95166f v310.251002.0 2025-10-02 16:55:53 +02:00
jenkins build server
d80cb804ee update translations 2025-10-02 16:53:24 +02:00
bir
3a4d087e27 Set state colors for whitelabel
State colors don't use Material 3 color tokens, instead they are based
on outline_variant with varying alphas.

For whitelabel, we use the generated theme's outline_variant and the
same alpha values as the base theme.

Close #9744

Co-authored-by: ivk <ivk@tutao.de>
Co-authored-by: hrb-hub <hrb-hub@users.noreply.github.com>
2025-10-02 16:06:18 +02:00
paw
435fb98bd0 Use offline storage for autosaved drafts
We want to use offline storage (SQLite) for autosaved drafts as it will
allow the locally saved draft to be accessible when offline. This means
we refactor ConfigurationDatabase.

We still want to use the same encoding methods, as it provides a simple
way to put in data without having to worry about making some complex
schema.

Closes #9743

Co-authored-by: ivk <ivk@tutao.de>
2025-10-02 13:17:57 +02:00
ivk
0237809e66 [android] Fix suspension handling, normalize headers in RestClient
Close #9740

Co-authored-by: hrb-hub <hrb-hub@users.noreply.github.com>
2025-10-02 13:15:50 +02:00
ivk
7ebdd2896e [ios] Fix suspension handling, normalize headers in RestClient
Previously suspension handling was not working because SDK was
expecting the headers to be lowercased. This was not the case
for Swift impl of RestClient.

Additionally, fix logging errors from notification extension by
setting correct privacy modifier when interpolating.

Close #9740

Co-authored-by: hrb-hub <hrb-hub@users.noreply.github.com>
2025-10-02 13:15:50 +02:00
hrb-hub
c61dff79e8 Fix failed AssertNotNull when reply shortcut is triggered to quickly
Caused by the shortcut being enabled when the `MailViewerViewModel` is
`null`, which happens because `!viewModel()?.isDraftMail()` is true when
the mail is not draft but also when the viewModel is null.

The same is true for the replyAll and forward shortcuts.

Close #9736
2025-10-02 13:15:50 +02:00
wrd
b8ee0971d4 Update UI for dialog asking user about overwriting draft
Close #9711
2025-10-02 13:15:50 +02:00
hrb-hub
f6d7ec8ae6 [desktop] Remove downloaded file names from logs
private-issues: 268
2025-10-02 10:33:07 +02:00
sas
d9d49daad2
fix asymmetricCrypto used before assigned in WorkerLocator
tuta#2467
2025-10-01 15:59:49 +02:00