Commit graph

393 commits

Author SHA1 Message Date
ppom
3beca6d7a5
Document state_directory
Fixes #71
2025-06-05 12:00:00 +02:00
ppom
b53044323c
Add small doc for C helpers 2025-06-05 12:00:00 +02:00
ppom
02f13a263e
fix release 2025-06-05 12:00:00 +02:00
ppom
d4fb820cb7
version 2.0.0 v2.0.0 2025-06-05 12:00:00 +02:00
ppom
da884029e6
Fix git not sorting tags correctly 2025-06-05 12:00:00 +02:00
ppom
bc078471a9
NLnet statement 2025-06-01 12:00:00 +02:00
ppom
daf1bf3818
Update configuration
- README.md: update minimal example (was outdated since a long time 🤐)
- README.md: link to Security part of the wiki
- README.md: fix OpenBSD broken link
- README.md: add `state_directory` option
- activitywatch & server configs: remove from config directory. only
  available in the wiki.
- heavy-load: move to new bench directory
- test: move to tests directory

Fix #121
2025-05-31 12:00:00 +02:00
ppom
ebad317a97
Config: fix state_directory and concurrency merging 2025-05-28 12:00:00 +02:00
ppom
9152c95b03
Change error wording again 2025-05-28 12:00:00 +02:00
ppom
8ffbcad1b9
Add test-config man page 2025-05-28 12:00:00 +02:00
ppom
f7bd28e46e
Config: cleanup code 2025-05-28 12:00:00 +02:00
ppom
73ffcb97ab
Update CLI help messages 2025-05-28 12:00:00 +02:00
ppom
283d5c0f13
delete note files 2025-05-28 12:00:00 +02:00
ppom
1a5548c871
Reorder Config fields for test-config command
This order looks more readable to me, with:
- one line options at the beginning of config
- streams at the end of config
- filters at the end of streams
- actions at the end of filters
2025-05-28 12:00:00 +02:00
ppom
4f2eac2788
Change error wording 2025-05-28 12:00:00 +02:00
Baptiste Careil
231c9f8a99
Add test-config sub-command 2025-05-28 12:00:00 +02:00
Baptiste Careil
cf96fa30f1
Add test directory for reading configuration files from a directory 2025-05-28 12:00:00 +02:00
Baptiste Careil
28b3a173bb
Add ability to read config from multiple files in a same directory 2025-05-28 12:00:00 +02:00
ppom
5260b6b9b1
Clean handle_order
Had bits that were relevant in LMDB but are not anymore.
Use of keys().cloned() instead of custom map()
Clone after filtering.
2025-05-28 12:00:00 +02:00
ppom
a5bbd7641a
reaction {show,flush}: do not display empty pending actions
We were creating an action array even if it was past.
2025-05-28 12:00:00 +02:00
ppom
14aa859e2d
cargo update 2025-05-28 12:00:00 +02:00
ppom
c5dbb4e29c
Remove bincode & fjall crates 2025-05-28 12:00:00 +02:00
ppom
3ffb2b5bed
Adapt to Time being serialized as rfc3339
And all tests pass! 🎉
2025-05-28 12:00:00 +02:00
ppom
3d05815263
Daemon runs the database task
Completely forgot to run the database task, no wonder why the test didn't pass 😅

Now the database asks for the daemon to stop if it errors.
A lot better than panicking, as it was until now.

It also quits with all the other tasks, thanks to its ShutdownToken.

It uses a oneshot channel to tell the daemon about a potential error.
2025-05-28 12:00:00 +02:00
ppom
78ca0025df
Document daemon::shutdown. ShutdownToken is now able to ask for shutdown. 2025-05-28 12:00:00 +02:00
ppom
0c56acfd81
Fix FilterManager unit tests
e2e test failing for now
2025-05-28 12:00:00 +02:00
ppom
1783bf8062
Use treedb in FilterManager
Not tested yet, tests must be adapted
2025-05-28 12:00:00 +02:00
ppom
037b3498bc
rename waltree into treedb
WAL was a wrong name. It's not a Write Ahead Log, but a "Write Behind
Log" (new concept haha), so it made no sense to keep wal.
And wbl is not unpronounceable.
2025-05-28 12:00:00 +02:00
ppom
5f21db5279
WriteDB: reuse write buffer 2025-05-28 12:00:00 +02:00
ppom
fe5cd70f7a
use tokio::time::interval instead of sleep, for guaranteed flushes 2025-05-28 12:00:00 +02:00
ppom
c93f4e34c1
LoadedDB internal to Database, open_tree uses LoadedDB. More doc! 2025-05-28 12:00:00 +02:00
ppom
482df254ba
implement Tree::fetch_update 2025-05-28 12:00:00 +02:00
ppom
ec0d8c72e9
Reorganise waltree. Test log rotation. Add tokio channel. Implement Tree operations. 2025-05-28 12:00:00 +02:00
ppom
6cdad37588
Finish tests on lowlevel db (and fix bug) 2025-05-28 12:00:00 +02:00
ppom
924c8e8635
test ReadDB::read 2025-05-28 12:00:00 +02:00
ppom
a029e28812
Tested ReadDB and WriteDB 2025-05-28 12:00:00 +02:00
ppom
779e5e5d86
more WIP, split into low level / high level 2025-05-28 12:00:00 +02:00
ppom
bb9b17761a
WIP custom db 2025-05-28 12:00:00 +02:00
ppom
9c8be2f2de
Working db rewrite using fjall 2025-05-28 12:00:00 +02:00
ppom
2facac9fbd
WIP fjall 2025-05-28 12:00:00 +02:00
ppom
11459f7ee4
lmdb NO_SYNC 2025-05-28 12:00:00 +02:00
ppom
68e35ed021
Benchmark after heed/lmdb rewrite 2025-05-28 12:00:00 +02:00
ppom
db0e480daa
arbitrary max size
To avoid a runtime Error(MapSize) by heed

FIXME How to know which value is high enough?
2025-05-28 12:00:00 +02:00
ppom
a056efe770
Fix warning until next heed release 2025-05-28 12:00:00 +02:00
ppom
d952a61f83
Fix last lmdb bug
Hopefully the last. Tests pass!
I was using a read transaction outdated by recent changes in an
uncommited write transaction. So I use the same write transaction to
read things now.

I also checked that other uses of a read transaction is fine.
They're fine because they're only used to iterate on a whole db, while
the write transaction drops some keys/values.
2025-05-28 12:00:00 +02:00
ppom
64fdc52e9a
Better use of transactions, React enum for more precise tests, test macros
Transactions are now opened at higher level, to
1. Ensure a logical transaction is a complete action.
   (ex: when exec, remove match & add corresponding trigger)
2. Fix issue where multiple Rw / Ro transactions are open at the same
   time.

So now they're open only by:
- FilterManager's task in handle_*,
- FilterManager's task in cleanup_*,
- the scheduled task for after actions.

handle_line returns an enum (no match, match, exec) to have more precise
tests. So maybe the tests could care less about database state and only
about FilterManager's behavior.

Use of macros to have more readable tests and reduce duplication and
remove complex generic functions

Tests still failing, behavior seems to be altered.
2025-05-28 12:00:00 +02:00
ppom
ea1bc033c5
WIP 2025-05-28 12:00:00 +02:00
ppom
8393ebf20e
Temporary fix while we can't remove heed databases
Also remove sled related code
2025-05-28 12:00:00 +02:00
ppom
77f001f860
Add debug log before closing DB 2025-05-28 12:00:00 +02:00
ppom
e321e3ea0b
collecting maps before any remove_match to avoid iterators holding a read
But now the tasks don't finish so I introduced another bug?
2025-05-28 12:00:00 +02:00