Commit graph

465 commits

Author SHA1 Message Date
ppom
f4d002c615
Fix trigger count on start
schedule_exec was called before inserting the data in triggers,
resulting in action count being set again after decrement in schedule
exec.
This could lead to:
- trigger not disappearing after done
- second action with no "after" not being run
- ...
2025-08-05 12:00:00 +02:00
ppom
f477310a29
duplicates: Add failing tests for Deduplication on start 2025-08-05 12:00:00 +02:00
ppom
773eb76f92
Update README to advertise ip-specific features 2025-08-04 12:00:00 +02:00
ppom
59c7bfdd1d
Move action filtering logic from daemon to concepts and use at 3 places
Used in Filter::schedule_exec, Filter::handle_order, State::add_trigger
Add proper testing.
This also fix previously failing test.
2025-08-04 12:00:00 +02:00
ppom
cebdbc7ad0
ipv4 regex: do no accept numbers 0[0-9]
The Rust std won't accept it anyway, as it interprets numbers starting
with 0 as octal numbers and forbid that.
2025-08-04 12:00:00 +02:00
ppom
0b2bfe533b
Update example configs to get rid of ip46tables 2025-08-04 12:00:00 +02:00
ppom
a0b804811b
Refacto: make all Config structures' fields public
Config is 'static after setup anyways.
I don't need to hide all this, it's just cumbersome for tests.
2025-08-04 12:00:00 +02:00
ppom
6f63f49acd
Add failing test for flushing ipvXonly actions 2025-08-04 12:00:00 +02:00
ppom
b927ba4fdf
Add ipv4only/ipv6only logic to actions 2025-08-04 12:00:00 +02:00
ppom
e4e50dd03b
cargo clippy 2025-08-04 12:00:00 +02:00
ppom
0a9c7f97df
Split IP pattern code in 3 files 2025-08-04 12:00:00 +02:00
ppom
130607d28f
Add test for pattern deserialization 2025-08-04 12:00:00 +02:00
ppom
19e3b2bf98
Make IP regex much more robust and add tests
IP will be correctly extracted in any regex line, even if it is
surrounded by greedy catch-all: .*<ip>.*

This what actually hard to do!
2025-08-04 12:00:00 +02:00
ppom
421002442e
Add ip tests on daemon::filter
Fix PatternType deserialization
Fix regex deserialization (now optional)
Tests currently failing
2025-08-04 12:00:00 +02:00
ppom
4f79b476aa
Cut ip regexes in smaller blocks and add tests 2025-08-04 12:00:00 +02:00
ppom
6cde89cc4b
rename file 2025-08-04 12:00:00 +02:00
ppom
43f8b66870
Update config documentation 2025-08-04 12:00:00 +02:00
ppom
94b40c4a0b
Add more tests
Done: Tests on PatternIp.
Todo: Tests on Pattern.

Fixed a bug in is_ignore.
Checked a new possible misconfiguration.
2025-08-04 12:00:00 +02:00
ppom
a5f616e295
WIP pattern ip
add ipv{4,6}mask
factorize redundant code in util functions
normalize match
most tests done
2025-08-04 12:00:00 +02:00
ppom
04b5dfd95b
ip: Add includes, tests, more setup constraints 2025-08-04 12:00:00 +02:00
ppom
44e5757ae3
WIP pattern ip 2025-08-04 12:00:00 +02:00
ppom
ea0452f62c
Fix components starting order
Now Database and Socket components are created before start commands are
executed. So in case of error, start commands are not executed.

Also socket syscalls are now async instead of blocking, for better
integration with the async runtime.

New start order:
- DB
- Socket
- Start commands
- Streams
2025-08-04 12:00:00 +02:00
ppom
6b970e74c5
Update configuration reference 2025-07-14 12:00:00 +02:00
ppom
d8db2a1745
Add extensive test on Duplicate and fix related bug 2025-07-14 12:00:00 +02:00
ppom
6f346ff371
Test existing FilterManager tests for each Duplicate enum 2025-07-14 12:00:00 +02:00
ppom
81e5fb4c42
add State tests and fix trigger persistance
Triggers were only persisted for retry duration, instead of longuest
action duration.
As retry is often shorter than after, this would make reaction forget
most triggers on restart.
entry_timeout is now set to longuest_action_duration.
2025-07-14 12:00:00 +02:00
ppom
270a1a9bdf
Duplicate: Fix tests, more tests 2025-07-14 12:00:00 +02:00
ppom
d9842c2340
Duplicate::Extend: Re-Trigger only after actions
- implement schedule_exec's only_after
2025-07-14 12:00:00 +02:00
ppom
22384a2cb4
rename React::Exec to React::Trigger 2025-07-14 12:00:00 +02:00
ppom
2cebb733b5
WIP duplicates
- change remove_trigger to remove all triggers for a Match
- schedule_exec will take only_after boolean
2025-07-14 12:00:00 +02:00
ppom
881fc76bf9
WIP duplicates
- new duplicate option
- change triggers Tree structure to keep O(log(n)) querying:
  now we need to know if a match already has a trigger.
- triggers migration
- triggers adaptations in State & FilterManager
2025-07-14 12:00:00 +02:00
ppom
4ddaf6c195
v2.1.2 v2.1.2 2025-07-14 12:00:00 +02:00
ppom
b62f085e51
Fix trigger persistance
Triggers were only persisted for retry duration, instead of longuest
action duration.
As retry is often shorter than after, this would make reaction forget
most triggers on restart.
entry_timeout is now set to longuest_action_duration.

Cherry picked from the duplicate branch.
2025-07-14 12:00:00 +02:00
ppom
fd0dc91824
Get rid of useless Buffer wrapper for Vec<u8>
Write is already implemented on Vec<u8>
2025-07-11 12:00:00 +02:00
ppom
d880f7338b
Get rid of low-level async with Poll
use futures::stream::try_unfold to create a Stream from an async closure
2025-07-11 12:00:00 +02:00
ppom
e0609e3c3e
Move rewrite section 2025-07-08 12:00:00 +02:00
ppom
28f136f491
README update
project status update (rust rewrite ok)

contributing: separate ideas & code
2025-07-08 12:00:00 +02:00
ppom
5d9f2ceb6a
v2.1.1 v2.1.1 2025-07-07 12:00:00 +02:00
ppom
bba113b6ab
Remove newline at the end of stream lines
Bug introduced by !24 which kept trailing `\n` and fed it to filters.
Thus regexes ending with `$` couldn't match anymore.

Fixes #128
2025-07-07 12:00:00 +02:00
ppom
5bf67860f4
Fix Filter::regex for StreamManager::compiled_regex_set
regexes were pushed multiple times, with pattern names not completely
replaced by their corresponding regexes.
Now only pushed when pattern replacement is finished.
2025-07-07 12:00:00 +02:00
ppom
39bf662296
Fix example configs
- Fix comma issues
- Fix regex syntax doc
- Add ssh regexes
2025-06-28 12:00:00 +02:00
ppom
359957c58c
README: Add trigger command 2025-06-24 12:00:00 +02:00
ppom
3f3236cafb
v2.1.0 v2.1.0 2025-06-24 12:00:00 +02:00
ppom
78056b6fc5
src/client/request.rs rename and ARCHITECTURE.md update 2025-06-24 12:00:00 +02:00
ppom
6a778f3d01
cargo fmt, cargo clippy --all-targets 2025-06-24 12:00:00 +02:00
ppom
35862d32fa
Fix trigger command
- Force STREAM.FILTER one the command line
- Fix typo
2025-06-24 12:00:00 +02:00
ppom
283d1867b8
Benchmark: Add real-life configuration file and benchmark wrapper
Performance on this real-life configuration:

Before last commit:
Service runtime: 2min 22.669s
CPU time consumed: 3min 44.299s
Memory peak: 50.7M (swap: 0B)

With last commit:
Service runtime: 7.569s
CPU time consumed: 21.998s
Memory peak: 105.6M (swap: 0B)
2025-06-23 12:00:00 +02:00
ppom
ad6b0faa30
Performance: Use a RegexSet for all regexes of a Stream
StreamManager is now a struct that has its own RegexSet created from all
the regexes inside its Filters. Instead of calling
FilterManager::handle_line on all its FilterManagers, resulting in m*n
regex passes, it matches on all the regexes with its RegexSet.
It then only calls FilterManager::handle_line on matching Filters.

This should increase performance in those cases:
- Streams with a lot of filters or a lot of regexes
- Filters that match a small proportion of their Stream lines

This may decrease performance when most of lines are matched by all
Filters of a Stream.
2025-06-23 12:00:00 +02:00
ppom
55ed7b9c5f
Amend heavy-load test
- Add wrapper script
- Add non-matching lines
- Put two filters on the same stream, where either one of them matches
2025-06-23 12:00:00 +02:00
Baptiste Careil
d12a61c14a Fix #124: discard invalid utf8 sequences from input streams 2025-06-23 19:13:49 +00:00