since 4521732ebbf3 ("monitor: missing cache and set handle initialization")
these fields are set via handle_merge(), so don't clobber those
fields in json output case:
==31877==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 16 byte(s) in 2 object(s) allocated from:
#0 0x7f0cb9f29d4b in strdup asan/asan_interceptors.cpp:593
#1 0x7f0cb9b584fd in xstrdup src/utils.c:80
#2 0x7f0cb9b355b3 in handle_merge src/rule.c:127
#3 0x7f0cb9ae12b8 in netlink_events_setelem_cb src/monitor.c:457
Seen when running tests/monitor with asan enabled.
Fixes: 4521732ebbf3 ("monitor: missing cache and set handle initialization")
Signed-off-by: Florian Westphal <fw@strlen.de>
The existing documentation briefly mentioned that the handle field can be
used for positioning, but the behavior was ambiguous. This commit clarifies:
- ADD with handle: inserts rule AFTER the specified handle
- INSERT with handle: inserts rule BEFORE the specified handle
- Multiple rules added at the same handle are positioned relative to the
original rule, not to previously inserted rules
- Explicit commands (with command wrapper) use handle for positioning
- Implicit commands (without command wrapper, used in export/import)
ignore handle for portability
This clarification helps users understand the correct behavior and avoid
confusion when using the JSON API for rule management.
Signed-off-by: Alexandre Knecht <knecht.alexandre@gmail.com>
Signed-off-by: Phil Sutter <phil@nwl.cc>
Now that JSON parser respects rule handles in explicit add commands, the
still present rule handle causes an error since the old rule does not
exist anymore.
Fixes: 50b5b71ebeee3 ("parser_json: Rewrite echo support")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Add comprehensive test for JSON handle-based rule positioning to verify
the handle field correctly positions rules with explicit add/insert
commands while being ignored in implicit format.
Test coverage:
1. ADD with handle positions AFTER the specified handle
2. INSERT with handle positions BEFORE the specified handle
3. INSERT without handle positions at beginning
4. Multiple commands in single transaction (batch behavior)
5. Implicit format ignores handle field for portability
The test uses sed for handle extraction and nft -f format for setup
as suggested in code review. Final state is a table with two rules
from the implicit format test.
Signed-off-by: Alexandre Knecht <knecht.alexandre@gmail.com>
Signed-off-by: Phil Sutter <phil@nwl.cc>
Add comprehensive test for JSON add/insert/delete/replace/create
operations on all object types to ensure the handle field changes
don't break non-rule objects.
Tests coverage:
- ADD operations: table, chain, rule, set, counter, quota
- INSERT operations: rule positioning
- REPLACE operations: rule modification
- CREATE operations: table creation with conflict detection
- DELETE operations: rule, set, chain, table
The test verifies that all object types work correctly with JSON
commands and validates intermediate states. Final state is an empty
table from the CREATE test.
Signed-off-by: Alexandre Knecht <knecht.alexandre@gmail.com>
Signed-off-by: Phil Sutter <phil@nwl.cc>
This patch enables handle-based rule positioning for JSON add/insert
commands by using a context flag to distinguish between explicit and
implicit command formats.
When processing JSON:
- Explicit commands like {"add": {"rule": ...}} set no flag, allowing
handle fields to be converted to position for rule placement
- Implicit format (bare objects like {"rule": ...}, used in export/import)
sets CTX_F_IMPLICIT flag, causing handles to be ignored for portability
This approach ensures that:
- Explicit rule adds with handles work for positioning
- Non-rule objects (tables, chains, sets, etc.) are unaffected
- Export/import remains compatible (handles ignored)
The semantics for explicit rule commands are:
ADD with handle: inserts rule AFTER the specified handle
INSERT with handle: inserts rule BEFORE the specified handle
Implementation details:
- CTX_F_IMPLICIT flag (bit 10) marks implicit add commands
- CTX_F_EXPR_MASK uses inverse mask for future-proof expression flag filtering
- Handle-to-position conversion in json_parse_cmd_add_rule()
- Variables declared at function start per project style
Link: https://patchwork.ozlabs.org/project/netfilter-devel/patch/20251029224530.1962783-2-knecht.alexandre@gmail.com/
Suggested-by: Phil Sutter <phil@nwl.cc>
Suggested-by: Florian Westphal <fw@strlen.de>
Signed-off-by: Alexandre Knecht <knecht.alexandre@gmail.com>
Signed-off-by: Phil Sutter <phil@nwl.cc>
The meta keyword currently accepts 'STRING' arguments.
This was originally done to avoid pollution the global token namespace.
However, nowadays we do have flex scopes to avoid this.
Add the tokens currently handled implciitly via STRING within
META flex scope.
SECPATH is a compatibility alias, map this to IPSEC token.
IBRPORT/OBRPORT are also compatibility aliases, remove those tokens
and handle this directly in scanner.l.
This also avoids nft from printing tokens in help texts that are only
there for compatibility with old rulesets.
meta_key_parse() is retained for json input parser.
Signed-off-by: Florian Westphal <fw@strlen.de>
Acked-by: Phil Sutter <phil@nwl.cc>
Now, on syntax errors, e.g., 'nft create fable filter', the user sees:
Error: syntax error, unexpected string
create fable filter
^^^^^
The patch builds an error message that lists what the parser expects
to see, in that case it would print:
Error: syntax error, unexpected string
expected any of: synproxy, table, chain, set, element, map,
flowtable, ct, counter, limit, quota, secmark
create fable filter
^^^^^
The obvious purpose of this is to help people who learn nft syntax.
The messages are still not as explanatory as one wishes, for it may
list parser token names such as 'string', but it's still better than
no hints at all.
Heed that the list of possible items on the parser's side is not
always consistent with expectations.
For instance, lexer/parser recognizes 'l4proto' in this command:
nft add rule ip F I meta l4proto tcp
as a generic '%token <string> STRING', while 'iifname' in
nft add rule ip F I meta iifname eth0
is recognized as a '%token IIFNAME'
In such case the parser is only able to say that right after 'meta'
it expects 'iifname' or 'string', rather than 'iifname' and 'l4proto'.
This 'meta STRING' is a historic wart and can be resolved in
a followup patch.
[ fw@strlen.de: minor coding style changes and rewordings ]
Signed-off-by: Jan Kończak <jan.konczak@cs.put.poznan.pl>
Signed-off-by: Florian Westphal <fw@strlen.de>
This is a first exclusive start condition, i.e. one which rejects
unscoped tokens. When tokenizing, flex all too easily falls back into
treating something as STRING when it could be split into tokens instead.
Via an exclusive start condition, the string-fallback can be disabled as
needed.
With rates in typical formatting <NUM><bytes-unit>/<time-unit>,
tokenizer result depended on whitespace placement. SCANSTATE_RATE forces
flex to split the string into tokens and fall back to JUNK upon failure.
For this to work, tokens which shall still be recognized must be enabled
in SCANSTATE_RATE (or all scopes denoted by '*'). This includes any
tokens possibly following SCANSTATE_RATE to please the parser's
lookahead behaviour.
Signed-off-by: Phil Sutter <phil@nwl.cc>
Reviewed-by: Florian Westphal <fw@strlen.de>
Introduce scoped tokens for "kbytes" and "mbytes", completing already
existing "bytes" one. Then generalize the unit for byte values and
replace both quota_unit and limit_bytes by a combination of NUM and
bytes_unit.
With this in place, data_unit_parse() is not called outside of
datatype.c, so make it static.
Signed-off-by: Phil Sutter <phil@nwl.cc>
Reviewed-by: Florian Westphal <fw@strlen.de>
Since log statement is scoped already, it's just a matter of declaring
the tokens in that scope and using them. This eliminates the redundant
copy of log level string parsing in parser_bison.y - the remaining one,
namely log_level_parse() in statement.c is used by JSON parser.
Signed-off-by: Phil Sutter <phil@nwl.cc>
Reviewed-by: Florian Westphal <fw@strlen.de>
Use the already existing SCANSTATE_TYPE for keyword scoping.
This is a bit of back-n-forth from string to token and back to string
but it eliminates the helper function and also takes care of error
handling.
Note that JSON parser does not validate the type string at all but
relies upon the kernel to reject wrong ones.
Signed-off-by: Phil Sutter <phil@nwl.cc>
Reviewed-by: Florian Westphal <fw@strlen.de>
There already is a start condition for "monitor" keyword and also a
DESTROY token. So just add the missing one and get rid of the
intermediate string buffer.
Keep checking the struct monitor::event value in eval phase just to be
on the safe side.
Signed-off-by: Phil Sutter <phil@nwl.cc>
Reviewed-by: Florian Westphal <fw@strlen.de>
Add tests to exercise packet path for rbtree and hash set types.
We check both positive (added address is matched) and negative
matches (set doesn't indicate match for deleted address).
For ranges, also validate that addresses preceeding or trailing
a range do not match.
Pipapo has no test to avoid duplicating what is already in
kernel kselftest (nft_concat_range.sh).
Signed-off-by: Florian Westphal <fw@strlen.de>
bitmap sets don't support 'counter' flag, so we can only check
'match' vs 'no match', but we can't tell which set element has
matched.
Static test, counter validation via dumps.
Signed-off-by: Florian Westphal <fw@strlen.de>
The rework to reduce memory consumption has introduced a bug that result
in spurious EEXIST with large batches.
The code that tracks the start and end elements of the interval can add
the same element twice to the batch. This works with the add element
command, since it ignores EEXIST error, but it breaks the the create
element command.
Update this codepath to ensure both sides of the interval fit into the
netlink message, otherwise, trim the netlink message to remove them.
So the next netlink message includes the elements that represent the
interval that could not fit.
Fixes: 91dc281a82ea ("src: rework singleton interval transformation to reduce memory consumption")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
commit 91dc281a82ea ("src: rework singleton interval transformation to
reduce memory consumption") duplicates singleton interval elements when
the netlink message gets full, this results in spurious EEXIST errors
when creating many elements in a set.
This patch extends the existing test to cover for this bug.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
When called from another directory without specifying test cases, an
incorrect regexp was used to glob all tests and no test was run at all:
| # ./tests/monitor/run-tests.sh
| echo: running tests from file *.t
| ./tests/monitor/run-tests.sh: line 201: testcases/*.t: No such file or directory
| monitor: running tests from file *.t
| ./tests/monitor/run-tests.sh: line 201: testcases/*.t: No such file or directory
| json-echo: running tests from file *.t
| ./tests/monitor/run-tests.sh: line 201: testcases/*.t: No such file or directory
| json-monitor: running tests from file *.t
| ./tests/monitor/run-tests.sh: line 201: testcases/*.t: No such file or directory
Fixes: 83eaf50c36fe8 ("tests: monitor: Become $PWD agnostic")
Signed-off-by: Phil Sutter <phil@nwl.cc>
CONFIG_SHELL=/bin/dash ./configure
breaks with:
./config.status: 2044: Syntax error: Bad for loop variable
Fixes: 64c07e38f049 ("table: Embed creating nft version into userdata")
Signed-off-by: Jan Palus <jpalus@fastmail.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
On a kernel with broken (never upstreamed) patch this fails with:
Accepted bad ruleset with jump from filter type to masquerade (3)
and
Accepted bad ruleset with jump from prerouting to masquerade
... because bogus optimisation suppresses re-validation of 'n2', even
though it becomes reachable from an invalid base chain (filter, but n2
has nat-only masquerade expression).
Another broken corner-case is validation of the different hook types:
When it becomes reachable from nat:prerouting in addition to the allowed
nat:postrouting the validation step must fail.
Improve test coverage to ensure future optimisations catch this.
Signed-off-by: Florian Westphal <fw@strlen.de>
Added cases for SNAT or DNAT only for active and passive modes.
Signed-off-by: Andrii Melnychenko <a.melnychenko@vyos.io>
Signed-off-by: Florian Westphal <fw@strlen.de>
Refactored the setup of nft rulesets, now it is possible to set up an
SNAT or DNAT-only ruleset for future tests.
Presented the testcase function to test passive or active modes.
Signed-off-by: Andrii Melnychenko <a.melnychenko@vyos.io>
Signed-off-by: Florian Westphal <fw@strlen.de>
Pablo reports 'make distcheck' got broken due to a bogus source file
added in the afl split:
make *** No rule to make target '-I./include', needed by 'distdir-am'. Stop.
Get rid of this line.
Fixes: 32c994f84904 ("src: move fuzzer functionality to separate tool")
Reported-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
Older kernels do not support netdev basechain without device, add it so
this works.
Alternative is to skip it by adding:
# NFT_TEST_REQUIRES(NFT_TEST_HAVE_netdev_chain_without_device)
but it seems easier to support it.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Florian Westphal <fw@strlen.de>
v2:
- Switched to range syntax instead of two matches as suggested by Phil.
Signed-off-by: Yi Chen <yiche@redhat.com>
Reviewed-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Florian Westphal <fw@strlen.de>
Set declaration + set flush results in a crash because CMD_OBJ_SETELEMS
does not expect no elements. This internal command only shows up if set
contains elements, however, evaluation flushes set content after the set
expansion. Skip this command CMD_OBJ_SETELEMS if set is empty.
Fixes: d3c8051cb767 ("rule: rework CMD_OBJ_SETELEMS logic")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
This means some loss of functionality since you can no longer combine
--fuzzer with options like --debug, --define, --include.
On the upside, this adds new --random-outflags mode which will randomly
switch --terse, --numeric, --echo ... on/off.
Update README to reflect this change.
Signed-off-by: Florian Westphal <fw@strlen.de>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
Tunnel object listing support was missing. Now it is possible to list
tunnels. Example:
sudo nft list tunnel netdev x y
table netdev x {
tunnel y {
id 10
ip saddr 192.168.2.10
ip daddr 192.168.2.11
sport 10
dport 20
ttl 10
erspan {
version 1
index 2
}
}
}
Fixes: a937a5dc02db ("src: add tunnel statement and expression support")
Signed-off-by: Fernando Fernandez Mancera <fmancera@suse.de>
Signed-off-by: Florian Westphal <fw@strlen.de>
afl comes with a compiler frontend that can add instrumentation suitable
for running nftables via the "afl-fuzz" fuzzer.
This change adds a "--with-fuzzer" option to configure script and enables
specific handling in nftables and libnftables to speed up the fuzzing process.
It also adds the "--fuzzer" command line option.
afl-fuzz initialisation gets delayed until after the netlink context is set up
and symbol tables such as (e.g. route marks) have been parsed.
When afl-fuzz restarts the process with a new input round, it will
resume *after* this point (see __AFL_INIT macro in main.c).
With --fuzzer <stage>, nft will perform multiple fuzzing rounds per
invocation: this increases processing rate by an order of magnitude.
The argument to '--fuzzer' specifies the last stage to run:
1: 'parser':
Only run / exercise the flex/bison parser.
2: 'eval': stop after the evaluation phase.
This attempts to build a complete ruleset in memory, does
symbol resolution, adds needed shift/masks to payload instructions
etc.
3: 'netlink-ro':
'netlink-ro' builds the netlink buffer to send to the kernel,
without actually doing so.
4: 'netlink-rw':
Pass generated command/ruleset will be passed to the kernel.
You can combine it with the '--check' option to send data to the kernel
but without actually committing any changes.
This could still end up triggering a kernel crash if there are bugs
in the valiation / transaction / abort phases.
Use 'netlink-ro' if you want to prevent nft from ever submitting any
changes to the kernel or if you are only interested in fuzzing nftables
and its libraries.
In case a kernel splat is detected, the fuzzing process stops and all further
fuzzer attemps are blocked until reboot.
Signed-off-by: Florian Westphal <fw@strlen.de>
Document the syntax of this meta-object used by "list" and "flush"
commands only.
Fixes: 872f373dc50f7 ("doc: Add JSON schema documentation")
Signed-off-by: Phil Sutter <phil@nwl.cc>
In cmd_obj enum hooks, tunnel and tunnels elements documentation were
missing.
Signed-off-by: Fernando Fernandez Mancera <fmancera@suse.de>
Signed-off-by: Florian Westphal <fw@strlen.de>
The kernel can form aggregate packets whether or not GSO is enabled.
Disabling GSO is not a useful suggestion in this case.
Fixes: 05628cdd677d (doc: describe behaviour of {ip,ip6} length)
Signed-off-by: Florian Westphal <fw@strlen.de>
If the systemd service file is not installed, currently the related man-page
and example nft file are still installed. Instead only install them when the
service file is installed.
Fixes: 107580cfa85c ("build: disable --with-unitdir by default")
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Florian Westphal <fw@strlen.de>
- Clarify that a terminating statement also prevents the execution of later
statements in the same rule and give an example about that.
- Correct that `accept` won’t terminate the evaluation of the ruleset (which is
generally used for the whole set of all chains, rules, etc.) but only that of
the current base chain (and any regular chains called from that).
Indicate that `accept` only accepts the packet from the current base chain’s
point of view.
Clarify that not only chains of a later hook could still drop the packet, but
also ones from the same hook if they have a higher priority.
- Various other minor improvements/clarifications to wording.
Link: https://lore.kernel.org/netfilter-devel/3c7ddca7029fa04baa2402d895f3a594a6480a3a.camel@scientia.org/T/#t
Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>
Signed-off-by: Florian Westphal <fw@strlen.de>
Relieve callers from having to suffix their messages with a newline
escape sequence, have the macro append it to the format string instead.
This is mostly a fix for (the many) calls to BUG() without a newline
suffix. Adjust the previously correct ones since they emit an extra
newline now.
Signed-off-by: Phil Sutter <phil@nwl.cc>
_get() functions must not be used when refcnt is 0, as expr_free()
releases expressions on 1 -> 0 transition.
Also, check that a refcount would not overflow from UINT_MAX to 0.
Use INT_MAX to also catch refcount leaks sooner, we don't expect
2**31 get()s on same object.
This helps catching use-after-free refcounting bugs even when nft
is built without ASAN support.
v3: use a macro + BUG to get more info without a coredump.
Signed-off-by: Florian Westphal <fw@strlen.de>
While its correct that the queue statement is internally implemented
via the queue verdict, this is an implementation detail.
We don't list "stolen" as a verdict either.
nft ... queue will always use the nft_queue statement, so move the
reinject detail from statements to queue statement and remove this.
Signed-off-by: Florian Westphal <fw@strlen.de>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
While executing the test suite from tests/shell folder, the following error
is displayed many times:
tests/shell/testcases/maps/vmap_timeout: line 48: [: : integer expected
Looking at the script, a non-existing variable (expires) is tested instead of
the existing one (expire).
Reproduction:
tests/shell/run-tests.sh -v
Fixes: db80037c0279 ("tests: shell: extend vmap test with updates")
Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com>
Signed-off-by: Florian Westphal <fw@strlen.de>
Overhaul the description of `jump`/`goto`/`return`.
`jump` only explains what the statement causes from the point of view of the
new chain (that is: not, how the returning works), which includes that an
implicit `return` is issued at the end of the chain.
`goto` is explained in reference to `jump`.
`return` describes abstractly how the return position is determined and what
happens if there’s no position to return to (but not for example where an
implicit `return` is issued).
List and explain verdict-like statements like `reject` which internally imply
`accept` or `drop`.
Further explain that with respect to evaluation these behave like their
respectively implied verdicts.
Link: https://lore.kernel.org/netfilter-devel/3c7ddca7029fa04baa2402d895f3a594a6480a3a.camel@scientia.org/T/#t
Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>
Signed-off-by: Florian Westphal <fw@strlen.de>
Included bogons assert:
Assertion `!expr_is_constant(*expr) || expr_is_singleton(*expr)' failed
This is because the "foo*" + prefix combination causes expr_evaluate
to replace the binop + string expression with another prefix that
gets allocated while handling "foo*" (wildcard).
This causes expr_evaluate_prefix to build
a prefix -> prefix -> binop chain.
After this, we get:
Error: Right hand side of relational expression ((null)) must be constant
a b ct helper "2.2.2.2.3*1"/80
~~~~~~~~~~^^^^^^^^^^^^^^^^
Error: Binary operation (&) is undefined for prefix expressions
a b ct helper "2.2.2.****02"/80
^^^^^^^^^^^^^^^^^
for those inputs rather than hitting assert() in byteorder_conversion()
later on.
Signed-off-by: Florian Westphal <fw@strlen.de>
Callers of netlink_gen_{key,data}() pass an uninitialized auto-variable,
avoid misinterpreting garbage in fields "left blank".
Signed-off-by: Phil Sutter <phil@nwl.cc>
