Phil Sutter 9d105581b5 scanner: Introduce SCANSTATE_RATE
This is a first exclusive start condition, i.e. one which rejects
unscoped tokens. When tokenizing, flex all too easily falls back into
treating something as STRING when it could be split into tokens instead.
Via an exclusive start condition, the string-fallback can be disabled as
needed.

With rates in typical formatting <NUM><bytes-unit>/<time-unit>,
tokenizer result depended on whitespace placement. SCANSTATE_RATE forces
flex to split the string into tokens and fall back to JUNK upon failure.
For this to work, tokens which shall still be recognized must be enabled
in SCANSTATE_RATE (or all scopes denoted by '*'). This includes any
tokens possibly following SCANSTATE_RATE to please the parser's
lookahead behaviour.

Signed-off-by: Phil Sutter <phil@nwl.cc>
Reviewed-by: Florian Westphal <fw@strlen.de>
2026-01-20 16:03:12 +01:00
..
2026-01-20 16:03:12 +01:00