49 Commits

Author SHA1 Message Date
Rob Landley
58500b8029 Adjust various tests to pass on TEST_HOST debian bookworm. 2025-01-06 09:47:13 -06:00
Rob Landley
16387c91d0 Switch testing->testcmd. 2023-04-05 17:19:09 -05:00
Rob Landley
cb2e4cc6e8 Add a sed test. 2023-04-05 17:00:31 -05:00
Rob Landley
abd8b1b221 Fix s/// tests get skipped before realloc() allocates new string memory.
(They tried to copy the unchanged data to a destination that didn't exist
yet: skip copy for null pointer and defer copy to allocation time.)
2023-02-17 15:35:18 -06:00
Rob Landley
05069bdf6a Couple more sed tests. 2022-10-18 01:00:32 -05:00
Rob Landley
51815ade0b Once again, mksh is missing bash features. 2022-10-18 00:35:15 -05:00
Rob Landley
49329a11f7 Add test for sed -e -f interleaving. 2022-10-13 00:54:36 -05:00
Rob Landley
5262c53874 Add sed --tarxform mode to wrap a simple protocol around transforms,
and have tar --xform use it.
2022-09-28 01:27:26 -05:00
Rob Landley
0ceee5c382 Fix some sed -z cases and add tests. 2022-09-24 19:39:44 -05:00
Rob Landley
4fe59859d4 Match another sed posix violation (l escapes \n), more tests, tweak docs. 2022-09-13 08:31:33 -05:00
Rob Landley
f2f35fe829 Fix sed -z P with test. 2022-09-13 08:29:28 -05:00
Rob Landley
5147183a21 Fix sed -z and add test cases. 2022-09-13 08:02:03 -05:00
Rob Landley
b607b57909 Fix sed bug with multiple w to same file. 2022-01-20 13:30:45 -06:00
Rob Landley
773d404ee2 Bugfix: sed s command couldn't skip initial match. 2021-04-26 05:09:57 -05:00
Elliott Hughes
50d8ed89b1 sed: add 'x' flag to the 's' command.
The GNU tar manual, when talking about the `tar --transform` option that I
need to implement, describes the 'x' flag by saying "regexp is an extended
regular expression (see section 'Extended regular expressions' in GNU sed)".

Only it turns out that even the latest GNU sed doesn't actually have
that flag. It's unique to `tar --transform`. That link is just telling
you that the sed manual will explain extended regular expressions, not
that GNU sed also supports the 'x' flag.

So I don't know whether we want this in toybox sed after all. (It made
sense that sed would have such a flag, but no sed that I know of
actually does.)
2020-12-07 00:46:54 -06:00
Rob Landley
c0dca293c1 The bionic/clang asan plumbing slows the test down >10x, so expand timeout. 2020-06-27 03:14:49 -05:00
Rob Landley
2e2f76ed5a Annotate sed tests with toyonly. 2020-05-29 00:31:03 -05:00
Rob Landley
8098f58af1 Add tests for sed -s 2020-05-29 00:28:47 -05:00
Elliott Hughes
92975af3f2 sed s///: support POSIX I as synonym for i
POSIX chose I rather than i as the case-insensitive flag for s///,
because apparently more seds support I than i. We're allegedly alone in
only supporting i. (On the Mac, sed supports neither.)

Strictly this isn't *currently* in POSIX, but it's been accepted for
issue 8.

Bug: https://austingroupbugs.net/view.php?id=779#c2050
2020-03-28 00:57:53 -05:00
Rob Landley
6d2205e813 Teach sed about +N range ending. 2019-05-26 08:15:44 -05:00
Rob Landley
48162c4ee3 Greatly simplify and speed up regexec0() using REG_STARTEND.
This is a 15 year old freebsd extension (presumably thus also available on
MacOS) that glibc adopted in 2004, uClibc adopted in 2005, and bionic
supports. The only thing that DOESN'T support it is musl, once again
because its maintainer explicitly decided not to
(https://www.openwall.com/lists/musl/2013/01/15/26), so add an #ifdef
to let musl stay uniquely broken. (It'll stop at first NUL, everything
else can match NULs).

Finally fixes "s/x/y/g on a megabyte line of x's takes forever" issue.
2019-05-06 13:16:24 -05:00
Elliott Hughes
b928ec480c sed: fix endless loop in "b loop" test.
Very few places actually check for errors from emit, and I actually see
the same endless loop from "sed (GNU sed) 4.4" on current Debian, so I'm
not sure this isn't Broken As Designed, but an endless loop spewing
"short write" (or saying nothing, in the case of GNU sed) really doesn't
feel like useful behavior in face of EPIPE, which really isn't going to
fix itself. Certainly not being able to run the sed tests to completion
is pretty annoying --- which is why, unless we remove this test as
invalid, we should probably also add a SKIP_HOST=1 to the "b loop" test.

Note that even with this fix you'll see the error twice:

  sed: short write: Broken pipe
  sed: short write: Broken pipe

Once from the first = command to fail, and then another from the !FLAG(n)
flush of the pattern space.
2019-02-13 05:51:27 -06:00
Elliott Hughes
7079a558d6 sed: fix substitution of empty capturing groups.
The test for \N where N was larger than the number of capturing groups
in the regular expression was incorrect, and firing for cases such as
matching __(ARM_)?NR_([a-z]*) against __NR_read, where the first group is
empty (because it failed to match) but the second group did match "read".

Use regex_t's re_nsub for the error check, and treat rm_so == -1 as a
signal to just copy nothing into the result.

(Found trying to build minijail in AOSP.)
2019-02-12 21:31:20 -06:00
Elliott Hughes
411f7fc478 sed: add -z.
Used to construct SELinux policies in the AOSP build.

I left loopfiles_lines with its hard-coded '\n' because although cut(1)
also has a -z option, I can't find any case where it's used in any of
the codebases searchable by me. (And fmt(1), the other user, doesn't
even have the option.) YAGNI.

Bug: http://b/122744241
2019-01-12 13:37:30 -06:00
Rob Landley
1f0cfeef3c Fix b ending with }
For a definition of "fix" that's even _more_ of a deviation from posix, but
matches what debian does...
2019-01-11 22:01:44 -06:00
Elliott Hughes
323779dde2 sed: add test for AOSP build breakage.
Bug: http://b/122744241
2019-01-11 21:11:59 -06:00
Rob Landley
64cbbab181 A few more sed tests. 2016-10-17 23:52:51 -05:00
Rob Landley
337c072ac0 Teach sed s/// how to handle [:space:] type sequences.
Or more accurately, s@[[:space:]@]@replace@ which can't treat the @ in
[] as a delimiter but has to know about nested [[]] to make that decision.
2016-09-06 00:14:24 -05:00
Rob Landley
c7f0d6ce89 Common test infrastructure says command name, don't have sed.tests say 'sed sed' 2016-07-21 16:37:03 -05:00
Rob Landley
f9b9f8a1a4 sed -f - should read from stdin. 2016-04-11 11:32:36 -05:00
Rob Landley
af51034338 Debian bug 635570 did something horribly nonstandard (depending on a side
effect of "sed -e 'a\'" with no trailing line). But there's an actual user,
and it's not hard to implement, and it's not hard to implement, and there
isn't an obvious _other_ way to do it, so add the behavior and a test for it.

Fix some bad/missing comments while I was there, and add a couple TODOs.
2016-03-24 16:20:36 -05:00
Andy Chu
516f3d620f Fix bug where all tests aren't being run with 'make test'.
The tests/*.test files shouldn't explicitly exit, because they are
sourced in scripts/test.sh.  No tests after sed were being run.
2016-03-17 14:13:30 -05:00
Andy Chu
96a5ed118c Fix segfault in sed -e 'c\'.
Found by afl-fuzz.
2016-03-10 13:59:32 -06:00
Rob Landley
336c44adca Factor out command name at the start of test name, have runtest.sh print it. 2016-03-02 15:20:04 -06:00
Rob Landley
32b3587af2 The perl build's attempt to escape spaces and such in LD_LIBRARY_PATH is _SAD_.
It uses a sed expression that assumes you can escape - to use it as a literal
(you can't, it has to be first or last char of the range), and assumes
you have to escape delimiters in sed [] context (you don't), and/or that
non-printf escapes become the literal character (they don't, the backslash
is preserved as a literal), meaning it winds up doing "s/[\-\]//" which is
a length 1 range, which is officially undefined behavior according to posix,
and regcomp errors out.

But if we don't accept it (like other implementations do) the perl build
breaks. So collapse [A-A] into just [A].

Testcae taken from perl 5.22.0 file Makefile.SH line 8.

(While we're at it, remove an unused argument from a function.)
2016-02-17 19:21:44 -06:00
Rob Landley
2081ec658c Add a sed test from the posix mailing list. 2016-01-21 13:04:51 -06:00
Rob Landley
88d207790e Replace duplicate sed test with case we hadn't covered yet. 2015-12-04 15:18:15 -06:00
Rob Landley
769341fdd7 Fix sed bug where any ] right after [ was skipped, not just first one in range. 2015-11-05 00:13:27 -06:00
Rob Landley
8132ad2f4e Fix sed bug reported by Isabella Parakiss, where sed -e "/x/c\" -e "y" added
an extra newline because the test for whether we have an existing string to
append a newline to was checking if struct step had data appended to it,
and the /x/ regex is data appended to it. Change test to check for null
terminator at ->arg1 offset.
2015-10-29 01:30:58 -05:00
Rob Landley
dec4669fa1 Set the executable bits on all the *.test files. (Wasn't consistent, is now.) 2015-05-12 13:24:26 -05:00
Rob Landley
a64e35b336 Fix sed bug David Halls hit trying to compile libiconv. 2015-03-28 20:21:03 -05:00
Rob Landley
c09b79dc71 Another sed bug. (The e2fsprogs build uses multiple line continuations on the same command.) 2014-12-21 23:17:06 -06:00
Rob Landley
807a50d899 Make sed a\ line continuations work properly for different pattern input modes. 2014-12-14 13:51:28 -06:00
Rob Landley
e7835d79c5 sed: c needs to trigger range logic like d, D works like d when there isn't anything left in the line, and more tests. 2014-11-27 20:38:21 -06:00
Rob Landley
0c558f09d9 Debugging pass on sed: make the existing test suite pass. 2014-11-15 16:16:29 -06:00
Rob Landley
1d5f48f3c1 Add a few more sed tests. 2014-11-14 16:44:21 -06:00
Rob Landley
eece7ed546 Yet more sed tests. 2014-11-09 14:16:33 -06:00
Rob Landley
b069d8da8c More sed tests. 2014-11-06 16:26:59 -06:00
Rob Landley
448c874348 First batch of sed tests.
Only good for TEST_HOST=1 at the moment because the test infrastructure itself
depends on sed, so if an unfinished sed is in the $PATH it goes boing. But
hey, corner cases!

I have... more.
2014-10-29 18:44:33 -05:00