* basicdefs.h: Do not include alloca.h.
* bootstrap.conf (gnulib_modules): Remove alloca, stdalign,
c-ctype, closeout, mbrlen, verify.
Add version-etc.
* lib/.gitignore, m4/.gitignore: Omit now-unused files.
* po/POTFILES.in: Remove closeout.c.
* testsuite/test-mbrtowc.c: Do not include closeout.h.
(main): Close stdout by hand, to avoid a dependency on closeout.
Fix some some longstanding but unlikely integer overflows.
Internally, 'sed' now more often prefers signed integer arithmetic,
which can be checked automatically via 'gcc -fsanitize=undefined'.
* basicdefs.h (countT): Remove. All uses replaced
with a more-specific signed type, e.g., idx_t.
Similarly, change uses of types like size_t to
signed types like idx_t when appropriate.
(REALLOC): Remove; no longer used. We now use xpalloc
because that detects integer overflow in size calculations.
Also, we no longer use XCALLOC since the code never relies
on the storage being zero, and leaving it uninitialized is
more likely to catch errors when debugging implementations
are used. We use XNMALLOC instead, or xpalloc.
* bootstrap.conf (gnulib_modules): Add stdckdint, strtoimax.
* lib/.gitignore, m4/.gitignore: Update for new Gnulib modules.
* sed/compile.c: Include stdckdint.h.
(VECTOR_ALLOC_INCREMENT): Remove; no longer used.
(in_integer): Return maximal value if integer overflow.
All callers changed to expect this.
(next_cmd_entry): Use xpalloc instead of reallocating by hand,
which might suffer integer overflow.
(normalize_text): Don’t rely on system-defined conversion
of out-of-range size_t to int.
(next_cmd_entry): Arg is now pointer, not pointer-to-pointer.
All uses changed.
* sed/debug.c (debug_print_function): Don’t attempt to
fwrite a null pointer with a zero size.
* sed/execute.c: Include <stdckdint.h>, "minmax.h".
(resize_line): LEN arg is now increment, not total length,
to avoid overflow when calculating total length.
All uses changed. Do not assume lb->alloc * 2 cannot overflow.
(resize_line, line_copy): Use xpalloc instead of doing realloc by
hand, which might suffer integer overflow.
(str_append_modified): Do not add n to to->length until
after it's known this cannot overflow.
(read_file_line): Don’t assume ssize_t fits in long.
(get_backup_file_name): Don’t assume string length fits in int.
Do not assume PTR-1+1 works; behavior is undefined if PTR
is at buffer start. Check for integer overflow in buffer
size calculation.
(read_pattern_space): Check for line number overflow.
(match_address_p): Check for address overflow.
(debug_print_line): Omit unnecessary test for in->active being null.
(execute_program): Check for Q overflow.
* sed/regexp.c: Include <stdckdint.h>.
(match_regex): Don’t assume TYPE_MAXIMUM (regoff_t) == INT_MAX.
* sed/sed.c: Include inttypes.h, for strtoimax.
(main): Use strtoimax, not atoi.
* sed/utils.c (init_buffer): Use xmalloc and xpalloc
instead of guessing sizes ourselves, and unnecessarily
initializing.
(resize_buffer): Remove; all callers changed to use xpalloc.
(free_buffer): Don’t call free (NULL), since we already
test whether the pointer is null.
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* sed/regexp.c (compile_regex_1): Do away with use of DFA_CASE_FOLD.
This gnulib update pulls in a dfa module in which that symbol no
longer exists. Instead, it uses RE_ICASE in syntax bits.
* m4/.gitignore: Add files created by running bootstrap.
* lib/.gitignore: Likewise.
The following definitions caused trouble for no gain. They caused
some .o file names to have a long additional prefix and even
resulted in some .Po files not being removed by "make distclean"
when building with recent automake. The only reason to use these
definitions was -- long ago -- to require different compiler options
for files in lib/ than elsewhere. That is no longer necessary.
* lib/local.mk (lib_libsed_a_CPPFLAGS): Remove definition.
(lib_libsed_a_CFLAGS, lib_libsed_a_LIBADD): Likewise.
(lib_libsed_a_DEPENDENCIES): Likewise.
Sed incorrectly matches the anchor '^' to a newline character
(ASCII 10) despite using NUL line terminators (-z).
See https://bugs.gnu.org/24615 .
Previous (incorrect) behavior:
$ printf "a\0b\0" | sed -z 'N;s/^/X/mg;' | od -An -a
X a nul b nul
With this fix:
$ printf "a\0b\0" | sed -z 'N;s/^/X/mg;' | od -An -a
X a nul X b nul
Current regex engine can not handle NUL multiline anchors.
Implement such search manually (by breaking the pattern into separate
NUL-terminated lines) and searching each one individually.
* NEWS: Mention it.
* sed/regexp.c (compile_regex_1): Don't use newline_anchor of regex, if
the buffer delimiter is not newline character.
(match_regex): Do above case line-by-line.
* testsuite/zero-anchor.good, testsuite/zero-anchor.inp,
testsuite/zero-anchor.sed: New test.
* testsuite/local.mk: Add the test.
* testsuite/Makefile.tests: Add the test.
* bootstrap.conf: Add memrchr.
* lib/.gitignore, m4/.gitignore: Ignore memrchr files.
* doc/local.mk (distclean-local): Remove rule. Moved to ...
* Makefile.am (distclean-local): ...here, and add a command to
remove all .deps directories. It could have stayed in that .mk
file, but since with this change, it is no longer specific to doc/,
the top level makes more sense.
* lib/local.mk (MOSTLYCLEANFILES): Remove this. Covered by the above.
Reported by Assaf Gordon in
https://lists.gnu.org/archive/html/sed-devel/2016-08/msg00019.html
* doc/local.mk (doc/sed.1): Depend not on sed.c, but on the binary
that help2man must run. Also depend on .version, to be sure we
pick up a version change. Also, be careful not to write directly
to $@: instead write to $@-t, and make read-only to mark as generated,
and then, only if all is well, rename atomically to $@.
Also, prefix commands with $(AM_V_GEN) and $(AM_V_at) and ensure
that doc/ exists via $(MKDIR_P).
(distclean-local): New target. Use this instead of ...
(dist-hook-man-page): ... this. Remove rule.
(SEDBIN): Remove unnecessary $(top_builddir)/ prefix.
(doc/sed.texi, doc/s-texi): Remove some uses of $(top_builddir).
Replace others with $(srcdir).
* Makefile (dist-hook): remove dependency on dist-hook-man-page.
* lib/local.mk (MOSTLYCLEANFILES): Arrange for lib/.deps/*.Po
to be removed as part of "make distclean", so that make distcheck
no longer fails when using latest automake.
* .gitignore: Ignore '.dirstamp' files
* bootstrap.conf: Add 'non-recursive-gnulib-prefix-hack' gnulib module.
Add missing 'stdalign' module.
(bootstrap_post_import_hook): Use module to adjust gnulib.mk file.
* configure.ac: Add 'subdir-objects' automake option, don't generate
makefiles in subdirectories. Adjust path of XFAIL tests.
* Makefile.am: Include 'local.mk' files, adjust for non-recursive make.
* doc/Makefile.am: Rename to ...
* doc/local.mk: Adjust paths. Rename SED to
SEDBIN variable, to avoid overriding the default system-wide SED value
which is automatically set by autotools.
* lib/Makefile.am: Rename to ...
* lib/local.mk: Adjust paths.
* lib/.gitignore: Ignore 'stdalign.h' from gnulib module.
* sed/Makefile.am: Rename to ...
* sed/local.mk: Adjust paths.
* testsuite/Makefile.am: Rename to ...
* testsuite/local.mk: Adjust paths.
* testsuite/init.cfg: Move to ...
* init.cfg: ... here. Moved to avoid changing the path in init.sh, and to
keep the same directory structure as coreutils.
(require_en_utf8_locale_): Adjust path for get-mb-cur-max executable.
(require_ja_shiftjis_locale_,require_valid_ja_shiftjis_locale_): Adjust
path for test-mbrtowc executable.
* m4/.gitignore: Add gnulib's non-recusive-make module.
* testsuite/runtest: Adjust paths of Makefile.tests and dir variables,
Skip utility program 'get-mb-cur-max' which is not a standalone test.
* testsuite/Makefile.tests: Adjust path of sed exeutable.
* testsuite/eval.sed, testsuite/eval.good: Adjust path of sed executable.
* testsuite/cmd-R.sh,
testsuite/cmd-l.sh,
testsuite/colon-with-no-label.sh,
testsuite/comment-n.sh,
testsuite/compile-errors.sh,
testsuite/compile-tests.sh,
testsuite/convert-number.sh,
testsuite/execute-tests.sh,
testsuite/follow-symlinks-stdin.sh,
testsuite/follow-symlinks.sh,
testsuite/help-version.sh,
testsuite/in-place-hyphen.sh,
testsuite/in-place-suffix-backup.sh,
testsuite/invalid-mb-seq-UMR.sh,
testsuite/mb-bad-delim.sh,
testsuite/mb-charclass-non-utf8.sh,
testsuite/mb-match-slash.sh,
testsuite/mb-y-translate.sh,
testsuite/normalize-text.sh,
testsuite/nulldata.sh,
testsuite/panic-tests.sh,
testsuite/posix-char-class.sh,
testsuite/posix-mode-N.sh,
testsuite/posix-mode-addr.sh,
testsuite/posix-mode-bad-ref.sh,
testsuite/posix-mode-s.sh,
testsuite/range-overlap.sh,
testsuite/recursive-escape-c.sh,
testsuite/regex-errors.sh,
testsuite/stdin-prog.sh,
testsuite/subst-mb-incomplete.sh,
testsuite/subst-options.sh,
testsuite/subst-replacement.sh,
testsuite/temp-file-cleanup.sh,
testsuite/unbuffered.sh: Adjust paths of init.sh, sed executable.
Sed tests purposefully use invalid non-utf8 multibyte sequences to
test sed's handling of invalid input/program.
Some implementations wrongly accept invalid sequences, causing false
alarms. Add a test program to detect such buggy implementations and skip
the tests if needed.
See discussion in:
http://lists.gnu.org/archive/html/sed-devel/2016-07/msg00005.htmlhttp://lists.gnu.org/archive/html/sed-devel/2016-07/msg00000.htmlhttp://lists.gnu.org/archive/html/sed-devel/2016-06/msg00031.htmlhttp://lists.gnu.org/archive/html/sed-devel/2016-06/msg00014.html
* bootstrap.conf: Add gnulib's closeout module.
* m4/.gitignore, lib/.gitignore, po/POTFILES.in: Adjust after adding
closeout module.
* testsuite/Makefile.am (check_PROGRAMS): Add new program.
* testsuite/test-mbrtowc.c: New program.
* testsuite/init.cfg (require_valid_ja_eucjp_locale_)
(require_valid_ja_shiftjis_locale_): New functions. Call test-mbrtowc
with known invalid multibyte sequences to ensure implementation is not
buggy and rejects them; Skip test otherwise.
* testsuite/invalid-mb-seq-UMR.sh: Use require_valid_ja_eucjp_locale_.
* testsuite/mb-charclass-non-utf8.sh: Use require_valid_ja_shiftjis_locale_.
* ChangeLog-2014: Renamed from ChangeLog.
* po/ChangeLog-2014: Renamed from po/ChangeLog.
* Makefile.am (EXTRA_DIST): Add the two new file names.
(dist-hook): Generate ChangeLog at tarball-creation time.
(gen-ChangeLog): New rule. Just like the one in coreutils and grep.
* bootstrap.conf (gnulib_modules): Add the modules, readme-release
and gitlog-to-changelog, and revamp to be much more like grep and
coreutils.
* lib/Makefile.am: Update.
2010-07-16 Paolo Bonzini <bonzini@gnu.org>
* autoboot.conf (gnulib_modules): Replace mkstemp with mkostemp.
* bootstrap.sh.in: Compile mkostemp.c instead of mkstemp.c.
* gnulib: Update.
* NEWS: Mention change.
* sed/utils.c (ck_mkstemp): Use mkostemp to work around Cygwin
oddity. Reported by Stepan Kasal.
