It's illegal for the user-supplied function to modify the hash table
while executing hash_map*() methods. snap_file() may add elements
due to EXTRA_PREREQS, so we can't call it via hash_map_args().
Diagnosis and test by Shim Manning <shimmanning@gmail.com>
Possible fix by Dmitry Goncharov <dgoncharov@users.sf.net>
* src/hash.h (hash_table): Remember when we're in hash_map*().
* src/hash.c (hash_init): Initialize the in-map value to false.
(hash_map): Set the in-map value to true when walking the map.
(hash_map_arg): Ditto.
(hash_insert_at): Assert we're not in hash_map*()
(hash_free_items): Ditto.
(hash_delete_items): Ditto.
(hash_free): Ditto.
* src/file.c (snap_deps): Use hash_dump() to create a copy of the
hash and walk that to call snap_file(), so we can add more files.
(snap_file): Since we're calling it directly, don't use void* args.
When using Perl's backticks make sure that the path to the command
invoked is quoted. On Windows, in particular, paths to commands such
as diff, etc. may contain whitespace.
It's clear that this change causes too many problems to be made
without warning. Revert the change disallowing conditional lines to
start with TAB.
Instead, generate warnings whenever a directive line begins with a
TAB character. Make this change for all directives, not just
conditional directives: define, undefine, export, unexport, vpath,
load, include, etc.
* NEWS: Update the backward-compatibility warning.
* src/read.c (eval): Track whether the line starts with a TAB.
If so then whenever we recognize a directive, emit a warning.
Revert the previous change for this bug.
(parse_var_assignment): Accept a file location if the line begins
with TAB; show a warning if we discover a directive.
(conditional_line): Warn about lines starting with TAB.
* tests/scripts/...: Add tests to verify warnings for initial TAB.
Avoid Emacs local variable settings to disable whitespace cleanup,
by creating markup tokens for TAB characters (#TAB#) and space
characters (#SPACE#) (for end-of-line spaces). Modify all the
tests that use TABs to use the new markup. This requires changing
some old-fashioned tests to use the modern run_make_test(). Also
remove some of the comments regarding test numbers now that the
framework keeps track.
Set the jobserver pipe to non-blocking before writing tokens; if a
token write fails the user's jobserver value is too large so fail.
Original implementation: Dmitry Goncharov <dgoncharov@users.sf.net>
* src/posixos.c (set_blocking): Split into force_blocking() which
always enables/disables blocking; set_blocking() may call it.
(jobserver_setup): Set the write side of the pipe to non-blocking
before writing tokens. If it fails with EAGAIN we know the pipe
is full: create a fatal error.
* tests/scripts/features/jobserver: Test a too-large jobserver.
Make sure the jobserver named pipe is deleted even if the
jobserver_setup() function calls fatal() and exits early.
* src/main.c (clean_jobserver): Always reset_jobserver().
* src/posixos.c (jobserver_setup): Set job_root up front.
* tests/test_driver.pl: Print out the logfile pathname on error.
For tests with regex matches this might be the only file available.
Don't print the base filename unless one is created.
An explicitly mentioned double-colon target cannot be intermediate.
* src/read.c (record_files): Set file->is_explicit in the case of an
explicitly mentioned file built by a double colon rule.
* src/file.c (print_file): Have print_file print whether a file is
explicitly mentioned.
* tests/scripts/features/patternrules: Add tests.
* src/file.c (remove_intermediates): Fix an error message about a
failure to remove an intermediate file.
* tests/scripts/features/patternrules: Add tests.
Suggested patch by Hannes Domani <ssbssa@yahoo.de>
* src/remake.c (notice_finished_file): When run with -n, mark
also_make targets as updated.
* tests/scripts/options/dash-n: Test pattern and grouped targets.
Make enters an infinite loop when some option and MAKEFLAGS=<value>
are specified on the command line. For example,
$ make -r MAKEFLAGS=hello=world
If decode_switches() runs handle_non_switch_argument() from within
the getopt() loop, it would recursively call decode_switches() to
enter a new getopt() loop, corrupting the state of the outer loop.
* src/main.c (decode_switches): Save up non-option arguments and run
handle_non_switch_argument() only after we're done with getopt().
* tests/scripts/variables/MAKEFLAGS: Add tests.
Add a warning to control circular dependency detection. Use "warn" as
the default action to be backward-compatible.
* src/warning.h (enum warning_type): Add warning type wt_circular_dep.
* src/warning.c (warn_init): Set default wt_circular_dep to w_warn.
* src/remake.c (update_file_1): Consult the circular-dep warning to
handle circular dependencies.
* tests/scripts/options/warn: Test --warn circular-dep flag.
* tests/scripts/variables/WARNINGS: Test .WARNINGS circular-dep flag.
* doc/make.texi: Document circular-dep warning.
* doc/make.1: Ditto.
Make crashes when -r and MAKEFLAGS= are specified on the command line.
On startup make begins to process command line arguments.
During processing of "MAKEFLAGS=" make calls reset_makeflags, which in
turn calls disable_builtins, which dereferences null suffix_file.
Here is the backtrace.
0 disable_builtins main.c:3482
1 reset_makeflags main.c:3104
2 set_special_var variable.c:1325
3 do_variable_definition variable.c:1693
4 try_variable_definition variable.c:1889
5 handle_non_switch_argument main.c:3021
6 decode_switches main.c:3150
7 main main.c:1621
* src/main.c (disable_builtins): Avoid dereferencing null suffix_file.
* tests/scripts/features/suffixrules: Add a test.
* src/function.c (func_intcmp): Reverse comparison direction when
both arguments are negative.
* tests/scripts/functions/intcmp: Add tests and remove useless ones.
POSIX requires that a conforming makefile should not use -e if
1) make is invoked with -i
2) A .IGNORE target exists with no prerequisites
3) The current target is a prerequisite of .IGNORE
* src/job.c (start_job_command): Add the target's flags when
constructing argv so it can check (3) above.
(construct_command_argv_internal): Don't set shellflags if it's not
set: this only happens if we're parsing for the slow path and we
don't need them.
(construct_command_argv): Don't allocate buffers if not needed.
When detecting "-ec", check the global ignore_errors_flag and the
current command line flags.
* tests/scripts/targets/IGNORE: Add tests for .IGNORE.
* tests/scripts/targets/POSIX: Add tests for the three cases above.
Appending to a pattern specific variable produces an incorrect value
in the presence of a command line definition or an env override of
the variable. Also, fix pattern/target-specific appending to a
variable with origin override.
* At parse time record_target_var sets the value of a pattern
specific variable to the value defined on command line or to the
value of the env override.
* Later, at build time, recursively_expand_for_file appends this
value of the variable (set in record_target_var) to the command
line value again, regardless of the origin of the variable.
This patch modifies recursively_expand_for_file to avoid appending,
unless the origin of the variable beats or equals the origin of one
of the parent definitions of this variable.
Reported by Rob <robw9739@gmail.com>,
Brian Vandenberg <phantall@gmail.com>,
Markus Oberhumer <markus@oberhumer.com>.
* NEWS: Note the change.
* src/variable.c (do_variable_definition): Avoid merging a
pattern-specific variable with the parent definition when a command
line or env override is present.
* src/expand.c (recursively_expand_for_file): Avoid appending to a
pattern-specific variable, unless the origin of this pattern-specific
variable beats or equals the origin of one of the parent definitions
of this variable.
* doc/make.texi (Override Directive): Add missing cross-reference.
* tests/scripts/variables/append: Add tests.
Ensure the origin of all variables inherited from the environment is
"environment override" if -e is given. Previously only variables
that were set in the makefile had this origin.
PDS: Most of these changes are from Dmitry but I slightly modified
the algorithm: instead of rearranging the way in which MAKEFLAGS is
parsed we reset the env_override value to the default before we
re-parse MAKEFLAGS, then we set the origin of all env vars to the
correct value based on the new setting.
* NEWS: Mention the change for backward-compatibility.
* src/main.c (main): Ensure MAKEFLAGS is always marked special.
(reset_makeflags): Set env_overrides back to default before parsing
MAKEFLAGS.
(decode_switches): Call reset_env_override() to check for changes.
* src/variable.h (reset_env_override): Declare a new function.
* src/variable.c (reset_env_override): Go through all env variables
and ensure the origin is correct based on env_overrides.
(set_env_override): Helper function for the hash.
* tests/scripts/functions/foreach: Fix tests.
* tests/scripts/functions/let: Ditto.
* tests/scripts/functions/origin: Ditto.
* tests/scripts/options/dash-e: Add tests.
Rework the single "?=" operator to instead allow a "?" modifier to be
prepended to ANY assignment operator. If "?" is given then the
variable is assigned (using whatever operator comes next) if and only
if the variable is not already defined. If it is defined then no
action is taken (the right-hand side is not expanded, etc.)
* NEWS: Announce this new feature.
* doc/make.texi: Modify the documentation around assignment operators.
* src/variable.h: Remove the f_conditional variable flavor.
(do_variable_definition): Add an argument specifying conditional.
* src/variable.c (parse_variable_definition): Use the existing flag
"conditional" to remember if we saw "?" rather than the flavor.
When we see "?" skip it and continue trying to parse an assignment.
(try_variable_definition): Pass condition to do_variable_definition().
(initialize_file_variables): Ditto.
(do_variable_definition): Check for conditional up-front: quit if set.
Remove handling of obsolete f_conditional flavor.
* src/read.c (eval_makefile): MAKEFILE_LIST is not conditional.
(do_define): Unset conditional for define with no operator. Pass the
conditional flag to do_variable_definition().
(construct_include_path): .INCLUDE_DIRS is not conditional.
* src/load.c (load_file): .LOADED is not conditional.
* tests/scripts/variables/conditional: Add new tests.
* src/read.c (make_word_type): Remove w_varassign from the enum.
(get_next_mword): This function is never called on a string in a place
where we might find a variable assignment. Any variable assignments
have already been handled via try_variable_definition(), so we don't
need to check for them here.
* tests/scripts/variables/flavors: Check triple-colon assignment in
target-specific variables.
Reported-by: MIAOW Miao <guoyr_2013@hotmail.com>
Patch from: Henrik Carlqvist <hc981@poolhem.se>
Test from: Dmitry Goncharov <dgoncharov@users.sf.net>
* src/expand.c (recursively_expand_for_file): Check the variable name
before checking for equality so we don't overrun the buffer.
* tests/scripts/functions/shell: Add a test with a very long variable.
Add an option to print a list of targets defined in the makefiles.
Don't print targets of implicit rules, or special targets. To
support this remember which files are deemed suffix rule targets.
Add a missing warning for single-suffix targets with prerequisites.
Suggested by many. Sample implementation by Tim <tdhutt@gmail.com>.
* NEWS: Announce the new option and single-suffix warning.
* doc/make.1: Add --print-targets to the man page.
* doc/make.texi: Add --print-targets to the documentation. Clean up
the text around the definition of suffix rules.
* src/main.c (print_targets_flag): New variable for --print-targets.
(switches): Add a new long option --print-targets.
(main): If the option was provided call print_targets() and exit.
* src/filedef.h (struct file): Add a "suffix" boolean value. Remove
print_prereqs() since it's static. Add new print_targets().
* src/file.c (rehash_file): Merge the new suffix value.
(print_prereqs): Used only locally: change to static.
(print_target): Print targets which are not suffix rule targets and
are not special targets.
(print_targets): Call print_target() on each file.
* src/rule.c (convert_to_pattern): Make maxsuffix local; it doesn't
need to be static. Emit ignoring prerequisites for single-suffix
rules as well as double-suffix rules. Remember which files are
actually suffix rules.
* tests/scripts/features/suffixrules: Test single-suffix behavior.
* tests/scripts/options/print-targets: Add tests for --print-targets.
Ensure that we correctly skip the entirety of a macro or function
reference when searching for the "," separator in an ifeq/ifneq
conditional, including using "$," and also "${foo,bar}". Note that
this change means that parenthesis OTHER than those used for variable
expansion are not considered special, any longer.
* NEWS: Announce the change.
* src/read.c (conditional_line): Skip variable references when looking
for "," and ensure that we match closing parens/braces properly.
* tests/scripts/features/conditionals: Add tests for this behavior.
* src/rule.c (snap_implicit_rules): Set need_2nd_expansion of each
prerequisite of global .EXTRA_PREREQS.
* src/file.c (snap_file): Set need_2nd_expansion of each prerequisite
of target-specific .EXTRA_PREREQS.
* tests/scripts/variables/EXTRA_PREREQS: Add tests.
Reported by Daniel Gerber <dg@atufi.org>.
Modify areas dealing with large command lines to use the heap rather
than relying on alloca / stack space.
* src/main.c (main): Allocate potentially large buffers with xmalloc.
(decode_env_switches): Ditto.
* src/function.c (func_error): Replace alloca with xmalloc/free.
* tests/scripts/features/expand: Add a newline for readable diffs.
* NEWS: Mention this change.
* src/read.c (eval): Check for ignoring for any line even if not
in a rule context.
* tests/scripts/features/conditionals: Write new tests.
If a loaded object defines a symbol <object>_gmk_unload, assume it's
a function and invoke it whenever the loaded object is unloaded.
Original implementation by Dmitry Goncharov <dgoncharov@users.sf.net>
* NEWS: Announce the change.
* doc/make.texi: Describe the behavior.
* src/gnumake.h: Add information to the comments.
* src/makeint.h (unload_all): Declare a new function.
* src/main.c (die): Invoke unload_all().
* src/load.c (unload_func_t): Declare a new type for unload.
(struct load_list): Remember the unload symbol if it exists.
(load_object): Move the parsing of the object name from load_file.
Check for the _gmk_unload symbol and if found, remember it.
(load_file): Allow load_object to do object filename parsing.
(unload_file): Remove the load_list entry when unloading the object.
(unload_all): Unload all the loaded objects.
* tests/scripts/features/loadapi: Test the unload function.
Add an ABI version both to the header file and passed to the setup
function. Unfortunately this itself is an ABI break and I couldn't
find a good way to avoid it.
* NEWS: Announce the ABI is not a preview and the incompatibility.
* doc/make.texi: Remove the preview warnings for object loading.
Document the new ABI version argument.
* src/gnumake.h (GMK_ABI_VERSION): Set the ABI version to 1.
Add comments documenting the format of the setup function.
* src/load.c (setup_func_t): Rename from load_func_t.
(load_file): Pass the ABI version to the setup function.
* tests/scripts/features/load: Rework the setup function.
* tests/scripts/features/loadapi: Ditto.
Previously "load:" worked but "load :" failed. Allow the latter as
well. This doesn't fix all issues; "load foo :" is still treated
as a load operation for "foo" and ":". Avoids SV 50413 as well.
* tests/scripts/features/load: Write tests.
When the expanded value of the variable in buf occupies more space
than available in variable_buffer, function variable_buffer_output
reallocates variable_buffer: return a pointer into the new memory,
not the old memory.
* src/expand.c (expand_variable_buf): Preserve the offset of buf and
return that offset into the (potentially reallocated) buffer.
* tests/scripts/features/expand: Add tests.
Disable builtin variables and rules immediately, when -R or -r is
added to MAKEFLAGS inside the makefile.
* src/main.c (disable_builtins): Add new function disable_builtins().
(main): Call disable_builtins().
(reset_makeflags): Call disable_builtins().
* tests/scripts/options/dash-r: Add tests.
* tests/scripts/variables/MAKEFLAGS: Update tests.
Avoid cross-contamination between test files by creating a new
working directory for each file, and setting it as the current
directory before starting the tests in that file.
Rename the test output as tNNN.{base,log,diff,mk} where NNN is
a test number starting with 001 for the first test. It is
slightly more annoying to find diff files since you can't use
autocomplete directly but it is simpler to match things.
Detect the source directory as the location of the test_driver.pl
script, so remove the separate -srcdir option.
* Makefile.am: Remove hacks to create symlinks when building
out-of-tree, and remove -srcdir option from run_make_tests.
* tests/test_driver.pl: Locate $srcpath based on __FILE__, then
compute $toppath as its parent. Set $scriptpath under $srcpath
and $workpath under the current directory. Toss $*_filename
and modify get_logfile() etc. to use the suffix directly. Add
a chdir() around the invocation of the test.
* tests/run_make_tests.pl: Throw out the -srcdir option and use
$srcpath set in test_driver.pl. The #WORK# helper is no longer
useful so remove it. Set #PWD# to the current working dir. Always
search the local directory and $srcpath for config-flags.pm.
Use $srcpath for finding the thelp.pl script.
* tests/scripts/features/vpath: Don't put things in work/ as it
is no longer a subdirectory.
* tests/scripts/features/vpathgpath: Ditto.
* tests/scripts/features/vpathplus: Ditto.
* tests/scripts/misc/general1: Ditto.
* tests/scripts/misc/general2: Ditto.
* tests/scripts/options/dash-k: Ditto.
* tests/scripts/options/symlinks: Use $testpath as the working
directory.
* tests/scripts/variables/GNUMAKEFLAGS: Use the test helper to
display env var values (grepping for GNUMAKEFLAGS finds extra things
now that it is our current working directory).
Given this setup:
$ cat Makefile
A::; @echo A-1 && sleep 1 && echo A-1 done
A::; @echo A-2 && sleep 1 && echo A-2 done
A::; @echo A-3 && sleep 1 && echo A-3 done
B::; @echo B-1 && sleep 1 && echo B-1 done
B::; @echo B-2 && sleep 1 && echo B-2 done
B::; @echo B-3 && sleep 1 && echo B-3 done
$ make -j8 A .WAIT B
All recipes for A should be started sequentially and complete before
any recipe for B is started, then all recipes for B should be started
sequentially. This wasn't happening because the double-colon target
was getting pruned too early.
* src/remake.c (update_file): Don't prune a target if it's a double
colon rule which is complete, but there are other recipes to run for
this target: we want those other recipes to be run first.
* tests/scripts/targets/WAIT: Test .WAIT with double colon rules.
* src/main.c (handle_non_switch_argument): Return 1 if arg is .WAIT.
(decode_switches): Set wait_here for a goal that follows .WAIT.
* src/remake.c (update_goal_chain): Honor wait_here for a command
line goal. Don't allow double-colon targets to continue if .WAIT is
given for one of them.
* tests/scripts/targets/WAIT: Add .WAIT tests.
If -j1 is given explicitly on the command line don't warn about it.
* src/main.c (main): Skip the warning if -j1.
* tests/scripts/features/jobserver: Add a test for this behavior.
Change error and fatal messages to start with lowercase and not
end with a period. Note a few very common messages were left
as-is, just in case some other tools parse them.
Also modify the test known-good-output to satisfy the messages.
Create a new special variable, .WARNINGS, to allow per-makefile
control over warnings. The command line settings will override
this.
Move the handling of warning flags to a new file: src/warning.c.
Allow the decode to work with generic strings, and call it from
decode_switches().
* Makefile.am: Add new file src/warning.c.
* build_w32.bat: Ditto.
* builddos.bat: Ditto.
* po/POTFILES.in: Ditto.
* src/makeint.h: #define for the .WARNINGS variable name.
* src/warning.h: Add declarations for methods moved from main.c.
Rename the enum warning_state to warning_action.
* src/warning.c: New file. Move all warning encode/decode here
from main.c.
* src/main.c: Move methods into warning.c and call those methods
instead.
(main): Set .WARNINGS as a special variable.
* src/job.c (construct_command_argv): Rename to warning_action.
* src/read.c (tilde_expand): Ditto.
* src/variable.c (set_special_var): Update warnings when the
.WARNINGS special variable is set.
* tests/scripts/options/warn: Check invalid warning options.
* tests/scripts/variables/WARNINGS: Add tests for the .WARNINGS
special variable.
The "invalid-var" warning triggers if the makefile attempts to assign
a value to an invalid variable name (a name containing whitespace).
The "invalid-ref" warning triggers if the makefile attempts to
reference an invalid variable name. Both new warnings have a default
action of "warn".
* NEWS: Add these new warnings.
* doc/make.1: Document them in the man page.
* doc/make.texi (Warnings): Document them in the user's manual.
* src/warning.h: Add enum values for the new warning types.
* src/main.c (initialize_warnings): Initialize the new warnings.
* src/variable.h (undefine_variable_in_set, undefine_variable_global):
Ask callers to provide a struct floc specifying where the variable
is undefined.
* src/read.c (do_undefine): Pass floc when undefining.
* src/variable.c (check_valid_name): If invalid-var is enabled, check
the variable name.
(define_variable_in_set): Call it.
(undefine_variable_in_set): Ditto.
(check_variable_reference): If invalid-ref is enabled, check the
variable reference.
(lookup_variable): Call it.
(lookup_variable_in_set): Ditto.
* tests/scripts/options/warn: Add tests for the new warning types.
