Delete do_source(), replacing blocking recursive calls with FILE *source

Browse Source

in sh_fcall stack. The main processing loop moved back at the end of
sh_main() doing read/parse/run on TT.ff->source, popping the fcall
stack at EOF and exiting when the stack is empty.

The motivation was "trap" needed to call do_source() from a signal handler
(which it couldn't, even with longjmp) but the result is a noticeably
cleaner design removing several special cases.

Each sh_fcall entry now has a FILE *source which (if not NULL) is where to
read the next line from: "source" is now basically TT.ff->source = fopen(name)
and and "eval" is fmemopen(s, strlen(s), "r"). The starting script and
/etc/profile work like "source", the -c argument works like "eval".
When run_lines() runs out of parsed lines (I.E. TT.ff->pl==NULL),
if TT.ff->source isn't NULL it returns to sh_main(), otherwise it
pops the fcall stack and continues. When sh_main() hits EOF/error on
TT.ff->source it fclose()s it and sets it to NULL so the next call
to run_lines() can pop it and handle trailing cleanup like "eval x && y".
When everything is popped (TT.ff==NULL) the loop exits.

The old design would block until things completed, the new design means
things like "source" add TODO items to the fcall stack which have not
even been started when source_main() returns, so a lot of the
lifetime rules changed, sometimes subtly. For example, the old
do_source(fmemopen("cd .")) style calls in the setup code now run after
the initialization functions return, so the initial value of $_ (the shell
name) was getting overwritten with the "." argument to cd. (It's now
effectively doing "_= cd ." with a prefix assignment to absorb/discard
the update.) The new ff->_ member ("_" is a legal C variable name) solves
a similar problem of $_ updates at the end of "source" and "eval" calls
happening long after source_main() and eval_main() return.

Added reference counting to sh_process, and added sh_process *pp to sh_fcall
to track the TT.ff->arg string lifetimes because free_process() frees them but
when to call that is non-obvious (pp could be backgrounded, etc).
Also removed TT.ff->delete and just have its users always use
TT.ff->pp->delete: the previous code had deletion lists in both places
and the prefix assignment case transplanted pp->delete into ff->delete, which
is one such tangle this redesign smoothed out.

Previously the sh_process's filehandle unredirect list was always processed
at the end of run_command(), but since "command | thingy" and
"function | thingy" do very different things now that doesn't work.
Both need to unredirect() before running "thingy", and most commands finish
before run_command() exits, but source/eval/functions need the redirects to
persist until the fcall context is eventually popped after running the body.
We can't move pp->urd into ff->urd because the redirects are performed
when creating the process (parsing the command line or preparing a pipe
to the next command) before run_command() is called, the relevant fcall
context doesn't exist yet. Answer: have end_fcall() flush and clear urd
for the attached process, if ff->pp isn't NULL end_fcall will
unredirect(ff->pp->urd) of any ff->pp attached to the popped fcall.
(So delete is processed when the last user drops it, but urd is processed
when the FIRST user drops it.)

And run_subshell() sets ff->pp to (void *)1 in the CFG_TOYBOX_FORK child
codepath to signal that end_fcall() should _exit() instead of returning
when this fcall is popped, because a child process running in a subshell
still needs the full fcall stack to resolve layers of local variables,
but should NOT proceeed past the end of its domain to run shell script
that belongs to the parent process. (The previous logic for figuring this
out was one of the most opaque sharp edges in the codebase. There's also
a new TT.forkchild which run_subshell() longjmp()s back to from there,
because the forked child sequencing is unusual at both ends.)

The new design replaces the previous implicit rules to figure out what each
fcall entry meant with much simpler explicit rules. No more "funconly"
argument to end_fcall(): now it always frees one fcall entry (even the last
one, leaving TT.ff null). Each fcall has ->function set if it's a function()
call, ->source set if it's a source of text script lines ("eval" leaves
->name null, "source" sets it), and it's still got the ->var entry with
whiteouts for local variable processing. Calling run_command() creates ONE
fcall entry, which is always there, containing prefix assignments and getting
its source set by source_main() and eval_main(). This means lots of nofork
commands messing with TT.ff got adjusted to look at TT.ff->next instead
because the command's transient function context is not what "break",
"return", "set", or "shift" should be messing with. The old magic "is
ff->var null or not" tests now just check for ff->function.

More cleanup: removed recfile[] (it fclose()s each TT.ff->source instead),
lineno now lives in TT.ff->lineno instead of TT.lineno (no more "oldlineno"
shuffle to match bash behavior), fewer nommu recursion limit checks
(recalculate() still recurses on the C stack but eval eval eval
eval eval eval does not; $(a=$(b=$(c)) forks (subshell) children and waits
for them), parse_line() no longer takes an sh_pipeline argument (just uses
TT.ff->pl) and no longer has a second instance of cleanup code in its
error path (sh_main handles it), get_next_line() no longer returns (void *)1
for "invalid" but just NULL or a string (things like sourcing an ELF binary
now force invalid UTF-8 sequence detection), get_next_line() no longer
takes a special NULL argument meaning "stdin but interactive": when the
first argument is NULL it returns NULL (meaning EOF/error) immediately,
nommu_entry() and subshell_setup() behave similarly: both return instead
of one blocking and the other returning, each cues up the appropriate
input source(s) in the TT.ff stack, and the setup logic has mostly moved
into the setup functions instead of being at the start of sh_main().

Deleting do_source() removed the need for a function prototype (alas
free_pipeline()/free_function() and setvar()/recalculate() can still loop),

New sherror_msg() function to display source/function and lineno,
replacing calls to perror_msg() and being called by syntax_err().
Switch perror_msg() to sherror_msg() to display source/function and lineno.

Cleaned up setvar_found()'s handling of the freeable argument,
call_function() now returns the new TT.ff so we can just -> tweak it
right after the call.

...

This commit is contained in:

Rob Landley 2024-12-30 18:44:15 -06:00

parent 027a3c459c

commit 7a2f81c82d

1 changed files with 351 additions and 341 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Show Stats Download Patch File Download Diff File Expand all files Collapse all files

692

toys/pending/sh.c

File diff suppressed because it is too large Load Diff