To make the test files XML-compliant, and the expected results
possibly easier to manage by keeping them in `.c` files.
Non-XML-compliant files are down to 36 after this patch.
Also:
- make all macro expansions apply to `%includetext` contents.
Closes#19799
- URLs specified outside of the markdown []() are now extracted and
checked
- also check TODO, FAQ and KNOWN_BUGS
- more aggressive avoiding to check github.com/curl/curl, all uses of
example domains and some more established URLs on the curl.se site
- list all errors in the end to make them easier to spot in CI logs
Closes#19848
Before this patch curl used the C preprocessor to override standard
memory allocation symbols: malloc, calloc, strdup, realloc, free.
The goal of these is to replace them with curl's debug wrappers in
`CURLDEBUG` builds, another was to replace them with the wrappers
calling user-defined allocators in libcurl. This solution needed a bunch
of workarounds to avoid breaking external headers: it relied on include
order to do the overriding last. For "unity" builds it needed to reset
overrides before external includes. Also in test apps, which are always
built as single source files. It also needed the `(symbol)` trick
to avoid overrides in some places. This would still not fix cases where
the standard symbols were macros. It was also fragile and difficult
to figure out which was the actual function behind an alloc or free call
in a specific piece of code. This in turn caused bugs where the wrong
allocator was accidentally called.
To avoid these problems, this patch replaces this solution with
`curlx_`-prefixed allocator macros, and mapping them _once_ to either
the libcurl wrappers, the debug wrappers or the standard ones, matching
the rest of the code in libtests.
This concludes the long journey to avoid redefining standard functions
in the curl codebase.
Note: I did not update `packages/OS400/*.c` sources. They did not
`#include` `curl_setup.h`, `curl_memory.h` or `memdebug.h`, meaning
the overrides were never applied to them. This may or may not have been
correct. For now I suppressed the direct use of standard allocators
via a local `.checksrc`. Probably they (except for `curlcl.c`) should be
updated to include `curl_setup.h` and use the `curlx_` macros.
This patch changes mappings in two places:
- `lib/curl_threads.c` in libtests: Before this patch it mapped to
libcurl allocators. After, it maps to standard allocators, like
the rest of libtests code.
- `units`: before this patch it mapped to standard allocators. After, it
maps to libcurl allocators.
Also:
- drop all position-dependent `curl_memory.h` and `memdebug.h` includes,
and delete the now unnecessary headers.
- rename `Curl_tcsdup` macro to `curlx_tcsdup` and define like the other
allocators.
- map `curlx_strdup()` to `_strdup()` on Windows (was: `strdup()`).
To fix warnings silenced via `_CRT_NONSTDC_NO_DEPRECATE`.
- multibyte: map `curlx_convert_*()` to `_strdup()` on Windows
(was: `strdup()`).
- src: do not reuse the `strdup` name for the local replacement.
- lib509: call `_strdup()` on Windows (was: `strdup()`).
- test1132: delete test obsoleted by this patch.
- CHECKSRC.md: update text for `SNPRINTF`.
- checksrc: ban standard allocator symbols.
Follow-up to b12da22db1f11da51082977dc21a7edee7858911 #18866
Follow-up to db98daab05aec251bcb6615d2d38dfebec291736 #18844
Follow-up to 4deea9396bc7dd25c6362fa746a57bf309c74ada #18814
Follow-up to 9678ff5b1bfea1c847aee4f9edf023e8f01c9293 #18776
Follow-up to 10bac43b873fe45869e15b36aac1c1e5bc89b6e0 #18774
Follow-up to 20142f5d06f7120ba94cbcc25c998e8d81aec85b #18634
Follow-up to bf7375ecc50e857760b0d0a668c436e208a400bd #18503
Follow-up to 9863599d69b79d290928a89bf9160f4e4e023d4e #18502
Follow-up to 3bb5e58c105d7be450b667858d1b8e7ae3ded555 #17827Closes#19626
Example of old text:
--dns-ipv4-addr requires that libcurl is built to support c-ares.
New version:
For --dns-ipv4-addr to work, it requires that the underlying libcurl is
built to support c-ares.
Closes#19665
To complement the existing `curlx_fopen()` internal API.
It's used by the curl's `--stderr` option.
`curlx_freopen()` adds two features to the bare `freopen()`:
- tracing for debug-enabled builds.
- Unicode and long-filename support for Windows builds.
In effect this adds long-filename and enables Unicode support for
the `--stderr <filename>` curl command-line option on Windows.
Also add to checksrc.
Follow-up to 2f17a9b654121dd1ecf4fc043c6d08a9da3522db #10673Closes#19598
Also:
- replace `manpage` with `man page`, add to `badwords.txt`.
- badwords.pl: import `-w` feature from curl-www, syncing the two
scripts fully.
- badwords.txt: import missing items from curl-www, syncing the two
files fully.
- pyspelling.words: drop `cURL` allowed word.
Closes#19468
On Windows this requires Perl 5.22 from year 2015.
Also:
- mdlinkcheck: delete redundant error handling logic.
Follow-up to 77be4a7ab2b5a0c633b9107fd286bda1f57e4725 #19437Closes#19461
To make it simpler to move them around, create and delete them without
syncing with `REUSE.toml`.
Also:
- checksrc: allow empty lines in `.checksrc`.
- comment on why curl printfs are disallowed in examples.
Closes#19024
Add infrastructure to colled and dispatch notifications for transfers
and the multi handle in general. Applications can register a callback
and en-/disable notification type the are interested in.
Without a callback installed, notifications are not collected. Same when
a notification type has not been enabled.
Memory allocation failures on adding notifications lead to a general
multi failure state and result in CURLM_OUT_OF_MEMORY returned from
curl_multi_perform() and curl_multi_socket*() invocations.
Closes#18432
After this patch, the codebase no longer overrides system printf
functions. Instead it explicitly calls either the curl printf functions
`curl_m*printf()` or the system ones using their original names.
Also:
- drop unused `curl_printf.h` includes.
- checksrc: ban system printf functions, allow where necessary.
Follow-up to db98daab05aec251bcb6615d2d38dfebec291736 #18844
Follow-up to 4deea9396bc7dd25c6362fa746a57bf309c74ada #18814Closes#18866
Also:
- tests/server: replace local `sstrerror()` with `curlx_strerror()`.
- tests/server: show the error code next to the string, where missing.
- curlx: use `curl_msnprintf()` when building for src and tests.
(units was already using it.)
- lib: drop unused includes found along the way.
- curlx_strerror(): avoid compiler warning (and another similar one):
```
In file included from servers.c:14:
../../lib/../../lib/curlx/strerr.c: In function ‘curlx_strerror’:
../../lib/../../lib/curlx/strerr.c:328:32: error: ‘snprintf’ output may be truncated before the last format character [-Werror=format-truncation=]
328 | SNPRINTF(buf, buflen, "%s", msg);
| ^
../../lib/../../lib/curlx/strerr.c:47:18: note: ‘snprintf’ output 1 or more bytes (assuming 2) into a destination of size 1
47 | #define SNPRINTF snprintf
| ^
../../lib/../../lib/curlx/strerr.c:328:7: note: in expansion of macro ‘SNPRINTF’
328 | SNPRINTF(buf, buflen, "%s", msg);
| ^~~~~~~~
```
Follow-up to 45438c8d6f8e70385d66c029568524e9e803c539 #18823Closes#18840
By quoting the search expression to be replaced. This avoid the issue
when the code leading up to a banned function contained regex characters
that the script did not explicitly handle, e.g. `+`.
Assisted-by: Daniel Stenberg
Ref: https://perldoc.perl.org/functions/quotemeta
Follow-up to dd37d6970cfd8b4cf47ebd469f03772813b92c23 #18775Closes#18845
By making them defaults, then fixing and/or reshuffling remaining
exceptions as necessary.
- checksrc: ban by default: `snprintf`, `vsnprintf`, `sscanf`, `strtol`.
- examples: replace `strtol` with `atoi` to avoid a checksrc exception.
- tests/libtest: replace `strtol` with `atol`.
- tests/server: replace most `strtol` with `atol`.
- tests/server: replace most `strtoul` with `atol`/`atoi`.
- tests/server: drop no longer used `util_ultous`.
- fix typo in checksrc rules: `vsnprint` -> `vsnprintf`.
- update local exceptions.
Also:
- examples: ban curl printf functions. They're discouraged in user code.
- examples: replace curl printf with system printf.
Add `snprintf` workaround for <VS2015.
- examples/synctime: fix `-Wfloat-equal`.
- examples/synctime: exclude for non-Windows and non-UWP Windows.
- examples/synctime: build by default.
Closes#18823
Fixing:
```
Unmatched ) in regex; marked by <-- HERE in m/ \*buffer_len = \(ssize_t) <-- HERE
strtol\(/ at /home/runner/work/curl/curl/scripts/checksrc.pl line 916, <$R> line 380.
```
Ref: https://github.com/curl/curl/actions/runs/18209824275/job/51848079550#step:3:5
Also add a test case.
Follow-up to 684f4cdd3ef0cc41c547fce0e45d8a059a3058b3 #18779
Cherry-picked from #18823Closes#18836
Replace them by `curlx_open()` and `curlx_stat()`.
To make it obvious in the source code what is being executed.
Also:
- tests/server: stop overriding `open()` for test servers.
This is critical for the call made from the signal handler.
For other calls, it's an option to use `curlx_open()`, but
doesn't look important enough to do it, following the path
taken with `fopen()`.
Follow-up to 10bac43b873fe45869e15b36aac1c1e5bc89b6e0 #18774
Follow-up to 20142f5d06f7120ba94cbcc25c998e8d81aec85b #18634
Follow-up to bf7375ecc50e857760b0d0a668c436e208a400bd #18503Closes#18776
If the source line had square brackets before the match, the stripping
of the banned function left the original line intact, and repeated the
check on it forever. E.g. with banned function `open` in `lib518.c`:
```c
t518_testfd[0] = open(DEV_NULL, O_RDONLY);
```
Closes#18775
By introducing wrappers for them in the curlx namespace:
`curlx_fopen()`, `curlx_fdopen()`, `curlx_fclose()`.
The undefine/redefine/`(function)()` methods broke on systems
implementing these functions as macros. E.g. AIX 32-bit's `fopen()`.
Also:
- rename `lib/fopen.*` to `lib/curl_fopen.*` (for `Curl_fopen()`)
to make room for the newly added `curlx/fopen.h`.
- curlx: move file-related functions from `multibyte.c` to `fopen.c`.
- tests/server: stop using the curl-specific `fopen()` implementation
on Windows. Unicode isn't used by runtests, and it isn't critical to
run tests on longs path. It can be re-enabled if this becomes
necessary, or if the wrapper receives a feature that's critical for
test servers.
Reported-by: Andrew Kirillov
Bug: https://github.com/curl/curl/issues/18510#issuecomment-3274393640
Follow-up to bf7375ecc50e857760b0d0a668c436e208a400bd #18503
Follow-up to 9863599d69b79d290928a89bf9160f4e4e023d4e #18502
Follow-up to 3bb5e58c105d7be450b667858d1b8e7ae3ded555 #17827Closes#18634
Add script to run all Perl sources through `perl -c` to ensure no
issues, and run this script via GHA/checksrc in CI.
Fallouts:
- fix two repeated declarations.
- move `shell_quote()` from `testutil.pm` to `pathhelp.pm`, to
avoid circular dependency in `globalconfig.pm`.
Closes#18745
