GCC 15.2 warns when assigning computed "all" bitmask macros to 32-bit
flag types because negated masks expand to the full width of unsigned
long/long on 64-bit platforms.
Mask these macros to a 32-bit domain so they do not set high bits and
avoid -Woverflow/-Wconversion warnings in callers.
Reported-by: Patrick Monnerat
Fixes#20242Closes#20416
And a few variables around.
There remain cases where the accepted pointer is const, yet the returned
pointer is written to.
Partly addressing (glibc 2.43):
```
* For ISO C23, the functions bsearch, memchr, strchr, strpbrk, strrchr,
strstr, wcschr, wcspbrk, wcsrchr, wcsstr and wmemchr that return
pointers into their input arrays now have definitions as macros that
return a pointer to a const-qualified type when the input argument is
a pointer to a const-qualified type.
```
Ref: https://lists.gnu.org/archive/html/info-gnu/2026-01/msg00005.html
Reported-by: Rudi Heitbaum
Ref: #20420Closes#20421
Seen in GHA/codeql builds.
One of them also affected wolfSSL < 3.9.10 builds.
61093e2a819d26b7ddf309baef264b9e50c6c56f #20353
Cherry-picked from #20404Closes#20417
- Use upper limit INT_MAX instead of UINT_MAX.
UINT_MAX doesn't work as the max value for the variable since it is
passed as a long and becomes -1 on platforms that have same sized
int and long, like Windows.
Closes https://github.com/curl/curl/pull/20414
- drop leading indent from Markdown.
- switch to Markdown section markers where missing.
- move `&&` and `||` to the end of the line (C, Perl).
- openssl: add parenthesis to an if sub-expression.
- misc clang-format nits.
- unfold Markdown links.
- SSL-PROBLEMS.md: drop stray half code-fence.
Closes#20402
To enable known linker options dropping unused, dead, code and data from
the executables built.
Useful to reduce binary sizes for curl, libcurl shared lib and apps
linking static libcurl. It's effective on both "unity" and non-unity
builds. Aligning "unity" build sizes with default, non-unity ones.
Supported platforms: Apple, MSVC, llvm/clang and GCC on all tested
platforms: Linux, BSDs, Windows, MSYS2/Cygwin, Android, MS-DOS.
Notes:
- Static libraries grow 20-30% with non-Apple toolchains.
This effect is controlled by separate, optional compiler flags on
non-Apple. This patch enables them automatically for public binaries
(libcurl and curl tool), and leaves them off for internal/test ones.
- MSVC enables this option by default for 'Release' configurations.
The curl build option has no effect on it.
- Observed effect on VS2010 is negligible. VS2012+ is recommended.
- Works with LTO, Fil-C.
- No observed/conclusive effect on build speed.
- On Windows with clang/gcc (mingw-w64/MSYS2/Cygwin) it also enables
`-fno-asynchronous-unwind-tables` as a workaround to make
the toolchain options actually work.
Ref: https://sourceware.org/bugzilla/show_bug.cgi?id=11539
Thanks-to: Andarwinux
Also:
- GHA: enable in Linux and MinGW jobs to test it. Size changes:
- linux aws-lc H3:
curl: 2000000 -> 1937152, libcurl.a: 2065724 -> 2716532 bytes
- macos clang HTTP-only:
curl: 1364376 -> 128799 bytes, libcurl.a: unchanged
- macos llvm MultiSSL:
curl: 410056 -> 405720, libcurl.dylib: 1350336 -> 1348480 bytes
- mingw schannel c-ares U:
curl: 1588736 -> 1507328, libcurl-d.a: 3322040 -> 3884746 bytes
bld: 34 -> 35MB
- GHA: enable in MSVC and Apple jobs to reduce disk footprint, with no
obvious downside. Size changes:
- AppVeyor CI VS2019:
curl: 2339840 -> 1295872, libcurl-d.dll: 3155968 -> 1900544 bytes
bld: 161 -> 97MB
- AppVeyor CI VS2022 clang-cl:
curl: 2933248 -> 2332160, libcurl-d.lib: 4762688 -> 5511330 bytes
bld: 133 -> 121MB
- AppVeyor CI VS2022 HTTP-only:
curl: 3514368 -> 2177024, libcurl-d.lib: 2538420 -> 3151740 bytes
bld: 137 -> 83MB
- GHA intel:
curl: 2629120 -> 2023424, libcurl-d.lib: 4366652 -> 5350670 bytes
bld: 86 -> 69MB
- GHA arm64:
curl: 2832896 -> 2063872, libcurl-d.lib: 4690616 -> 5597250 bytes
bld: 82 -> 66MB
Refs:
https://maskray.me/blog/2021-02-28-linker-garbage-collectionhttps://web.archive.org/web/20110811230637/msdn.microsoft.com/en-us/library/bxwfs976.aspx (VS2010)
https://learn.microsoft.com/cpp/build/reference/opt-optimizationshttps://learn.microsoft.com/cpp/build/reference/gy-enable-function-level-linkingCloses#20357
MSVC does not advertise itself as C99 via `__STDC_VERSION__`, but
supports variadic macros in all curl-supported versions. Fix by
explicitly enabling C99 verbose string logic for MSVC.
With verbose logging enabled (default), this makes logging perform
better, on par with clang/gcc (and other C99) builds. (With the cost
of extra binary size.) With verbose logging disabled, it excludes all
verbose logging related strings and code from the binary. Before this
patch, MSVC used the C89 fallback code in both configs, which used
a fixed function call, with the called function deciding to actually
log or not, while also retaining the verbose log string in both configs.
Size comparison (bytes), schannel, static, debug, VS2022, local build:
curl-before-verbose.exe 4,024,832
curl-before-noverbose.exe 4,013,056
curl-after-verbose.exe 4,117,504
curl-after-noverbose.exe 3,928,064
In CI with non-verbose:
Before:
```
3274240 bytes: ./_bld/lib/Debug/libcurl-d.dll
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/53408629/job/htj7ps88q83ew9ww#L224
After:
```
3155968 bytes: ./_bld/lib/Debug/libcurl-d.dll
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/53408771/job/tp9epgjpef098vsr#L224
Idea-by: Arnav Purushotam
Ref: #20367
Ref: #20341
Follow-up to 61093e2a819d26b7ddf309baef264b9e50c6c56f #20353Closes#20387
- schannel: fix mixed-up declaration. (originally fenced infof for
verbose, then changed to failf with the fence kept, then fence
removed and variable marked as verbose, when in fact it's not, but
not tested and caught in CI.
- fix two other fallouts.
- GHA/windows: disable verbose strings in a mingw job.
- appveyor: disable verbose strings in an MSVC job.
- appveyor: add way to pass any CMake option per-job.
Cherry-picked from #20387
Follow-up to 61093e2a819d26b7ddf309baef264b9e50c6c56f #20353Closes#20388
When the compiler supports C99.
- map logging functions to macro stubs when verbose logging is disabled
and the compiler is C99. Make sure these stubs silence unused variable
warnings for non-variadic arguments.
Before this patch they mapped to function stubs, the same codepath
used for C89 compiler in this configuration.
- introduce new macros to tell the compiler which code to include
when verbose code is active, or inactive:
- `CURLVERBOSE`: defined when verbose code is active.
To enclose blocks of code only used for verbose logging.
- `VERBOSE(statement);`:
compile statement when verbose code is active.
To mark code lines only used for verbose logging.
- `NOVERBOSE(statement);`:
compile statement when verbose code is inactive.
To suppress warnings for arguments passed to logging functions via
printf masks, e.g. `NOVERBOSE((void)ipaddress);`, yet keeping
the warning in verbose builds.
Note these macros are not the same as `CURL_DISABLE_VERBOSE_STRINGS`.
Verbose code is always active in C89 mode (without variadic macro
support).
- drop existing uses of `CURL_DISABLE_VERBOSE_STRINGS` where redundant,
or replace with the above macros. Ending up reducing the number of
`#ifdef`s, and also the number of lines.
Assisted-by: Daniel Stenberg
Assisted-by: Jay Satiro
Reported-by: Dan Fandrich
Fixes#20341
Refs: #12105#12167Closes#20353
In some legacy systems IPv6 might dynamically work/not work and thus
curl needs to check/probe to see if it should indeed be used.
This change moves the probe that checks for working IPv6 to the multi
handle setup function instead of delaying it to when the first name
resolve is performed. This avoids a later tricky error path if the
socket cannot be created due to OOM.
Closes#20383
Use FormatMessageA() to get the error message as multibyte-character
string (local codepage) directly, instead of using FormatMessageW()
and then convert the string from Unicode (UTF-16) to multi-byte (local
codepage) manually.
Prior to this change we used FormatMessageW + conversion because some
Windows CE did not have FormatMessageA. Since curl no longer supports
Windows CE, FormatMessageA can be used.
Closes https://github.com/curl/curl/pull/20261
Use non-deprecated CRT function variants on Windows.
- introduce `curlx_fdopen()`, `curlx_close()` and use them. Map them to
non-deprecated, underscored, CRT functions on Windows.
- replace `close()` uses with either `sclose()` (for sockets) or
`curlx_close()` (for files).
- map `fileno`, `unlink`, `isatty` to their non-deprecated, underscored,
versions on Windows.
- tool_dirhie: map `mkdir` to `_mkdir` on Windows.
- easy: use `_strdup()` on Windows, regardless of how `HAVE_STRDUP` is
set.
- cmake: assume `HAVE_STRDUP` on Windows. To allow dropping a detection
hack using `_CRT_NONSTDC_NO_DEPRECATE` with MSVC. Windows always has
`_strdup()` which the code uses, but also needs `HAVE_STRDUP` defined
to disable curl's own `strdup()` implementation.
- curl_setup.h: drop `_CRT_NONSTDC_NO_DEPRECATE` as no longer necessary.
Closes#20212
When checking a transfer for being expired via `Curl_timeleft_ms()`,
eleminate the `bool connecting` parameter and have the function check
the `mstate` of the transfer instead.
Advantages:
* eleminate the caller needing awareness if the transfer is
connecting or in a later state
* fix pingpong timeout handling to check the correct timeout
during "proto_connect" phases
* avoid using "connecting" timeouts during establishing a secondary
connection (e.g. FTP) since this would use the timestamp from
the original, primary connect and thus be wrong
Reported-by: Wyuer on github
Fixes#20347Closes#20354
New define USE_SO_NOSIGPIPE in curl_setup.h, for now set whenever
SO_NOSIGPIPE is defined. Maybe overridden in the future on systems where
this does not work.
With USE_SO_NOSIGPIPE defined, set SO_NOSIGPIPE on all sockets created
by libcurl and fail the creation when setsockopt() fails.
Closes#20370
Extend two existing local suppressions to GCC, and add another
GCC-specific one as a replacement.
Before this patch suppressing this warning was odd with clang, because
after this option, `-Wformat=2` is used, which re-enables it.
Also:
- mprintf: minimize scope of a warning suppression.
- tests/server: suppress this warning for a system `vsnprintf()` call
where it could trigger in C89 builds or with
`CFLAGS=-DCURL_NO_FMT_CHECKS` set. Seen with Apple clang 17:
```
curl/tests/server/util.c:114:37: warning: format string is not a string literal [-Wformat-nonliteral]
114 | vsnprintf(buffer, sizeof(buffer), msg, ap);
| ^~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/secure/_stdio.h:124:69: note: expanded from macro 'vsnprintf'
124 | #define vsnprintf(str, len, ...) __vsnprintf_chk_func (str, len, 0, __VA_ARGS__)
| ^~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/secure/_stdio.h:81:65: note: expanded from macro '__vsnprintf_chk_func'
81 | __builtin___vsnprintf_chk (str, len, flag, __darwin_obsz(str), format, ap)
| ^~~~~~
```
Ref: #20363Closes#20366
```
lib/rtsp.c:1073:26: error: no previous extern declaration for non-static variable 'Curl_scheme_rtsp' [-Werror,-Wmissing-variable-declarations]
1073 | const struct Curl_scheme Curl_scheme_rtsp = {
| ^
lib/rtsp.c:1073:7: note: declare 'static' if the variable is not intended to be used outside of this translation unit
1073 | const struct Curl_scheme Curl_scheme_rtsp = {
| ^
```
Ref: https://github.com/curl/trurl/actions/runs/21157411659/job/60844860592?pr=425#step:3:3036
Follow-up to 8edc0338f30f458f812f9ea355de1240771fa343 #20351Closes#20365
This allows builds know about all schemes - but only have the protocol
implementations for those actually built-in.
It further allows multiple protocols to reuse the same protocol setup
and functions for both TLS and non-TLS implementations instead of
needing two (or more) structs.
The scheme information is now in 'struct Curl_scheme' and all the
function pointers for each scheme/protocol implementation are in struct
Curl_protocol.
The URL API now always work with all known protocols.
Closes#20351
Drop separate `TrackMemory` (aka `CURLDEBUG`) debug feature.
After recent changes (thread-safety,
193cb00ce9b47e75d42157c650cc3de3fd96d35d, and updates leading up to
it), `TrackMemory` is unlikely to cause build or runtime issues.
To simplify builds and debug options, enable `TrackMemory`
unconditionally for debug-enabled (aka `DEBUGBUILD`) builds. Before
this patch, this was already the default, with an option to disable
it, or enable it in non-debug-enabled builds.
Note, in practice these two debug options already went hand in hand. It
was not possible to toggle them separately for a long time due to bugs,
before 59dc9f7e69c399102e9ebe3670360ef52706ff23 (2024-05-28) fixed it.
This patch also removes/deprecates separate knobs and feature flags for
`TrackMemory`:
- autotools: `--enable-curldebug`/`--disable-curldebug`
- cmake: `-DENABLE_CURLDEBUG=ON`/`OFF`
- C macro: `CURLDEBUG`
- libcurl: `CURL_VERSION_CURLDEBUG` symbol deprecated in favor
of `CURL_VERSION_DEBUG`. They always return the same value after this
patch.
Also:
- drop `TrackMemory` from `curl -V` output.
- rename internal `CURLDEBUG` macro to `CURL_MEMDEBUG` internally.
To avoid confusion with `DEBUGBUILD`, but to keep guarding
`TrackMemory`-related internals for readability.
- runtests: bind `TrackMemory` to debug feature. Keep it a separate
test feature requirement, for clarity.
- CI: drop test builds for combinations of the two options.
- GHA/linux: no longer disable TrackMemory in the TSAN job.
Ref: https://github.com/curl/curl/pull/20328#issuecomment-3754528407Closes#20331
Define `struct Curl_sigpipe_ctx` that can be passed as argunent
to "lower" functions so that applying a transfers 'no_signal'
setting can be delayed as much as possible and sometimes avoided
alltogether.
Fixes#20326Closes#20329
Reported-by: Dag Haavi Finstad
- It is slower and uses more memory than the alternatives and is only
experimental in curl.
- We disable a few tests for OpenSSL-QUIC because of flakiness
- It gets little attention from OpenSSL and we have no expectation of the
major flaws getting corrected anytime soon.
- No one has spoken up for keeping it
- curl users building with vanilla OpenSSL can still use QUIC through the
means of ngtcp2
Closes#20226
With the same semantics as Apple SecTrust, in both libcurl and the curl
tool, when using non-Schannel TLS backends. In practice it means that
it makes TLS work without manually or implicitly configuring a CA bundle
`.crt` file, such as `curl-ca-bundle.crt`.
To enable:
- autotools: `--enable-ca-native`
- cmake: `-DCURL_CA_NATIVE=ON`
- CPPFLAGS: `-DCURL_CA_NATIVE`
When enabled:
- enables `CURLSSLOPT_NATIVE_CA` (libcurl) / `--ca-native`
and `--proxy-ca-native` (curl tool) options by default.
- unsafe search for an on-disk CA bundle gets disabled by default.
Equivalent to `--disable-ca-search` with autotools,
`-DCURL_DISABLE_CA_SEARCH=ON` with CMake.
- build-time detection of CA bundle and CA path gets disabled. As with
Apple SecTrust. This was already the default for Windows.
- native CA can be disabled at run-time with the `--no-ca-native`
and/or `--no-proxy-ca-native` command-line options.
Rationale: This build option:
- has a repeat and active interest from packagers and users.
- helps integrating curl with Windows for those who need this.
- it also applies to macOS: #17525
Shipped in curl 8.17.0.
- makes it trivial to use custom certs configured on the OS.
- frees applications/packagers/users from the task of securely
distributing, and keeping up-to-date, a CA bundle.
- frees potentially many curl tool from configuring a CA bundle manually
to access HTTPS (and other TLS) URLs. This is traditionally difficult
on Windows because there is no concept of a universal, protected,
non-world-writable, location on the file system to securely store
a CA bundle.
- allows using modern features regardless of Windows version. Some of
these features are not supported with Schannel (e.g. HTTP/3, ECH) on
any Windows version.
- is necessary for HTTP/3 builds, where bootstrapping a CA bundle is not
possible with Schannel, because MultiSSL is not an option, and HTTP/3
is not supported with Schannel.
Ref: #16181 (previous attempt)
Ref: https://github.com/curl/curl/discussions/9348
Ref: https://github.com/curl/curl/issues/9350
Ref: https://github.com/curl/curl/pull/13111
Ref: https://github.com/microsoft/vcpkg/pull/46459#issuecomment-3162068701
Ref: 22652a5a4cb6a4cc1c0f4ff3ebc4f9768f6663cd #14582
Ref: eefd03c572996e5de4dec4fe295ad6f103e0eefc #18703Closes#18279
Add casts to `bool`, or use `bit` type in local variables, where
neccessary to avoid MSVC compiler warnings C4242.
Note: There may remain places needing the above updates, where not
tested in CI, and missed in manual review.
Also:
- urldata: convert struct field `connect_only` to bitfield to match its
counterpart in another struct.
- rename curl-specific `bit` type to `curl_bit`.
Closes#20142
After this patch curl requires targeting Vista or newer, and a toolchain
with Vista support.
Supported MSVC compilers (VS2010+) all support Vista:
- VS2012+ target Win8 (or later) by default.
- VS2010 targets Win7 by default.
Supported mingw-w64 versions (v3+) all support Vista:
- mingw-w64 v9+ target Win10 by default.
- mingw-w64 v8 and older target Server 2003 (~XP) by default.
After this patch it may be necessary to override the default Windows
target version to Vista (or newer) via:
autotools: `CPPFLAGS=-D_WIN32_WINNT=0x0600`
cmake: `-DCURL_TARGET_WINDOWS_VERSION=0x0600`
- mingw-w64 v6+ allow changing the default at toolchain build-time.
Notes:
- For non-MSVC, non-mingw-w64 toolchains, `if_nametoindex` needs to be
allowlisted in `curl_setup.h`, if they do support it.
Fixes#17985 (discussion)
Closes#18009
Declutter the ifdefs in socketpair.h. Introduce Curl_wakeup_*()
function that encapsulate the details about how the socketpair
is implemented.
This moves the EVENTFD specials from the using code into socketpair
implemenatation, avoiding duplications in three places.
Closes#20340
When a download size is known and rate limiting is in effect, adjust the
duration of each measurement step and its rate for maximum precision.
Since it is unpredictable how long the last bytes of a download will
take, download speed can be thrown off if the "last bytes" are a
significant amount of the total download. Make the "last bytes" small in
comparision to the rest and "stretch" the rate limit intervals to
accommodate the difference.
Fix ngtcp2 receive data acknowldgements to be based on a local window
size tracking. This allows window updates controlled by rate limits.
Fix ratelimit wait time calculation to accomodate negative tokens.
h3 rate limit, update timeers
Make download rate limits work correctly in ngtcp2. Fix multi handling
of rate limits to set a timer for when limits will update again.
Without running the transfer on limit updates, protocols like h2/h3 may
stall if the server does not send due to stream windows being too small.
scorecard: measure download speedlimits
When running scorecard with --limit-rate=n, show the reported download
speed averages plus percentage deviation from the limit.
Closes#20228
Both guard the use of `CURL_DNS_SERVER` debug env.
Follow-up to df2b4ccc229c9de61dd798e4b3a7cf74a073144e #18157
Follow-up to 02e9690c3ee2feb7c78c93c1c544c898df733383 #17015
Follow-up to 59dc9f7e69c399102e9ebe3670360ef52706ff23 #13718Closes#20328
Most by moving functions around. Also delete unused ones.
Reducing their number from 83 to 33.
Remaining ones due to:
- circular dependencies.
- H3 code, that I did not attempt to update and likely the above applies.
- static declarations with attributes (`CURL_PRINTF`, `WARN_UNUSED_RESULT`).
- OS400 code.
Closes#20321
