[Daniel's week] April 25, 2025

[Daniel Stenberg]

Hi there,

Another packed week passed by.

## malloc

I compared curl of today vs curl 8 years ago on malloc count + memory use to
download a single 512MB file over clear text HTTP on localhost:

- 129 mallocs, which is exactly the same.

- Maximum amount of memory allocated now: 135566. 17,681 bytes *less* than
   eight years ago.

I found it inspiring. It could be added that 100K of the allocated memory is
for the download buffer.

## getaddrinfo

Last week I wrote a bit about my work on writing a mock DNS server for the
curl test suite, and that I made curl built with c-ares able to use that.
This, because when c-ares is used for name resolving it is easy to add special
debug-mode that makes it use our dedicated server instead of the "actual" one
that resolv.conf tells it to use.

Changing the builds that use the good old getaddrinfo() for name resolving
would be harder I wrote, as this function is not that easy to trick into
speaking to the test server on demand.

But over the past weekend I came up with the most excellent fix for this: I
provide a getaddrinfo() mock-up replacement that in curl's view works
identically, but under the hood uses c-ares and that then can be told to use
the test DNS server. Such execution then still uses the exact same
getaddrinfo() code flow in curl, which is the one we want to test.

I only had to write a getaddrinfo() replacement using c-ares. It was actually
surprisingly easy to do, perhaps because I could do most of it by copy and
pasting pieces of already existing code and just make sure it was done
correctly. I made it a separate implementation I call repladdrinfo [1] that I
then could double-check by running the real implementation and my clone
implementation on the same host names and verify that they report the same
contents back to the application.

I then subsequently took that implementation and converted it into a "fake
getaddrinfo" for libcurl testing purposes [2] and I did a few initial manual
tests quite successfully.

While working on that, I realized that I now need a better way to tell the
test suite that we have a curl build that can "override DNS" so to speak, so
that the test runner know when such a test should be invoked and be expected
to work, and when to skip those tests because the build isn't fit. That took
me into another detour on cleaning up how the test suite gets such build
information, and right note the state is that it will *probably* become a
'curlinfo' tool in src/ directory that outputs detailed build information [3].

## typecheck

Someone dropped a comment in IRC and after a quick verification it become
clear that the typecheck-gcc macros we have were not working. I don't know at
what point they stopped working as we don't really test them, but I suspect
they have been broken for quite some time.

These macros uses gcc/clang special magic instructions to attempt to add type
checks to some libcurl function calls that otherwise completely lack those
checks. This, because they use "vararg" function calls and then by definition
C allows anything to be used as argument, while the libcurl API insists that
they should use particular dedicated types.

It was quick to fix the immediate reported bug, but then I had a second look
at the file and I spotted that it did not check all options. In fact, it was
missing a whole bunch of them. It took me several hours to write the
additional checks for twenty something options and clean up the file - and add
a test that verifies that all existing options also have checks in the
typecheck-gcc header [4]. In the end I think it was pretty cool improvement
and with these fixes done, I subsequently found twenty something minor API
abuses in the curl repository - in code, docs and examples. Not bad.

## drop deprecation markers?

As I worked on polishing the typecheck macros I realized that the error
messages from that setup is somewhat ruined by the deprecation marker stuff we
have for deprecated options in the public curl header files. I figured maybe
they don't actually do much good and asked on the mailing list whether we
should consider dropping them. It would at least make the typecheck macros
slightly better.

The answers to my question were mixed, and there was certainly no consensus in
my favor so I dropped my proposed PR that was going to remove them. Oh well,
another time maybe.

## DDoS

We have had our "origin server" DDoS twice over the past week. Both times the
amount of network traffic was way over bandwidth capability which made our ISP
null-route the machine while the storm was going. Both times the tsunami was
over in roughly one hour.

The server they targeted is the origin server for the curl website, my private
website and some twenty other sites. It also runs a DNS server, mailing lists
and a whole slew of other Internet services for me and my friends at Haxx.

There have been no signals or indications as to why someone did this, nor what
they exactly were targeting. The attacks did not affect the public appearance
of the websites curl.se and daniel.haxx.se as Fastly graciously host them on
their CDN.

The attacks made us start a conversation about the fragility of this setup and
if we should possibly take precautions or actions to change some project
infrastructure. It's a tough question but I don't think we should overreact
too much based on just these two events since this is a setup that have worked
for a few decades already and it has been a long time since we had DDoS
problems.

## curl up

curl up 2025 [5] is already next week. I have started to put together slides
for my presentations.

## location-mode

I brought up my location-mode PR [6] for feedback, to see if there is interest
enough out there to try to get it merged within this feature window. It looks
so. Let's see what we can make of it!

## rc tags

At the curl+distro meeting a while back we arrived at the conclusion that
having tags set in git for the curl release candidates would make it easier
for some to use them.

I propose we name them 'rc-[version]-[num]', where '[version]' is the same
style as the normal release tags where dots are replaced with underscores.
[num] would be 1, 2 and 3. Example: rc-8_14_0-2

As the rc tarballs are meant to be ephemeral and temporary, so are the tags.
They will not remain set forever.

My proposal exists as a PR [7] for a documentation update where the format and
details are explained.

## tpm2

I have been working with a customer over the last few weeks to get a tpm2
OpenSSL provider to work fine with curl. Providers is the concept in OpenSSL
v3 that replaces the "engines" of OpenSSL v1.

This week we finally made some break-troughs and I have a fresh PR [8] up with
the necessary curl changes needed to get things to work correctly.

Lots of the struggle can be attributed to sparse documentation, no example
codes and completely incomprehensible error messages. In other words: business
as usual.

## wcurl

There was a wcurl release [11] and this version of the script has now been
imported [10] into the curl source tree to be shipped as part of the regular
curl release going forward.

The wcurl development, which is expected to be rather slow going forward, will
continue in its separate dedicated repository but we decided to ship it as
part of curl as a convenience to the world. It is imported into the curl git
repository this way also partly to make sure that the release tarballs remain
100% reproducible.

## CNA

I summed up in a blog post how it has been for us to work as a CNA [9] over
the last year or so.

## Coming up

- curl up happens next weekend
- final OpenSSL provider PR cleanups and merge
- last week of curl feature window for this cycle

## links

[1] = https://github.com/bagder/repladdrinfo
[2] = https://github.com/curl/curl/pull/17134
[3] = https://github.com/curl/curl/pull/17187
[4] = https://github.com/curl/curl/pull/17143
[5] = https://github.com/curl/curl-up/wiki/2025
[6] = https://github.com/curl/curl/pull/16543
[7] = https://github.com/curl/curl/pull/17177
[8] = https://github.com/curl/curl/pull/17165
[9] = https://daniel.haxx.se/blog/2025/04/24/how-the-cna-thing-is-working-out/
[10] = https://github.com/curl/curl/pull/17035
[11] = https://github.com/curl/wcurl/releases/tag/v2025.04.20

-- 

  / daniel.haxx.se