[Daniel's week] January 10, 2025

Fri Jan 10 16:33:04 CET 2025

Hi friends,

Another work week ends. Things happened.

## communication

I added a chapter [1] on communication in Uncurled [2], and while I was back
there I polished and edit a range of other pages as well. Some of them were
also extended somewhat. It just shows that coming back to a text after a while
of absence is pretty good as it gives you a different perspective on things.

I might edit it a bit more in the coming weeks.

## infrastructure

It struck me that we had no good collected documentation of the curl project's
infrastructure, servers and services we run and maintain for the project, so I
created one [3]. Turns out we have quite a collection of things.

## Korean apps

A friendly user emailed me screenshots of curl credits shown in three Korean
apps and I of course put them up on the curl credits page[4] (check the bottom
of the now fairly long page). Being totally ignorant of the Korean landscape I
had to have someone tell me that these are some of the most popular apps in
the region. KakaoTalk alone is likely to have a user base of over a hundred
million users. Fun!

## test infra

This week I introduced *three* new graphs on the curl dashboard[5].

The first one is counting how many lines of code and data we have in the
tests/ directory in curl. Basically everything that we can call test
infrastructure in source tree[6].

The two following one is than using that initial plot line and compares it with
other project plots. Lines of test infra per line of product code [7] and
lines of test infra per separate actual test case [8].

No earth shattering revelations in there, just mildly curious data.

## OpenSSL QUIC

This week we merged a fix [20] for the OpenSSL QUIC backend that should put an
end to the need and risk for busy-loops in the upload handling. We previously
had that ugly work-around because of missing OpenSSL APIs, which may have been
because of a lack of understanding. Still, even with this fix merged, OpenSSL
QUIC performance remains *abysmal* compared to the competition. In fact, the
fix made no performance boost at all but actually made instead some tests a
tad bit slower. This fact did not surprise me, because the API they provide is
not very clever or crafted for high performance.

The ngtcp2 backend is 2-4 times faster, and it also turns out that in some use
cases the OpenSSL QUIC backend uses 25(!) times more memory than the ngtcp2
one does.

We still have the OpenSSL QUIC backend marked experimental for curl, but the
question is if it should remain so. We previously did that because of the
busy-loop situation, but with that out of the way the question is now if we
should consider the serious speed problem a reason for us to keep this label
just to warn users off this or if we should rather let users decided what they
want. Removing the label might allow a few more users to enable HTTP/3 even if
super slow.

The question has not yet been decided, but I am leaning towards removing the
experimental label and leaving the performance problem for the OpenSSL team to
work out and users to test.

I remain advocating using the ngtcp2 backend when doing QUIC and HTTP/3 for
real with curl.

## msh3

While on the topic of QUIC (and HTTP/3) backends in curl, we recently started
debating msh3 again after they released new versions of the library that
changed the API so that curl cannot just seamlessly upgrade. Checking out the
status of msh3 in curl it is clear that no substantial work has been put into
it for almost two years now - while still not working correctly. Clearly
people are not caring much for this backend or making it work for curl, so the
conclusion has to be that it is only a friction in the machinery and we have
now decided that unless something drastic happens before then we will remove
support msh3 by the summer of 2025.

We still support three other QUIC and HTTP/3 backends, all which work as a
comparison to the msh3 one, so it should not be a major blow to anyone. Only
one of the backends is not labeled experimental though (ngtcp2).

There is also the additional possible future QUIC backend using
QUIC-in-kernel, but that one is still only around as a potential PR[10]. Still
feels rather distant.

## backends

For a presentation I am about to do a coming week I went back and refreshed an
old slide of mine, showing all "libcurl backends" [9]. Removing hyper from
curl simplified it a bit since there is no longer any HTTP/1 or HTTP/2
backends. We have also added AppleIDN since I last showed this slide.

## winbuild

On the topic of deprecating things, we have also started a discussion around
possibly removing support for the "winbuild" set of build scripts [11]
provided in the curl source tree, since all of those builds can be done using
cmake now and it would simplify our maintainer lives a bit by removing one of
the build methods and instead putting a stronger focus on a smaller set.

This simplifies documentation and testing.

winbuild is for Windows users and use "nmake" directly using a set of
handcrafted Makefiles.

## CVE warnings

I have yapped about "security scanners" before and now they have stricken
again. Previously we have mostly experienced how users on Windows have gotten
reports about how the curl.exe shipped by Microsoft contains code vulnerable
to a CVE. This week a user on macOS reported how they use Crowd Strike and
this week it alerts them that libcurl on macOS is vulnerable to CVE-2024-9681
[12].

The user then comes asking us for advice on what to do and there really is
nothing we can do because that file is a component of macOS and it is
maintained and managed by Apple. The libcurl Apple ships is version 8.7.1 and
that version can be vulnerable to up to five different curl CVEs [13], but
only the top two actually affect the macOS version due to build conditions.

The curl security team set CVE-2024-9681 to severity LOW because it seems
highly unlikely to trigger for many users (and even when it does, not with too
significant drawbacks). NVD and CISA both set it to MEDIUM with CVSS around
six. It seems silly to warn users much about this, but hey, someone sells
these scanners and someone buys them. There is business to be made there.

Of course it is also interesting to see that people are apparently still using
Crowd Strike.

## exported sessions

As one of the large changed merged in this curl release cycle, Stefan
Eissing's new code adds experimental support for exporting and importing SSL
sessions [14].

Put shortly: reusing a saved session makes the TLS handshake faster and it
offers users the ability to use TLS early data so that curl can send data
earlier in the handshake process, thus completing faster.

Stefan's blog shows with some performance numbers that it can actually make a
difference and make curl complete its task faster.

This feature is marked EXPERIMENTAL for now, to allow us to tweak the API and
behavior for a while going forward based on feedback and experiences we get.
It also means that you need to explicitly enable this feature in your build to
get it, and we strongly discourage users from shipping it in production.

## 6265bis

The IETF document known as 6265bis[16] has been submitted to become an RFC.
This is the updated HTTP cookies document. I have a pending blog post draft to
publish once this gets a number and becomes an actual RFC.

In curl I this week landed a change that makes cookie expiration dates capped
at 400 days into the future [15], because that is what the new spec says a
client SHOULD do. This is a change I have worked with on and off during quite
some time and what caused the me the biggest headache was how to get it tested
properly without relying on a debug build (where we can set a fake artificial
time). Eventually I think I managed to come up with an approach that is
decent. See the commit for details if your curios on the specifics.

I believe all other changes in the spec update has already been taken care of
in the curl code since a long time back.

There is already work started on yet another update after this one. I will
most likely write about that as well in a future email.

## Getting started

I did a webinar called "getting started with libcurl" on Thursday, and there
is now a YouTube recording of it [17]. I have done basically the same
presentation before so some of you might recognize it. It should work as a
decent intro to people who are new to libcurl and doing internet transfers with
it.

## WorstFit

Well known security hacker Orange Tsai wrote this extensive (to say the least)
post about problems with Unicode in Windows [18], in which it is mentioned
that curl on Windows is affected - at least the build we provide on curl.se
is, the version Microsoft ships seems to be fine.

The post also features two quotes by me from the HackerOne report [19] they
filed about this issue a while back. I still maintain that this is problem
caused outside of curl that makes curl a victim. Of course, we might still
eventually have to do something to reduce the impact.

## Coming up

- curl feature freeze starts tomorrow for the 8.12.0 release
- I do a talk about curl and Open Source in digital infra on Thursday

## Links

[1] = https://un.curl.dev/people/communication.html
[2] = https://un.curl.dev/
[3] = https://github.com/curl/curl/blob/master/docs/INFRASTRUCTURE.md
[4] = https://daniel.haxx.se/blog/2016/10/03/screenshotted-curl-credits/
[5] = https://curl.se/dashboard.html
[6] = https://curl.se/dashboard1.html#testinfra
[7] = https://curl.se/dashboard1.html#testinfra-kloc
[8] = https://curl.se/dashboard1.html#testinfra-test
[9] = https://mastodon.social/@bagder/113792696623049894
[10] = https://github.com/curl/curl/pull/14313
[11] = https://github.com/curl/curl/pull/15920
[12] = https://curl.se/docs/CVE-2024-9681.html
[13] = https://curl.se/docs/vuln-8.7.1.html
[14] = https://github.com/icing/blog/blob/main/curl-sessions-earlydata.md
[15] = https://github.com/curl/curl/pull/15937
[16] = https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-rfc6265bis-19
[17] = https://youtu.be/W-0yEqLMkwg
[18] = https://blog.orange.tw/posts/2025-01-worstfit-unveiling-hidden-transformers-in-windows-ansi/
[19] = https://hackerone.com/reports/2550951
[20] = https://curl.se/bug/?i=15909

-- 

  / daniel.haxx.se