[Daniel's week] August 2, 2024

Fri Aug 2 23:12:19 CEST 2024

Hello friends!

Another week ended. I did some stuff.

## libuv

I started the week gently [1] by doing a little work on the curl command line
tool. The idea is to have a special debug-version of curl to be able to do
parallel transfers using the event-based libcurl API and to do that proper it
needs to use an event-library. I chose to base my work on libuv. I picked
libuv mostly because that's the event library I worked with last and not with
much other research or scrutiny. I think it works.

With the special debug-version of curl and using a debug-only command line
option, curl will then use the `curl_multi_socket_action()` function when
`--parallel` is used. To enable us to run test cases doing parallel transfers
that then test drives and verifies the libcurl socket_action API.

The PR is not merged yet since I got busy with other things the rest of the
week.

## patch release

During our Monday curl sync-up meeting with Dan Fandrich and Stefan Eissing we
decided that some of the early regressions we had gotten reported seemed
frequent and annoying enough to warrant a patch release. We decided to ship
that on Wednesday to give me a little time to do the release correctly in no
rush.

Tuesday morning we received a report about a security vulnerability that we
quickly could confirm was real. That gave us a busy day to work on a fix,
write a security advisory, realize the first fix was not good enough, do a
second fix etc. It was not until late evening my time I could finally merge
the update and confirmed fine fix - with a set of bonus unit tests for the
function that previously had the security problem.

It was not the calm and controlled follow-up release we thought just the day
before.

On Wednesday morning I packaged curl 8.9.1, did the release video stream and
now a few days later I think we can say that it in spite of everything it went
pretty well.

## trurl 0.14

With the new URL API flag that was introduced in libcurl 8.9.0 that trurl can
use, it also made sense to make a new trurl release to allow everyone the
chance to use this. I mean, I'm pretty sure basically not a single soul out
there actually know or have used that particular niche feature yet but still,
it felt right.

The feature? Now trurl can set a scheme for URLs conditionally if it is not
previously set, and leave it as-is if it was already set to something.

Of course, immediately after the 0.14 release it was reported to me about
build issues on Windows so now we have a reason to ship trurl 0.15 soon.

## Podcasts

I was a guest in two separate podcast episode recordings this week. I will
save mentioning their names until they actually go public.

## backdoors

Triggered by a discussion I had earlier this week in one of the podcast
episodes I participated in, I decided to add a section to the "security"
section in everything curl about what we do in the curl project to mitigate
the backdoor risk [2]. I posted about it on Mastodon, received great feedback
that made me do a few extra updates and clarifications to the page. I think it
now pretty clearly describes what we do. There is no silver bullets or special
magic, just plain old engineering and doing everything we *should* and *can*
do.

One little thing I added this week is a CI job that scans the curl git
repository for what appears to be binary files and errors out if one is
detected that is not already whitelisted with a checksum [5]. This is not
all-covering nor does it completely protect us from risks of malicious
content, but it is a little automated helper along the way.

## reproducible builds

How good is reproducible builds and the *possibility* of verifying release
builds if on one actually does it? I suppose most people assume, expect, hope
or wish someone else does it.

I decided to help said "someone" to verify curl release tarballs, and when I
started working on a script to do the job I realized that our existing
tarballs were actually missing a few files so they could not be recreated
without having a git checkout. Unfortunate, but I fixed that now [3] so it
should be better in the future.

My new verify-release script [4] simply unpacks a tarball, rebuilds it and
finally verifies that the final tarball is identical to the initial one! I
also created a CI job that verifies that the script itself works and that
tarballs actually are created in a reproducible manner.

If a few people start running this (or a similar) script in their machines and
surroundings when they get new curl releases, it should be a fairly good way
to make sure that nothing unintended was injected into the tarballs in the
release process in the machine used for making the release. Like if my account
or local development machines were ever breached.

Put simply: it would detect xz style attacks.

## QUIC

A potential 5th QUIC backend for curl was submitted as a PR this week [6].

This effort is using a Linux kernel module for doing the QUIC protocol as a
socket and nghttp3 for the HTTP/3 part - and I believe the idea seems to be to
be somewhat generic in the TLS layer. It made me update my curl HTTP/3 backend
slide [7] for how it might soon look like in curl internals. The slide is a
little presumptuous because right now it does not work exactly like that -
this initial PR is done using GnuTLS only.

## Enterprise

For unknown reasons GitHub rearranged things and curl was converted into a
GitHub Enterprise Account. This means that curl is now registered as
enterprise [8] which owns the organization curl [9] which owns the git
repository curl [10] which builds the command line tool curl.

Enter jokes about enterprises.

## curl maintainers BoF

The Debian curl maintainers talked curl related topics at DebConf24 in South
Korea. See the video [11].

## Coming up

- the curl feature window opens Saturday. There is a long queue of pull
   requests waiting to get merged. Buckle up! I suspect the following two weeks
   can get busy.

## Links

[1] = https://github.com/curl/curl/pull/14298
[2] = https://everything.curl.dev/project/security.html#backdoors-and-supply-chain-risks
[3] = https://github.com/curl/curl/pull/14336
[4] = https://github.com/curl/curl/pull/14350
[5] = https://github.com/curl/curl/pull/14333
[6] = https://github.com/curl/curl/pull/14313
[7] = https://mastodon.social/@bagder/112881751025255409
[8] = https://github.com/enterprises/curl
[9] = https://github.com/curl
[10] = https://github.com/curl/curl
[11] = https://meetings-archive.debian.net/pub/debian-meetings/2024/DebConf24/

-- 

  / daniel.haxx.se