[Daniel's week] August 23, 2024

Fri Aug 23 14:51:11 CEST 2024

Hello,

So another week ends.

## blogposts

I continued to publish blog posts about upcoming new features that ship in
curl 8.10.0: "skip a curl transfer"[1] and "a file name when none exists"[2]
are the third and fourth articles about features this version introduces. It
was not quite intention but clearly this single release is going to ship a
whole bunch of new goodies for curl command line users.

I might do a fifth one as well later to explain the new and somewhat subtle
embedded CA store feature.

## hackerone

We received a new security vulnerability report over at hackerone[3] the other
day and it was quickly proved to be valid. We currently consider it to be
severity medium and are working with the reporter to make sure we patch it
correctly and document all the necessary details in the security advisory.
This vulnerability is scheduled to be announced in sync with the pending curl
8.10.0 release on September 11.

## CVE math

Speaking of CVEs. Greg Kroah-Hartman did a presentation [4] at OSS China this
week where he mentioned that the Linux kernel security team now publishes 60
CVEs weekly.

The Linux kernel is 38 million lines of code, curl is 170 thousand lines. The
Kernel is thus 223 times larger. 60 CVEs per week makes 3100 CVEs per year.
3100 / 223 = 14.

Looking at curl stats and its rate of average number of CVEs filed per year
the last five years [5], it is currently 13. Which I think is amazingly close.

It makes about one CVE per year per every 13K LOC for either project, give or
take.

## full curl

I was given a "full curl" [6] Canadian whiskey as a gift by Andreas Scherbaum
this week. I gave him a curl tshirt in return.

## build times

For the fun of it I decided to measure how fast I can build curl these days.
Especially since I knew Viktor Szakats has been working on making sure "unity
builds" work fine (meaning concatenating all files into a single one to build
just the entire thing in one single go).

On my local dev machine:

Running cmake to generate a unity build takes 11.1 seconds. "make -sj" then
takes 2.3 seconds to build curl and libcurl.

Running configure to setup a fairly full-stocked build takes 9.7 seconds,
"make -sj" then takes 3.4 seconds. This is then not a unity build.

Conclusion: curl builds pretty fast no matter what.

## a google frontend bug

We received a curl bug report [7] on a HTTP/2 problem when doing a range
request. Stefan Eissing took it for a spin and had soon figured out the
details. The problem in this case using a URL on the https://go.dev site, is
in the server side. The so called "Google Frontend" server.

The server side misbehaves when it gets a range request over HTTP/2 that it
can't fulfill. The response is just wrong so curl complains.

I reached out to someone I know who works with HTTP related things at the Goog
and pointed out this issue as I have no idea how to properly or officially
report these things.

## setopt testing

In another curl discussion it was noted that we had shipped a regression in a
recent libcurl release. The problem was that setting a particular option to
NULL caused `curl_easy_setopt()` to return error, when it never had done that
in the past and really it should not.

We fixed the regression quickly, but the reason we had not discovered the
problem and let it slip in took in on a journey.

We have this test case 1521 that generates a huge C program that sets *all*
300+ libcurl options to a few different values to make sure nothing strange
happens when it does. It should have detected this regression but did not. It
took me on a rewrite of the test [8] to make it much pickier about what errors
each setopt accepts so that ideally the next time we accidentally break such a
return code, this test should detect it.

## code style

Yet another issue revealed itself this week when for an inexplicable reason we
managed to merge a PR that introduced code style warnings [9]. The idea is
that we verify code style in CI jobs so we should always detect and fix such
mistakes before merging but here it had for some reason failed.

This made me decide to make a new stand-alone code-style-check-all-C-code CI
job [10] to completely remove the risk that this happens again. The upside
with this new job is that it checks some code that previously was never
checked at all so now we can be sure exactly all our C code follow our
standards.

## scorecard

An old curl discussion thread [11] about running the OpenSSF Scorecard [12]
and displaying its findings came back to life this week.

The scorecard seems to say that we score 8.3 or 9.5 out of 10, but there are
multiple findings I take issue with and I truly believe we should score even
higher. I think the tool is biased against the style of project we are and
development we do and I do not think we should publish the scorecard output as
it might give the impression it is something else than just an opinion by a
script.

I will of course value and appreciate more thoughts on this topic there.

## DoH options

It was suggested that we lack libcurl options to specify TLS preferences for
DoH transfers. Specifically the user wanted different options for the "real"
transfer than they got for the DoH ones as curl currently inherit quite a
number of TLS options to DoH from the main transfer.

We have a significant amount of TLS related options already, and it would be
unfortunate to have to add as many new options to become DoH-[options]. We
already mostly have that situation with proxy-[options] for HTTPS proxies.

A fresh idea popped up during this discussion: maybe we should allow an
application to create a "DoH settings easy handle" and a "proxy settings easy
handle" for which the application can set the specific TLS options
accordingly, and then we add just two new options for
copy-TLS-options-from-this-handle for DoH and proxy correspondingly. Something
to think about as the feature window is closed for a few more weeks anyway.

That solution does however not address the challenge with command line
options; if tool users want the same TLS flexibility when using DoH.

## Coming up

  - work on curl - we're not done yet

## Links

[1] = https://daniel.haxx.se/blog/2024/08/17/skip-a-curl-transfer/
[2] = https://daniel.haxx.se/blog/2024/08/19/a-filename-when-none-exists/
[3] = https://hackerone.com/curl
[4] = https://git.sr.ht/~gregkh/presentation-security/blob/main/security-stuff.pdf
[5] = https://curl.se/dashboard1.html#vulns-per-year
[6] = https://mastodon.social/@bagder/112994023935070735
[7] = https://github.com/curl/curl/issues/14620
[8] = https://github.com/curl/curl/pull/14634
[9] = https://github.com/curl/curl/pull/14624
[10] = https://github.com/curl/curl/pull/14625
[11] = https://github.com/curl/curl/discussions/12609
[12] = https://scorecard.dev/

-- 

  / daniel.haxx.se