The script that publishes articles on this site tests every article on a local copy of the site first, at https://linkhub.ddev.site. That local site runs in DDEV, and its HTTPS certificate is signed by a local certificate authority that mkcert created. On the Ubuntu install where the pipeline started, that just worked. When I moved it to my Linux From Scratch install, every request to the local site failed TLS verification, while requests to the real site were unaffected.
Why it worked on Ubuntu
mkcert -install adds its root certificate to the system trust store. mkcert's README lists the stores it knows how to install into, and on Linux that means distributions with update-ca-trust or update-ca-certificates, Ubuntu among them. Once the root was in the system store, anything that trusted the system store trusted the local site.
On Ubuntu, Python's requests did. That's not upstream behavior; it's packaging. The requests.certs module says there is only one default CA bundle, the one from the certifi package, and that distributions packaging requests can change where it points to use a separately packaged bundle. The pipeline used the distribution's package there, Debian's patched python3-requests, which reads the system store, so mkcert's root was included without anyone asking.
Why it failed on LFS
On the LFS install, the pipeline's tools were set up per user, without root, and requests came from pip install --user. That's upstream requests, and upstream requests trusts certifi's bundle, which on this machine is a file under ~/.local/lib/python3.14/site-packages/certifi/. That bundle knows every public CA and nothing about a CA created on my laptop. The local site's certificate failed verification; the real site's, signed by a public CA, didn't.
The system store wouldn't have helped on its own anyway: mkcert's root wasn't in it. On this LFS system the system bundle is /etc/pki/tls/certs/ca-bundle.crt, the path the BLFS book's certificate tooling uses, and not Debian's /etc/ssl/certs/ca-certificates.crt.
The fix: one bundle per run
Instead of changing the system, the wrapper script builds a combined bundle each time it runs:
- find the system bundle, trying the BLFS path and then the Debian path
- find mkcert's root with
$(mkcert -CAROOT)/rootCA.pem - concatenate the two into a temporary file
- point three environment variables at that file
The three variables cover three different TLS stacks:
REQUESTS_CA_BUNDLE, which requests checks first when choosing a CA bundle, beforeCURL_CA_BUNDLEand its certifi defaultCURL_CA_BUNDLE, which curl uses as its--cacertvalueSSL_CERT_FILE, which OpenSSL documents as the default file containing CA certificates, for anything else built on it
The wrapper's cleanup function deletes the temporary file when the run ends, along with stopping the other things it started.
Why production verification is unaffected
Pointing requests at a custom bundle replaces certifi for that run, which sounds like it could weaken verification of the real site. It doesn't, because the bundle starts with every certificate in the system bundle: 121 of them on this machine. The real site's certificate verifies against a public CA, exactly as before. The only thing the bundle adds is one root that exists on this laptop.
The shortcut alternatives, verify=False or a switch that skips TLS checks for local runs, would make the local test behave differently from production, which is the opposite of what a test against a local copy is for.
What to take from it
If a Python tool trusts your local CA on one machine and not on another, check where its requests came from before anything else. A distribution's package may read the system store; the same version from pip reads certifi's bundle. python3 -c "import certifi; print(certifi.where())" shows which file pip's requests trusts by default, and REQUESTS_CA_BUNDLE is the documented way to point it somewhere else.
Leave a comment
Comments are reviewed before they appear.