What Racket does with package checksums, and how it got that way

There is a thread on the Racket Discourse right now about whether raco pkg verifies the packages it downloads, and what the checksums attached to every package are actually for. I had a small hand in the way raco pkg treats checksums in certain uncommon cases, and most of the backstory is scattered across a GitHub issue, two Fossil forum threads and some Discord messages. This post collects it in one place, and then lays out what Racket currently does with checksums when you install and update packages.

What I was trying to do

I use Fossil for some of my projects, and I wanted to publish Racket packages straight from a self-hosted Fossil repository, with no build or upload step: commit, sync, done.

raco pkg doesn’t know anything about Fossil, but it does support what the docs call manual deployment. You put two files on a web server: pkg.zip, containing the package, and pkg.zip.CHECKSUM, containing the package’s checksum. Then you register the URL of the zip file on the package server.

At the time, the Racket docs defined the checksum like this:

a string that identifies different releases of a package. A package can be updated when its checksum changes, whether or not its version changes. The checksum normally can be computed as the SHA1 of the package’s content.

“Normally can be” suggested that any string unique to a release would do. Fossil can serve a zip of the latest check-in on any branch, so the zip file was easy. The checksum file was the hard part.

January 2024: asking around

I asked on the Fossil forum whether Fossil had a URL that served only the hash of the latest check-in on a branch. It didn’t. Daniel Dumitriu pointed out that Racket wanted the SHA-1 of the package. I pointed back at the “normally can be” wording, and the thread moved on to other ideas. Warren Young and Andy Bradford noted that /raw/trunk serves the manifest of the latest trunk check-in: a block of text, several lines long, that is unique to that check-in. If any unique string would do, the manifest would do.

Fossil’s URL aliases (configured in the repository settings) only allow one path element on each side, so /raw/trunk couldn’t be an alias target. But it turned out that /raw?name=trunk works the same way, and that one can be. So I could set up two aliases:

• /pluto.zip → /zip?name=pluto.zip
• /pluto.zip.CHECKSUM → /raw?name=trunk

Around the same time I asked on the Racket Discord whether the checksum had to be SHA-1. Matthew Flatt went back and forth on it for a few minutes before settling on: “I’m back to thinking that you get to pick any method.” In a separate conversation, Philip McGrath suggested that a fossil+https:// package source could be built as a thin layer over the zip-plus-checksum approach. With the aliases working, that seemed unnecessary.

Then I set the whole thing aside for a year.

March 2025: it doesn’t work

On March 20, 2025, I finally set up a test package called pluto in a Fossil repo with both aliases and registered it on the package server. It failed:

pkg: mismatched checksum on package
  package source: https://joeldueck.com/code/pluto/pluto.zip
  expected: "C Initial\\scommit\nD 2025-03-20T17:26:52.192\nF LICENSE-APACHE 147adf8d…
  got: "610bdaf3cc73ed4a6f31bf12a0a549bbc4e62ad1"

In practice, raco pkg computed the SHA-1 of the downloaded zip and refused to continue unless it matched the contents of the .CHECKSUM file. The “any string” reading of the docs held for some package sources but not this one.

On Discord, Sam Tobin-Hochstadt asked what would happen if I left out the .CHECKSUM file entirely. Answer:

raco pkg install: remote package had no checksum
  package: https://joeldueck.com/code/mercury/pluto.zip

Sam agreed that the checksum is mainly there to detect updates, and that the package system ought to support this case somehow, though it wasn’t obvious how. I asked whether raco pkg could support fossil+http:// URLs by shelling out to fossil; Sam’s view was that a package source that exists by default shouldn’t depend on an external program being installed. (raco pkg can install from Git sources without git installed; you only need git for --clone.)

The next day I did three things:

1. Opened racket/racket#5231, changing “normally can be computed as the SHA1” to “must be computed as the SHA1” in the docs, so they at least matched reality.
2. Posted an update to the Fossil forum thread. Richard Hipp offered pointers for adding a JSON endpoint myself, but Andy Bradford got there first: within a day he had a branch where /whatis/trunk?hash returns only the full hash of the named check-in. That was merged to trunk on March 25 and shipped in Fossil 2.26.
3. Opened racket/racket#5232, proposing that raco pkg stop verifying downloaded zip files against their .CHECKSUM files.

The Racket side: issue #5232

My argument in the issue was that checking the zip against the .CHECKSUM file doesn’t add any security: anyone who can tamper with pkg.zip (on the server or in transit) can just as easily tamper with pkg.zip.CHECKSUM sitting next to it. So the check might as well go, and the checksum could go back to being any string that identifies a release. That would let Fossil’s SHA3-256 hashes (or manifests) work.

Matthew was receptive at first. After trying the change, he found two problems with it:

• When a checksum is supplied up front (for example with --checksum, or from a catalog), raco pkg never downloads the .CHECKSUM file at all. In that case the comparison against the zip is the only check there is.
• Downloading two separate files is inherently racy. If the package is updated between the two requests, you get a checksum for one release and a zip for another. Comparing the zip’s own hash catches that.

He proposed the opposite approach. Keep requiring the SHA-1 of the archive, but make the .CHECKSUM file optional: if it isn’t there, download the zip and compute the SHA-1 directly. To avoid downloading the whole package every time someone checks for updates, cache the server’s ETag for the zip alongside the computed SHA-1, and send If-None-Match on the next request. If the server answers 304 Not Modified, use the cached SHA-1.

He asked whether Fossil’s ETags would cooperate. I checked: the ETag on Fossil’s /zip response changed after a commit to trunk, and stayed the same after a commit to a different branch. Good enough.

I offered to write the patch, but Matthew had already done it in commit d920ad5 (Racket 8.16.0.4, so first released in 8.17). He also rewrote the checksum definition in the docs:

The checksum must be computed as the SHA-1 hash of the package’s archive when the package is distributed in archive form. A package can be installed in a way that it has no checksum, but then the package installation does not support updating.

In May 2025, after Matthew upgraded the package server to 8.17, I was able to register and install a package served directly from Fossil, with no .CHECKSUM file.

The Fossil side: ETags

While testing, I noticed that Racket never wrote anything to its ETag cache when talking to my Fossil server. That led to two small Fossil bugs:

• Per the HTTP spec, an If-None-Match value is enclosed in double quotes. Fossil compared the header value as-is, quotes included, against its ETag, so a spec-following client never got a 304.
• Fossil sent its own ETag header without the quotes. Racket’s parser follows the spec and ignores an unquoted ETag, so it never cached anything.

Florian Balmer committed fixes for both within days (on May 26 and May 30). They shipped in Fossil 2.27, released September 30, 2025.

What Racket does with checksums today

Here is what raco pkg does as of Racket 9.3, based on reading the current source and some testing.

A package’s checksum is a string that identifies a release. Its main job is to let raco pkg update tell whether a newer release exists. Where the checksum comes from depends on the kind of package source:

• Archive URL (https://host/pkg.zip): the SHA-1 of the archive. A .CHECKSUM file next to the archive is optional, but if it exists it must contain that SHA-1.
• Git or GitHub: the commit ID that the branch or tag currently points to.
• Remote directory with a MANIFEST: whatever is in the directory’s .CHECKSUM file, computed any way you like. This is the one case where the old “any string” reading still holds.
• Catalog name: whatever checksum the catalog has on record. The package server gets that value by polling the package’s real source, using the same raco pkg code as everything above.

For a URL, this is the procedure for finding the current checksum:

When you install a package, raco pkg downloads the archive and compares its SHA-1 against the checksum it expected. A mismatch is an error:

For a package installed by catalog name, the expected checksum comes from the catalog. So if the author publishes a new release after the package server’s last poll but before the next one, an install will fail with “unexpected checksum” until the catalog catches up. (I hit this once during testing and couldn’t explain it at the time; this is my best guess.)

When you update, raco pkg compares the checksum it recorded at install time against the current one, and reinstalls if they differ:

Note that for a package installed from the catalog, checking for updates never touches the server hosting the package. Only the package server polls that.

Packages installed as links (or from a local directory or file) have no checksum, so raco pkg update never reinstalls them. But they aren’t ignored either: with --all or --update-deps, their dependencies still get checked and updated.

A few practical notes if you want to serve a package from Fossil without a .CHECKSUM file:

• Use an alias like /pkg.zip → /zip?name=pkg.zip. Don’t register a URL of the form /zip/trunk/pkg.zip: Fossil treats the last path element there as just a download filename, so it will happily answer a request for /zip/trunk/pkg.zip.CHECKSUM with the zip file itself, and raco pkg will treat the zip’s bytes as the checksum. With the alias, the .CHECKSUM request gets a 404, which is what you want.
• You need Fossil 2.27 or later for the ETag shortcut. Older versions still work, but every update check downloads the whole zip.
• Fossil’s zip of a given check-in is byte-for-byte stable across requests, so the SHA-1 only changes when the branch gets a new check-in.
• Without a .CHECKSUM file, the first install of a package downloads the zip twice: once to compute the checksum and once to install it.

What the checksum doesn’t do

Going back to the Discourse thread: the comparison during install is a consistency check. It makes sure the zip you received is the release the checksum describes. It doesn’t make sure that release is what the package’s author intended, because the checksum comes from the same place as the zip: either a file sitting next to it on the same server, or a catalog that computed it by downloading from that server. For Git sources, as Sam pointed out in the thread, Racket uses the commit ID as the checksum but doesn’t check the downloaded content against it.

The proposal in that thread, where authors register a hash with the package server and clients verify against it, would be a different mechanism with a different purpose. The changes described here neither help nor hinder it, as far as I can tell.