--- title: What Racket does with package checksums, and how it got that way date: 2026-10-07 slug: racket-package-checksums --- There is [a thread on the Racket Discourse][disc] right now about whether `raco pkg` verifies the packages it downloads, and what the checksums attached to every package are actually for. I had a small hand in the way `raco pkg` treats checksums in certain uncommon cases, and most of the backstory is scattered across a GitHub issue, two Fossil forum threads and some Discord messages. This post collects it in one place, and then lays out what Racket currently does with checksums when you install and update packages. [disc]: https://racket.discourse.group/t/how-do-we-specify-verify-checksums-in-package-dependencies-today/4416 ## What I was trying to do I use [Fossil](https://fossil-scm.org) for some of my projects, and I wanted to publish Racket packages straight from a self-hosted Fossil repository, with no build or upload step: commit, sync, done. `raco pkg` doesn’t know anything about Fossil, but it does support what the docs call [manual deployment][md]. You put two files on a web server: `pkg.zip`, containing the package, and `pkg.zip.CHECKSUM`, containing the package’s *checksum*. Then you register the URL of the zip file on the package server. At the time, the Racket docs [defined][concepts] the checksum like this: > a string that identifies different releases of a package. A package can be updated when its > checksum changes, whether or not its version changes. The checksum normally can be computed as the > SHA1 of the package’s content. “Normally can be” suggested that any string unique to a release would do. Fossil can serve a zip of the latest check-in on any branch, so the zip file was easy. The checksum file was the hard part. [md]: https://docs.racket-lang.org/pkg/getting-started.html#%28part._manual-deploy%29 [concepts]: https://docs.racket-lang.org/pkg/Package_Concepts.html ## January 2024: asking around I asked [on the Fossil forum][ff1] whether Fossil had a URL that served only the hash of the latest check-in on a branch. It didn’t. Daniel Dumitriu pointed out that Racket wanted the SHA-1 of the package. I pointed back at the “normally can be” wording, and the thread moved on to other ideas. Warren Young and Andy Bradford noted that `/raw/trunk` serves the *manifest* of the latest trunk check-in: a block of text, several lines long, that is unique to that check-in. If any unique string would do, the manifest would do. [ff1]: https://fossil-scm.org/forum/forumpost/4b11cfc8fac93800 Fossil’s URL aliases (configured in the repository settings) only allow one path element on each side, so `/raw/trunk` couldn’t be an alias target. But it turned out that `/raw?name=trunk` works the same way, and that one *can* be. So I could set up two aliases: * `/pluto.zip` → `/zip?name=pluto.zip` * `/pluto.zip.CHECKSUM` → `/raw?name=trunk` Around the same time I asked on the Racket Discord whether the checksum had to be SHA-1. Matthew Flatt went back and forth on it for a few minutes before settling on: “I’m back to thinking that you get to pick any method.” In a separate conversation, Philip McGrath suggested that a `fossil+https://` package source could be built as a thin layer over the zip-plus-checksum approach. With the aliases working, that seemed unnecessary. Then I set the whole thing aside for a year. ## March 2025: it doesn’t work On March 20, 2025, I finally set up a test package called `pluto` in a Fossil repo with both aliases and registered it on the package server. It failed: ``` pkg: mismatched checksum on package package source: https://joeldueck.com/code/pluto/pluto.zip expected: "C Initial\\scommit\nD 2025-03-20T17:26:52.192\nF LICENSE-APACHE 147adf8d… got: "610bdaf3cc73ed4a6f31bf12a0a549bbc4e62ad1" ``` In practice, `raco pkg` computed the SHA-1 of the downloaded zip and refused to continue unless it matched the contents of the `.CHECKSUM` file. The “any string” reading of the docs held for some package sources but not this one. On Discord, Sam Tobin-Hochstadt asked what would happen if I left out the `.CHECKSUM` file entirely. Answer: ``` raco pkg install: remote package had no checksum package: https://joeldueck.com/code/mercury/pluto.zip ``` Sam agreed that the checksum is mainly there to detect updates, and that the package system ought to support this case somehow, though it wasn’t obvious how. I asked whether `raco pkg` could support `fossil+http://` URLs by shelling out to `fossil`; Sam’s view was that a package source that exists by default shouldn’t depend on an external program being installed. (`raco pkg` can install from Git sources without `git` installed; you only need `git` for `--clone`.) The next day I did three things: 1. Opened [racket/racket#5231][pr], changing “normally can be computed as the SHA1” to “must be computed as the SHA1” in the docs, so they at least matched reality. 2. Posted [an update][ff2] to the Fossil forum thread. Richard Hipp offered pointers for adding a JSON endpoint myself, but Andy Bradford got there first: within a day he had a branch where `/whatis/trunk?hash` returns only the full hash of the named check-in. That was [merged to trunk][f226] on March 25 and shipped in Fossil 2.26. 3. Opened [racket/racket#5232][issue], proposing that `raco pkg` stop verifying downloaded zip files against their `.CHECKSUM` files. [pr]: https://github.com/racket/racket/pull/5231 [ff2]: https://fossil-scm.org/forum/forumpost/4b11cfc8fac93800 [f226]: https://fossil-scm.org/home/info/49567895 [issue]: https://github.com/racket/racket/issues/5232 ## The Racket side: [issue #5232][issue] My argument in the [issue] was that checking the zip against the `.CHECKSUM` file doesn’t add any security: anyone who can tamper with `pkg.zip` (on the server or in transit) can just as easily tamper with `pkg.zip.CHECKSUM` sitting next to it. So the check might as well go, and the checksum could go back to being any string that identifies a release. That would let Fossil’s SHA3-256 hashes (or manifests) work. Matthew was receptive at first. After trying the change, he found two problems with it: * When a checksum is supplied up front (for example with `--checksum`, or from a catalog), `raco pkg` never downloads the `.CHECKSUM` file at all. In that case the comparison against the zip is the only check there is. * Downloading two separate files is inherently racy. If the package is updated between the two requests, you get a checksum for one release and a zip for another. Comparing the zip’s own hash catches that. He proposed the opposite approach. Keep requiring the SHA-1 of the archive, but make the `.CHECKSUM` file optional: if it isn’t there, download the zip and compute the SHA-1 directly. To avoid downloading the whole package every time someone checks for updates, cache the server’s [`ETag`][et] for the zip alongside the computed SHA-1, and send `If-None-Match` on the next request. If the server answers `304 Not Modified`, use the cached SHA-1. [et]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/ETag He asked whether Fossil’s ETags would cooperate. I checked: the ETag on Fossil’s `/zip` response changed after a commit to trunk, and stayed the same after a commit to a different branch. Good enough. I offered to write the patch, but Matthew had already done it in [commit d920ad5][d920] (Racket 8.16.0.4, so first released in 8.17). He also rewrote the checksum definition in the docs: > The checksum must be computed as the SHA-1 hash of the package’s archive when the package is > distributed in archive form. A package can be installed in a way that it has no checksum, but then > the package installation does not support updating. [d920]: https://github.com/racket/racket/commit/d920ad59a9124ce2c6c4912edb3704d4b379c6d7 In May 2025, after Matthew upgraded the package server to 8.17, I was able to register and install a package served directly from Fossil, with no `.CHECKSUM` file. ## The Fossil side: ETags While testing, I noticed that Racket never wrote anything to its ETag cache when talking to my Fossil server. That led to [two small Fossil bugs][ff3]: * Per the HTTP spec, an `If-None-Match` value is enclosed in double quotes. Fossil compared the header value as-is, quotes included, against its ETag, so a spec-following client never got a `304`. * Fossil sent its own `ETag` header *without* the quotes. Racket’s parser follows the spec and ignores an unquoted ETag, so it never cached anything. [ff3]: https://fossil-scm.org/forum/forumpost/93f8690efee7309b8d061e0549d8637ba097f2a5692bbb90d48dce9b9ba9d3c1 Florian Balmer committed fixes for both within days (on May 26 and May 30). They shipped in Fossil 2.27, released September 30, 2025. ## What Racket does with checksums today Here is what `raco pkg` does as of Racket 9.3, based on reading the current source and some testing. A package’s checksum is a string that identifies a release. Its main job is to let `raco pkg update` tell whether a newer release exists. Where the checksum comes from depends on the kind of package source: * **Archive URL** (`https://host/pkg.zip`): the SHA-1 of the archive. A `.CHECKSUM` file next to the archive is optional, but if it exists it must contain that SHA-1. * **Git or GitHub**: the commit ID that the branch or tag currently points to. * **Remote directory with a `MANIFEST`**: whatever is in the directory’s `.CHECKSUM` file, computed any way you like. This is the one case where the old “any string” reading still holds. * **Catalog name**: whatever checksum the catalog has on record. The package server gets that value by polling the package’s real source, using the same `raco pkg` code as everything above. For a URL, this is the procedure for finding the current checksum: •figure["/posts/img/raco-pkg-checksum-lookup.svg"]{How `raco pkg` finds the current checksum of a package at a URL.} When you install a package, `raco pkg` downloads the archive and compares its SHA-1 against the checksum it expected. A mismatch is an error: •figure["/posts/img/raco-pkg-install-checksums.svg"]{Checksums during `raco pkg install`.} For a package installed by catalog name, the expected checksum comes from the catalog. So if the author publishes a new release after the package server’s last poll but before the next one, an install will fail with “unexpected checksum” until the catalog catches up. (I hit this once during testing and couldn’t explain it at the time; this is my best guess.) When you update, `raco pkg` compares the checksum it recorded at install time against the current one, and reinstalls if they differ: •figure["/posts/img/raco-pkg-update-checksums.svg"]{Checksums during `raco pkg update`.} Note that for a package installed from the catalog, checking for updates never touches the server hosting the package. Only the package server polls that. Packages installed as links (or from a local directory or file) have no checksum, so `raco pkg update` never reinstalls them. But they aren’t ignored either: with `--all` or `--update-deps`, their dependencies still get checked and updated. A few practical notes if you want to serve a package from Fossil without a `.CHECKSUM` file: * Use an alias like `/pkg.zip` → `/zip?name=pkg.zip`. Don’t register a URL of the form `/zip/trunk/pkg.zip`: Fossil treats the last path element there as just a download filename, so it will happily answer a request for `/zip/trunk/pkg.zip.CHECKSUM` with the zip file itself, and `raco pkg` will treat the zip’s bytes as the checksum. With the alias, the `.CHECKSUM` request gets a 404, which is what you want. * You need Fossil 2.27 or later for the ETag shortcut. Older versions still work, but every update check downloads the whole zip. * Fossil’s zip of a given check-in is byte-for-byte stable across requests, so the SHA-1 only changes when the branch gets a new check-in. * Without a `.CHECKSUM` file, the first install of a package downloads the zip twice: once to compute the checksum and once to install it. ## What the checksum doesn’t do Going back to the Discourse thread: the comparison during install is a consistency check. It makes sure the zip you received is the release the checksum describes. It doesn’t make sure that release is what the package’s author intended, because the checksum comes from the same place as the zip: either a file sitting next to it on the same server, or a catalog that computed it by downloading from that server. For Git sources, as Sam pointed out in the thread, Racket uses the commit ID as the checksum but doesn’t check the downloaded content against it. The proposal in that thread, where authors register a hash with the package server and clients verify against it, would be a different mechanism with a different purpose. The changes described here neither help nor hinder it, as far as I can tell.