Using Git LFS with Radicle

After a recent mention on social media, I wanted to verify whether Git LFS can really work with Radicle. I’d read a couple of times on the Radicle Zulip chat that “it should just work”.

Now, I know Radicle doesn’t (yet) ship an LFS server, and Git LFS doesn’t take that very well — it tries to upload the blobs to the rad remote as if it were an LFS endpoint, finds nothing there to receive them, and the whole push aborts.

But it is absolutely possible to use a custom transfer agent instead. I used lfs-s3, to push/pull LFS objects directly to/from an S3 bucket (MinIO in my case, but this also works with AWS S3, Cloudflare R2, … and any S3-compatible endpoint).

With it configured:

• Radicle replicates the repository and the small LFS pointer files just fine,
• lfs-s3 stores and serves the actual large blobs in the S3 bucket, with no LFS HTTP server required.

Here’s how I got this working, in case you want to follow along:

How it works

                    git push / rad init
   working copy  ─────────────────────────►   Radicle storage
  ┌────────────┐   pointer files               (repo + refs, gossiped
  │ bigfile.bin│   + code + history             to other nodes)
  │            │
  └─────┬──────┘
        │ clean/smudge filter
        │ + lfs-s3 transfer agent
        ▼
   ┌──────────────┐
   │  S3 bucket   │   the real big blobs live here
   │ (AWS/MinIO/…)│   keyed by their LFS oid (zstd-compressed)
   └──────────────┘

lfs-s3 is registered as a standalone transfer agent, which means Git LFS uses it for all transfers and never queries an LFS API server. So pushes to Radicle succeed (only pointers travel through Radicle) and the blobs go straight to S3.

Prerequisites

• A Radicle identity and a running node:

  rad auth --alias <your-alias>   # create an identity (set RAD_PASSPHRASE to skip the prompt)
  rad node start                  # start the node
  rad self                        # confirm DID / Node ID
  

• git and git-lfs installed (lfs-s3 supports git-lfs >= 3.3.0):

  sudo apt-get install -y git-lfs   # Debian/Ubuntu
  git lfs install                   # one-time, sets up the global LFS filter
  

• An S3 bucket and credentials (see the MinIO option below if you don’t have one).

Install lfs-s3

Grab a release binary from the releases page ( Linux, macOS, Windows) and put it on your PATH:

curl -fsSL -o ~/.local/bin/lfs-s3 \
  https://github.com/nicolas-graves/lfs-s3/releases/download/0.2.2/lfs-s3-linux
chmod +x ~/.local/bin/lfs-s3

Or build from source (requires Go — this is what I needed on arm64, since the release lfs-s3-linux binary is x86-64):

go install github.com/nicolas-graves/lfs-s3@latest   # installs to $(go env GOPATH)/bin

A quick gotcha: go install puts the binary in $(go env GOPATH)/bin (typically ~/go/bin), which is often not on PATH. Either add it (export PATH="$HOME/go/bin:$PATH") or use the absolute binary path as the value of lfs.customtransfer.lfs-s3.path in the next section.

Verify it runs:

lfs-s3 --help    # prints the -access_key_id / -bucket / -endpoint flags

Provision an S3 bucket

Any S3 provider works. I tested with MinIO because I already had it running next to my Radicle node:

# Run MinIO
podman run -d --name minio -p 9000:9000 \
  -e MINIO_ROOT_USER=minioadmin -e MINIO_ROOT_PASSWORD=minioadmin \
  quay.io/minio/minio server /data

# Create the bucket (using the MinIO client image)
podman run --rm --network host --entrypoint /bin/sh quay.io/minio/mc -c '
  mc alias set local http://localhost:9000 minioadmin minioadmin
  mc mb -p local/lfs-demo'

For AWS S3, just create a bucket and an IAM user/role with read/write access to it; the endpoint is the standard regional S3 endpoint and you can omit --use_path_style.

Wire it up

mkdir git-lfs-project && cd git-lfs-project
git init -b main

# Register lfs-s3 as the standalone transfer agent for this repo.
git config --add lfs.customtransfer.lfs-s3.path lfs-s3
git config --add lfs.standalonetransferagent lfs-s3
git config --add lfs.customtransfer.lfs-s3.args \
  "--access_key_id=minioadmin --secret_access_key=minioadmin \
   --bucket=lfs-demo --endpoint=http://localhost:9000 \
   --region=us-east-1 --use_path_style=true"

# Track the file types you want in LFS, then commit as usual.
git lfs track "*.bin"
git add .gitattributes
echo "# My project" > README.md
head -c 5242880 /dev/urandom > bigfile.bin   # a 5 MiB stand-in
git add README.md bigfile.bin
git commit -m "Add 5MiB binary tracked by git-lfs"

Don’t forget to commit the .gitattributes file — it’s what tells every clone which paths are LFS-managed. You can double-check what LFS is tracking with git lfs ls-files.

Heads up — credentials passed in lfs.customtransfer.lfs-s3.args end up in cleartext in .git/config. You might want to check out the lfs-s3 Alternative configuration method.

Initialise the Radicle repository

With the agent configured, rad init now succeeds: the pre-push hook hands the blobs to lfs-s3 (which uploads them to S3), and only the pointers go to Radicle.

rad init \
  --name "git-lfs-project" \
  --description "Demo: git-lfs (lfs-s3) with Radicle" \
  --default-branch main \
  --public --no-confirm

Expected output contains your Repository ID (newer rad versions print a few more informational lines around these — e.g. hints about rad publish and git push):

✓ Repository git-lfs-project created.
Your Repository ID (RID) is rad:z….

For later changes, push normally — the same wiring applies:

git add . && git commit -m "…"
git push

Verify what went where

The blob should be in S3 (zstd-compressed, keyed by its oid), and Radicle should hold only the pointer:

# Object in the bucket, e.g. de1b17e1….zstd, ~5 MiB:
podman run --rm --network host --entrypoint /bin/sh quay.io/minio/mc -c '
  mc alias set local http://localhost:9000 minioadmin minioadmin
  mc ls --recursive local/lfs-demo'

# Radicle stores the pointer, and NOT the 5 MiB file, as expected:
git cat-file -p main:bigfile.bin
# version https://git-lfs.github.com/spec/v1
# oid sha256:de1b17e1…
# size 5242880

Sharing with a collaborator: clone and verify

The repository travels over Radicle as usual (rad clone, seeding, etc.). The only extra requirement is that collaborators can reach the same S3 bucket and configure lfs-s3:

This is the bit that proves it actually works end-to-end. On a machine that can reach both the Radicle node and the S3 bucket:

# Create a new radicle profile
rad auth
# start a radicle node
rad node start

# After a few seconds where the radicle-node has had a chance to sync with the peer-to-peer 
# network, you will be able to clone your repo from Radicle 
rad clone rad://z…  git-lfs-project
cd git-lfs-project

# Right now bigfile.bin is just the pointer:
head -1 bigfile.bin

# Configure the same agent, then pull the blobs from S3:
git config --add lfs.customtransfer.lfs-s3.path lfs-s3
git config --add lfs.standalonetransferagent lfs-s3
git config --add lfs.customtransfer.lfs-s3.args \
  "--access_key_id=minioadmin --secret_access_key=minioadmin \
   --bucket=lfs-demo --endpoint=http://localhost:9000 \
   --region=us-east-1 --use_path_style=true"
git lfs pull

# The file is now the real 5 MiB and matches the original byte-for-byte:
sha256sum bigfile.bin   # equals the source oid de1b17e1…

Success ! 🎉🎉

Troubleshooting

A couple of issues I hit along the way:

A few things to keep in mind

Radicle isn’t an LFS host — it only carries the pointer files. If the S3 bucket is lost, the large files are gone, and Radicle can’t recover them. Unlike normal Radicle objects, LFS blobs don’t gossip across nodes either; distribution is entirely your bucket’s job. So a collaborator who can rad clone but can’t reach the bucket gets the code and pointers, but not the file contents.

lfs-s3 is also a deliberately lightweight agent — the author notes that the “existing repo” migration path is best-effort, and recommends lfs-dal if you need something more capable.

That’s it — happy seeding! 🌱