From e7910a9ffc1a4a16c4a08c7f21db88caae898490 Mon Sep 17 00:00:00 2001 From: hmirin Date: Wed, 14 Oct 2026 14:31:01 +0900 Subject: [PATCH] Rewrite the README and add a user guide - Rewrite the README for newcomers and experts: why end-to-end encryption, the three differences (fast, safe, no master key), an at-a-glance performance excerpt, a comparison with git-crypt and git-remote-gcrypt, the security model, and the full benchmark table. - Add illustrations (Tabler Icons and Griddy Icons, both MIT, licenses included). - Add docs/guide.md, CONTRIBUTING.md, and SECURITY.md. --- CONTRIBUTING.md | 60 +++ README.md | 743 +++++++++++++++++----------------- SECURITY.md | 20 + docs/art/LICENSE-griddy-icons | 21 + docs/art/LICENSE-tabler-icons | 21 + docs/art/overview.svg | 47 +++ docs/art/why-e2ee.svg | 48 +++ docs/guide.md | 277 +++++++++++++ 8 files changed, 867 insertions(+), 370 deletions(-) create mode 100644 CONTRIBUTING.md create mode 100644 SECURITY.md create mode 100644 docs/art/LICENSE-griddy-icons create mode 100644 docs/art/LICENSE-tabler-icons create mode 100644 docs/art/overview.svg create mode 100644 docs/art/why-e2ee.svg create mode 100644 docs/guide.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..29445f6 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,60 @@ +# Contributing + +Issues and pull requests are welcome. For security problems, follow +[SECURITY.md](SECURITY.md) instead of opening a public issue. + +## Ground rules + +- Keep the storage contract backend-neutral. Filesystem behavior must not leak + into encrypted repository semantics. +- Do not invent cryptographic primitives. Use established AEAD, signature, + hashing, and key-wrapping libraries. +- Treat storage as malicious for confidentiality and integrity, and document + availability, rollback, freeze, and equivocation limits precisely. +- Never commit key files or decrypted test data. Test secrets must live only in + temporary directories. +- Protocol changes must update [docs/spec.md](docs/spec.md) and, where relevant, + [docs/design.md](docs/design.md) in the same change. + +## Build and test + +```console +cargo test --all-targets +cargo clippy --all-targets -- -D warnings +cargo fmt --all -- --check +``` + +CI runs the same three commands. The ignored power-cut harness is not part of +the default run; its protocol, Windows commands, and durability limits are in +[docs/durability.md](docs/durability.md). + +The test suite covers: + +- incremental push and reconstruction into a fresh repository; +- native clone, fetch, pull, push, dry-run, refspec, and force-push behavior; +- rollback, same-generation fork, and ciphertext-tampering rejection; +- streaming-AEAD chunk boundaries, wrong keys or AAD, reordering, duplication, + truncation, trailing bytes, forged sizes, and legacy-format rejection; +- independent device add, read, and write, non-admin rejection, + genesis-substitution rejection, administrative rollback pinning, device + revocation without pack rewrites, and multi-generation offline catch-up; +- malformed predecessor-link, generation-key commitment, recipient-set, and + signed-but-incomplete Git object-graph rejection; +- two independent carrier writers racing real `git push` processes, with + exactly one winner; +- plaintext-absence checks across carrier history; +- applicable black-box scenarios independently reimplemented from Git + upstream's + [`t/t5801-remote-helpers.sh`](https://github.com/git/git/blob/master/t/t5801-remote-helpers.sh). + +A local bare Git repository is enough for deterministic carrier concurrency +tests: local-path pushes still run Git's real `receive-pack`, ref locking, and +fast-forward checks. Hosted smoke tests remain useful for authentication, +request limits, and provider-specific policy. + +## Benchmarks + +The opt-in harness in `scripts/` measures encryption, reconstruction, +verification, and incremental updates. See [docs/benchmarks.md](docs/benchmarks.md). +Benchmark outputs, repository keys, and reconstructed data must not be +committed. diff --git a/README.md b/README.md index c3787ae..d7011b9 100644 --- a/README.md +++ b/README.md @@ -1,389 +1,392 @@ -# git-remote-e2ee - -[![CI](https://github.com/combinatrix-ai/git-remote-e2ee/actions/workflows/ci.yml/badge.svg)](https://github.com/combinatrix-ai/git-remote-e2ee/actions/workflows/ci.yml) -[![License: MIT OR Apache-2.0](https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-blue.svg)](#license) - -End-to-end encrypted Git remotes on storage you do not have to trust with your -repository contents or Git metadata. - -`git-remote-e2ee` is a Git remote helper. You keep using ordinary `git clone`, -`git fetch`, `git pull`, and `git push`; the helper turns Git packs and refs into -authenticated ciphertext before they leave the client. The storage provider -sees opaque objects and an opaque latest-manifest pointer, not the inner branch -names, commit IDs, paths, authors, or messages. +

git-remote-e2ee

+ +

+ Git your host can't read.
+ End-to-end encrypted Git remotes. Keep using clone, pull, and push; + GitHub, a NAS, or any other storage only ever holds ciphertext. +

+ +

+ CI + License: MIT OR Apache-2.0 + Status: prototype +

+ +

+ Quick start + · How it compares + · Design + · Specification +

+ +

+ On your devices the repository is ordinary Git; the storage host only holds opaque encrypted objects +

> [!WARNING] -> This is an early prototype, not yet a safe backup system. The repository -> format is unstable and there are no compatibility guarantees between -> versions. Keep an independent copy of every repository and key. - -The implemented v4 protocol gives every published generation a fresh random -root key. Each active reader receives one small public-key envelope for that -generation; pack ciphertext is stored only once, independent of reader count, -and is authenticated in bounded-memory 1 MiB segments. -See the [design overview](docs/design.md) and [normative specification](docs/spec.md). - -## Why this exists - -Git hosting normally requires the server to understand the repository. That is -useful for web diffs, pull requests, search, and CI, but it also exposes the -entire object graph and most repository metadata to the host. - -`git-remote-e2ee` deliberately gives up server-side Git features. Git semantics -stay on trusted clients, while the server is reduced to two jobs: - -1. store immutable ciphertext objects; -2. atomically replace one opaque head value if it has not changed. - -This differs from [`git-crypt`](https://github.com/AGWA/git-crypt), which is -designed to encrypt selected files inside an otherwise normal repository. It is -closer to -[`git-remote-gcrypt`](https://github.com/spwhitton/git-remote-gcrypt), but uses -modern authenticated encryption, a signed policy and manifest chain, -per-device keys, incremental Git packs, and a backend-neutral compare-and-swap -storage contract. In -particular, the carrier-Git backend does not require uploading the entire inner -repository on every update. - -## Comparison with existing tools - -These projects solve two different problems. File-filter tools keep an ordinary -Git repository useful to its host while hiding selected blob contents. Encrypted -remote helpers hide the repository as a whole, which also removes server-side -diffs, pull requests, search, and CI over the plaintext. - -| | [`git-crypt`](https://github.com/AGWA/git-crypt) | [`transcrypt`](https://github.com/elasticdog/transcrypt) | [`git-remote-gcrypt`](https://github.com/spwhitton/git-remote-gcrypt) | `git-remote-e2ee` | -|---|---|---|---|---| -| Primary use | Encrypt selected files | Encrypt selected files | Encrypt a complete Git remote | Encrypt a complete Git remote | -| Integration | Git clean/smudge filters | Git clean/smudge filters | Git remote helper | Git remote helper | -| Hidden from host | Selected blob contents | Selected blob contents | Inner objects, refs, and encrypted manifest contents | Inner objects, refs, paths, authors, messages, and manifest contents | -| Host retains normal Git features | Yes, for visible repository data | Yes, for visible repository data | No | No | -| Update granularity | Per encrypted file; a changed encrypted file is stored again | Per encrypted file | Backend-dependent; Git and SFTP backends may retransmit full history | Incremental Git packs on filesystem and carrier-Git backends | -| Integrity model | Git repository integrity plus deterministic encrypted blobs | Git repository integrity plus encrypted blobs | Encrypted and signed manifest; ciphertext-addressed packs | AEAD packs/manifests, signed policy and manifest chains, and per-client history pinning | -| Key and collaborator model | Symmetric key or GPG users | Shared passphrase | GPG participants and symmetric mode | Per-repository device keys; any authorized device can decrypt independently; writers and administrators are separate roles | -| Maturity | Established | Established | Established | Experimental prototype | - -The closest comparison is `git-remote-gcrypt`. It already supports participant -management and several transports, making it the more mature choice today. -`git-remote-e2ee` is exploring a different storage protocol: immutable -incremental packs plus an explicit compare-and-swap head, modern AEAD, and a -signed history chain that returning clients pin locally. According to -`git-remote-gcrypt`'s -[`PERFORMANCE` documentation](https://manpages.debian.org/trixie/git-remote-gcrypt/git-remote-gcrypt.1.en.html#PERFORMANCE), -its arbitrary Git and SFTP transports upload the complete repository history on -each push; its rsync backend behaves differently. The comparison is therefore -backend-specific, not a claim that every `git-remote-gcrypt` update is a full -upload. - -If you need to protect a few secrets while retaining GitHub or GitLab features, -use a file-filter tool. If the storage provider must not learn the repository -structure or metadata, use a whole-remote encryption design—and, for now, treat -this project as research-grade software. - -## Current features - -- Normal Git remote-helper workflow for clone, fetch, pull, and push -- Per-device HPKE (X25519/HKDF-SHA-256/ChaCha20-Poly1305) generation-key envelopes -- A fresh random generation root for every successful HEAD publication -- Domain-separated HKDF subkeys and XChaCha20-Poly1305 payload encryption -- Streaming authenticated pack encryption and decryption with bounded Rust-side - memory rather than whole-pack buffers -- Ed25519-signed, append-only policy and manifest chains -- 1-of-N recipient access: each authorized device unlocks with only its own key -- Separate reader, writer, and administrator authorization -- Atomic device addition and revocation without rewriting historical packs -- An authenticated backward key chain: the current key unlocks earlier - generations, while an earlier key cannot unlock later generations -- Delta manifests and incremental Git packs rather than full repository snapshots -- Client-side fast-forward enforcement and explicit force push -- Atomic stale-writer rejection through compare-and-swap -- Per-client rollback and manifest-fork detection after first observation -- Complete signed history, ciphertext, and reconstructed Git object-graph - verification before refs or continuity pins move -- Filesystem storage backend -- Carrier-Git backend for GitHub, GitLab, a bare repository, or another ordinary - Git remote - -Push destinations are currently limited to branches under `refs/heads/*`. Tags -and branch deletion are rejected without publishing a new manifest. - -## Install - -The project currently builds two binaries: - -- `git-e2ee`: repository initialization, key generation, and diagnostic CLI -- `git-remote-e2ee`: helper invoked automatically by Git for `e2ee::` URLs - -```console -cargo install --path . -git-e2ee --help -``` - -Both binaries must be on `PATH` for native Git integration. - -## Quick start: filesystem backend - -Create a repository key and encrypted storage directory: - -```console -git-e2ee keygen --output /safe/place/repository.key.json -git-e2ee init \ - --storage /srv/encrypted/example \ - --key /safe/place/repository.key.json -``` - -Add it to an existing local Git repository: - -```console -git remote add private 'e2ee::/srv/encrypted/example' -git config remote.private.e2ee-key /safe/place/repository.key.json -git push -u private main -git fetch private -``` - -The key path is local Git configuration. The key file is never written to the -encrypted remote and must never be committed. - -## Add and revoke devices - -Each device has a different private key. Only the small public device file is -given to an administrator. If machine A created the repository, set up machine -B like this (substitute the repository root printed by A's `keygen`): - -```console -# Machine B -git-e2ee keygen \ - --repository-root \ - --output /safe/place/machine-b.key.json -git-e2ee device-export \ - --key /safe/place/machine-b.key.json \ - --output machine-b.public.json - -# Machine A, after receiving only machine-b.public.json -git-e2ee device-add \ - --storage /srv/encrypted/example \ - --key /safe/place/repository.key.json \ - --device machine-b.public.json -``` - -The default added device can read and write but cannot change policy. Pass -`--admin` to grant administration too. `git-e2ee device-list` prints opaque -device IDs; revoke one with: +> This is an early prototype. The repository format is unstable and may change +> without a migration path. Do not use it as your only copy of anything: keep an +> independent copy of every repository and every key. + +## Why end-to-end encryption? + +A private repository is private from other people, not from the host. Even +with encryption at rest, the host keeps the key next to the data, so whoever +breaks into the host, its storage, or its backups can read your code. +End-to-end encryption keeps the key on your devices: an intruder finds only +locked boxes, and anything they change is rejected. + +

+ Left: with a private repository your files sit on GitHub or GitLab in readable form, so a hacker who breaks in reads everything. Right: with git-remote-e2ee the host holds only locked boxes and the key stays on your laptop, so a hacker who breaks in finds nothing readable. +

+ +- **Good fits:** notes and journals, unpublished research, client or NDA work, + private configuration, backups on storage you do not control. +- **What you give up:** no web diffs, pull-request reviews, code search, or + hosted CI on the plaintext, and nobody can recover your data if you lose + every key. + +## Why git-remote-e2ee + +Encrypting Git is not new. File-level tools such as +[`git-crypt`](https://github.com/AGWA/git-crypt) encrypt selected files, and +[`git-remote-gcrypt`](https://github.com/spwhitton/git-remote-gcrypt) has +encrypted whole repositories with GnuPG for years. `git-remote-e2ee` hides the +whole repository like gcrypt, and fixes what makes that painful to live with: + +- **Fast: pushes upload only what changed.** A push uploads the new Git pack + plus a few KB of metadata on every backend: about 9 KB for a one-line + change in the [benchmark](#performance). With a Git or SFTP backend such as + GitHub, gcrypt re-uploads the entire encrypted history on every push: 921 MB + for the same change. +- **Safe: no silent force pushes.** Fast-forward checks run on the client, and + the storage moves `HEAD` only by compare-and-swap. When two people push at + once, one wins and the other gets an ordinary rejection. With gcrypt every + push is effectively a force push, so a push made without pulling first can + erase someone else's work. +- **No master key: nothing long-lived is shared.** git-crypt protects a + repository with one symmetric key that every collaborator ends up holding, + and it cannot revoke anyone. `git-remote-e2ee` creates a fresh random key for + every push and wraps it separately for each authorized device's own key. An + administrator can revoke one device without rewriting history, and it + receives nothing built on top of that revocation. + +It also does the rest of what an encrypted remote should: + +- **Hides who has access.** Device keys, roles, and membership changes are + encrypted. Storage sees only a padded device count, rounded up to a power of + two. +- **Read, write, and admin roles.** Give a CI runner or a reviewer a read-only + key. Only administrators can add or revoke devices. +- **Detects tampering, rollback, and forks.** A clone that has synced before + refuses an older or diverging state, and `git-e2ee recover` lets a writer + continue explicitly after someone corrupts `HEAD`. +- **Behaves like Git.** Branches, tags, remote branch deletion, and explicit + force pushes all work, and the remote grows only by what you add: 878 MiB + after five pushes, against 5.2 GiB for gcrypt. +- **Stays light.** A small push needs about 9 MiB of memory and takes about a + second. +- **Needs nothing else.** Two Rust binaries with all cryptography built in, no + GnuPG or OpenSSL. Storage can be any Git host (GitHub, GitLab, a bare + repository) or a local or mounted directory. + +Under the hood, Git runs on your machine as usual. The helper encrypts the +packs and refs Git hands it, uploads them as opaque immutable objects, and then +atomically moves one opaque `HEAD` pointer. + +### At a glance + +Godot repository (867 MiB of history), a local bare repository as the Git +host, Apple M1 Pro, median of three runs. Bold marks where `git-remote-e2ee` +beats `git-remote-gcrypt`. Full results in [Performance](#performance). + +| | Plain Git | `git-remote-gcrypt` | `git-remote-e2ee` | +|---|---:|---:|---:| +| Initial push | 28.4 s | 36.6 s | **22.9 s** | +| Fresh fetch | 27.7 s | 55.0 s | **40.0 s** | +| Tiny push | 0.08 s | 6.5 s | **0.97 s** | +| Tiny update (fetch) | 0.07 s | 22.6 s | **0.77 s** | +| Data sent per tiny push | 4.5 KB | 921 MB | **8.9 KB** | + +## Quick start + +Install from source (a Rust toolchain is required). This installs both +`git-e2ee`, the setup CLI, and `git-remote-e2ee`, the helper Git calls for +`e2ee::` URLs. Both must be on `PATH`. ```console -git-e2ee device-revoke \ - --storage /srv/encrypted/example \ - --key /safe/place/repository.key.json \ - --device-id +cargo install --git https://github.com/combinatrix-ai/git-remote-e2ee ``` -For a carrier-Git backend, use `--remote ` instead of `--storage` -with `device-add`, `device-list`, and `device-revoke`. - -Adding or revoking a device publishes a fresh generation key in one -compare-and-swap operation. Its header contains one small envelope for each -active reader, while all historical manifests and packs stay unchanged. A newly -added reader can use the current key's authenticated backward links to decrypt -the complete history. A revoked reader retains the snapshot it could already -decrypt but receives no key for the new generation or later ones. -The CLI keeps an administrative continuity pin next to the administrator key as -`.admin-state.json`. Preserve that file together with the key. - -This is deliberately **not multisig**. The current implementation accepts one -authorized administrator signature for a policy change. The wire format has an -administrator threshold and signature array so a future version can add M-of-N, -but this version fails closed on any threshold other than 1. - -## Carrier-Git backend - -An ordinary Git repository can act as the ciphertext carrier. It may be empty -or contain unrelated branches; `git-remote-e2ee` uses its own -`git-remote-e2ee` branch. +Use an empty GitHub repository as encrypted storage and push an existing +project to it: ```console -git-e2ee keygen --output /safe/place/carrier.key.json +git-e2ee keygen --output ~/.config/git-e2ee/notes.key.json git-e2ee carrier-init \ - --remote https://git.example/user/encrypted-carrier.git \ - --key /safe/place/carrier.key.json + --remote https://github.com/you/notes-encrypted.git \ + --key ~/.config/git-e2ee/notes.key.json -git remote add private \ - 'e2ee::git+https://git.example/user/encrypted-carrier.git' -git config remote.private.e2ee-key /safe/place/carrier.key.json +cd my-notes +git remote add private 'e2ee::git+https://github.com/you/notes-encrypted.git' +git config remote.private.e2ee-key ~/.config/git-e2ee/notes.key.json git push -u private main ``` -Clone by supplying the key path once: - -```console -git -c e2ee.key=/safe/place/carrier.key.json clone \ - 'e2ee::git+https://git.example/user/encrypted-carrier.git' -``` - -The helper persists the path as local `remote.origin.e2ee-key` configuration in -the new clone. Later `git fetch`, `git pull`, and `git push` need no wrapper. - -The carrier stores only the outer structure below: - -```text -refs/heads/git-remote-e2ee -└── e2ee/ - ├── HEAD - ├── objects/aa//00000000 - ├── manifests/bb//00000000 - └── policies/cc//00000000 -``` - -Ciphertext is divided into 32 MiB chunks so it can be carried as ordinary Git -blobs. The host can still observe outer commit times, chunk counts and sizes, -update frequency, and total growth. +From then on, `git pull` and `git push` work as usual. The key file stays on +your machine; never commit it, and keep a backup somewhere safe. + +To use a second machine, give it its own key and authorize its public half. +The [user guide](docs/guide.md#add-a-second-device) walks through it, as well as +the directory backend, cloning, and revoking devices. + +## How it compares + +There are two kinds of encrypted-Git tools. **File-level tools**, represented +here by [`git-crypt`](https://github.com/AGWA/git-crypt), encrypt the contents +of selected files inside an otherwise normal repository. **Encrypted remotes**, +[`git-remote-gcrypt`](https://github.com/spwhitton/git-remote-gcrypt) and +`git-remote-e2ee`, encrypt the whole repository, history included. + +### What the storage host can learn + +| | Private repo | `git-crypt` | `git-remote-gcrypt` | `git-remote-e2ee` | +|---|:---:|:---:|:---:|:---:| +| Contents of files you chose to protect | visible | hidden | hidden | hidden | +| Contents of all other files | visible | visible | hidden | hidden | +| File names and directory layout | visible | visible | hidden | hidden | +| Commit messages, authors, and dates | visible | visible | hidden | hidden | +| Branch names and the commit graph | visible | visible | hidden | hidden | +| Which files changed, their sizes, identical files | visible | visible¹ | hidden | hidden | +| When you push, and roughly how much | visible | visible | visible | visible | +| Collaborators | account list | key fingerprints² | count only³ | count only⁴ | + +### What each tool can do + +| | `git-crypt` | `git-remote-gcrypt` | `git-remote-e2ee` | +|---|:---:|:---:|:---:| +| Normal `clone`, `pull`, and `push` | ○ | ○ | ○ | +| GitHub or GitLab as storage | ○ | ○ | ○ | +| Hosted diffs, review, and search for unencrypted parts | ○ | × | × | +| Encrypt only selected files | ○ | × | × | +| Push uploads only new data | △⁵ | △⁶ | ○ | +| Concurrent pushes can't silently overwrite each other | ○ | ×⁷ | ○ | +| Tampering detected | △⁸ | ○ | ○ | +| No master key | ×⁹ | ○ | ○ | +| Revoke one collaborator | ×¹⁰ | ×¹¹ | ○¹² | +| Roles (read / write / admin) | × | ×¹³ | ○ | +| Detects a rolled-back or forked remote | × | × | △¹⁴ | +| No external dependencies | △¹⁵ | ×¹⁶ | ○ | +| Tags and remote branch deletion | ○ | ○ | ○ | +| Mature, stable format | ○ | ○ | × | +| Cryptography | AES-256-CTR, HMAC-SHA1 SIV | OpenPGP (GnuPG) | XChaCha20-Poly1305, HPKE (X25519), Ed25519 | + +○ supported · △ with caveats · × not supported + +
+Notes + +1. git-crypt's README states that it does not hide when a file changes, its + length, or whether two files are identical. +2. In git-crypt's GPG mode, the repository key is encrypted to each user and + committed under `.git-crypt/`, named by key fingerprint. +3. gcrypt hides recipient key IDs by default (`gpg -R`); the number of + encrypted-key packets is still observable. +4. The envelope count is padded to the next power of two, with a minimum of + four, so storage sees only an upper bound on the number of devices. Public + keys, roles, and membership changes are encrypted, though size patterns may + suggest a membership change. Do not reuse device keys between repositories. +5. A changed encrypted file becomes a new, unrelated ciphertext blob, so Git + cannot delta-compress it against the previous version. +6. gcrypt is incremental with its local and rsync backends, but its + [documentation](https://manpages.debian.org/trixie/git-remote-gcrypt/git-remote-gcrypt.1.en.html) + says a Git or SFTP backend uploads the entire history on every push, and it + may repack the remote without warning. +7. Every gcrypt push is effectively a force push. Its explicit-force option + prevents accidents but does not coordinate concurrent writers. +8. git-crypt authenticates encrypted file contents; file names, history, and + which files are encrypted are ordinary Git data. +9. A master key here means one long-lived key that every collaborator ends up + holding. git-crypt uses one symmetric repository key for the life of the + repository: GPG mode wraps that same key to each user's GPG key, and + without GPG everyone shares the same exported key file. gcrypt and + git-remote-e2ee encrypt each push under fresh keys delivered to each + recipient's own key. +10. git-crypt's README states that it does not support revoking access. +11. A participant can be dropped from `gcrypt.participants`, but gcrypt does + not document a revocation workflow. +12. Revocation publishes a fresh key without rewriting history. It cannot take + back data the device already had. +13. gcrypt's recipient list is the local `gcrypt.participants` setting of + whoever pushes, so any participant who can push decides who can read the + next state. +14. A clone that has synced before rejects rollback or a diverging history. A + brand-new clone cannot tell without an external anchor; see + [Security model](#security-model). +15. git-crypt is a C++ program linked against OpenSSL; GPG mode also needs + GnuPG. +16. gcrypt is a shell script that requires GnuPG. + +
+ +### When another tool is a better fit + +- **git-remote-gcrypt**: you want an established tool with a stable format, + already use GPG, and work alone or can coordinate pushes. +- **A file-level tool such as git-crypt**: only a few files are secret, and you + want GitHub or GitLab to keep working normally for the rest. +- **An ordinary private repository**: you trust the host and want pull + requests, search, previews, and CI. ## Security model -The storage provider is treated as malicious for confidentiality and integrity. -Authenticated encryption, ciphertext-addressed objects, signatures, and the -manifest hash chain detect modification. A returning client pins the newest -manifest it has observed and rejects a lower generation or a history that no -longer descends from that pin. - -The current security claim is: - -- confidentiality of inner repository contents and Git metadata; -- authenticity and integrity of fetched repository state; -- continuity from the state previously observed by the same local clone. -- future-generation exclusion after an atomic device revocation. - -It does **not** independently prevent: - -- rollback presented to a fresh client; -- equivocation between clients shown different valid histories; -- freezing a client at its last valid state; -- deletion or denial of service by storage; -- destructive changes made by an authorized writer; -- disclosure of data a revoked device could access before its revocation; -- disclosure of the complete historical snapshot at or before any leaked - generation key; -- a revoked writer and colluding storage presenting a pre-revocation fork to a - fresh or stale client; -- policy rollback, freezing, or equivocation presented to a fresh client. - -Global rollback and equivocation resistance require gossip or an external -transparency anchor. A returning clone pins both manifest and policy generation. -The administrative CLI also pins its last published state and does not -automatically retry a lost CAS race; inspect the winner and rerun the operation. - -Generation headers and policy objects must remain plaintext-structured so a -device can find its envelope without already knowing the generation key. -Consequently the storage host can see device count, per-repository public keys, -roles, policy changes, and generation numbers. Device keys should never be -reused between repositories. Inner refs, object IDs, paths, authors, messages, -generation keys, derived subkeys, and contents remain encrypted. - -A generation key is intentionally a transferable snapshot capability: leaking -`K_t` exposes generations `0..=t` through the backward links. It does not expose -generation `t+1` or later. Continued future disclosure therefore requires a -reader's private device key, repeated release of each later generation key or -plaintext, or continued access to an authorized device. This is key regression, -not forward secrecy for already published history. - -If two authorized writers race, each may locally create a valid generation key -and ciphertext, but compare-and-swap permits only one HEAD update. The loser -must discard its unpublished generation and retry from the winning HEAD. Anyone -who received the loser's key can decrypt that losing unpublished snapshot; CAS -prevents it from becoming repository history but cannot retract already shared -plaintext or keys. - -## Storage protocol - -The backend-neutral interface has four logical operations. Immutable object -writes are staged so the final ciphertext ID can be learned while streaming: - -```text -begin_object(kind) -> writable stage; stage.finish(id) -open_object(kind, id) -> reader -read_head() -compare_and_swap_head(expected, next) -``` - -The filesystem backend implements head CAS with an advisory lock and a rename -in the storage root. Object publication hard-links a staged file from -`.staging` into `objects//` on the same filesystem and does not replace -an existing id. File contents are flushed before the name is published, new -directories are flushed through the preexisting ancestor, and the parent -directory is flushed again afterwards. A flush error fails the call. Stages -stay inside the backend's own filesystem or checkout, so publication does not -cross filesystems. Crash and power-loss limits are in -[docs/durability.md](docs/durability.md). The carrier-Git backend -implements head CAS as a normal fast-forward push to the outer branch. A future -S3 backend can use multipart upload plus conditional writes, but each provider -must be capability-tested; “S3 compatible” does not by itself promise correct -compare-and-swap behavior. - -A killed filesystem writer can leave an unreachable file under `.staging/`. -No published object or `HEAD` points to it, and stale `.stage-*` entries may be -deleted when no writer is running. Automatic age-based cleanup belongs to -future garbage collection. - -See [docs/design.md](docs/design.md) for the protocol and threat-model details. - -## Tests - -```console -cargo test --all-targets -cargo clippy --all-targets -- -D warnings -cargo fmt --all -- --check -``` - -The ignored power-cut harness is not part of that default run. Protocol, -Windows commands, and durability limits are in [docs/durability.md](docs/durability.md). - -The test suite includes: - -- incremental push and reconstruction into a fresh repository; -- native clone, fetch, pull, push, dry-run, refspec, and force-push behavior; -- rollback, same-generation fork, and ciphertext-tampering rejection; -- streaming-AEAD chunk boundaries, wrong keys/AAD, reordering, duplication, - truncation, trailing bytes, forged sizes, and legacy-format rejection; -- independent device add/read/write, non-admin rejection, genesis-substitution - rejection, administrative rollback pinning, device revocation without pack - rewrites, and multi-generation offline catch-up; -- malformed predecessor-link, generation-key commitment, recipient-set, and - signed-but-incomplete Git object-graph rejection; -- two independent carrier writers racing real `git push` processes, with - exactly one winner; -- plaintext-absence checks across carrier history; -- applicable black-box scenarios independently reimplemented from Git - upstream's - [`t/t5801-remote-helpers.sh`](https://github.com/git/git/blob/master/t/t5801-remote-helpers.sh). - -A local bare Git repository is sufficient for deterministic carrier concurrency -tests: local-path pushes still execute Git's real `receive-pack`, ref locking, -and fast-forward checks. Hosted smoke tests remain useful for authentication, -request limits, and provider-specific policy. - -## Roadmap - -- M-of-N administrative authorization (future format version; threshold 1 only today) -- Recovery and device-key replacement workflows -- Automatic stale-push fetch/retry workflow -- Persistent partial-clone cache for large carrier repositories -- S3 conditional-write backend and provider compatibility suite -- Safe compaction and garbage collection -- Signed checkpoint transitions for compaction without ambiguous key-chain semantics -- Optional gossip or transparency-log anchoring -- Shallow and partial clone support - -## Performance benchmarks - -An opt-in local harness measures initial encryption, fresh reconstruction, -verification, and incremental updates without contacting the source -repository's configured remote. See [docs/benchmarks.md](docs/benchmarks.md). Benchmark -outputs, repository keys, and reconstructed data must not be committed. +The storage provider is treated as malicious. `git-remote-e2ee` aims to +guarantee: + +- confidentiality of inner contents and Git metadata: files, paths, refs, + object IDs, authors, and messages; +- authenticity and integrity of every fetched state: only devices with write + authority under the authenticated policy history can author a state that + conforming clients accept; +- continuity for a clone that has synced before: it rejects any state older + than, or diverging from, its continuity floor, the newest state it accepted + after a complete connectivity check or its own successful push; +- that a revoked device gets no keys for generations that conforming writers + build on top of its revocation. + +Write authority is about what clients accept, not about who can put bytes on +the storage. Anyone who can write to the storage, including a read-only member +or someone with only push access to the carrier repository, can still replay +old valid states, hide updates, or corrupt `HEAD`. Clients reject what they +cannot verify, so this can deny service but cannot make unauthorized history +accepted. Restrict storage write access as a separate layer, for example with +read-only repository permissions and branch protection on GitHub. + +It does **not** prevent: + +- the host or anyone with storage write access deleting data, corrupting + `HEAD`, or refusing service; +- a brand-new clone, or a clone that has not synced since, being shown an old + or forked history, or different clients being shown different histories + (that needs an external anchor, which is on the roadmap); +- a revoked writer or administrator creating a valid-looking fork from before + its revocation, which clients that never saw the revocation cannot tell apart; +- an authorized writer making destructive changes; +- a revoked device reading what it could read before revocation; +- traffic analysis: update times, object sizes, total growth, and the padded + reader count are visible; +- future quantum attacks: key exchange uses X25519. + +Concurrent pushes cannot silently overwrite each other as long as the storage +honors compare-and-swap, as a Git host's fast-forward check or the directory +backend's lock does. + +Keys work as a backward chain. The current key can decrypt all earlier +history, which lets a new device read the whole repository, but an old key +cannot decrypt anything newer. This is key regression, not forward secrecy for +history already published. See [docs/design.md](docs/design.md) and [docs/spec.md](docs/spec.md) +for the full threat model, and [SECURITY.md](SECURITY.md) to report an issue. + +## Performance + +Committing is ordinary Git: encryption happens only when you push or fetch. +Small pushes and fetches upload or download only the change, even when GitHub +or another Git host is the storage. + +Measured on the Godot repository (867 MiB of history), with a local bare +repository standing in for the Git host, on an Apple M1 Pro with 32 GiB RAM. +Median of three runs; tiny rows are a one-line change, repeated five times. +Bold marks where `git-remote-e2ee` beats `git-remote-gcrypt`. + +| | Plain Git | `git-remote-gcrypt` | `git-remote-e2ee` | +|---|---:|---:|---:| +| Initial encryption (to a local directory) | 28.6 s | 11.0 s | **8.1 s** | +| Initial push | 28.4 s | 36.6 s | **22.9 s** | +| Fresh fetch | 27.7 s | 55.0 s | **40.0 s** | +| Tiny commit | 0.09 s | 0.09 s | 0.09 s | +| Tiny push | 0.08 s | 6.5 s | **0.97 s** | +| Tiny update (fetch) | 0.07 s | 22.6 s | **0.77 s** | +| Data sent per tiny push | 4.5 KB | 921 MB | **8.9 KB** | +| Peak memory, initial push | 1.27 GiB | 0.98 GiB | 1.27 GiB | +| Peak memory, tiny push | 37 MiB | 887 MiB | **9 MiB** | +| Remote size after five tiny pushes | 891 MiB | 5,272 MiB | **878 MiB** | +| Client disk after fresh fetch (`.git` plus cache) | 891 MiB | 1,780 MiB | 1,779 MiB | + +With a Git backend, gcrypt sends the whole encrypted history again on every +push, so the remote grows by about the repository size each time. +`git-remote-e2ee` sends the new pack plus a few KB of metadata. On the client, +`git-remote-e2ee` keeps a local cache of the encrypted carrier next to your +repository, which roughly doubles disk use, the same as gcrypt's local copy. +Peak memory for full transfers is dominated by Git's own pack generation. Plain Git +stays faster on small operations because a Git server understands the +repository; an encrypted remote has to verify and decrypt on the client. These +are local measurements without network latency. + +To reproduce these numbers, run `scripts/reproduce-benchmark.sh` from a clone +of this repository on macOS or Linux. It fetches the pinned inputs (about +1 GB), runs three rounds, and writes this table plus raw data and environment +details to `bench-results/`. Expect a few hours and about 12 GB of free disk. +[docs/benchmarks.md](docs/benchmarks.md) has the method, a Linux aarch64 +cross-check, per-push series, phase breakdown, and caveats. + +## FAQ + +**What does the GitHub repository look like?** +One branch, `git-remote-e2ee`, holding ciphertext under `e2ee/`. It can live +next to unrelated branches. Do not edit it by hand. + +**What if I lose my key?** +If no remaining device has access, the data is gone. There is no recovery +service by design. Keep an offline backup of at least one administrator key and +its `.admin-state.json` file. + +**Does revoking a device hide old history from it?** +No. It keeps whatever it could already decrypt. It gets no keys for anything +published on top of the revocation. A writer who never saw the revocation, or +was shown a stale state, could still publish to the old reader set; see +[Security model](#security-model). + +**Can I review pull requests?** +Not on the host. Review happens on a machine that has a key: fetch the branch +and diff locally, or give a CI runner its own read-only device key. + +**What if someone corrupts `HEAD`?** +Clients refuse it and stop. A writer can then run `git-e2ee recover`, which +shows what it found and continues from the last verified state, or from a newer +legitimate state it discovers in the carrier history. See the +[user guide](docs/guide.md#recover-from-a-broken-head). + +**Can the host roll my repository back?** +It can serve old data. A clone that has synced before refuses anything older +than, or diverging from, what it already accepted. A fresh clone, or a clone +that has not synced since a newer state was published, cannot tell yet. + +## Status and roadmap + +Working today: clone, fetch, pull, and push, including tags and remote branch +deletion; the directory and Git-host backends; incremental pushes and fetches +(the Git-host backend keeps a local cache of the carrier); per-device keys with +read, write, and admin roles; adding and revoking devices; rollback and race +detection; and explicit recovery from a corrupted `HEAD`. + +Planned: M-of-N administrator approval, key recovery and replacement, automatic +retry after losing a push race, an S3 conditional-write backend, garbage +collection and compaction, shallow and partial clone, optional +transparency-log anchoring, and an optional file-level mode that encrypts only +selected files while keeping per-device keys and revocation. + +## Documentation + +- [User guide](docs/guide.md): backends, cloning, device management, and the + storage layout +- [docs/design.md](docs/design.md): design overview and threat model +- [docs/spec.md](docs/spec.md): normative protocol specification +- [docs/benchmarks.md](docs/benchmarks.md): benchmark method and results +- [docs/durability.md](docs/durability.md): crash and power-loss guarantees of the filesystem backend +- [CONTRIBUTING.md](CONTRIBUTING.md): building, testing, and what the test + suite covers ## License -Licensed under either of - -- [Apache License, Version 2.0](LICENSE-APACHE), or -- [MIT License](LICENSE-MIT) - -at your option. +Licensed under either of the [Apache License, Version 2.0](LICENSE-APACHE) or +the [MIT License](LICENSE-MIT), at your option. diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..c8ea0bd --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,20 @@ +# Security + +Please report vulnerabilities privately to +**git-remote-e2ee@combinatrix.ai** rather than opening a public issue. + +`git-remote-e2ee` is an early prototype and has not been independently +audited. Its intended guarantees and known limits are listed in the +[README](README.md#security-model) and specified in [docs/design.md](docs/design.md) and +[docs/spec.md](docs/spec.md). Reports are especially welcome about: + +- plaintext, refs, paths, or other inner metadata reaching storage; +- accepting tampered, rolled-back, or forked state that a returning clone + should reject; +- a revoked device obtaining keys for later generations; +- authorization bypass between reader, writer, and administrator roles; +- key material leaking into logs, Git configuration, or the remote. + +Limits already documented, such as rollback shown to a fresh clone or metadata +visible to storage, are not vulnerabilities by themselves. Proposals to close +them are still welcome as issues. diff --git a/docs/art/LICENSE-griddy-icons b/docs/art/LICENSE-griddy-icons new file mode 100644 index 0000000..79255e2 --- /dev/null +++ b/docs/art/LICENSE-griddy-icons @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2025 Griddy Icons + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/docs/art/LICENSE-tabler-icons b/docs/art/LICENSE-tabler-icons new file mode 100644 index 0000000..3e82379 --- /dev/null +++ b/docs/art/LICENSE-tabler-icons @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2020-2026 Paweł Kuna + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/docs/art/overview.svg b/docs/art/overview.svg new file mode 100644 index 0000000..1a4e913 --- /dev/null +++ b/docs/art/overview.svg @@ -0,0 +1,47 @@ + + What your devices see and what the storage host sees + On your devices the repository is ordinary Git with branches, commits, and files. It is encrypted before it leaves the device, so the storage host only holds opaque ciphertext objects and one opaque head pointer. + + + + + + On your devices + ordinary Git, fully readable + $ git log --oneline + a1f3c9e Fix login redirect + 7c02d41 Add customer notes + 3e9b7aa Initial import + $ ls + src/auth.rs notes/acme.md + branches: main, feature/sso + key: only on this device + + + + + encrypt + before + upload + + + + On the storage host + GitHub, a NAS, a bare repo… + e2ee/HEAD + 9f4c…e21a + e2ee/objects/ + 3f/9a1c07… (1.2 MiB) + 7c/e04b5d… (3.4 KiB) + b8/51f2aa… (80 KiB) + no names, paths, messages, + authors, or branches + diff --git a/docs/art/why-e2ee.svg b/docs/art/why-e2ee.svg new file mode 100644 index 0000000..2d60bff --- /dev/null +++ b/docs/art/why-e2ee.svg @@ -0,0 +1,48 @@ + + A hacker breaks into the Git host + Left, a private repository: your files sit on GitHub or GitLab in readable form, so a hacker who breaks in reads everything. Right, git-remote-e2ee: the host stores only locked boxes and the key stays on your laptop, so a hacker who breaks in finds nothing readable. Icons from Tabler Icons (MIT) and Griddy Icons (MIT). + + + + + + + + + + Private repository + + + + + + + GitHub, GitLab, … + + + + + + + + A break-in exposes everything. + Your files sit on the server in readable form. + + git-remote-e2ee + + + + + + + GitHub, GitLab, … + + + + + + + ? + A break-in finds only locked boxes. + The key never leaves your devices. + diff --git a/docs/guide.md b/docs/guide.md new file mode 100644 index 0000000..dec6457 --- /dev/null +++ b/docs/guide.md @@ -0,0 +1,277 @@ +# User guide + +This guide covers everyday use. For why the tool exists and how it compares to +alternatives, see the [README](../README.md). For the protocol, see +the [design overview](design.md) and the [specification](spec.md). + +## Install + +The project builds two binaries: + +- `git-e2ee`: key generation, repository setup, device management, and + diagnostics +- `git-remote-e2ee`: the remote helper Git runs automatically for `e2ee::` URLs + +```console +cargo install --git https://github.com/combinatrix-ai/git-remote-e2ee +git-e2ee --help +``` + +On AArch64 (Apple Silicon or 64-bit Arm Linux), the ChaCha20 dependency selects +its NEON backend automatically, including for `cargo install --git` builds. +Other architectures use the backend selected for their target. + +From a checkout, use `cargo install --path .` instead. Both binaries must be on +`PATH` for Git to find the helper. + +## Keys + +`git-e2ee keygen` writes a private device key file. The first key created for a +repository is its administrator. The key path is stored only in local Git +configuration. The key file is never written to the encrypted remote and must +never be committed. + +The CLI keeps an administrative continuity pin next to an administrator key, +named `.admin-state.json`. Back up that file together with the key. + +## Git host as storage (carrier-Git backend) + +Any ordinary Git repository can carry the ciphertext: GitHub, GitLab, or a bare +repository. It may be empty or contain unrelated branches. `git-remote-e2ee` +uses its own `git-remote-e2ee` branch. + +```console +git-e2ee keygen --output /safe/place/carrier.key.json +git-e2ee carrier-init \ + --remote https://git.example/user/encrypted-carrier.git \ + --key /safe/place/carrier.key.json + +git remote add private \ + 'e2ee::git+https://git.example/user/encrypted-carrier.git' +git config remote.private.e2ee-key /safe/place/carrier.key.json +git push -u private main +``` + +The carrier holds only this outer structure: + +```text +refs/heads/git-remote-e2ee +└── e2ee/ + ├── HEAD + ├── objects/aa//00000000 + ├── manifests/bb//00000000 +``` + +Ciphertext is split into 32 MiB chunks so it can be stored as ordinary Git +blobs. The host can still observe outer commit times, chunk counts and sizes, +update frequency, and total growth. + +Each helper process fetches the carrier branch into a persistent local bare +object cache, then uses a disposable checkout that borrows those cached Git +objects. The cache is shared by helper and `git-e2ee` processes for the same +normalized remote URL. It is stored at +`$XDG_CACHE_HOME/git-remote-e2ee/carrier/` or, when +`XDG_CACHE_HOME` is unset, `~/.cache/git-remote-e2ee/carrier/`. +Set `GIT_REMOTE_E2EE_CACHE_DIR` to override the parent directory that contains +the per-remote cache directories; tests and isolated runs should set it to a +temporary directory. Fetches are serialized across processes, and each refresh +downloads only missing carrier objects. The cache stores carrier Git objects +and metadata only: no keys, plaintext, or decrypted data. + +## Directory as storage (filesystem backend) + +A local or mounted directory can hold the encrypted repository: + +```console +git-e2ee keygen --output /safe/place/repository.key.json +git-e2ee init \ + --storage /srv/encrypted/example \ + --key /safe/place/repository.key.json + +git remote add private 'e2ee::/srv/encrypted/example' +git config remote.private.e2ee-key /safe/place/repository.key.json +git push -u private main +``` + +Do not point two machines at the same directory through a file-sync service. +Concurrent writers rely on the backend's atomic compare-and-swap, which a sync +service does not provide across machines. Use the carrier-Git backend instead. + +## Clone + +Supply the key path once: + +```console +git -c e2ee.key=/safe/place/carrier.key.json clone \ + 'e2ee::git+https://git.example/user/encrypted-carrier.git' +``` + +The helper saves the path as `remote.origin.e2ee-key` in the new clone. Later +`git fetch`, `git pull`, and `git push` need nothing extra. + +## Add a second device + +Each device has its own private key, and only the small public device file ever +leaves it. If machine A created the repository, set up machine B like this. +Substitute the repository root printed by A's `keygen`. + +```console +# Machine B +git-e2ee keygen \ + --repository-root \ + --output /safe/place/machine-b.key.json +git-e2ee device-export \ + --key /safe/place/machine-b.key.json \ + --output machine-b.public.json + +# Machine A, after receiving only machine-b.public.json +git-e2ee device-add \ + --remote https://git.example/user/encrypted-carrier.git \ + --key /safe/place/carrier.key.json \ + --device machine-b.public.json +``` + +Machine B can then clone with its own key, as shown above. + +`device-add --role` grants one of three roles: `read` can clone, fetch, and +pull; `write` adds push access; and `admin` also allows device membership +changes. The default is `write`. For example, add a read-only device with +`--role read`. `device-list` prints the same role names. For the directory +backend, use `--storage ` instead of `--remote ` with +`device-add`, `device-list`, and `device-revoke`. + +## Revoke a device + +`git-e2ee device-list` prints opaque device IDs. Revoke one with: + +```console +git-e2ee device-revoke \ + --remote https://git.example/user/encrypted-carrier.git \ + --key /safe/place/carrier.key.json \ + --device-id +``` + +Adding or revoking a device publishes a fresh generation key in a single +compare-and-swap. The new generation carries one small envelope per active +reader, and all historical manifests and packs stay unchanged. A newly added +device can use the current key's authenticated backward links to decrypt the +complete history. A revoked device keeps what it could already decrypt, but it +receives no key for the new generation or any later one. + +Membership changes need one administrator signature. This is deliberately +**not multisig** yet: the wire format has an administrator threshold and a +signature array so a future version can add M-of-N approval, but this version +refuses any threshold other than 1. + +The administrative CLI does not retry automatically when it loses a +compare-and-swap race. Inspect the winning state and run the command again. + +## Recover from a broken HEAD + +If a storage writer replaces `HEAD` with an invalid manifest, normal fetches +and pushes stop. Run recovery from a Git repository that has already pinned a +known good state for that remote. The default command prints a report and does +not publish or advance the continuity floor: + +```console +git-e2ee recover \ + --remote https://git.example/user/encrypted-carrier.git \ + --key /safe/place/carrier.key.json \ + --repo /work/my-clone \ + --remote-name private +``` + +For a directory backend, use `--storage ` instead of `--remote`. +The local floor is read from +`/.git/git-remote-e2ee//state.json`; use the same repository +and remote name that previously fetched or pushed. A fresh clone has no floor, +so recovery reports that freshness and fork identity are unverified and will +not publish. + +Review the offered manifest IDs, generations, verified signers, replays, and +any fork report. Recovery is allowed only when the current head is demonstrably +invalid. It refuses valid, unverifiable, unsupported, unavailable, and +discontinuous heads. A signer extracted from an invalid head is labelled only +as a claim until its signature is verified. Read-only devices can inspect the +report but cannot publish recovery. + +To publish from the default base, add `--publish`: + +```console +git-e2ee recover \ + --remote https://git.example/user/encrypted-carrier.git \ + --key /safe/place/carrier.key.json \ + --repo /work/my-clone \ + --remote-name private \ + --publish +``` + +The default base is the newest authenticated descendant found after the local +floor, or the floor if there is no descendant. Recovery refuses conflicting +authenticated descendants. Selecting an older offered base requires both +`--base ` and `--discard-newer`; this can omit legitimate changes +or a revocation. A lost compare-and-swap fails. Start a new command and review +the fresh report before trying again. + +The directory backend has no carrier commit history, so it can offer only the +local floor. Publishing requires `--accept-stale-floor` and prints this warning: + +> Updates after this floor may be omitted. This may also omit revocations and +> disclose subsequently published content to devices excluded by a newer +> policy. + +Carrier recovery restores required ciphertext from prior outer commits when +available, checks each content ID, verifies the selected refs' complete Git +object graph in temporary local storage, then publishes a normal fast-forward +carrier commit. If required ciphertext is no longer available, recovery stops. +After successful recovery, run the usual `git fetch` or `git pull` to update +your local remote-tracking refs. + +## Limits today + +- Pushes support branches under `refs/heads/*` and tags under `refs/tags/*`; + other namespaces are rejected. Branches keep fast-forward checks, while an + existing tag needs `--force` to move. Branch and tag deletion publish a new + encrypted generation without a pack. The helper advertises `main` as the + default branch while it exists, so it refuses to delete `refs/heads/main`. +- After deleting a remote ref, use `git fetch --prune --prune-tags` on a + returning clone to remove its stale local remote-tracking branch or tag. +- A push that loses a race with another writer fails. Fetch, integrate, and + push again. +- A new or deleted local carrier cache downloads the complete carrier branch. + Later operations reuse its objects and fetch only new carrier objects. + `GIT_REMOTE_E2EE_CACHE_DIR` overrides the cache parent directory. +- Recovery discovery has fixed limits of 2,048 carrier commits, 256 distinct + heads, 64 MiB of manifest data, and 1,000,000 estimated cryptographic + operations. It fails closed if any limit is exhausted. +- Shallow and partial clones are not supported. A fresh clone downloads and + verifies the complete history. + +## Storage contract + +Backends implement four logical operations. Immutable object writes are staged +so the final ciphertext ID can be learned while streaming: + +```text +begin_object(kind) -> writable stage; stage.finish(id) +open_object(kind, id) -> reader +read_head() +compare_and_swap_head(expected, next) +``` + +The filesystem backend implements head compare-and-swap with an advisory lock +and a rename in the storage root. Object publication hard-links a staged file +from `.staging` into `objects//` on the same filesystem and never +replaces an existing ID. File contents are flushed before a name is published, +new directories are flushed through the preexisting ancestor, and the parent +directory is flushed again afterwards; a flush error fails the call. Crash and +power-loss limits are in [durability.md](durability.md). The carrier-Git +backend implements compare-and-swap as a normal fast-forward push to the outer +branch. A future S3 backend can use multipart upload plus conditional writes, +but each provider must be capability-tested: "S3 compatible" does not by itself +promise correct compare-and-swap behavior. + +A killed filesystem writer can leave an unreachable file under `.staging/`. +Nothing published points to it, and stale `.stage-*` entries may be deleted +when no writer is running. Automatic cleanup belongs to future garbage +collection.