This commit is contained in:
Silas Brack 2026-03-07 11:05:05 +01:00
parent 2fad27efc6
commit 1461b41a36
2 changed files with 14 additions and 42 deletions

2
.gitignore vendored
View file

@ -1,2 +1,2 @@
/target
*.db
*.db*

54
TODO.md
View file

@ -5,55 +5,27 @@
- [x] SQLite schema + pragmas + ReadPool + WriterHandle (`db.rs`)
- [x] Config parsing with validation (`config.rs`)
- [x] Error types with axum IntoResponse (`error.rs`)
- [x] Consistent hash ring with virtual nodes (`hasher.rs`)
- [x] Volume HTTP client with .key sidecar writes (`volume.rs`)
- [x] Consistent hash ring with virtual nodes + stability tests (`hasher.rs`)
- [x] Volume HTTP client — key-as-path, no sidecar files (`volume.rs`)
- [x] Health checker background task (`health.rs`)
- [x] HTTP handlers: GET (302 redirect), PUT (fan-out), DELETE, HEAD, list (`server.rs`)
- [x] HTTP handlers with pure decision functions extracted (`server.rs`)
- [x] CLI with clap subcommands (`main.rs`)
- [x] Wiring: state, routes, health checker startup
- [x] Rebuild tool — recursive nginx autoindex walk (`rebuild.rs`)
- [x] Rebalance tool with --dry-run (`rebalance.rs`)
- [x] Integration tests — 8 tests against real nginx (Docker Compose)
- [x] Unit tests — 17 tests (hasher, server pure fns, rebalance planning)
- [x] Key-as-path simplification (removed content-addressed paths + .key sidecars)
## Remaining
### Core features
- [ ] **Rebuild tool** (`tools/rebuild.rs`)
- Scan all volumes for `.key` sidecar files (via nginx autoindex JSON)
- Read each `.key` file to recover the original key name
- Verify the blob exists alongside each `.key`
- Reconstruct SQLite index, merging replicas (same key on multiple volumes)
- Uses `db::bulk_put` for efficient batch inserts
- [ ] **Rebalance tool** (`tools/rebalance.rs`)
- Load current index, compute desired placement via new ring config
- Diff current vs desired volumes per key
- Copy blobs to new volumes, delete from old ones
- Update SQLite records
- `--dry-run` flag to preview without acting
- Progress output: keys moved, bytes transferred, errors
### Refactoring
- [ ] **Extract pure decision logic from handlers** (see PHILOSOPHY.md)
- `put_key`: separate "compute placement" from "execute IO"
- `get_key`: separate "pick healthy volume" from "build redirect"
- Make fan-out results processing a pure function
### Worth doing
- [ ] **Typed volume errors** — replace `String` errors in `volume.rs` with a proper enum
### Testing
- [ ] **Unit tests for `db.rs`** — CRUD with an in-memory SQLite
- [ ] **Integration test harness** (see TESTING.md)
- Docker Compose with 3 nginx volume containers
- Rust integration tests that spin up the full server
- PUT/GET/DELETE/HEAD happy path
- Failure test: kill a volume mid-test, verify reads still work from replica
- Rebalance test: add 4th volume, rebalance, verify all keys accessible
- [ ] **Property-based tests for hasher** — verify ring invariants under add/remove
### Polish
- [ ] **Unit tests for `db.rs`** — CRUD round-trip with in-memory SQLite
- [ ] **Graceful shutdown** — drain in-flight requests, flush writer
### Nice to have
- [ ] **Metrics** — request count, latency histograms, volume error rates
- [ ] **Request ID / tracing spans** — per-request trace context
- [ ] **Config reload** — SIGHUP to reload config without restart