Progress
This commit is contained in:
parent
2fad27efc6
commit
1461b41a36
2 changed files with 14 additions and 42 deletions
2
.gitignore
vendored
2
.gitignore
vendored
|
|
@ -1,2 +1,2 @@
|
|||
/target
|
||||
*.db
|
||||
*.db*
|
||||
|
|
|
|||
54
TODO.md
54
TODO.md
|
|
@ -5,55 +5,27 @@
|
|||
- [x] SQLite schema + pragmas + ReadPool + WriterHandle (`db.rs`)
|
||||
- [x] Config parsing with validation (`config.rs`)
|
||||
- [x] Error types with axum IntoResponse (`error.rs`)
|
||||
- [x] Consistent hash ring with virtual nodes (`hasher.rs`)
|
||||
- [x] Volume HTTP client with .key sidecar writes (`volume.rs`)
|
||||
- [x] Consistent hash ring with virtual nodes + stability tests (`hasher.rs`)
|
||||
- [x] Volume HTTP client — key-as-path, no sidecar files (`volume.rs`)
|
||||
- [x] Health checker background task (`health.rs`)
|
||||
- [x] HTTP handlers: GET (302 redirect), PUT (fan-out), DELETE, HEAD, list (`server.rs`)
|
||||
- [x] HTTP handlers with pure decision functions extracted (`server.rs`)
|
||||
- [x] CLI with clap subcommands (`main.rs`)
|
||||
- [x] Wiring: state, routes, health checker startup
|
||||
- [x] Rebuild tool — recursive nginx autoindex walk (`rebuild.rs`)
|
||||
- [x] Rebalance tool with --dry-run (`rebalance.rs`)
|
||||
- [x] Integration tests — 8 tests against real nginx (Docker Compose)
|
||||
- [x] Unit tests — 17 tests (hasher, server pure fns, rebalance planning)
|
||||
- [x] Key-as-path simplification (removed content-addressed paths + .key sidecars)
|
||||
|
||||
## Remaining
|
||||
|
||||
### Core features
|
||||
|
||||
- [ ] **Rebuild tool** (`tools/rebuild.rs`)
|
||||
- Scan all volumes for `.key` sidecar files (via nginx autoindex JSON)
|
||||
- Read each `.key` file to recover the original key name
|
||||
- Verify the blob exists alongside each `.key`
|
||||
- Reconstruct SQLite index, merging replicas (same key on multiple volumes)
|
||||
- Uses `db::bulk_put` for efficient batch inserts
|
||||
|
||||
- [ ] **Rebalance tool** (`tools/rebalance.rs`)
|
||||
- Load current index, compute desired placement via new ring config
|
||||
- Diff current vs desired volumes per key
|
||||
- Copy blobs to new volumes, delete from old ones
|
||||
- Update SQLite records
|
||||
- `--dry-run` flag to preview without acting
|
||||
- Progress output: keys moved, bytes transferred, errors
|
||||
|
||||
### Refactoring
|
||||
|
||||
- [ ] **Extract pure decision logic from handlers** (see PHILOSOPHY.md)
|
||||
- `put_key`: separate "compute placement" from "execute IO"
|
||||
- `get_key`: separate "pick healthy volume" from "build redirect"
|
||||
- Make fan-out results processing a pure function
|
||||
### Worth doing
|
||||
|
||||
- [ ] **Typed volume errors** — replace `String` errors in `volume.rs` with a proper enum
|
||||
|
||||
### Testing
|
||||
|
||||
- [ ] **Unit tests for `db.rs`** — CRUD with an in-memory SQLite
|
||||
- [ ] **Integration test harness** (see TESTING.md)
|
||||
- Docker Compose with 3 nginx volume containers
|
||||
- Rust integration tests that spin up the full server
|
||||
- PUT/GET/DELETE/HEAD happy path
|
||||
- Failure test: kill a volume mid-test, verify reads still work from replica
|
||||
- Rebalance test: add 4th volume, rebalance, verify all keys accessible
|
||||
- [ ] **Property-based tests for hasher** — verify ring invariants under add/remove
|
||||
|
||||
### Polish
|
||||
|
||||
- [ ] **Unit tests for `db.rs`** — CRUD round-trip with in-memory SQLite
|
||||
- [ ] **Graceful shutdown** — drain in-flight requests, flush writer
|
||||
|
||||
### Nice to have
|
||||
|
||||
- [ ] **Metrics** — request count, latency histograms, volume error rates
|
||||
- [ ] **Request ID / tracing spans** — per-request trace context
|
||||
- [ ] **Config reload** — SIGHUP to reload config without restart
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue