mkv/TODO.md

mkv TODO

Done

• SQLite schema + pragmas + ReadPool + WriterHandle (db.rs)
• Config parsing with validation (config.rs)
• Error types with axum IntoResponse (error.rs)
• Consistent hash ring with virtual nodes (hasher.rs)
• Volume HTTP client with .key sidecar writes (volume.rs)
• Health checker background task (health.rs)
• HTTP handlers: GET (302 redirect), PUT (fan-out), DELETE, HEAD, list (server.rs)
• CLI with clap subcommands (main.rs)
• Wiring: state, routes, health checker startup

Remaining

Core features

• Rebuild tool (tools/rebuild.rs)

  • Scan all volumes for .key sidecar files (via nginx autoindex JSON)
  • Read each .key file to recover the original key name
  • Verify the blob exists alongside each .key
  • Reconstruct SQLite index, merging replicas (same key on multiple volumes)
  • Uses db::bulk_put for efficient batch inserts

• Rebalance tool (tools/rebalance.rs)

  • Load current index, compute desired placement via new ring config
  • Diff current vs desired volumes per key
  • Copy blobs to new volumes, delete from old ones
  • Update SQLite records
  • --dry-run flag to preview without acting
  • Progress output: keys moved, bytes transferred, errors

Refactoring

• Extract pure decision logic from handlers (see PHILOSOPHY.md)

  • put_key: separate "compute placement" from "execute IO"
  • get_key: separate "pick healthy volume" from "build redirect"
  • Make fan-out results processing a pure function

• Typed volume errors — replace String errors in volume.rs with a proper enum

Testing

• Unit tests for db.rs — CRUD with an in-memory SQLite
• Integration test harness (see TESTING.md)
  • Docker Compose with 3 nginx volume containers
  • Rust integration tests that spin up the full server
  • PUT/GET/DELETE/HEAD happy path
  • Failure test: kill a volume mid-test, verify reads still work from replica
  • Rebalance test: add 4th volume, rebalance, verify all keys accessible
• Property-based tests for hasher — verify ring invariants under add/remove

Polish

• Graceful shutdown — drain in-flight requests, flush writer
• Metrics — request count, latency histograms, volume error rates
• Request ID / tracing spans — per-request trace context
• Config reload — SIGHUP to reload config without restart
• Rate limiting on list endpoint — prefix scan can be expensive