Delete unused files
This commit is contained in:
parent
ec408aff29
commit
984953502b
5 changed files with 0 additions and 249 deletions
169
TESTING.md
169
TESTING.md
|
|
@ -1,169 +0,0 @@
|
||||||
# Testing Strategy
|
|
||||||
|
|
||||||
## Layers
|
|
||||||
|
|
||||||
### 1. Unit tests (no IO)
|
|
||||||
|
|
||||||
Pure logic that can be tested with `cargo test`, no containers needed.
|
|
||||||
|
|
||||||
**`hasher.rs`** (already done)
|
|
||||||
- Deterministic placement for the same key
|
|
||||||
- All volumes appear when requesting full replication
|
|
||||||
- Even distribution across volumes
|
|
||||||
- Correct key path format
|
|
||||||
|
|
||||||
**`db.rs`** (TODO)
|
|
||||||
- Open an in-memory SQLite (`:memory:`)
|
|
||||||
- Test put/get/delete/list_keys/all_records round-trip
|
|
||||||
- Test upsert behavior (put same key twice)
|
|
||||||
- Test soft delete (deleted flag)
|
|
||||||
- Test bulk_put
|
|
||||||
|
|
||||||
**Pure decision functions** (TODO, after refactor)
|
|
||||||
- Given a record and a set of healthy volumes, which volume to redirect to?
|
|
||||||
- Given fan-out results (list of Ok/Err), which volumes succeeded? Should we rollback?
|
|
||||||
- Given current vs desired volume placement, what needs to move?
|
|
||||||
|
|
||||||
### 2. Volume client tests (mock HTTP)
|
|
||||||
|
|
||||||
Use a lightweight HTTP server in-process (e.g. `axum` itself or `wiremock`) to test `volume.rs` without real nginx.
|
|
||||||
|
|
||||||
- PUT blob + .key sidecar → verify both requests made
|
|
||||||
- GET blob → verify body returned
|
|
||||||
- DELETE blob → verify both blob and .key deleted
|
|
||||||
- DELETE non-existent → verify 404 is treated as success
|
|
||||||
- Health check → respond 200 → healthy; timeout → unhealthy
|
|
||||||
|
|
||||||
### 3. Integration tests (real nginx)
|
|
||||||
|
|
||||||
Full end-to-end with Docker containers. These are slower but catch real issues.
|
|
||||||
|
|
||||||
#### Setup
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
# docker-compose.test.yml
|
|
||||||
services:
|
|
||||||
vol1:
|
|
||||||
image: nginx:alpine
|
|
||||||
volumes:
|
|
||||||
- ./tests/nginx.conf:/etc/nginx/conf.d/default.conf
|
|
||||||
- vol1_data:/data
|
|
||||||
ports: ["3101:80"]
|
|
||||||
|
|
||||||
vol2:
|
|
||||||
image: nginx:alpine
|
|
||||||
volumes:
|
|
||||||
- ./tests/nginx.conf:/etc/nginx/conf.d/default.conf
|
|
||||||
- vol2_data:/data
|
|
||||||
ports: ["3102:80"]
|
|
||||||
|
|
||||||
vol3:
|
|
||||||
image: nginx:alpine
|
|
||||||
volumes:
|
|
||||||
- ./tests/nginx.conf:/etc/nginx/conf.d/default.conf
|
|
||||||
- vol3_data:/data
|
|
||||||
ports: ["3103:80"]
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
vol1_data:
|
|
||||||
vol2_data:
|
|
||||||
vol3_data:
|
|
||||||
```
|
|
||||||
|
|
||||||
```nginx
|
|
||||||
# tests/nginx.conf
|
|
||||||
server {
|
|
||||||
listen 80;
|
|
||||||
root /data;
|
|
||||||
|
|
||||||
location / {
|
|
||||||
dav_methods PUT DELETE;
|
|
||||||
create_full_put_path on;
|
|
||||||
autoindex on;
|
|
||||||
autoindex_format json;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
```toml
|
|
||||||
# tests/test_config.toml
|
|
||||||
[server]
|
|
||||||
port = 3100
|
|
||||||
replication_factor = 2
|
|
||||||
virtual_nodes = 100
|
|
||||||
|
|
||||||
[database]
|
|
||||||
path = "/tmp/mkv-test/index.db"
|
|
||||||
|
|
||||||
[[volumes]]
|
|
||||||
url = "http://localhost:3101"
|
|
||||||
|
|
||||||
[[volumes]]
|
|
||||||
url = "http://localhost:3102"
|
|
||||||
|
|
||||||
[[volumes]]
|
|
||||||
url = "http://localhost:3103"
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Test cases
|
|
||||||
|
|
||||||
**Happy path**
|
|
||||||
1. PUT `/hello` with body `"world"` → 201
|
|
||||||
2. HEAD `/hello` → 200, Content-Length: 5
|
|
||||||
3. GET `/hello` → 302 to a volume URL
|
|
||||||
4. Follow redirect → body is `"world"`
|
|
||||||
5. GET `/` → list contains `"hello"`
|
|
||||||
6. DELETE `/hello` → 204
|
|
||||||
7. GET `/hello` → 404
|
|
||||||
|
|
||||||
**Replication verification**
|
|
||||||
1. PUT `/replicated` → 201
|
|
||||||
2. Read SQLite directly, verify 2 volumes listed
|
|
||||||
3. GET blob from both volume URLs directly, verify identical content
|
|
||||||
|
|
||||||
**Volume failure**
|
|
||||||
1. PUT `/failtest` → 201
|
|
||||||
2. Stop vol1 container
|
|
||||||
3. GET `/failtest` → should still 302 to vol2 (healthy replica)
|
|
||||||
4. PUT `/new-during-failure` → should fail if replication_factor can't be met, or succeed on remaining volumes depending on ring placement
|
|
||||||
5. Restart vol1
|
|
||||||
|
|
||||||
**Rebuild**
|
|
||||||
1. PUT several keys
|
|
||||||
2. Delete the SQLite database
|
|
||||||
3. Run `mkv rebuild`
|
|
||||||
4. GET all keys → should all still work
|
|
||||||
|
|
||||||
**Rebalance**
|
|
||||||
1. PUT several keys with 3 volumes
|
|
||||||
2. Add a 4th volume to config
|
|
||||||
3. Run `mkv rebalance --dry-run` → verify output
|
|
||||||
4. Run `mkv rebalance` → verify keys migrated
|
|
||||||
5. GET all keys → should all work
|
|
||||||
|
|
||||||
#### Running integration tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Start volumes
|
|
||||||
docker compose -f docker-compose.test.yml up -d
|
|
||||||
|
|
||||||
# Wait for nginx to be ready
|
|
||||||
sleep 2
|
|
||||||
|
|
||||||
# Run integration tests
|
|
||||||
cargo test --test integration
|
|
||||||
|
|
||||||
# Tear down
|
|
||||||
docker compose -f docker-compose.test.yml down -v
|
|
||||||
```
|
|
||||||
|
|
||||||
The integration test binary (`tests/integration.rs`) starts the mkv server
|
|
||||||
in-process on a random port, runs all test cases sequentially (shared state),
|
|
||||||
then shuts down.
|
|
||||||
|
|
||||||
### 4. What we don't test
|
|
||||||
|
|
||||||
- Performance / benchmarks (follow-up)
|
|
||||||
- TLS, auth (not implemented)
|
|
||||||
- Concurrent writers racing on the same key (SQLite serializes this correctly by design)
|
|
||||||
- Blobs > available RAM (streaming is a follow-up)
|
|
||||||
32
TODO.md
32
TODO.md
|
|
@ -1,32 +0,0 @@
|
||||||
# mkv TODO
|
|
||||||
|
|
||||||
## Done
|
|
||||||
|
|
||||||
- [x] SQLite schema + pragmas + ReadPool + WriterHandle (`db.rs`)
|
|
||||||
- [x] Config parsing with validation (`config.rs`)
|
|
||||||
- [x] Error types with axum IntoResponse (`error.rs`)
|
|
||||||
- [x] Consistent hash ring with virtual nodes + stability tests (`hasher.rs`)
|
|
||||||
- [x] Volume HTTP client — key-as-path, no sidecar files (`volume.rs`)
|
|
||||||
- [x] Health checker background task (`health.rs`)
|
|
||||||
- [x] HTTP handlers with pure decision functions extracted (`server.rs`)
|
|
||||||
- [x] CLI with clap subcommands (`main.rs`)
|
|
||||||
- [x] Rebuild tool — recursive nginx autoindex walk (`rebuild.rs`)
|
|
||||||
- [x] Rebalance tool with --dry-run (`rebalance.rs`)
|
|
||||||
- [x] Integration tests — 8 tests against real nginx (Docker Compose)
|
|
||||||
- [x] Unit tests — 17 tests (hasher, server pure fns, rebalance planning)
|
|
||||||
- [x] Key-as-path simplification (removed content-addressed paths + .key sidecars)
|
|
||||||
|
|
||||||
## Remaining
|
|
||||||
|
|
||||||
### Worth doing
|
|
||||||
|
|
||||||
- [x] **Typed volume errors** — replace `String` errors with a proper enum
|
|
||||||
- [ ] **Unit tests for `db.rs`** — CRUD round-trip with in-memory SQLite
|
|
||||||
- [x] **Graceful shutdown** — drain in-flight requests on SIGINT/SIGTERM
|
|
||||||
|
|
||||||
### Nice to have
|
|
||||||
|
|
||||||
- [ ] **Metrics** — request count, latency histograms, volume error rates
|
|
||||||
- [ ] **Request ID / tracing spans** — per-request trace context
|
|
||||||
- [ ] **Config reload** — SIGHUP to reload config without restart
|
|
||||||
- [ ] **Rate limiting on list endpoint** — prefix scan can be expensive
|
|
||||||
|
|
@ -1,16 +0,0 @@
|
||||||
[server]
|
|
||||||
port = 3000
|
|
||||||
replication_factor = 2
|
|
||||||
virtual_nodes = 100
|
|
||||||
|
|
||||||
[database]
|
|
||||||
path = "/tmp/mkv-bench/index.db"
|
|
||||||
|
|
||||||
[[volumes]]
|
|
||||||
url = "http://localhost:3101"
|
|
||||||
|
|
||||||
[[volumes]]
|
|
||||||
url = "http://localhost:3102"
|
|
||||||
|
|
||||||
[[volumes]]
|
|
||||||
url = "http://localhost:3103"
|
|
||||||
16
config.toml
16
config.toml
|
|
@ -1,16 +0,0 @@
|
||||||
[server]
|
|
||||||
port = 3000
|
|
||||||
replication_factor = 2
|
|
||||||
virtual_nodes = 100
|
|
||||||
|
|
||||||
[database]
|
|
||||||
path = "/tmp/mkv/index.db"
|
|
||||||
|
|
||||||
[[volumes]]
|
|
||||||
url = "http://localhost:3001"
|
|
||||||
|
|
||||||
[[volumes]]
|
|
||||||
url = "http://localhost:3002"
|
|
||||||
|
|
||||||
[[volumes]]
|
|
||||||
url = "http://localhost:3003"
|
|
||||||
|
|
@ -1,16 +0,0 @@
|
||||||
[server]
|
|
||||||
port = 3100
|
|
||||||
replication_factor = 2
|
|
||||||
virtual_nodes = 100
|
|
||||||
|
|
||||||
[database]
|
|
||||||
path = "/tmp/mkv-test/index.db"
|
|
||||||
|
|
||||||
[[volumes]]
|
|
||||||
url = "http://localhost:3101"
|
|
||||||
|
|
||||||
[[volumes]]
|
|
||||||
url = "http://localhost:3102"
|
|
||||||
|
|
||||||
[[volumes]]
|
|
||||||
url = "http://localhost:3103"
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue