Comprehensive Multi-Scenario Load Testing
Budget: $250 – $750 USD
I am planning a small-scale performance study that compares six distinct backend architectures and protocols. The core job is to spin up each environment from scratch, generate meaningful traffic, and record latency, throughput, error-rate, uptime, and CPU/memory figures so I can make an apples-to-apples comparison afterward.
The six target scenarios
1. ANN Faiss + Postgres
2. Two-tower retrieval strategy backed by Redis
3. Streaming pipeline on ScyllaDB
4. Pure FastAPI REST service
5. FastAPI service exposed by gRPC
6. FastAPI function deployed to AWS Lambda
Right now nothing is provisioned, so each stack must be containerised or scripted for repeatable deployment. Docker-Compose or Terraform will work fine—whichever lets me rerun tests quickly.
What matters most to me is the first scenario (Faiss + Postgres); please be ready to iterate on that one if we find unexpected bottlenecks before moving on to the others.
Deliverables
• Infrastructure scripts or dockerfiles for every scenario
• Test plans and command-line artefacts (Locust, k6, JMeter, wrk, vegeta—your call) that show exactly how traffic was generated
• A single results spreadsheet or dashboard summarising the required metrics for each run
• Brief write-up highlighting key findings and configuration notes
Acceptance criteria
• Each run captures the five metrics listed above with a clearly documented sampling interval
• Tests are reproducible on a clean machine using your provided scripts
• Raw logs and dashboards are included for verification
If you enjoy building clean benchmarking harnesses and can work independently across multiple databases and API frameworks, I’d love to hear how you would tackle this.
The six target scenarios
1. ANN Faiss + Postgres
2. Two-tower retrieval strategy backed by Redis
3. Streaming pipeline on ScyllaDB
4. Pure FastAPI REST service
5. FastAPI service exposed by gRPC
6. FastAPI function deployed to AWS Lambda
Right now nothing is provisioned, so each stack must be containerised or scripted for repeatable deployment. Docker-Compose or Terraform will work fine—whichever lets me rerun tests quickly.
What matters most to me is the first scenario (Faiss + Postgres); please be ready to iterate on that one if we find unexpected bottlenecks before moving on to the others.
Deliverables
• Infrastructure scripts or dockerfiles for every scenario
• Test plans and command-line artefacts (Locust, k6, JMeter, wrk, vegeta—your call) that show exactly how traffic was generated
• A single results spreadsheet or dashboard summarising the required metrics for each run
• Brief write-up highlighting key findings and configuration notes
Acceptance criteria
• Each run captures the five metrics listed above with a clearly documented sampling interval
• Tests are reproducible on a clean machine using your provided scripts
• Raw logs and dashboards are included for verification
If you enjoy building clean benchmarking harnesses and can work independently across multiple databases and API frameworks, I’d love to hear how you would tackle this.