Dockerized NLP Evaluation Task
Budget: $10 – $30 USD
I am putting together a research-grade, self-contained coding exercise that will later be used to benchmark large-language-model agents. Your job is to design the entire task environment around a classical Natural Language Processing problem built with Sci-kit Learn, then wrap everything so it runs identically on any machine through Docker.
Here is what I need from you:
• A clean Dockerfile that installs every dependency and reproduces the same results bit-for-bit.
• A solve.sh script that serves as the oracle reference solution. When executed inside the container it must train, infer and print whatever artefacts or metrics you decide are necessary to prove correctness.
• A deterministic test suite (pytest preferred, but any deterministic runner is fine) that automatically verifies a candidate submission’s output against the expected results. Random seeds and dataset splits must be fixed so CI results never vary.
• The task itself should revolve around an NLP objective using a text corpus—think classic sentiment analysis, topic classification or similar, but feel free to suggest something novel as long as it fits within Sci-kit Learn’s ecosystem (no deep-learning frameworks here).
• Clear README instructions explaining how to build, run, and evaluate the task from scratch.
Absolutely no WordPress or generic web-dev experience is relevant here; I am looking for someone comfortable with Python, Docker, and reproducible research workflows.
Acceptance criteria
1. `docker build . && docker run …` completes without manual tweaks on a fresh machine.
2. `./solve.sh` exits with code 0 and produces the benchmarked artefacts.
3. `pytest` (or equivalent) passes with 100 % success for the oracle solution and fails gracefully when output diverges.
4. The entire repo (code, data preparation script, README) is under 200 MB once built.
To be considered, include a public GitHub link showing previous work where you combined Docker and Python. Proposals without this proof will be ignored.
Here is what I need from you:
• A clean Dockerfile that installs every dependency and reproduces the same results bit-for-bit.
• A solve.sh script that serves as the oracle reference solution. When executed inside the container it must train, infer and print whatever artefacts or metrics you decide are necessary to prove correctness.
• A deterministic test suite (pytest preferred, but any deterministic runner is fine) that automatically verifies a candidate submission’s output against the expected results. Random seeds and dataset splits must be fixed so CI results never vary.
• The task itself should revolve around an NLP objective using a text corpus—think classic sentiment analysis, topic classification or similar, but feel free to suggest something novel as long as it fits within Sci-kit Learn’s ecosystem (no deep-learning frameworks here).
• Clear README instructions explaining how to build, run, and evaluate the task from scratch.
Absolutely no WordPress or generic web-dev experience is relevant here; I am looking for someone comfortable with Python, Docker, and reproducible research workflows.
Acceptance criteria
1. `docker build . && docker run …` completes without manual tweaks on a fresh machine.
2. `./solve.sh` exits with code 0 and produces the benchmarked artefacts.
3. `pytest` (or equivalent) passes with 100 % success for the oracle solution and fails gracefully when output diverges.
4. The entire repo (code, data preparation script, README) is under 200 MB once built.
To be considered, include a public GitHub link showing previous work where you combined Docker and Python. Proposals without this proof will be ignored.