Advanced Software Engineer for AI Evaluation
Budget: $30 – $250 USD
Hi, I’m looking for a highly experienced software engineer to collaborate with me on the project.
The work involves creating and repairing advanced software-engineering tasks used to evaluate and train AI coding agents. Each task is based on a real GitHub repository, pull request, or commit and must include accurate instructions, strong regression tests, a correct oracle solution, reproducible Docker configuration, and complete metadata.
I need someone who is strong in software architecture, Git, Docker, CI/CD, automated testing, and debugging across different programming languages. Experience with AI evaluation, coding agents, benchmark development, or adversarial test design would be especially valuable.
The engineer must be able to identify subtle failures—such as mismatches between instructions and tests, incomplete test coverage, broken oracle solutions, non-reproducible environments, and packaging problems—and deliver tasks that pass strict automated and human review.
This is detailed, hands-on engineering work for someone who values accuracy, reproducibility, and high-quality technical standards. I’m interested in building a reliable long-term working relationship with the right senior engineer. If this matches your background, I’d be glad to discuss the workflow, expectations, availability, and compensation.
The work involves creating and repairing advanced software-engineering tasks used to evaluate and train AI coding agents. Each task is based on a real GitHub repository, pull request, or commit and must include accurate instructions, strong regression tests, a correct oracle solution, reproducible Docker configuration, and complete metadata.
I need someone who is strong in software architecture, Git, Docker, CI/CD, automated testing, and debugging across different programming languages. Experience with AI evaluation, coding agents, benchmark development, or adversarial test design would be especially valuable.
The engineer must be able to identify subtle failures—such as mismatches between instructions and tests, incomplete test coverage, broken oracle solutions, non-reproducible environments, and packaging problems—and deliver tasks that pass strict automated and human review.
This is detailed, hands-on engineering work for someone who values accuracy, reproducibility, and high-quality technical standards. I’m interested in building a reliable long-term working relationship with the right senior engineer. If this matches your background, I’d be glad to discuss the workflow, expectations, availability, and compensation.