Robot Simulation Benchmark Development
Budget: $30 – $250 USD
A MuJoCo task in this repo is a robotics/simulation benchmark where the agent must produce a verifiable artifact, usually an MJCF model file like /tmp/output/model.xml or a controller like /tmp/output/policy.py. The artifact is graded by scorer/compute_score.py using deterministic MuJoCo checks: compile the model, inspect joints/bodies/sensors, run fixed rollouts, measure stability/control/objective success, and return a score from 0 to 1.
The task package lives under problems/<task_id>/ and includes the public prompt (instruction.md), task metadata/config (task.toml), public data/helpers (data/), hidden grader logic/data (scorer/), oracle solution (solution/solve.sh), and for MuJoCo, a required reviewer video generated by solution/render.sh. The oracle must score exactly 1.0, and the video must be committed under .bigben/ground_truth/rendering.mp4 at exactly 1280x720.
In plain terms: you are designing a small robotics challenge for an AI to solve, plus an automatic judge that proves whether the solution really works in simulation. Example: examples/mujoco-pendulum asks the agent to create a damped pendulum MJCF with specific physical properties; the scorer checks that it compiles, has the right hinge/body/sensors/mass/COM, and remains stable during rollout.
https://github.com/mikaelli4444-collab/mujoco-rlhf
The task package lives under problems/<task_id>/ and includes the public prompt (instruction.md), task metadata/config (task.toml), public data/helpers (data/), hidden grader logic/data (scorer/), oracle solution (solution/solve.sh), and for MuJoCo, a required reviewer video generated by solution/render.sh. The oracle must score exactly 1.0, and the video must be committed under .bigben/ground_truth/rendering.mp4 at exactly 1280x720.
In plain terms: you are designing a small robotics challenge for an AI to solve, plus an automatic judge that proves whether the solution really works in simulation. Example: examples/mujoco-pendulum asks the agent to create a damped pendulum MJCF with specific physical properties; the scorer checks that it compiles, has the right hinge/body/sensors/mass/COM, and remains stable during rollout.
https://github.com/mikaelli4444-collab/mujoco-rlhf