Roblox LLM Scenario Developer Needed
Budget: $750 – $1,500 USD
Project Type: Contract / Task-based
Category: Game Development, LuaU Programming, QA & Test Engineering
About the Project
We are looking for skilled taskers to help build reinforcement learning evaluation scenarios for Roblox's LLM Assistant. The LLM Assistant is an AI tool built into RobloxStudio that helps developers write and modify game code. Our goal is to evaluate and improve this assistant's performance using a framework called OpenGameEval, which runs the LLM against sample scenarios and assesses how well it completes prompted tasks.
Your role is to take skeleton evaluation scripts — each containing a base game state and a prompt describing a development task — and complete them into fully functional evaluation scenarios.
What You'll Be Doing
For each assigned scenario, you will:
Analyze the prompt and game environment — Open the provided .rbxl file in RobloxStudio, understand the game logic, and determine what a correct solution to the prompt looks like.
Write a reference ("golden") solution — Implement an ideal LuaU solution to the prompt inside the eval's reference function.
Design a rubric with evaluation criteria — Plan correctness, completeness, and precondition criteria that map directly to assert statements in your test harness.
Build unit tests — Write assert-based tests across three evaluation stages: static checks (check_scene), server-side runtime checks (serverCheck), and client-side runtime checks (clientChecks).
Provide instance guidance — Identify which script and non-script instances in the DataModel are relevant to evaluating the prompt.
Document your approach — Write a markdown explanation covering your reference code, unit tests, blockers, and methodology.
Required Skills & Experience
Strong proficiency in LuaU / Lua and experience scripting within the Roblox ecosystem
Solid understanding of Roblox's DataModel, including services such as Workspace, Players, ReplicatedStorage, ServerScriptService, StarterPlayer, StarterPack, and StarterGui
Familiarity with Roblox's client-server architecture — understanding the distinction between Scripts, LocalScripts, and ModuleScripts, and how replication works
Experience with RobloxStudio — navigating the Explorer, reading instance properties/attributes, and using testing modes (Test, Server & Clients)
Unit testing mindset — ability to write tests that validate functionality without overfitting to a specific implementation. Tests must accept all reasonable solutions, not just your own.
Familiarity with the Roblox API — especially Instance, Player, Humanoid, Tool, RemoteEvent, ProximityPrompt, and ValueBase classes
Nice to Have
Experience with Roblox game templates or community-published games
Familiarity with reinforcement learning evaluation concepts
Prior experience with test harness design or QA automation
Knowledge of VirtualInputManager for simulating player inputs
Tools You'll Use
RobloxStudio with the Assistant Eval Studio Plugin
OpenGameEval framework (for remote validation)
Standard text editor for .lua script development
Deliverables Per Task
Completed .lua eval script with all fields filled in (reference, instance guidance, check_scene, serverCheck, clientChecks)
Rubric file (.json) following the provided schema specification
Documentation file (.md) explaining your reference and testing approach
Key Principles to Keep in Mind
Tests must be solution-agnostic. If an average Roblox developer would consider a solution acceptable, your tests should pass it. Brittle, overfitting tests are not acceptable.
Design tests before writing the reference to avoid biasing your harness toward your own implementation.
All validation must use Lua's assert function — other testing frameworks or error-throwing approaches won't work with the OpenGameEval pipeline.
The reference function must be self-contained — no dependencies on the testing suite or EvalUtils modules.
Avoid including any PII in submissions.
How to Apply
Please include the following in your application:
A brief summary of your Roblox development experience, including any published games or plugins
Examples of LuaU code you've written (game scripts, tools, systems)
Any relevant experience with test design, QA, or evaluation frameworks
Your availability and estimated capacity (tasks per week)
Write "Lua" at the beginning of your proposal.
We look forward to hearing from experienced Roblox developers who enjoy solving puzzles, writing clean code, and thinking critically about what makes a good evaluation!
Category: Game Development, LuaU Programming, QA & Test Engineering
About the Project
We are looking for skilled taskers to help build reinforcement learning evaluation scenarios for Roblox's LLM Assistant. The LLM Assistant is an AI tool built into RobloxStudio that helps developers write and modify game code. Our goal is to evaluate and improve this assistant's performance using a framework called OpenGameEval, which runs the LLM against sample scenarios and assesses how well it completes prompted tasks.
Your role is to take skeleton evaluation scripts — each containing a base game state and a prompt describing a development task — and complete them into fully functional evaluation scenarios.
What You'll Be Doing
For each assigned scenario, you will:
Analyze the prompt and game environment — Open the provided .rbxl file in RobloxStudio, understand the game logic, and determine what a correct solution to the prompt looks like.
Write a reference ("golden") solution — Implement an ideal LuaU solution to the prompt inside the eval's reference function.
Design a rubric with evaluation criteria — Plan correctness, completeness, and precondition criteria that map directly to assert statements in your test harness.
Build unit tests — Write assert-based tests across three evaluation stages: static checks (check_scene), server-side runtime checks (serverCheck), and client-side runtime checks (clientChecks).
Provide instance guidance — Identify which script and non-script instances in the DataModel are relevant to evaluating the prompt.
Document your approach — Write a markdown explanation covering your reference code, unit tests, blockers, and methodology.
Required Skills & Experience
Strong proficiency in LuaU / Lua and experience scripting within the Roblox ecosystem
Solid understanding of Roblox's DataModel, including services such as Workspace, Players, ReplicatedStorage, ServerScriptService, StarterPlayer, StarterPack, and StarterGui
Familiarity with Roblox's client-server architecture — understanding the distinction between Scripts, LocalScripts, and ModuleScripts, and how replication works
Experience with RobloxStudio — navigating the Explorer, reading instance properties/attributes, and using testing modes (Test, Server & Clients)
Unit testing mindset — ability to write tests that validate functionality without overfitting to a specific implementation. Tests must accept all reasonable solutions, not just your own.
Familiarity with the Roblox API — especially Instance, Player, Humanoid, Tool, RemoteEvent, ProximityPrompt, and ValueBase classes
Nice to Have
Experience with Roblox game templates or community-published games
Familiarity with reinforcement learning evaluation concepts
Prior experience with test harness design or QA automation
Knowledge of VirtualInputManager for simulating player inputs
Tools You'll Use
RobloxStudio with the Assistant Eval Studio Plugin
OpenGameEval framework (for remote validation)
Standard text editor for .lua script development
Deliverables Per Task
Completed .lua eval script with all fields filled in (reference, instance guidance, check_scene, serverCheck, clientChecks)
Rubric file (.json) following the provided schema specification
Documentation file (.md) explaining your reference and testing approach
Key Principles to Keep in Mind
Tests must be solution-agnostic. If an average Roblox developer would consider a solution acceptable, your tests should pass it. Brittle, overfitting tests are not acceptable.
Design tests before writing the reference to avoid biasing your harness toward your own implementation.
All validation must use Lua's assert function — other testing frameworks or error-throwing approaches won't work with the OpenGameEval pipeline.
The reference function must be self-contained — no dependencies on the testing suite or EvalUtils modules.
Avoid including any PII in submissions.
How to Apply
Please include the following in your application:
A brief summary of your Roblox development experience, including any published games or plugins
Examples of LuaU code you've written (game scripts, tools, systems)
Any relevant experience with test design, QA, or evaluation frameworks
Your availability and estimated capacity (tasks per week)
Write "Lua" at the beginning of your proposal.
We look forward to hearing from experienced Roblox developers who enjoy solving puzzles, writing clean code, and thinking critically about what makes a good evaluation!