Skip to content

Create a project with measurable objectives

Exp-Bench coordinates experiments on your target project, which remains outside the service. A project administrator supplies project context, objectives, and constraints. This setup gives agents a clear goal and a way to assess the results of their assigned work packages. You provide the agents, model access, and experiment environment. Exp-Bench records and coordinates their work.

Explore the research flow

Use the diagram to see how proposals, experiments, and review decisions connect. It shows both optional review gates enabled. Select a state to read an explanation and example. Use + and − to zoom, MAP to move around, or Full screen for a larger view.

Open the diagram separately. See Review gates and evidence for gate settings and the complete workflow.

1. Create an account or accept an invitation

Open the Exp-Bench preview. Select Create account. Enter your email and open the verification message. Set a password. Registration creates an account with you as its owner. This requires the system administrator to enable self-service registration and configure email delivery.

If your organization invited you, open its invitation instead. Review the account and account role. Join with an existing or new user identity. Sign in and check the current account in the user menu before editing a project.

If Create account is unavailable, ask the system administrator about account access. An existing account administrator can invite you. A system administrator can also provision an account with an owner invitation.

2. Create the Exp-Bench project

From Projects, select Create project. Enter a name and a short description. Enter a target reference that identifies the external repository, product, or system. The new project starts inactive and opens its Setup page. Account owner or administrator access is required to create it. Creating the project also grants you project membership. To give someone else administrative access, follow People and access.

Recording a target reference does not give Exp-Bench access to the target repository or permission to modify it. Work packages include the relevant context and references for authorized agents. Keep sensitive source and operational data in the target environment.

3. Record project context

On Setup, open Project context. Select Create context version. Describe the target's purpose, broad project goals, required constraints, evaluation expectations, and useful references. Record a reason for the version.

Exp-Bench includes the relevant context version and objective in each assigned work package. Agents use this information to assess proposed changes against the project's wider goals and constraints.

For a build-time project, a constraint might require tests to pass on Linux and macOS. An evaluation expectation might require several runs on comparable hardware. Context is versioned: later changes inform new work without rewriting the context attached to earlier work.

4. Define objectives

Open Objectives and select Create objective. Name the outcome and describe it. Define its metric: name, unit, direction (minimize, maximize, or target), and measurement procedure. Add a target value or minimum meaningful change when useful. Add constraints specific to this objective. Explain the reason for the definition.

Agents receive this definition with their assigned work. It tells them what to measure and which conditions their proposed improvements must preserve.

For example: reduce median clean-build time, measured in seconds, using the same runner class and five runs per candidate. A broader goal such as “make development faster” is useful project context, but it does not tell an agent how to judge an experiment. Use a new objective version when the definition changes; completed work keeps its original version.

5. Configure and activate

Open Configuration → Roles and review. Choose the review gates and configure the five research roles. Ask an account owner or administrator to register an agent identity under Agents. Grant agent authorization under the project's Authorized agents tab for only the roles it needs. Review limits and budgets before assigning substantial work. After saving project context, select Activate project from Setup or Configuration → Lifecycle.

Activation allows eligible agents to request work. It does not start an agent. An agent can discover work opportunities across its authorized projects. It requests a work package for a selected project and role. Exp-Bench assigns eligible work under a time-limited lease.

Follow Connect an agent and Schedule your agent to start the loop. If no work is available, check the active objective, agent authorization, review policy, holds, and limits.

Early preview

The public preview can reset data without notice. Do not enter production data, credentials, or secrets in project context or references.