Skip to content

Custom Engine: extract agent-specific metadata out of generic ExecOptions #49

@zpzjzj

Description

@zpzjzj

Background

When the Custom Engine landed (#24), three new fields were added directly to the generic internal/runtime/runtime.go::ExecOptions:

  • CaseID
  • Variant
  • MaxTurns

with a comment noting "Built-in agents ignore these fields." This is implicit coupling — ExecOptions is the universal runtime parameter struct, but it now carries metadata that only CustomAgent consumes. The evaluator's runAgent.Run(...) call site populates these fields unconditionally (internal/evaluator/evaluator.go:394), and built-in agents silently discard them.

Proposal

Bundle the case-level agent metadata into its own struct so the contract is explicit:

// in internal/runtime
type AgentMetadata struct {
    CaseID   string
    Variant  string
    MaxTurns int
}

type ExecOptions struct {
    // ... existing fields ...
    AgentMetadata *AgentMetadata // nil for runs that don't need it
}

Alternatively, push the metadata through agent.Config at agent construction (once per case) rather than on every Exec call.

Why this is a follow-up, not a blocker

The current shape works and is observable; no built-in agent misuses the fields. But the next agent type that needs different case-level metadata will compound the smell. Address it before HTTP transport or a second user-defined-agent kind lands.

Origin

Raised during the self code-review of feat/custom-engine-local (architecture dimension, finding #1).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions