Skip to content

docs(agentic): define trajectory observability contract - #472

Open
YAO-001 wants to merge 1 commit into
inclusionAI:mainfrom
YAO-001:codex/issue-49-trajectory-observability
Open

docs(agentic): define trajectory observability contract#472
YAO-001 wants to merge 1 commit into
inclusionAI:mainfrom
YAO-001:codex/issue-49-trajectory-observability

Conversation

@YAO-001

@YAO-001 YAO-001 commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Defines the agentic trajectory observability contract in the existing observability guide.

The documentation now:

  • maps AgentTrajectory / AgentTrajectoryTurn to the merged per-prompt/sample RewardRecord;
  • defines inspection fields for messages, assistant responses, tool calls/results, response tokens/logprobs, rewards, loss masks, trace events, and filtered trajectories;
  • cites the existing agentic CPU tests that exercise the contract;
  • keeps a complete JSONL trajectory writer out of scope and distinguishes the existing bounded rollout sample diagnostic from a canonical artifact;
  • documents privacy and safety requirements for prompts and tool outputs in logs.

No runtime or artifact-writing behavior changes.

Related issue

Closes #49

Type of change

  • Documentation update

How was it tested?

  • python -m pytest tests/test_agentic_cpu.py -q (46 passed)
  • python -m sphinx -W --keep-going -b html docs dist/docs-issue49 (succeeded without warnings)

CPU and documentation validation only; no GPU validation is required for this documentation-only change.

Checklist

  • The PR title summarizes the contribution.
  • Linked the related issue in the description.
  • Relevant existing tests pass.
  • The documented behavior is covered by existing agentic tests cited in the guide.
  • Described the test commands run and hardware limitations.
  • No public API or CLI behavior changes.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7fffb7d2c6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

multiple model requests. Agent code returns an ``AgentTrajectory`` containing
one or more ``AgentTrajectoryTurn`` objects. AReno merges turns that carry the
same ``AgentItem`` and exposes the completed trajectory to the reward function
as one ``RewardRecord``. This is the v0.0.3 observability boundary; it defines

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use the actual release version for this contract

The reviewed tree declares version 0.0.6 in pyproject.toml, so labeling this as the v0.0.3 observability boundary—and repeating that label in the artifact section—misidentifies which release the documentation describes. This can lead users of older releases to expect the documented agentic types and fields; update both version references to the intended current release or explicitly identify this as a separately versioned contract.

Useful? React with 👍 / 👎.

Comment on lines +232 to +234
- Ordered normalized ``request``, ``assistant_text``,
``assistant_tool_call``, ``tool_result``, ``finish``, and ``error``
events. Events carry only fields relevant to their type.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not describe the current trace as event-ordered

For a multi-turn trajectory containing tool results, the current trace is not in conversation order: _append_sample_response concatenates each call's request/assistant/finish events, while _trace_with_tool_results inserts all tool-result events before the first finish. The resulting trace can place a result before its turn finishes and move later results into the first turn, so consumers relying on the documented ordering will misassociate events; either correct trace construction and assert order in the cited test, or document that messages is the ordering source.

Useful? React with 👍 / 👎.

@YAO-001

YAO-001 commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

Hi maintainers, the GitHub Actions checks for this fork PR are currently blocked with action_required, and no jobs have run. I’ve addressed the review feedback and verified locally (46 passed; Sphinx -W build succeeded). Could someone please approve and run the workflows when convenient? Thanks! @adohe

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Define the trajectory observability contract

1 participant