Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 29 additions & 21 deletions src/praisonai-mobile/docs/gaps.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,19 +8,21 @@ fact rather than something a reader has to infer from a `false` in a struct.

### The one root cause

Upstream `Agent.streamEvents()` emits a three-variant union:
Upstream `Agent.streamEvents()` emits a five-variant union:

```ts
type AgentEvent =
| { type: 'text'; delta: string }
| { type: 'finish'; text: string }
| { type: 'error'; error: Error }
| { type: 'text'; delta: string }
| { type: 'tool_call'; callId: string; name: string; args: Record<string, unknown> }
| { type: 'tool_result'; callId: string; name: string; ok: boolean; output: string }
| { type: 'finish'; text: string }
| { type: 'error'; error: Error }
```

Protocol v2 has **eleven** events. Nothing in the upstream channel carries a
tool call, a tool result, an approval request, reasoning, or usage — so this
engine cannot produce them, and the five conformance scenarios below are
declared unsupported rather than faked.
Protocol v2 has **eleven** events. The channel now carries tool calls and
results, so this engine reports tool activity. What it still cannot carry is an
approval request, reasoning, or usage — so the two approval conformance
scenarios below are declared unsupported rather than faked.

### What that means per capability

Expand All @@ -29,11 +31,17 @@ declared unsupported rather than faked.
| `streaming` | `true` | `text` deltas map straight to `delta`. |
| `cancellation` | `true` | Via `AgentStreamOptions.signal` (added upstream in #4426). |
| `reasoning` | `false` | No reasoning channel upstream. |
| `tools` | **`false`** | See the note below — this one is easy to misread. |
| `tools` | **`true`** | Since upstream gained `tool_call`/`tool_result`. Was false — see the note below. |
| `approvals` | `false` | `ApprovalManager` exists upstream but cannot reach the event channel. |
| `attachments` | `false` | `streamEvents` takes a prompt string only. |

**`tools: false` does not mean tools do not run.** praisonai-ts executes tools
**RESOLVED.** Upstream now emits `tool_call` and `tool_result`, so the flag is
`true` and three conformance scenarios that were declared unsupported
(`tool_ok`, `tool_failed`, `tool_unresolved`) are produced and passing. The
original note is kept below because the reasoning is what made the flag
trustworthy in the first place.

**`tools: false` did not mean tools do not run.** praisonai-ts executes tools
Comment thread
coderabbitai[bot] marked this conversation as resolved.
normally; it just never *announces* them. The flag describes what the engine can
**report**, because a UI that renders tool rows from a `true` flag would render
nothing and look broken. A tool call that silently failed would then be
Expand All @@ -44,21 +52,21 @@ indistinguishable from a normal answer — which is the exact defect protocol v2

| Scenario | Reason |
|---|---|
| `tool_ok` | no `tool_call`/`tool_result` variant upstream |
| `tool_failed` | no `tool_result`, so `ok: false` cannot be reported |
| `tool_unresolved` | no `tool_call`, so there is no row to leave unresolved |
| `approval` | `ApprovalManager` cannot reach the event channel |
| `approval` | `ApprovalManager` gates tool execution upstream but cannot reach the event channel |
| `two_approvals` | same as `approval` |

The suite prints every one of these on each run, so a contract that quietly
shrinks is visible in the output rather than silently green.
`tool_ok`, `tool_failed` and `tool_unresolved` were listed here until upstream
gained `tool_call`/`tool_result`; they are now produced and passing. The suite
prints every remaining omission on each run, so a contract that quietly shrinks
is visible in the output rather than silently green.

### Closing the gap
### Closing the remaining gap

This needs an upstream change: `AgentEvent` gaining tool and approval variants,
mirroring Python's `StreamEventType` (17 members). Until then the mobile app
either runs without tool visibility, or uses the `remote-http` engine — which
speaks the full vocabulary because the desktop server already emits it.
Approvals need a further upstream change: `AgentEvent` gaining an approval
variant, mirroring Python's `StreamEventType` (17 members). Until then the
mobile app either runs without approval prompts, or uses the `remote-http`
engine — which speaks the full vocabulary because the desktop server already
emits it.

## Node-only globals on the Agent import graph

Expand Down
32 changes: 27 additions & 5 deletions src/praisonai-mobile/engines/src/praisonai-ts/agent-api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,21 +9,43 @@
* without a provider, a network, or an API key.
* 2. It documents the coupling. This file IS the list of what would have to
* be re-implemented to swap frameworks. Right now that list is one class,
* three methods and a three-variant union -- which is the strongest
* three methods and a five-variant union -- which is the strongest
* evidence available that the seam is real.
* 3. It fails fast on drift. The adapter assigns the real `Agent` to this
* type at composition, so a signature change upstream is a typecheck error
* here rather than a runtime surprise on a device.
*
* The `AgentEvent` union really does have only three variants upstream --
* `text`, `finish`, `error`. Protocol v2 has eleven. That gap is not an
* oversight in this file; it is why `capabilities` below declares `reasoning`,
* `approvals` and `attachments` false, and it is recorded in gaps.md.
* The `AgentEvent` union has five variants upstream -- `text`, `tool_call`,
* `tool_result`, `finish`, `error`. Protocol v2 has eleven. The remaining gap
* is not an oversight in this file; it is why `capabilities` below declares
* `reasoning`, `approvals` and `attachments` false, and it is recorded in
* gaps.md.
*/

/** Upstream: `praisonai`'s `AgentEvent`, verbatim. */
export type PraisonAgentEvent =
| { readonly type: "text"; readonly delta: string }
/** Upstream gained these two, so tools are no longer invisible to a consumer
* of the event channel. Before them praisonai-ts executed tools perfectly
* well and never said so, and a UI had to infer tool activity from the
* model's own prose -- which is how a tool call that silently failed still
* looks like a normal answer. */
Comment on lines +28 to +32

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Stale event contract docs

The new tool variants contradict nearby documentation that still describes a three-variant upstream union with no tool events. The same obsolete account remains in engine.ts and gaps.md, including the list of tool scenarios as unsupported, leaving maintainers with conflicting capability guidance.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

| {
readonly type: "tool_call";
readonly callId: string;
readonly name: string;
readonly args: Record<string, unknown>;
}
| {
readonly type: "tool_result";
readonly callId: string;
readonly name: string;
/** THE signal of success. Never inferred from a non-empty `output`: a
* tool that failed with a message is byte-identical to one that
* succeeded with a message. */
readonly ok: boolean;
readonly output: string;
}
| { readonly type: "finish"; readonly text: string }
| { readonly type: "error"; readonly error: Error };

Expand Down
50 changes: 39 additions & 11 deletions src/praisonai-mobile/engines/src/praisonai-ts/conformance.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,15 @@
* Passing this IS the definition of implementing the agent-framework seam --
* the same suite the remote-http engine and the scripted fake already run.
*
* Five scenarios are declared unsupported, with reasons. That is not the suite
* Two scenarios are declared unsupported, with reasons. That is not the suite
* being lenient: `unsupported` prints every omission, so a shrinking contract
* is visible in the output rather than silently green. The five are all one
* fact -- upstream `streamEvents` emits three variants (text/finish/error)
* against protocol v2's eleven, so tool and approval events cannot be produced
* by this engine at all. It is recorded in gaps.md and reflected in
* `capabilities`, which the suite checks in the negative direction: an engine
* declaring approvals:false must never emit an approval_request.
* is visible in the output rather than silently green. Both are one fact --
* upstream `streamEvents` now emits tool_call/tool_result (so the three tool
* scenarios are produced and passing) but still has no approval channel, so
* `approval` and `two_approvals` cannot be produced by this engine. It is
* recorded in gaps.md and reflected in `capabilities`, which the suite checks
* in the negative direction: an engine declaring approvals:false must never
* emit an approval_request.
*/
import { describeEngineContract, type ScenarioName } from "../conformance.ts";
import { createPraisonTsEngine } from "./engine.ts";
Expand Down Expand Up @@ -46,6 +47,33 @@ const SCRIPTS: Partial<Record<ScenarioName, {
events: [{ type: "text", delta: "par" }],
stop: "cancelled",
},
tool_ok: {
events: [
{ type: "text", delta: "Checking. " },
{ type: "tool_call", callId: "c1", name: "search", args: { q: "x" } },
{ type: "tool_result", callId: "c1", name: "search", ok: true, output: "42" },
{ type: "finish", text: "Checking. The answer is 42." },
],
stop: "completed",
},
tool_failed: {
events: [
{ type: "tool_call", callId: "c1", name: "search", args: {} },
// A non-empty output on a FAILED call: the case that proves the suite
// reads `ok` rather than inferring success from content.
{ type: "tool_result", callId: "c1", name: "search", ok: false, output: "upstream is down" },
{ type: "finish", text: "That tool is unavailable." },
],
stop: "completed",
},
tool_unresolved: {
events: [
// A call with no result. The row must end unresolved, never successful.
{ type: "tool_call", callId: "c1", name: "search", args: {} },
{ type: "finish", text: "I could not finish that." },
],
stop: "completed",
},
};

function agentFor(scenario: ScenarioName): PraisonAgent {
Expand Down Expand Up @@ -79,10 +107,10 @@ describeEngineContract({
newMsgId: () => `m${++counter}`,
}),
unsupported: {
tool_ok: "upstream streamEvents has no tool_call/tool_result variant -- tools run but are never announced",
tool_failed: "same: no tool_result, so `ok: false` cannot be reported",
tool_unresolved: "same: no tool_call, so there is no row to leave unresolved",
approval: "ApprovalManager exists upstream but cannot reach the event channel",
// The three tool scenarios lived here until upstream gained tool_call and
// tool_result. They are produced above now -- which is the point of a map
// that records what an engine cannot do rather than what it never will.
approval: "ApprovalManager gates tool execution upstream, but its prompt cannot reach the event channel",
two_approvals: "same as approval",
},
});
9 changes: 6 additions & 3 deletions src/praisonai-mobile/engines/src/praisonai-ts/engine.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -266,9 +266,12 @@ test("dispose aborts everything still running and is idempotent", async () => {
});

test("capabilities describe what the engine can REPORT, not what it does inside", async () => {
// praisonai-ts executes tools but never announces them. A UI rendering tool
// rows from a true flag would render nothing and look broken.
assert.equal(PRAISONAI_TS_CAPABILITIES.tools, false);
// `tools` was false for a real reason and is true for a real reason. It was
// false because praisonai-ts EXECUTED tools and never announced them, so a
// UI rendering rows from a true flag would render nothing and look broken.
// Upstream now emits tool_call/tool_result, so the flag and the behaviour
// agree again -- which is the only thing the flag ever claimed.
assert.equal(PRAISONAI_TS_CAPABILITIES.tools, true);
assert.equal(PRAISONAI_TS_CAPABILITIES.approvals, false);
assert.equal(PRAISONAI_TS_CAPABILITIES.streaming, true);
assert.equal(PRAISONAI_TS_CAPABILITIES.cancellation, true);
Expand Down
46 changes: 36 additions & 10 deletions src/praisonai-mobile/engines/src/praisonai-ts/engine.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,17 @@
* -- the run controller, the transcript reducer, the views -- imports nothing
* from here and nothing from `praisonai`.
*
* THE CAPABILITY GAP IS DELIBERATE AND DECLARED.
* THE REMAINING CAPABILITY GAP IS DELIBERATE AND DECLARED.
*
* Upstream `streamEvents` emits three variants: `text`, `finish`, `error`.
* Protocol v2 has eleven. praisonai-ts *executes* tools -- it just does not
* announce them on the event channel -- so a tool call is invisible to any
* consumer of this engine. `capabilities.tools` is therefore FALSE: the flag
* describes what the engine can report, not what it can do internally, because
* a UI that renders tool rows from a flag would render nothing and look broken.
* See gaps.md, and the conformance harness's `unsupported` map, which prints
* every scenario this engine cannot be driven into.
* Upstream `streamEvents` now emits five variants: `text`, `tool_call`,
* `tool_result`, `finish`, `error`. Protocol v2 has eleven. Tool activity is
* therefore reportable and `capabilities.tools` is TRUE. What is still absent
* is the approval channel: praisonai-ts's `ApprovalManager` gates tool
* execution upstream but never surfaces its prompt on the event channel, so
* `capabilities.approvals` stays FALSE. The flag describes what the engine can
* report, not what it can do internally. See gaps.md, and the conformance
* harness's `unsupported` map, which prints every scenario this engine cannot
* be driven into.
*
* `end.userIndex` cannot be computed here and is not invented. It comes from
* the injected `RunPersistence`, which returns null when the write failed --
Expand Down Expand Up @@ -52,7 +53,11 @@ export interface RunPersistence {
export const PRAISONAI_TS_CAPABILITIES: EngineCapabilities = {
streaming: true,
reasoning: false, // upstream has no reasoning channel
tools: false, // executed, but never announced -- see the header
// True since upstream gained tool_call/tool_result. It was false for a real
// reason rather than caution: praisonai-ts executed tools and never
// announced them, so a UI rendering rows from a `true` flag would have
// rendered nothing and looked broken.
tools: true,
approvals: false, // ApprovalManager exists upstream but cannot reach the stream
cancellation: true, // AgentStreamOptions.signal, added for this port
attachments: false,
Expand Down Expand Up @@ -108,6 +113,27 @@ export function createPraisonTsEngine(options: PraisonEngineOptions): AgentEngin
if (event.delta === "") continue;
answer += event.delta;
yield { type: "delta", msgId, text: event.delta };
} else if (event.type === "tool_call") {
// Announced before the tool runs, so a view can show a call in
// progress rather than materialising a finished row out of nowhere.
yield { type: "tool_call", msgId, callId: event.callId, name: event.name, args: event.args };
} else if (event.type === "tool_result") {
// `ok` is passed straight through and never re-derived. Inferring it
// from a non-empty output is the exact defect the protocol's own
// comment was written against.
//
// `seconds: null` is honest rather than lazy: upstream does not
// report a duration, and null means "unknown", which the view
// renders differently from zero.
yield {
type: "tool_result",
msgId,
callId: event.callId,
name: event.name,
ok: event.ok,
output: event.output,
seconds: null,
};
} else if (event.type === "finish") {
// `finish` carries the FULL text. Trust it over the accumulation:
// upstream may normalise, and a turn that never streamed still
Expand Down
Loading