[sdk/python] Preserve events when optional promoted fields are unset

sdk/python/CHANGELOG.md

@@ -18,10 +18,30 @@ it ships.
 
 ## 0.0.1b2 — 2026-08-25
 
-Open for the next release. `0.0.1b1` published on 2026-08-24 and the `bump` job
-moved the version here automatically; nothing has landed against `0.0.1b2` yet.
-Add entries as changes merge — this section becomes the GitHub Release body when
-it ships.
+### A promoted column passed as `None` no longer costs the event
+
+- **`None` on a promoted column is now dropped and warned about, not refused.**
+  A promoted key left at `None` in `**fields` reached the wire as an explicit
+  JSON `null`, so `_validate_promoted_string` refused it outright. But `None` is
+  how a caller says *I have no value*, and the refusal landed inside their emit
+  helper — which swallows telemetry errors, because telemetry must not break a
+  run. The event vanished with nothing logged.
+
+  `agent_end(error_type=None)` is the shape **every successful run** produces:
+  `error_type` is populated only on a failing outcome. Found against a real
+  multi-agent app, where it silently dropped `agent_end` for every session that
+  succeeded — leaving each one with a dangling `agent_start`, no outcome, and no
+  evaluation, since the server triggers evaluation on `agent_end`.
+
+- **The same fix closes the mirror bug on promoted numerics.** `_build` omits
+  `None` only from a dataclass's named `specifics`; `extra` is merged verbatim.
+  So `duration_ms=None` was dropped when passed as a named parameter and written
+  as an explicit `null` when passed through `**fields` — the same value, two
+  outcomes, decided by which door it came through.
+
+  Both paths now agree: for a promoted column, no value means no key. Nothing
+  that worked before changes, and no explicit `null` reaches a promoted column
+  from either direction.
 
 - Retire the old inbound evaluator boundary and add evaluator authoring plus the
   outbound-only v2 worker runtime under the lazy `failproofai_sdk.evaluator`

sdk/python/failproofai_sdk/_events.py

@@ -99,6 +99,11 @@ def _validate_promoted_string(name: str, value) -> None:
     own call. `_build` copies the base dict verbatim, so a `None` here reached
     the wire as an explicit JSON `null`, the row was accepted at 200 OK, and the
     column was empty for some events and not others with nothing logged anywhere.
+
+    `**fields` no longer reaches this holding a `None`: `_validate_fields` drops
+    the key and warns, so an optional column the caller simply does not have
+    costs a log line rather than the whole event. The raise below stays as the
+    backstop for any direct caller.
     """
     if value is None:
         raise ValueError(
@@ -320,6 +325,21 @@ def _validate_fields(self, fields: dict) -> None:
         bad = _RESERVED & fields.keys()
         if bad:
             raise ValueError(f"Reserved field names cannot be used as custom fields: {sorted(bad)}")
+        # `_build` omits None only from a dataclass's named `specifics`; `extra`
+        # is merged verbatim, so a promoted key left at None reaches the wire as
+        # an explicit JSON null — accepted at 200 OK, stored as NULL, invisible
+        # to every filter on that column. For a promoted column "no value" has to
+        # mean "no key", so drop it here, the one place holding the caller's own
+        # dict. Warned rather than silent: passing None is still a mistake worth
+        # hearing about, it just must not cost the event.
+        for name in (_PROMOTED_NUMERIC | _PROMOTED_STRING) & fields.keys():
+            if fields[name] is None:
+                logger.warning(
+                    "%s was passed as None and has been omitted from the event; "
+                    "pass a value, or omit the argument entirely to silence this.",
+                    name,
+                )
+                del fields[name]
         for name in _PROMOTED_NUMERIC & fields.keys():
             _validate_promoted_numeric(name, fields[name])
         for name in _PROMOTED_STRING & fields.keys():

sdk/python/tests/test_server_contract.py

@@ -427,6 +427,66 @@ def test_the_promoted_numeric_set_matches_the_one_ingest_lifts():
     assert _events._PROMOTED_NUMERIC == PROMOTED_NUMERIC
 
 
+# ─────────────────────────────────────────────────────────────────────────────
+# A promoted column the caller does not have costs a warning, not the event
+# ─────────────────────────────────────────────────────────────────────────────
+
+
+@pytest.mark.parametrize("name", sorted(_events._PROMOTED_STRING))
+def test_a_promoted_string_passed_as_none_is_omitted_not_refused(name):
+    """None means "I have no value", and that must not cost the whole event.
+
+    `agent_end(error_type=None)` is the shape every successful run produces: the
+    field is populated only on a failing outcome, so the ordinary success path
+    passes None. Refusing it raised inside the caller's emit helper, and a helper
+    that swallows telemetry errors — which every one of them does, because
+    telemetry must not break a run — turned that into a silently missing event.
+    """
+    recorder = _Recorder()
+    EventNamespace(recorder).agent_start(session_id="s", agent_id="a", **{name: None})
+    assert name not in recorder.entries[0]
+
+
+@pytest.mark.parametrize("name", sorted(_events._PROMOTED_NUMERIC))
+def test_a_promoted_numeric_passed_as_none_in_fields_is_omitted_too(name):
+    """The named-parameter path already dropped None; `**fields` did not.
+
+    `_build` omits None only from a dataclass's own `specifics` — `extra` is
+    merged verbatim — so the same value went to the wire as an explicit JSON
+    null depending only on which door it came through.
+    """
+    recorder = _Recorder()
+    EventNamespace(recorder).agent_start(session_id="s", agent_id="a", **{name: None})
+    assert name not in recorder.entries[0]
+
+
+def test_a_dropped_promoted_column_is_warned_about(caplog):
+    """Silently dropping it would hide a real mistake; refusing it costs the event."""
+    recorder = _Recorder()
+    with caplog.at_level("WARNING", logger="failproofai_sdk._events"):
+        EventNamespace(recorder).agent_end(session_id="s", agent_id="a", error_type=None)
+    assert "error_type" in caplog.text
+
+
+def test_agent_end_on_a_successful_run_is_still_emitted():
+    """The regression this all exists for, in the shape the caller actually sends."""
+    recorder = _Recorder()
+    EventNamespace(recorder).agent_end(
+        session_id="s", agent_id="a", outcome="success", summary=None, error_type=None
+    )
+    assert [e["type"] for e in recorder.entries] == ["agent_end"]
+    assert recorder.entries[0]["outcome"] == "success"
+    assert "error_type" not in recorder.entries[0]
+
+
+@pytest.mark.parametrize("name", sorted(_events._PROMOTED_STRING))
+def test_a_promoted_string_with_a_real_value_still_goes_through(name):
+    """Dropping None must not also drop the values that matter."""
+    recorder = _Recorder()
+    EventNamespace(recorder).agent_start(session_id="s", agent_id="a", **{name: "real"})
+    assert recorder.entries[0][name] == "real"
+
+
 # ─────────────────────────────────────────────────────────────────────────────
 # The MEASURED duration is bound by the same u32 range a caller is held to
 # ─────────────────────────────────────────────────────────────────────────────