tests: extend install smoke test with a runtime call

This a follow-up for #1019 to prevent similar regressions. Existing install smoke tests are updated to actually run some code with the dependencies listed in the profile. This is different from the unit tests which are executed with dev deps and all extras.

In addition this PR restructures this test harness and moves it under `tests/` folder. Although those files do not use pytest and are not executed by pytest their current location under `scripts/` makes less sense as other files in this folder are helpers for local runs. A proper readme is added to clarify how it should be used.

Author: ishymko

.github/workflows/install-smoke.yml

@@ -8,7 +8,7 @@ on:
       - 'src/**'
       - 'pyproject.toml'
       - 'uv.lock'
-      - 'scripts/test_install_smoke.py'
+      - 'tests/install_smoke/**'
       - 'scripts/test_install_smoke.sh'
       # Self-callout: re-run when this workflow changes so YAML edits are validated in PRs.
       - '.github/workflows/install-smoke.yml'
@@ -58,5 +58,5 @@ jobs:
       - name: List installed packages
         run: VIRTUAL_ENV=.venv-smoke uv pip list
 
-      - name: Run import smoke test
-        run: .venv-smoke/bin/python scripts/test_install_smoke.py ${{ matrix.profile.name }}
+      - name: Run smoke test (imports + runtime checks)
+        run: .venv-smoke/bin/python -m tests.install_smoke ${{ matrix.profile.name }}

scripts/test_install_smoke.sh

@@ -2,11 +2,11 @@
 # Local equivalent of .github/workflows/install-smoke.yml.
 #
 # For each install profile, builds the wheel and installs it into a
-# clean venv (no dev deps), then runs the import smoke test for that
-# profile. By default runs every known profile; pass a profile name
-# to run just one.
+# clean venv (no dev deps), then runs the smoke test for that profile
+# (imports + any per-profile runtime checks). By default runs every
+# known profile; pass a profile name to run just one.
 #
-# Available profiles (must match those in scripts/test_install_smoke.py):
+# Available profiles (must match those in tests/install_smoke/profiles.py):
 #   base         -- `pip install a2a-sdk`
 #   http-server  -- `pip install a2a-sdk[http-server]`
 #   grpc         -- `pip install a2a-sdk[grpc]`
@@ -87,8 +87,10 @@ for profile in "${PROFILES[@]}"; do
     echo "--- Installed packages ---"
     VIRTUAL_ENV="$venv_dir" uv pip list
 
-    echo "--- Running import smoke test ---"
-    if ! "$venv_dir/bin/python" scripts/test_install_smoke.py "$profile"; then
+    echo "--- Running smoke test (imports + runtime checks) ---"
+    # Invoked as `python -m` from REPO_ROOT so the harness package is
+    # importable from the smoke venv (which has no pytest / dev deps).
+    if ! "$venv_dir/bin/python" -m tests.install_smoke "$profile"; then
         FAILED_PROFILES+=("$profile")
     fi
 done

tests/install_smoke/README.md

@@ -0,0 +1,39 @@
+# Install-smoke harness
+
+This package is **not** a pytest test suite. It is invoked as a
+standalone module from a freshly-installed, dev-deps-free venv:
+
+```bash
+python -m tests.install_smoke <profile>
+```
+
+The smoke venv is created by either
+[`scripts/test_install_smoke.sh`](../../scripts/test_install_smoke.sh)
+(local) or
+[`.github/workflows/install-smoke.yml`](../../.github/workflows/install-smoke.yml)
+(CI). The harness has no pytest dependency and uses only the Python
+standard library plus the freshly-installed `a2a-sdk` wheel for the
+profile under test.
+
+For a given install profile (`base`, `http-server`, `grpc`,
+`telemetry`, `sql`) it runs two phases:
+
+1. **Imports** — every module listed for the profile in `__main__.py`
+   must import cleanly. Catches missing deps and accidental top-level
+   imports of optional extras.
+2. **Runtime checks** (`runtime/`) — small public-API exercises that
+   actually call into the SDK. These catch regressions where imports
+   succeed but a real call fails.
+
+## Adding a new runtime check
+
+1. Drop a module under `tests/install_smoke/runtime/` exposing two
+   names:
+   - `NAME: str` — short human-readable label.
+   - `check() -> None` — callable that raises on failure.
+2. Register it in `RUNTIME_CHECKS` in
+   [`__main__.py`](./__main__.py) under each profile whose extras it
+   needs.
+
+Use only the dependencies guaranteed by the target profile. Do not import 
+`pytest` or any dev-deps.

tests/install_smoke/__init__.py

@@ -1,43 +1,34 @@
-#!/usr/bin/env python3
-"""Smoke test for installations of a2a-sdk with various extras.
-
-This script verifies that the public API modules associated with a
-given installation profile can be imported without pulling in modules
-that belong to other (uninstalled) optional extras.
-
-It is designed to run WITHOUT pytest or any dev dependencies -- just
-a clean venv with `pip install a2a-sdk[<profile>]`.
-
-Usage:
-    python scripts/test_install_smoke.py [profile]
-
-    profile defaults to "base" and selects which set of modules to
-    smoke-test. Available profiles:
-      base        -- `pip install a2a-sdk`
-      http-server -- `pip install a2a-sdk[http-server]`
-      grpc        -- `pip install a2a-sdk[grpc]`
-      telemetry   -- `pip install a2a-sdk[telemetry]`
-      sql         -- `pip install a2a-sdk[sql]`
+"""Entry point: ``python -m tests.install_smoke <profile>``.
 
 Exit codes:
-    0 - All imports for the profile succeeded
-    1 - One or more imports failed
+    0 - All imports and runtime checks for the profile succeeded.
+    1 - One or more imports or runtime checks failed.
+
+See README.md for design notes and how to add new runtime checks.
 """
 
 from __future__ import annotations
 
 import importlib
 import sys
 
+from typing import TYPE_CHECKING
+
+from tests.install_smoke.runtime import base_send_message
+
+
+if TYPE_CHECKING:
+    from collections.abc import Callable
+
 
 # Core modules that MUST be importable with only base dependencies.
 # These are the public API surface that every user gets with
 # `pip install a2a-sdk` (no extras).
 #
 # Do NOT add modules here that require optional extras (grpc,
-# http-server, sql, signing, telemetry, vertex, etc.).
-# Those modules are expected to fail without their extras installed
-# and should use try/except ImportError guards internally.
+# http-server, sql, signing, telemetry, vertex, etc.). Those modules
+# are expected to fail without their extras installed and should use
+# try/except ImportError guards internally.
 CORE_MODULES = [
     'a2a',
     'a2a.client',
@@ -69,10 +60,6 @@
 
 # Modules that MUST be importable with only the base + `http-server`
 # extras installed (no `grpc`, `sql`, `signing`, `telemetry`, etc.).
-#
-# A user building a Starlette/FastAPI A2A server with
-# `pip install a2a-sdk[http-server]` should be able to import these
-# without the gRPC stack being present on the system.
 HTTP_SERVER_MODULES = [
     'a2a.server.routes',
     'a2a.server.routes.agent_card_routes',
@@ -116,37 +103,72 @@
 }
 
 
-def main() -> int:
-    profile = sys.argv[1] if len(sys.argv) > 1 else 'base'
+# Per-profile runtime exercises. Each callable raises on failure and
+# returns None on success. These run after the import smoke succeeds
+# and are meant to invoke real public-API code paths against the
+# dependency versions resolved at install time.
+#
+# To add a new check: drop a module under `tests.install_smoke.runtime`
+# exposing `NAME` and `check()`, then add a tuple here for the
+# profile(s) whose extras it needs. See README.md.
+RUNTIME_CHECKS: dict[str, list[tuple[str, Callable[[], None]]]] = {
+    'base': [
+        (base_send_message.NAME, base_send_message.check),
+    ],
+}
+
+
+def main(argv: list[str]) -> int:
+    profile = argv[1] if len(argv) > 1 else 'base'
     if profile not in PROFILES:
         print(f'Unknown profile {profile!r}. Available: {sorted(PROFILES)}')
         return 1
 
     modules = PROFILES[profile]
-    failures: list[str] = []
-    successes: list[str] = []
-
+    import_failures: list[str] = []
     for module_name in modules:
         try:
             importlib.import_module(module_name)
-            successes.append(module_name)
         except Exception as e:  # noqa: BLE001, PERF203
-            failures.append(f'{module_name}: {e}')
+            import_failures.append(f'{module_name}: {e}')
 
     print(f'Profile: {profile}')
     print(f'Tested {len(modules)} modules')
-    print(f'  Passed: {len(successes)}')
-    print(f'  Failed: {len(failures)}')
+    print(f'  Passed: {len(modules) - len(import_failures)}')
+    print(f'  Failed: {len(import_failures)}')
 
-    if failures:
+    if import_failures:
         print('\nFAILED imports:')
-        for failure in failures:
+        for failure in import_failures:
             print(f'  - {failure}')
         return 1
 
     print('\nAll modules imported successfully.')
+
+    runtime_checks = RUNTIME_CHECKS.get(profile, [])
+    if not runtime_checks:
+        return 0
+
+    print(f'\nRunning {len(runtime_checks)} runtime check(s):')
+    runtime_failures: list[str] = []
+    for name, check in runtime_checks:
+        try:
+            check()
+        except Exception as e:  # noqa: BLE001, PERF203
+            runtime_failures.append(f'{name}: {type(e).__name__}: {e}')
+            print(f'  - FAIL: {name}')
+        else:
+            print(f'  - OK:   {name}')
+
+    if runtime_failures:
+        print('\nFAILED runtime checks:')
+        for failure in runtime_failures:
+            print(f'  - {failure}')
+        return 1
+
+    print('\nAll runtime checks passed.')
     return 0
 
 
 if __name__ == '__main__':
-    sys.exit(main())
+    sys.exit(main(sys.argv))

scripts/test_install_smoke.py tests/install_smoke/__main__.pyscripts/test_install_smoke.py renamed to tests/install_smoke/__main__.py

@@ -0,0 +1,88 @@
+"""Drive DefaultRequestHandler.on_message_send end-to-end with no transport.
+
+Uses only base dependencies (no http-server / gRPC transport).
+"""
+
+from __future__ import annotations
+
+import asyncio
+
+from a2a.helpers.proto_helpers import new_task_from_user_message
+from a2a.server.agent_execution import AgentExecutor
+from a2a.server.context import ServerCallContext
+from a2a.server.request_handlers import DefaultRequestHandler
+from a2a.server.tasks import InMemoryTaskStore
+from a2a.server.tasks.task_updater import TaskUpdater
+from a2a.types.a2a_pb2 import (
+    AgentCapabilities,
+    AgentCard,
+    Message,
+    Part,
+    Role,
+    SendMessageConfiguration,
+    SendMessageRequest,
+    Task,
+    TaskState,
+)
+
+NAME = 'DefaultRequestHandler.on_message_send roundtrip'
+
+
+class _HelloAgentExecutor(AgentExecutor):
+    async def execute(self, context, event_queue) -> None:  # type: ignore[no-untyped-def]
+        task = context.current_task
+        if not task:
+            assert context.message is not None
+            task = new_task_from_user_message(context.message)
+            await event_queue.enqueue_event(task)
+        updater = TaskUpdater(event_queue, task.id, task.context_id)
+        await updater.update_status(
+            TaskState.TASK_STATE_WORKING,
+            message=updater.new_agent_message([Part(text='I am working')]),
+        )
+        await updater.add_artifact(
+            [Part(text='Hello world!')], name='conversion_result'
+        )
+        await updater.complete()
+
+    async def cancel(self, context, event_queue) -> None:  # type: ignore[no-untyped-def]
+        pass
+
+
+async def _run() -> None:
+    handler = DefaultRequestHandler(
+        agent_executor=_HelloAgentExecutor(),
+        task_store=InMemoryTaskStore(),
+        agent_card=AgentCard(
+            name='smoke',
+            version='1.0',
+            capabilities=AgentCapabilities(
+                streaming=True, push_notifications=False
+            ),
+        ),
+    )
+    params = SendMessageRequest(
+        message=Message(
+            role=Role.ROLE_USER,
+            message_id='m1',
+            parts=[Part(text='hi')],
+        ),
+        configuration=SendMessageConfiguration(
+            accepted_output_modes=['text/plain']
+        ),
+    )
+    result = await handler.on_message_send(params, ServerCallContext())
+    if not isinstance(result, Task):
+        raise AssertionError(  # noqa: TRY004
+            f'expected Task result, got {type(result).__name__}'
+        )
+    if result.status.state != TaskState.TASK_STATE_COMPLETED:
+        raise AssertionError(
+            f'expected TASK_STATE_COMPLETED, got '
+            f'{TaskState.Name(result.status.state)}'
+        )
+
+
+def check() -> None:
+    """Run the roundtrip; raises on failure."""
+    asyncio.run(_run())