Conversation
Add LiveKit configuration, admission scheduling, token issuance, room workers, media bridging, and session lifecycle APIs. Route stream-serve through the LiveKit runtime while keeping the legacy transport available for the later removal commit. Verification: - .venv/bin/python -m pytest tests/unit/service/livekit --ignore=tests/unit/service/livekit/test_demo.py -q - 42 passed - git diff --cached --check
Use the standalone sageattention package as the only SageAttention backend source, avoiding the tf-kernel path that triggered CUDA misaligned-address failures. Verification: PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -m pytest tests/unit/ops/test_attention_backends.py tests/unit/service/livekit -q; ruff check and ruff format --check on the changed attention and LiveKit files; git diff --check.
Add the shared camera-control UI and LiveKit browser adapter with TURN relay configuration, reliable control messages, video playback, and streaming telemetry. Cover the rendered page and LingBot example defaults. Verification: - .venv/bin/python -m pytest tests/unit/service/livekit/test_demo.py tests/unit/pipelines/lingbot_world_fast/test_stream_example.py -q - 7 passed - git diff --cached --check
Keep actor-stage work asynchronous while reporting queue, pipeline, and applied-control latency separately. Preserve v2 camera intrinsics, rebalance translation controls, and make session stop and cache release behavior deterministic. Verification: - .venv/bin/python -m pytest tests/unit/pipelines/lingbot_world_fast/test_service_action_loop.py tests/unit/pipelines/lingbot_world_fast/test_service_metrics.py tests/unit/pipelines/lingbot_world_v2/test_service.py -q - 52 passed - git diff --cached --check
Make LiveKit the only stream transport, remove direct SDP and WebRTC API routes, delete the legacy aiortc session implementation and browser clients, and retain server-push support through the unified stream service contract. Verification: - .venv/bin/python -m pytest tests/unit/service -q - 182 passed - git diff --cached --check
Remove aiortc-specific LingBot stream benchmark contracts, configs, data, and helper scripts. Keep AIPerf documentation explicit that a validated LiveKit client adapter is required before streaming benchmarks are restored. Verification: - bash -n scripts/setup_aiperf_repo.sh - no references to removed stream benchmark assets - git diff --cached --check
Rewrite the English and Chinese stream guides around LiveKit sessions, data topics, deployment, and lifecycle. Add a root README quickstart that launches coturn, LiveKit, TeleFuser, and the browser controller with VS Code port forwarding, health checks, troubleshooting, and shutdown order. Verification: - .venv/bin/mkdocs build --strict --site-dir /tmp/telefuser-docs-livekit-rewrite - no aiortc, old SDP route, or legacy Demo references remain - git diff --cached --check
Add PyAV as a direct base dependency because the LiveKit media bridge imports it after aiortc is removed. Stop CI from requesting the deleted webrtc extra and install the standard development dependency set instead. Verification: - .venv/bin/python -m pip install --dry-run -e .[dev] - .venv/bin/python -m pytest tests/unit/service/livekit/test_app.py tests/unit/service/livekit/test_media_bridge.py tests/unit/service/livekit/test_runtime.py tests/unit/service/livekit/test_worker.py -q - 16 passed - GitHub Actions workflow YAML parsed successfully - git diff --cached --check
- add the TeleFuser-owned LiveKit adapter, pinned GitHub AIPerf installer, LingBot-World v2 one-minute workload, contracts, launchers, and focused adapter tests - preserve encode, DiT, and decode overlap with bounded directional prefetch and complete latent-chunk duration resolution - consolidate the user workflow and validated results in the benchmark README and localized docs while removing the retained AIPerf checkout flow Verification: - .venv/bin/python -m pytest tests/unit/pipelines/lingbot_world_fast/test_service_action_loop.py tests/unit/pipelines/lingbot_world_fast/test_streaming.py tests/unit/pipelines/lingbot_world_v2/test_service.py - env PYTHONPATH=benchmarks/telefuser_aiperf .venv-aiperf/bin/python -m pytest benchmarks/telefuser_aiperf/tests/test_livekit_adapter.py - ruff check benchmarks/telefuser_aiperf/telefuser_aiperf benchmarks/telefuser_aiperf/tests telefuser/pipelines/lingbot_world_fast/service.py tests/unit/pipelines/lingbot_world_fast/test_service_action_loop.py tests/unit/pipelines/lingbot_world_fast/test_streaming.py tests/unit/pipelines/lingbot_world_v2/test_service.py - ruff format --check benchmarks/telefuser_aiperf/telefuser_aiperf benchmarks/telefuser_aiperf/tests telefuser/pipelines/lingbot_world_fast/service.py tests/unit/pipelines/lingbot_world_fast/test_service_action_loop.py tests/unit/pipelines/lingbot_world_fast/test_streaming.py tests/unit/pipelines/lingbot_world_v2/test_service.py - bash -n scripts/setup_aiperf.sh benchmarks/telefuser_aiperf/scripts/run_stream_bench.sh - .venv/bin/python -m mkdocs build --strict - git diff --cached --check
Write fake-server output to a temporary log file so verbose startup logs cannot fill an unread subprocess pipe. Bypass environment proxies for the loopback readiness probe while retaining startup logs for failed fixtures. Verification: - Ruff 0.9.4 check, format check, and import check - Python 3.10, 3.11, and 3.12 unit CI jobs - Python 3.11 server CI job: 62 passed - Python 3.12 MkDocs build - pre-commit run --files tests/server/conftest.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR makes LiveKit the sole WebRTC transport for TeleFuser streaming.
It adds LiveKit-backed session APIs, admission scheduling, scoped tokens, room workers, native media bridging, reliable control/status messages, bounded telemetry, and browser reconnect handling for both
ServerPushServiceandBidirectionalService.It removes the legacy direct-aiortc implementation, SDP APIs, duplicate browser clients, and obsolete benchmark assets. It also adds a TeleFuser-owned LiveKit adapter that integrates with AIPerf without modifying or publishing code to the AIPerf repository.
Motivation
TeleFuser previously maintained two WebRTC paths with different session, control, media, and cache-lifecycle behavior. The direct-aiortc path lacked the room lifecycle, reconnection handling, admission control, and operational tooling needed by long-running interactive generation.
LiveKit provides a single transport with explicit ownership:
tf.controlandtf.statustopics.tf.metrics.Type of Change
Key Changes
telefuser stream-serveforServerPushServiceandBidirectionalService./v1/stream/sessionscreation, inspection, viewer-token, deletion, and health APIs.livekit_bidirectional_demo.pybrowser controller with reconnect handling, TURN relay support, controls, status, and latency telemetry.aiortc, direct SDP routes, the legacy WebRTC runtime, old browser clients, Streamlit streaming UI, and obsolete integration tests.telefuser_livekitAIPerf adapter, pinned Git installation helper, LingBot-World v2 workload, benchmark contract, tests, and reproduction documentation.sageattentionpackage instead oftf-kernel.Breaking Changes
telefuser stream-servenow requires LiveKit Cloud or a self-hosted LiveKit Server./v1/stream/webrtc/*and the direct SDP workflow have been removed./v1/stream/sessionssessions and join the returned LiveKit room.aiortcdependency and legacy WebRTC demos are removed.examples/stream_server/livekit_bidirectional_demo.py.Validation
git diff --check main..livekitManual end-to-end validation used four H100 80 GB GPUs, local coturn TCP relay, LiveKit Server, LingBot-World v2, forced TURN transport, and the browser control demo.
The validated AIPerf replay completed successfully with 60 chunks and 957 generated frames. The LiveKit client received 947 frames, with 6.253 s first-frame latency, 9.397 client FPS, and 3.399 / 3.901 s mean / p99 chunk pipeline residence.
These measurements distinguish client delivery from target compute and do not claim a model-compute speedup.
Checklist
pytest tests/executedpre-commit run --all-filespassesAdditional Notes
The local browser workflow uses development LiveKit credentials, loopback addresses, disabled TURN TLS,
--allow-loopback-peers, and forced TCP TURN relay for VS Code Remote SSH. These settings must not be used in production.AIPerf is installed from a pinned Git commit because the published 0.11.0 wheel does not contain the streaming runner. The LiveKit adapter remains TeleFuser-owned, is loaded from source through
PYTHONPATH, and requires no changes to the AIPerf repository.No CUDA or Triton kernels are added or modified. The end-to-end workflow was validated on SM90 H100 GPUs.