Conversation
… temporary creations Gallery and controls - Gallery cards are image-first square tiles with model and relative-time captions; status chips only appear for unfinished work; density is a Radio segmented control and phones keep two columns - Studio header uses the chat header-action icon recipe with tooltips; thread rename/delete move into an ellipsis options menu; result actions collapse to one row with icon-only download, expand and cover; the composer upload trigger is a paperclip like chat - Settings panel starts with a button that toggles between Generation history and New creation; document title follows the thread title Generated titles - media.titles config (enabled, endpoint, model, prompt, timeoutMs) falling back to endpoints.all title settings; packages/api/src/media/title.ts reuses chat provider resolution, sanitizeTitle and the PII title filter and records usage - Titles are applied with a conditional write that never overwrites a user rename; skipped for temporary creations Presets - Server-side Studio presets under /api/media/presets backed by a MediaPreset collection, media.limits.maxPresets, default preset seeding new drafts; dialog to save, apply, star and delete Model comparison - A second model picker sends the same prompt and references to two models sharing a comparisonId; the thread renders those turns side by side under one prompt Temporary creations - Header toggle flags a new thread as temporary; the server stamps expiresAt from temporaryChatRetention, hides it from the library, and the media worker retires it on expiry Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ventions - Eyebrow section labels (Label section variant) with quiet right-aligned actions replace stacked outline boxes - The history/new-creation toggle becomes a header-row action beside the settings heading - Presets are a labelled field that names the matching preset or reads Custom, with a Manage action for saving, defaults and deletion - Image/Edit/Video is one segmented Radio control; the compare row is a label with an Add model action and an inline picker - Advanced settings collapse under an eyebrow disclosure Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… Studio client Backend - Acquire deployment, integration and owner permits atomically so a job that fails its owner slot no longer parks in the queue holding deployment capacity for other users - Fold assistant replay content to text unless a part carries signed native media, so non-Google providers keep string AIMessages next to tool_calls; cover all call sites - Stop counting media debt as an account-deletion obligation; holds and pending settlements still block, and a debt-only balance can be deleted - Retry re-prepares the execution so a rotated key or refreshed catalog dispatches - Report queue-age expiry as queue_expired and leave provider cancellation without an error code; log router, worker and transport failures with a redacted cause - Reject unsupported or oversized uploads before they reach disk and sweep the staging directory from the worker tick; stream hosted references through a digest instead of buffering them - Assemble the media runtime once in createMediaRuntimeFromApp for both server entry points; resolve /api/config media permissions from the role; drop the app.locals reach in favour of the wiring layer; seed MEDIA permissions through the shared helper with ADMIN defaulting to use and create - Document the media configuration section in librechat.example.yaml Data schemas - Rewrite $facet lookups, the pipeline retainer update and raw createIndexes calls so the DocumentDB compatibility guard passes; add the retirement sweep index - Store native part expiry as a Date with a TTL index; pass settlement write concern per call; allowlist the deployment-wide permit table in the tenant-isolation guard Client - Stop receipt polling on a definitive 4xx and offer Dismiss for unresolved commands - Route thread, job, command and upload calls through useMutation with the existing keys - Gate Open in Studio once per chat view and mount the chat media host only when enabled - Register session cleanups through a store registry instead of importing Media from auth - Wire combobox labels and descriptions for assistive tech, use i18n locale for numbers and dates, restore the document title, remove the unused Composer prop and five unused translation keys, and add Studio route, handoff, cancel and session-cleanup coverage Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Reuse existing storage strategies, balances, transaction records and credentials. Add authenticated original and rendition delivery, durable Runway and Krea cancellation, and configuration checks before paid work. Validate with focused API, persistence and UI tests, four workspace typechecks, production builds, browser workflows and the unchanged Lighthouse performance gate.
Unify credentials, admission, billing, retention, observability and Insights with existing LibreChat services. Add operator recovery, resilient client transitions, native replay ownership, SDK contract verification and regression coverage for the integration audit.
Start title generation alongside provider submission while retaining the job's lease and accounting protections through cancellation and shutdown. Use the shared uncapped title-model options and normalize replies to plain text. Remove duplicated tests and unused aliases, simplify the media configuration example, and include all media browser scenarios in the test shortcut.
Reuse the shared deletion dialog, share form selection, and simplify provider key configuration. Batch thread file caching while preserving metadata and order. Remove unused persistence exports and forwarding wrappers, and consolidate duplicated tests while retaining failure, authorization, and replay coverage.
Preserve background receipt handling and both terminal-event projections. Keep the temporary media bridge on SDK 3.8.8 so conversation-isolated workspace routing remains intact.
Keeps the temporary @librechat/agents 3.9.0 pin: the native media patch does not apply to 3.9.1, and nothing on dev consumes a 3.9.1-only export. ChatRoute combines dev's route reconciliation with the branch's parallel message load, and its new spec mocks the two hooks that adds. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A substring lookup priced gemini-3.1-flash-image at the gemini-3.1 text rate, and Gemini usage dropped thoughtsTokenCount, so image jobs settled far below cost. Media pricing now accepts a pattern match only when it names the whole model, and thinking tokens count as output. In balance mode a provider-reported failure without a cost now releases its hold instead of waiting in requires_attention for an operator. Uncertain outcomes and cancellations stay unresolved. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…able A Google model listed in a media integration defaulted to TEXT+IMAGE, and start() threw when the chat surface was off, the user lacked MEDIA.CREATE or media was disabled, so the whole chat turn failed. A default image request that cannot be recorded now asks the provider for TEXT only; an explicit IMAGE request still fails as before. Native signature publishing also recovers after one failed metadata write instead of rejecting every later publication. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every account deletion ran media index builds, prerelease migrations and owner upserts, even on deployments that never enabled media. Deletion now skips the media fence entirely until media has been activated. The media completion step ran after the User row was deleted without its own error handling, so a transient failure skipped the remaining cleanup and returned an error for an account that no longer existed. It now logs and leaves the owner to media account reconciliation, which already completes prepared owners whose User is gone. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- deleteMessages reported an already-completed deletion as failed when the immediate media consumer reconcile threw; the periodic sweep now owns the retry. - Duplicate-key recovery in saveMessage returned another writer's mediaConsumerToken to the client. - The TS ban check reads IP bans without the proxy port, but the writer kept storing them under req.ip, whose assignment is a no-op in Express. - saveGeneratedImage rejected the raw base64 and generic data URLs tool artifacts could send before; native originals stay strict. - GET /api/messages/:conversationId and /:messageId skipped the public media file projection that the other message reads apply. - Azure and Firebase stream helpers dropped their failure log. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A file cached before the Files query loaded became the whole list, because the query never refetches on mount. The seeded list is now provisional, so the next observer loads every file and an in-flight first load restarts after the file exists. Saving or revoking a key waited for the media catalog to refetch from every provider before resolving; the refresh now runs in the background like the other invalidations. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…able - An upload switched the draft between image operations (the derived edit context dropping, or the first image enabling edit), which changed the reference owner and aborted the upload it started. The owner now tracks the media kind, not the exact operation. - A 202 rejected receipt stayed pending, so sending the same draft again replayed the rejection. A new send now supersedes it. - Older turn pages were keyed by the detail cursor, so every new turn collapsed loaded history to one page. They now refetch in place. - Open in Studio and Edit attachments could land on the gallery and hide the draft they had just seeded. - A command error from one thread stayed under the next thread. - The live event stream never reconnected after a clean server close, and event bursts kept cancelling the in-flight thread fetch. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- The worker advanced its due-scope cursor before a page that could stop at maxActiveTotal, so owners after the break waited a full cycle. - Unknown provider states read as running and polled forever outside balance mode; polling now shares recovery.attentionAfterMs. - Hosted URL imports had no client abort and only a socket idle timeout, so a dripping server could hold memory and capacity slots. The whole read is now bounded by timeouts.downloadMs. - Catalog discovery was cached per owner even for a shared credential. It is now keyed by tenant, binding and resolved headers, so users on one deployment key share discovery while user keys stay separate. - MiniMax server errors, or an error that still names a task, no longer release a hold as a definite rejection. - A key sent under a custom Authorization scheme (Sourceful) is now redacted from diagnostics. - OpenRouter bring-your-own-key usage adds the upstream inference cost to OpenRouter's fee. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The six media method factories this branch adds sat beside the existing methods/media directory as multi-word files. They now live inside it as accounting, consumers, native, preset, recovery and title, with their specs, and the package exports are unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The e2e cleanup helper assumed prepareMediaAccountDeletion always opened a media session; it now skips media reconciliation on runs that never activated media and passes the required log. The storage read helper imports its error metadata relatively, so /api specs that load it from source resolve it without the packages/api path alias. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Keeps the temporary @librechat/agents 3.9.0 pin over dev's ^3.9.3: the native media patch targets 3.9.0, and nothing on dev consumes an export added in 3.9.1-3.9.3. ChatView nests StudioProvider around dev's QueuedTurnPortalProvider and takes dev's z-10 composer stacking fix; dev's new stacking spec passes the branch's required messagesReady prop. ControlCombobox keeps the forwarded trigger ref and option actions and adds dev's popoverMaxHeight and unsearchedLimit. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…t path schemes Public share links passed Studio files through a blocklist that did not cover the media* File fields, so anonymous viewers received rendition storage keys, the owner's thread and conversation ids, and claim and deletion tokens. The share sanitizer now drops media* fields, matching what toPublicMessageFiles already strips on logged-in routes. mediaAssetSchema accepted any filepath, and toAbsoluteFilePath passes javascript: through to download links. Asset and rendition paths now accept only same-origin paths and http(s), read with URL so whitespace, tab and case variants are covered. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Collaborator
Author
|
Security review of Checked: authorization and owner/tenant scoping on every media route and query, admin routes, SSRF on user-submitted and provider-returned URLs, storage paths and file serving, credential handling, balance holds and settlement, agent tool, chat and share paths, frontend sinks, and the SDK patch, lockfile and CI changes. Found: nothing high-severity. Two low-severity issues:
Fixed in 168c72c:
Both fixes have tests that fail on the old code. |
Moves the native media bridge to @librechat/agents 4.0.1, the release dev targets: both consumers pin it exactly and patches/@librechat+agents+4.0.1.patch is regenerated from LibreChat-AI/agents#553 rebased onto 4.0.1, replacing the 3.9.0 patch. Conflict resolutions carry Studio onto dev's reshaped code: - "Create media" moves from the removed AttachFileMenu into the Attach and tools palette (useAttachItems), with the same chat/create gating. - Native media stream handling follows dev's move into hooks/SSE/steps. - Media file claims and native signatures wrap dev's single writeMessage path. - Cookie authentication for share, image and media files stays one module (images/cookies.ts) and gains dev's two-factor enrollment and token retirement checks; dev's parallel auth/share.ts is dropped. - Ban checks use dev's getBanIp without mutating req.ip. Studio components now satisfy dev's design lint: spacing and color move off Collapsible primitives onto plain wrappers, section actions use the section-action Button variant, image frames size through a --media-ratio custom property, and two shared variants cover needs other screens share (Input leading-icon for search fields, Button composer for composer-row controls; pressed header-action buttons fill through aria-pressed). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Keeps the native media factory beside dev's scheduled MCP execution options in both createRun calls, passes routePending to ChatForm now that dev has removed the composer tips setting, and adopts dev's split PixelCard frame loop and DiffRow rendering. Reduced-motion PixelCard stills reset the shared fill style cache, since they draw outside the animation frame that resets it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Resolves conflicts with dev's async local file saves, tool approval grants, UI scale, settings-open atom, running-chat rename and route re-nesting. Local storage keeps the branch's stream storage and unlink helpers alongside writeLocalFile; ControlCombobox keeps its forwarded ref and nested-dialog Escape handling and gains portalElement and rem-scaled rows; the studio routes move into dev's re-nested Root children. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Media Studio for creating and iterating on images and videos with persistent history and recoverable generation jobs. With
media.enabledconfigured, users can generate in Studio, reuse results in chat, and open chat media in Studio. Compatible follow-ups reuse the previous result; history cards show the latest available output.Depends on LibreChat-AI/agents#553. Related proposal: discussion #16140.
How it works
The API host supplies configuration, persistence, storage, and provider transports. Studio records its intent before a paid request; a worker publishes ordered File outputs and reconciles interrupted work. Native chat uses the SDK port and the existing chat accounting path.
Database contracts and queries live in
packages/data-schemas, runtime behavior inpackages/api/src/media, shared configuration and DTOs inpackages/data-provider/src/media, and Studio UI/state inclient/src/components/Media. Legacy/apicode wires the host into the application. Startup uses canonical user IDs, starts chat history alongside conversation metadata, and loads the Media activity snapshot before opening its event stream.Type of change
Testing
Current head:
d7e616f49d8f3a334d47139c2cbda645758dfb56.Manual browser testing also exercised configured Azure, Gemini, Vertex, and OpenRouter models and follow-up requests. CI browser flows use local fixtures without paid inference.
Risk / compatibility
@librechat/agents4.0.1, the releasedevtargets; the patch is generated from feat: add native media port for Google image output and host model-invocation tracing agents#553 rebased onto 4.0.1. It must be replaced by published SDK contracts before this PR is ready.Draft readiness
librechat.airepository as part of final review.