support claude 4.* models - #9
Conversation
|
Hello @jfbloom22! Thank you so much for your work on this. The community needs a better go to Anthropic integration that is supported/updated. If this is approved, I'll reach out to Justin to suggest this as a consideration for the update to our Pipe on the Open WebUI community site. One problem I have with your implementation of Opus4.1 (for all the API token rich out there): Claude Opus 4.1 has the same constraint and returns 400: “temperature and top_p cannot both be specified for this model.” Please gate Opus 4.x the same way as Sonnet 4.5 so only one sampler is sent. Suggested change: # Handle temperature/top_p settings
ONE_SAMPLER_PREFIXES = ("claude-sonnet-4-5", "claude-opus-4")
if api_model_name.startswith(ONE_SAMPLER_PREFIXES):
if is_thinking_model:
payload.pop("top_p", None)
payload["temperature"] = 1.0
elif self.valves.CLAUDE_45_USE_TEMPERATURE:
payload.pop("top_p", None)
payload["temperature"] = body.get("temperature", 0.8)
else:
payload.pop("temperature", None)
payload["top_p"] = body.get("top_p", 0.9)
else:
# Other Claude models support both
payload["temperature"] = body.get("temperature", 0.8)
payload["top_p"] = body.get("top_p", 0.9)This will prevent the 400 on Opus 4.1 while keeping existing behavior for 4.5. Thank you! |
- Updated valve naming for temperature settings to be more generic for Claude 4.x models. - Added a new method to determine if a model is a Claude 4.x generation model with temperature/top_p constraints.
|
@christian-taillon Thanks for the feedback. I am trying to make this a bit more future proof and as best I can tell Anthropic intends to only support top_p or temperature for all 4.* models. I added a dedicated pattern match: |
…logic for requests, and summary format extraction. Updated version to 0.7.0 and improved connection pooling for better performance.
|
fyi this is working really well for me. Haiku 4.5 started working automatically |
|
That is great. For reference, I'm using this integration hosted on my github for some extra features. Feel free to cannibalize anything you'd like. We (the community) definitely could up our game on Anthropic integration. I'd love to see oauth support added for Pro license users (some of my colleges): christian-taillon/open-webui-pipelines/pipe/anthropic.py |
|
@christian-taillon cool! thanks. Are you aware of any other pipes that have oauth working? maybe I can cannibalize 😄 |
|
I am not. However OpenCode supports it and maybe we can learn from how they are handling the Anthropic provider. |
|
I am not sure what happened, but Haiku stopped working for me and I realized I needed to add haiku to the regex. Not it is working for me. |
…t generation capabilities. Includes error handling, logging, and support for permissive safety settings. Initial version 0.1.4.
…iance, automatic resource management, and improved error handling. Refactor client creation methods and streamline model retrieval process. Add exponential backoff for transient errors and update documentation for clarity.
… streaming operations. Update description and features for clarity, emphasizing persistent async client usage and improved error handling. Streamline response handling for both streaming and non-streaming requests.
…voice selection, audio generation, and error handling. Implements API integration with ElevenLabs for voice fetching and speech synthesis, along with user interaction for voice selection.
…tom voice configurations and improved voice selection logic. Introduce a new README.md for setup instructions, voice management, and troubleshooting. Enhance error handling for voice selection and provide detailed voice descriptions for better user experience.
…ble audio output. Update README.md with instructions for setting audio format options (mp3, wav, pcm_16000, ulaw_8000) and enhance file generation logic to reflect selected format.
…ideo transcripts. Include README.md noting bot detection limitations. Implement video ID extraction and transcript fetching with language preferences and error handling.
…tatements with logging for request failures and JSON parsing issues. Introduce request ID extraction for better traceability of errors in streaming and non-streaming responses. Update timeout settings for requests to improve reliability.
|
I merged some things into my main branch. let me know if you there is interest in re-opening this. I will open it with an anthropic specific branch. |
|
went ahead and opened a new version of this PR: #19 |
Claude 4.5 (and Claude Opus 4.1) has a strict API constraint that prevents specifying both temperature and top_p parameters simultaneously.
updated your anthropic/main.py pipe with intelligent parameter handling
Smart Detection: The pipe detects if both parameters are provided by checking if they're different from their defaults
Priority Logic: If both are provided, it prioritizes temperature (most common use case)
Fallback: If only top_p is provided, it uses top_p instead