[Azure] Add AMQP-over-WebSockets support to Event Hub integrations - #20559
Open
zmoog wants to merge 5 commits into
Open
[Azure] Add AMQP-over-WebSockets support to Event Hub integrations#20559zmoog wants to merge 5 commits into
zmoog wants to merge 5 commits into
Conversation
Contributor
Elastic Docs Style Checker (Vale)Summary: 10 warnings found
|
| File | Line | Rule | Message |
|---|---|---|---|
| packages/azure_ai_foundry/data_stream/logs/manifest.yml | 174 | Elastic.QuotesPunctuation | Place punctuation inside closing quotation marks. |
| packages/azure_ai_foundry/data_stream/logs/manifest.yml | 187 | Elastic.QuotesPunctuation | Place punctuation inside closing quotation marks. |
| packages/azure_app_service/manifest.yml | 192 | Elastic.QuotesPunctuation | Place punctuation inside closing quotation marks. |
| packages/azure_app_service/manifest.yml | 205 | Elastic.QuotesPunctuation | Place punctuation inside closing quotation marks. |
| packages/azure_functions/_dev/build/docs/README.md | 178 | Elastic.QuotesPunctuation | Place punctuation inside closing quotation marks. |
| packages/azure_functions/_dev/build/docs/README.md | 182 | Elastic.QuotesPunctuation | Place punctuation inside closing quotation marks. |
| packages/azure_functions/data_stream/functionapplogs/manifest.yml | 175 | Elastic.QuotesPunctuation | Place punctuation inside closing quotation marks. |
| packages/azure_functions/data_stream/functionapplogs/manifest.yml | 188 | Elastic.QuotesPunctuation | Place punctuation inside closing quotation marks. |
| packages/azure_openai/data_stream/logs/manifest.yml | 174 | Elastic.QuotesPunctuation | Place punctuation inside closing quotation marks. |
| packages/azure_openai/data_stream/logs/manifest.yml | 187 | Elastic.QuotesPunctuation | Place punctuation inside closing quotation marks. |
The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.
Contributor
TL;DR
Remediation
Investigation detailsRoot CauseThis is a configuration/docs sync failure (not a runtime code bug). The lint step compares generated README content against the committed file and detected drift. In this PR, the template and rendered docs changed near the file end, including EOF/newline handling:
The Buildkite check reports the generated output wants extra trailing lines that are not present in the committed README. Evidence
Verification
Follow-upIf this exact mismatch reappears after regeneration, compare end-of-file newline/trailing blank-line handling between What is this? | From workflow: PR Buildkite Detective Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not. |
The new `transport` option is read only by the v2 Event Hub processor (`CreateEventHubConsumerClient` is called exclusively from `v2_input.go`; the v1 input uses the legacy azure-event-hubs-go SDK and ignores it). azure_ai_foundry, azure_openai, azure_functions and azure_app_service never emitted `processor_version`, so Beats applied its default, which is v1 on 8.19, 9.1 and 9.2. Selecting AMQP-over-WebSockets on those versions silently kept dialing AMQP on 5671/5672. Add a `processor_version` var defaulting to v2 and emit it from the agent templates, matching azure and azure_logs. Also fix the docs: name the processor version requirement explicitly and correct the supported agent versions per package (azure_ai_foundry cannot be installed before 9.2.4, azure_openai has no 9.1.x). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Bring azure_ai_foundry, azure_openai, azure_functions and azure_app_service
in line with azure and azure_logs by exposing the rest of the processor v2
settings: processor_update_interval, processor_start_position,
partition_receive_timeout, partition_receive_count and migrate_checkpoint.
migrate_checkpoint is emitted unconditionally rather than behind an
`{{#if}}` guard so that setting it to false is honoured; a guarded bool
renders nothing when false and the Beats default (true) would apply.
endpoint_suffix is deliberately left out: it is not an input option, only
a fragment of the storage_account_connection_string that azure and
azure_logs assemble by hand. These packages let Beats build that string,
and Beats derives the suffix from authority_host, which they already
expose.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
zmoog
marked this pull request as ready for review
August 6, 2026 17:09
Contributor
🚀 Benchmarks reportPackage
|
| Data stream | Previous EPS | New EPS | Diff (%) | Result |
|---|---|---|---|---|
logs |
1709.4 | 1308.9 | -400.5 (-23.43%) | 💔 |
To see the full report comment with /test benchmark fullreport
Contributor
|
✅ All changelog entries have the correct PR link. |
💚 Build Succeeded
History
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed commit message
Add AMQP-over-WebSockets transport support to the Azure Event Hub integrations (Azure OpenAI, Azure Functions, Microsoft Foundry, Azure App Service, and Custom Azure Logs).
transportdefaults toamqp;websockettunnels AMQP over HTTPS port443, enabling Event Hubs traffic through restrictive firewalls andHTTPS_PROXYproxies.Because
transportis a processor v2 option, this also exposes the processor v2 settings in the four packages that lacked them and flips their default processor version from v1 to v2 (matchingazure, #16955). On upgrade,migrate_checkpoint(defaulttrue) migrates v1 checkpoints once, so consumption resumes where v1 left off.This was requested for exactly these five integrations: Azure OpenAI, Azure Functions, Microsoft Foundry, Azure App Service, and Custom Azure Logs (which already used processor v2). Existing kibana constraints already require a stack supporting
transport(8.19.10 / 9.1.10 / 9.2.4+, per package), so no constraint changes are needed.Changes
transportselector (amqpdefault;websocketon port443) in eachazure-eventhubinput.processor_version,migrate_checkpoint,processor_update_interval,processor_start_position,partition_receive_timeout,partition_receive_count) and default to v2 for Azure OpenAI, Azure Functions, Microsoft Foundry, and Azure App Service; Custom Azure Logs already had them.HTTPS_PROXYenvironment variable and that it requireswebsockettransport.Impact on existing installations
The four packages above move from processor v1 to v2 on upgrade; the automatic v1→v2 checkpoint migration preserves the consumer position.
Verification
elastic-package buildandelastic-package checkfor all five packages.git diff --check.SVR00002; that dashboard is unchanged.Related issues