From 6fbc35f942c3b04c3f78afd057623f0724096161 Mon Sep 17 00:00:00 2001 From: Carson Ip Date: Tue, 21 Apr 2026 13:59:02 +0100 Subject: [PATCH 1/2] [APM] Add known issue for HTTP/2 strict client framing failures (#4979) * docs: add APM HTTP/2 strict client known issue for 8.x Document the HTTP/2 framing/protocol failure symptoms, workaround guidance, and 8.19.15 fix version in APM known issues without 9.x references. * docs: align HTTP/2 known-issue wording with apm-server Match the source known-issue details and wording for strict HTTP/2 client framing failures, while keeping the entry scoped to 8.x versions. Made-with: Cursor * docs: use source wording for HTTP/2 workaround Made-with: Cursor * docs: explicitly label HTTP/2 guidance as workaround Made-with: Cursor * docs: align workaround section with known-issues prose style Made-with: Cursor * Apply suggestions from code review Co-authored-by: Visha Angelova <91186315+vishaangelova@users.noreply.github.com> --------- Co-authored-by: Visha Angelova <91186315+vishaangelova@users.noreply.github.com> (cherry picked from commit 81a19502637ea8426d0d297915879be5dfdd408d) # Conflicts: # docs/en/observability/apm/known-issues.asciidoc --- .../observability/apm/known-issues.asciidoc | 23 +++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/docs/en/observability/apm/known-issues.asciidoc b/docs/en/observability/apm/known-issues.asciidoc index e5fa299dbc..0f82762b3f 100644 --- a/docs/en/observability/apm/known-issues.asciidoc +++ b/docs/en/observability/apm/known-issues.asciidoc @@ -22,6 +22,7 @@ _Versions: XX.XX.XX, YY.YY.YY, ZZ.ZZ.ZZ_ //// [discrete] +<<<<<<< HEAD == APM occasionally returning HTTP 502 "backend connection closed" or "use of closed network connection" _Elastic Stack versions: >=8.0.0 and <8.18.8 or <8.19.5, >=9.0.0 and <9.0.8 or <9.1.5_ @@ -67,6 +68,28 @@ To work around this issue you can either: In both cases, the settings of APM Integration are maintained. However, these workarounds will only keep APM Integration healthy until next integration policy change. This bug will be fixed in 8.19.1 and 9.1.1. +======= +== HTTP/2 connections can fail with strict clients due to framing errors + +_Elastic Stack versions: >=8.19.12 and <8.19.15_ + +APM Server can fail HTTP/2 requests from strict clients (for example, curl/nghttp2) after ALPN negotiates `h2`. + +In affected versions, APM Server can send inconsistent SETTINGS values at connection start (an initial empty/default SETTINGS frame followed by a different SETTINGS set), and strict clients treat that sequence as an HTTP/2 protocol error and close the connection. + +When this occurs, clients can report framing/connection errors such as `Error in the HTTP2 framing layer` or `Failure when receiving data from the peer`. Browser-based HTTP requests from the RUM agent can also fail with `net::ERR_HTTP2_PROTOCOL_ERROR`. + +When clients close the HTTP/2 connection due to this issue, APM Server logs an error: `http2: received GOAWAY [FrameHeader GOAWAY len=66], starting graceful shutdown`. + +For more information, check https://github.com/elastic/apm-server/issues/20887[issue #20887]. + +There are two workarounds: + +* Use HTTP/1.1 to communicate with APM Server on affected versions. +* If clients require HTTP/2, place a non-strict load balancer in front of APM Server and terminate HTTP/2 at the load balancer (for example, on some load balancers this can be done by switching from TCP mode to HTTP mode). + +This bug will be fixed in 8.19.15. +>>>>>>> 81a19502 ([APM] Add known issue for HTTP/2 strict client framing failures (#4979)) [discrete] == Tail Sampling may not compact / expired TTLs as quickly as desired, causing increased storage usage. From bc8bacad58c29b83b7b97c8a8f010e63119aeaa7 Mon Sep 17 00:00:00 2001 From: Carson Ip Date: Tue, 21 Apr 2026 16:27:55 +0100 Subject: [PATCH 2/2] Fix conflict --- docs/en/observability/apm/known-issues.asciidoc | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/en/observability/apm/known-issues.asciidoc b/docs/en/observability/apm/known-issues.asciidoc index 0f82762b3f..4f1cbb22ae 100644 --- a/docs/en/observability/apm/known-issues.asciidoc +++ b/docs/en/observability/apm/known-issues.asciidoc @@ -22,7 +22,6 @@ _Versions: XX.XX.XX, YY.YY.YY, ZZ.ZZ.ZZ_ //// [discrete] -<<<<<<< HEAD == APM occasionally returning HTTP 502 "backend connection closed" or "use of closed network connection" _Elastic Stack versions: >=8.0.0 and <8.18.8 or <8.19.5, >=9.0.0 and <9.0.8 or <9.1.5_ @@ -68,7 +67,8 @@ To work around this issue you can either: In both cases, the settings of APM Integration are maintained. However, these workarounds will only keep APM Integration healthy until next integration policy change. This bug will be fixed in 8.19.1 and 9.1.1. -======= + +[discrete] == HTTP/2 connections can fail with strict clients due to framing errors _Elastic Stack versions: >=8.19.12 and <8.19.15_ @@ -89,7 +89,6 @@ There are two workarounds: * If clients require HTTP/2, place a non-strict load balancer in front of APM Server and terminate HTTP/2 at the load balancer (for example, on some load balancers this can be done by switching from TCP mode to HTTP mode). This bug will be fixed in 8.19.15. ->>>>>>> 81a19502 ([APM] Add known issue for HTTP/2 strict client framing failures (#4979)) [discrete] == Tail Sampling may not compact / expired TTLs as quickly as desired, causing increased storage usage.